Sidecar Containers

Additional "sidecar" containers can be provided for source, udf and sink vertices.

The following example shows how to add a sidecar container to a udf vertex.

apiVersion: numaflow.numaproj.io/v1alpha1
kind: Pipeline
metadata:
  name: my-pipeline
spec:
  vertices:
    - name: my-udf
      sidecars:
        - name: my-sidecar
          image: busybox:latest
          command:
            [
              "/bin/sh",
              "-c",
              'echo "my-sidecar is running!" && tail -f /dev/null',
            ]
      udf:
        container:
          image: my-function:latest

There are various use-cases for sidecars. One possible use-case is a udf container that needs functionality from a library written in a different language. The library's functionality could be made available through gRPC over Unix Domain Socket. The following example shows how that could be accomplished using a shared volume.

It is the sidecar owner's responsibility to come up with a protocol that can be used with the UDF. It could be volume, gRPC, TCP, HTTP 1.x, etc.,

apiVersion: numaflow.numaproj.io/v1alpha1
kind: Pipeline
metadata:
  name: my-pipeline
spec:
  vertices:
    - name: my-udf-vertex
      volumes:
        - name: my-udf-volume
          emptyDir:
            medium: Memory
      sidecars:
        - name: my-sidecar
          image: alpine:latest
          command:
            [
              "/bin/sh",
              "-c",
              "apk add socat && socat UNIX-LISTEN:/path/to/my-sidecar-mount-path/my.sock - && tail -f /dev/null",
            ]
          volumeMounts:
            - mountPath: /path/to/my-sidecar-mount-path
              name: my-udf-volume
      udf:
        container:
          image: alpine:latest
          command:
            [
              "/bin/sh",
              "-c",
              'apk add socat && echo "hello" | socat UNIX-CONNECT:/path/to/my-udf-mount-path/my.sock,forever - && tail -f /dev/null',
            ]
          volumeMounts:
            - mountPath: /path/to/my-udf-mount-path
              name: my-udf-volume