Development

This doc explains how to set up a development environment for Numaflow.

Install required tools

1. go 1.19+.
2. git.
3. kubectl.
4. protoc 3.19 for compiling protocol buffers.
5. pandoc 2.17 for generating API markdown.
6. Node.js® for running the UI.
7. yarn.
8. A local Kubernetes cluster for development usage, pick either one of k3d, kind, or minikube.

Example: Create a k8s cluster with k3d

# Create a cluster with default name k3s-default
k3d cluster create -i rancher/k3s:v1.24.4-k3s1

# Get kubeconfig for the cluster
k3d kubeconfig get k3s-default

Useful Commands

• make start Build the source code, image, and install the Numaflow controller in the numaflow-system namespace.

• make build Binaries are placed in ./dist.

• make codegen Run after making changes to ./pkg/api/.

• make test Run unit tests.

• make image Build container image, and import it to k3d, kind, or minikube cluster if corresponding kubeconfig is sourced.

• make docs Convert the docs to Github pages, check if there's any error.

• make docs-serve Start an HTTP server on your local to host the docs generated Github pages.