Connector Syncs

Syncs rely on a Kubernetes cluster: each sync run executes as a Kubernetes pod, and scheduled syncs are managed as Kubernetes CronJobs created and reconciled by the syncctl service.

Warning

This page applies to the deprecated Docker Compose setup. With the recommended development Helm chart, syncctl runs inside the cluster and no extra configuration is needed.

The setup below attaches a cluster to the compose stack for testing.

How to set up

Connect the syncctl service to a Kubernetes cluster with the following variables in docker/.env.local:

  • SYNCS_ENABLED - set to true to enable syncs in the UI
  • SYNCCTL_KUBERNETES_CLIENT_CONFIG - kubeconfig of the cluster: either a path to a kubeconfig file mounted into the container, or the kubeconfig content itself in YAML format. The special value local (default) means in-cluster config and only works when syncctl itself runs inside Kubernetes.
  • SYNCCTL_KUBERNETES_CONTEXT - (optional) kubeconfig context to use, if not the current one

Without SYNCCTL_KUBERNETES_CLIENT_CONFIG set, the syncctl container exits gracefully and the rest of the stack works without syncs.

Minikube example

Pre-requisites

Start minikube

minikube start

Get the kubeconfig

kubectl config view --raw=true --minify=true --flatten=true

Copy the output into the SYNCCTL_KUBERNETES_CLIENT_CONFIG variable in docker/.env.local (keep the YAML indentation intact: use a line break right after the opening quotation mark). Note that the API server address in the kubeconfig must be reachable from inside the syncctl container — with minikube on the same machine, replace 127.0.0.1 with host.docker.internal.

Adjust docker/.env.local

SYNCS_ENABLED=true
SYNCCTL_KUBERNETES_CLIENT_CONFIG="
apiVersion: v1
kind: Config
...
"

Start Jitsu

docker compose -f ./docker/docker-compose.yml up --force-recreate

After it's running

The Connectors menu item appears in the Jitsu Console UI. Scheduled syncs materialize as CronJobs named sync-<syncId> (labeled jitsu.com/managed-by=syncctl) in the configured namespace; ad-hoc runs are started as regular pods.

To fill the connectors catalog, trigger the catalog-refresh console API endpoint in a browser with an authorized Jitsu user:

http://localhost:3000/api/admin/catalog-refresh