Configuration Reference

HTTP port where Ingest will be available.

Public url where ingest service is deployed and available from internet, usually it's load balancer or reverse proxy. E.g. https://data.jitsu.mycompany.com. Should contain protocol and port (if it's not default).

INGEST_REPOSITORY_URL is an URL of console's export endpoint that returns configuration of streams-with-destinations entities: https://$console-endpoint/api/admin/export/streams-with-destinations.

INGEST_REPOSITORY_AUTH_TOKEN is used to authorize request to console. It must start with service-admin-account: prefix. E.g.: service-admin-account

See also CONSOLE_AUTH_TOKENS of console configuration

Period in seconds for refreshing configuration from console's INGEST_REPOSITORY_URL.

List of Kafka brokers separated by comma. Each broker should be in format host

If SSL should be enabled for Kafka

Skip SSL verification of kafka server certificate.

Kafka authorization as JSON object. E.g.: {"mechanism": "SCRAM-SHA-256", "username": "user", "password": "password"}

URL template of function servers managed by the Operator. The default value matches the Kubernetes services the Operator creates. Used for running functions of device destinations. Replaces INGEST_ROTOR_URL of previous Jitsu versions.

Used to authorize HTTP-requests to function servers.
See ROTOR_AUTH_TOKENS, ROTOR_TOKEN_SECRET and ROTOR_RAW_AUTH_TOKENS in the Rotor section

Functions class assumed for workspaces without an explicit setting. See the Operator section.

Timeout in milliseconds for running functions of device destinations synchronously during ingestion.

Maximum allowed size of an ingested payload in bytes.

ClickHouse host and port to store incoming events log. E.g.: clickhouse.example.com:9440

ClickHouse database to store incoming events log.

ClickHouse username and password.

Enable SSL for Clickhouse connection

Format of application logs (that are written to stdout). Possible values: text or json

Failover logger is used to store events that failed to be sent to Kafka. Default values:

• INGEST_FAILOVER_LOGGER_ENABLED - false
• INGEST_FAILOVER_LOGGER_ROTATION_PERIOD_MINUTES - 60
• INGEST_FAILOVER_LOGGER_MAX_SIZE_MB - 100
• INGEST_FAILOVER_LOGGER_COMPRESS - true

Controls how failover logger files stored on local disk. Default values:

• INGEST_FAILOVER_LOGGER_BASE_PATH - /tmp/kafka_failover
• INGEST_FAILOVER_LOGGER_LOCAL_MAX_OLD_FILES - 10

Allows to set S3 destination for failover logger files.

HTTP port where Rotor will be available.

Role of this instance: rotor (Kafka consumer, the main event pipeline), functions (function server — set by the Operator) or profiles (profile builder server).

URL template of function servers managed by the Operator. Rotor calls function servers to execute user functions and profile builders.

Timeout in milliseconds for requests to function servers.

Timeout in milliseconds for fetch() calls made from user functions.

ROTOR_AUTH_TOKENS is a list of hashed auth tokens that authorizes user in HTTP interface separated by comma. Each must have format: $salt.$hash where $salt should be random string. Hash is hex(sha512($token + $salt + ROTOR_TOKEN_SECRET).

To hash token, use following command in the root of this repository: pnpm install && ROTOR_TOKEN_SECRET=xxxx pnpm tool

ROTOR_RAW_AUTH_TOKENS can be used instead of ROTOR_AUTH_TOKENS to provide a comma-separared list of raw takens instead of hases. It offers simplicity at cost of lower security.

REPOSITORY_BASE_URL is an URL of console's export endpoint that returns configuration of streams-with-destinations entities: https://$console-endpoint/api/admin/export.

REPOSITORY_AUTH_TOKEN is used to authorize request to console. It must start with service-admin-account: prefix. E.g.: service-admin-account

See also CONSOLE_AUTH_TOKENS of console configuration

Period in seconds for refreshing configuration from console's export endpoint.

ClickHouse host and port to store Events Log. E.g.: clickhouse.example.com:9440 Only HTTP(s) protocol is supported.

ClickHouse database to store Events Log.

ClickHouse username and password.

Enable SSL for Clickhouse connection

BULKER_URL is an URL of Bulker service. Rotor will use it to send event intended to warehouse destinations

BULKER_AUTH_KEY is user to authentificated HTTP-request to Bulker. Should be one of BULKER_AUTH_KEYS you configured in Bulker

Used for Events Logs feature.

List of Kafka brokers separated by comma. Each broker should be in format host

If SSL should be enabled for Kafka

Custom CA certificate for verifying the Kafka server certificate. KAFKA_SSL_CA is a certificate in PEM format, KAFKA_SSL_CA_FILE is a path to the file with the certificate.

Skip SSL verification of kafka server certificate.

Kafka authorization as JSON object. E.g.: {"mechanism": "SCRAM-SHA-256", "username": "user", "password": "password"}

MongoDB is used for Functions Persistent Storage and Identity Stitching.

The value starts with mongodb:// and has the following format: mongodb://$user:$password@$host:$port/$database

To smoothly migrate from REDIS_URL you need to set REDIS_URL and MONGODB_URL at the same time. Rotor will look for records both in MongoDB and Redis, but new records will be added only to MongoDB. When you are sure that MongoDB is populated with enough data, you can remove REDIS_URL from the configuration.

Redis connection string. Always start with redis://: redis://$user:$password:localhost:6379.
Redis Sentinel Address: sentinel1:26379,sentinel2:26379,sentinel3:26379
Can be used for Functions Persistent Storage as alternative to MongoDB.

Id of Bulker destination where Rotor will send event metrics. Only clickhouse destination is supported. If you don't want to send metrics, you can skip this option.

Number of events processed concurrently by each consumer.

Maximum number of retries for failed messages.

Base value for exponential backoff for failed messages. For example, if MESSAGES_RETRY_COUNT is 3 and base is 10, then retry delays will be 10, 100, 1000 minutes

Defines maximum possible retry delay in minutes. Default: 24 hours

MaxMind database used for GeoIP enrichment.

MaxMind database can be obtained from MaxMind servers using MAXMIND_LICENSE_KEY, from custom MAXMIND_URL or from S3 or compatible storage.

Use localized geographic names. Supported locales

Format of rotor application logs. Possible values: text or json

HTTP port where Console will be available.

This is a URL where Jitsu console will be publicly available, usually a load-balancer / reverse proxy address. E.g.: https://your-domain/

CONSOLE_AUTH_TOKENS is a list of hashed auth tokens that authorizes user in HTTP interface separated by comma. Each must have format: $salt.$hash where $salt should be random string. Hash is hex(sha512($token + $salt + CONSOLE_TOKEN_SECRET).

To hash token, use following command in the root of this repository: pnpm install && CONSOLE_TOKEN_SECRET=xxxx pnpm tool

CONSOLE_RAW_AUTH_TOKENS can be used instead of CONSOLE_AUTH_TOKENS to provide a comma-separared list of raw takens instead of hases. It offers simplicity at cost of lower security.

PostgreSQL connection string. postgres://$user:$password@localhost:$port/database?sslmode=no-verify&schema=newjitsu

schema must be newjitsu, and sslmode must be no-verify

BULKER_URL is an url of Bulker service. Used for pulling event logs. E.g.: https://bulker.your-domain.com

BULKER_AUTH_KEY is used to authorized HTTP-request to Bulker. See BULKER_AUTH_TOKENS, BULKER_TOKEN_SECRET and BULKER_RAW_AUTH_TOKENS in Bulker configuration

ROTOR_URL is an url of the Rotor service. E.g.: http://rotor:3401

ROTOR_AUTH_KEY is used to authorized HTTP-request to Rotor. See ROTOR_AUTH_TOKENS, ROTOR_TOKEN_SECRET and ROTOR_RAW_AUTH_TOKENS in the Rotor section

To enable GitHub OAuth for Jitsu.

You'll need to create a GitHub OAuth application to get those values:

1. Go to GitHub Developer settings » OAuth Apps » New OAuth App.
2. Put any value to Application name.
3. Set Homepage URL and Authorization callback URL with value of JITSU_PUBLIC_URL.
4. Press Register application button.
5. Press Generate a new client secret button.
6. Copy Client ID and Client Secret values to .env file to GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET variables respectively.

To enable OpenID Connect based authentication for Jitsu.

Expected json object with the following properties: issuer (the issuer domain in valid URL format), clientId, and clientSecret

The well-known configuration endpoint for the provider is automatically set based on the issuer, and the default authorization request includes scopes for OpenID, email, and profile information.

Auth0 Example: {"issuer":"https://{yourDomain}.us.auth0.com/","clientId":"***","clientSecret":"***"}

Optionally scopes the auth session cookie (jitsu-auth) to a parent domain so that sibling apps running on its subdomains can read the logged-in session. Example: jitsu.com makes the cookie available to use.jitsu.com, app.jitsu.com, etc.

If not set, the cookie is scoped to the Console's own host (e.g. use.jitsu.com) and is shared with that host's subdomains.

Security: widening the cookie to a parent domain shares the session across all of that domain's subdomains — only set this to a domain you fully control.

Url where Ingest service is publicly available. See INGEST_PUBLIC_URL param of ingest service for details.

Whet set to true enables Connectors Syncs feature in Jitsu Console UI. Requires syncctl service. Scheduled syncs are managed by syncctl as Kubernetes CronJobs — no external scheduler is needed (the GOOGLE_SCHEDULER_KEY variable of previous Jitsu versions was removed).

Required if SYNCS_ENABLED=true. SYNCCTL_URL is an URL of the main endpoint of Syncctl service. SYNCCTL_AUTH_KEY is Syncctl service authentication key: one of SYNCCTL_AUTH_KEYS or SYNCCTL_RAW_AUTH_TOKENS configured in Syncctl

Functions class assigned to workspaces that don't have an explicit setting: free (shared, sharded function-server deployments) or dedicated (one deployment per workspace). See the Operator section.

Maintenance mode: a JSON descriptor that switches the console API to read-only mode and shows a maintenance page. MAINTENANCE takes the JSON inline; MAINTENANCE_CONFIG_FILE takes a path to a JSON file (e.g. a mounted ConfigMap), so the mode can be toggled without restarting the console.

Replaces the JITSU_CONSOLE_READ_ONLY_UNTIL variable of previous Jitsu versions.

Per-minute sliding-window rate limiting of the console API. Enabled by default (MINUTE_RATE_LIMIT_ENABLED=true); MINUTE_RATE_LIMIT_BASE (default 60) is the base per-minute budget from which limits per auth type and HTTP method are derived.

When set to true, rejects signups with personal email addresses (gmail.com, etc.) — a work email is required.

ClickHouse host and port where Events Log is stored. E.g.: clickhouse.example.com:9440 Only HTTP(s) protocol is supported.

ClickHouse database to where Events Log is stored.

ClickHouse cluster id to properly create replicate tables for Events Log. E.g.: jitsu_cluster

ClickHouse username and password.

Enable SSL for Clickhouse connection

Email sending configuration:
SMTP_CONNECTION_STRING is a connection string to SMTP server in format: smtp://user

Format of console application logs. Possible values: text or json

HTTP port where Syncctl will be available.

A list of hashed auth tokens that authorizes user in HTTP interface separated by comma. Each must have format: $salt.$hash where $salt should be random string. Hash is hex(sha512($token + $salt + SYNCCTL_TOKEN_SECRET). $token must consist only of letters, digits, underscore and dash

SYNCCTL_RAW_AUTH_TOKENS can be used if you want to provide a comma-separared list of raw takens instead of hases. It offers simplicity at cost of lower security.

PostgreSQL connection string. postgres://$user:$password@localhost:$port/database?sslmode=no-verify&search_path=newjitsu. Should be the same as DATABASE_URL for console

search_path must be newjitsu, and sslmode must be no-verify

URL of the same PostgreSQL instance as it is reachable from kubernetes cluster. Required only if it is different from SYNCCTL_DATABASE_URL. E.g. if you use localhost in SYNCCTL_DATABASE_URL

Path to kubernetes config file or kubernetes config in yaml format. If syncctl service itself runs in kubernetes cluster, you can skip this option or use local value.

Name of kubernetes context if not the default one is used and name of kubernetes namespace where sync jobs will be created.

Maximum time in hours that sync job can run. After that time sync job will be terminated.

SYNCCTL_REPOSITORY_BASE_URL is the URL of the console's export endpoint: https://$console-endpoint/api/admin/export. Syncctl polls the syncs export and reconciles Kubernetes CronJobs (one per scheduled sync, named sync-<syncId>) to match the configured schedules. If not set, scheduled syncs are not managed — only ad-hoc runs work.

SYNCCTL_REPOSITORY_AUTH_TOKEN is used to authorize requests to console. It must start with the service-admin-account: prefix. E.g.: service-admin-account

Period in seconds for refreshing the syncs export from console.

Docker image of the sidecar container that accompanies each sync pod.

CronJob tuning: maximum job runtime in seconds (default 180000), Kubernetes retry limit for failed jobs (default 0) and maximum random start jitter in seconds (default 60).

Manual lever to force re-creation of all managed CronJobs (bump when the pod template must be re-rendered).

Sync task log retention: maximum number of log entries per task (default 3000) and retention in days (default 60). These settings were moved from console to syncctl.

OAuth token refresh for sources that authorize via Nango; the refresh runs as an init container of each sync pod.

Format of syncctl application logs. Possible values: text or json

HTTP port of the operator's status endpoints (/health, /ready, /status).

OPERATOR_REPOSITORY_BASE_URL is the URL of the console's export endpoint: https://$console-endpoint/api/admin/export. The operator polls workspaces, connections and functions from it.

OPERATOR_REPOSITORY_AUTH_TOKEN is used to authorize requests to console. It must start with the service-admin-account: prefix.

PostgreSQL connection string — the same database console uses. The operator maintains the FunctionsServer table there.

Kubernetes access: path to a kubeconfig file or inline kubeconfig YAML (default local — in-cluster config), context name, and the namespace where function-server deployments are created.

Docker image of function servers. The operator starts it with ROTOR_MODE=functions.

Port of the function-server Service. Must match the FUNCTIONS_SERVER_URL_TEMPLATE configured on Rotor, Ingest and Console.

Functions class assigned to workspaces without an explicit setting: free (shared, sharded deployments) or dedicated (one deployment per workspace).

Scaling of function-server deployments: number of shared deployments for the free class (default 1) and minimum replicas per deployment (default 2).

HorizontalPodAutoscaler settings for function-server deployments (disabled by default; max replicas 16; target CPU utilization 100).

Pod-level settings applied to function-server deployments: resource requests/limits (JSON), node selector, tolerations and service account.

MongoDB connection string used by function servers for Functions Persistent Storage and profile builders. When set, the operator adds a mongobetween connection-pooling sidecar to each function-server pod.

HTTP port where Ingest will be available.

Public url where admin service is deployed and available from internet, usually it's load balancer or reverse proxy. E.g. https://data.jitsu.mycompany.com. Should contain protocol and port (if it's not default).

ADMIN_REPOSITORY_URL is an URL of console's export endpoint that returns configuration of streams-with-destinations entities: https://$console-endpoint/api/admin/export/streams-with-destinations.

ADMIN_REPOSITORY_AUTH_TOKEN is used to authorize request to console. It must start with service-admin-account: prefix. E.g.: service-admin-account

See also CONSOLE_AUTH_TOKENS of console configuration

Period in seconds for refreshing configuration from console's ADMIN_REPOSITORY_URL.

List of Kafka brokers separated by comma. Each broker should be in format host

If SSL should be enabled for Kafka

Skip SSL verification of kafka server certificate.

Kafka authorization as JSON object. E.g.: {"mechanism": "SCRAM-SHA-256", "username": "user", "password": "password"}

Format of application logs (that are written to stdout). Possible values: text or json