Like Jitsu? Give us a star on ⭐ GitHub!

πŸ“œ Configuration

Configuration UI

πŸ‘©β€πŸ”¬ Extending Jitsu

Overview
Destination Extensions
Source Extensions
API Specs

Jitsu Internals

Configuration File

If Jitsu Server is deployed in a standalone mode, it could be configured via single YAML file (read more about deployment options here). The most popular options can be defined as environment variables too. We're aiming to make Jitsu 100% configurable with env variables, but we're not there yet. Besides, you can host Jitsu config file on an external storage - S3, HTTP etc - with JITSU_CONFIGURATOR_URL env var

We follow convention over configuration so the majority of parameters are optional. To get a base file, clone it from GitHub.

Redis

Jitsu Server requires Redis. Jitsu can run without it, however it will work in single-node mode without synchronization, queue persistence and other high-availability features. It's good enough for development, but not recommended to for production.

Redis version 5 and higher is supported. Here's a full list of features that will not work if Redis is not available

Redis credentials should be put into to meta.storage.redis section (see below)

Configuration file structure

Jitsu config file consists of the following sections:

  • server β€” General configuration parameters such as port, application logs configuration, singer bridge configuration, etc.
  • geo β€” Geo resolution data (extracting city/state information from the IP address). We currently only support MaxMind as a data provider. see Geo Data resolution
  • log β€” Jitsu writes all events locally and sends them to their destinations (in batch mode). This is where you configure your local temporary path and push frequency.
  • sql_debug_log β€” All SQL statements such as DDL and DML expressions can be stored in separated log files or in stdout. see SQL Query Logs
  • api_keys β€” A set of API Keys objects that identify incoming events JSONs and mapping between destinations is done based on them. see Authorization page
  • destinations β€” A set of targets where the final version of events will be stored. see Destinations Configuration
  • sources β€” A set of data sources to synchronize from. see Sources Configuration
  • users_recognition β€” Jitsu can update past events with user identifiers on user's identification event! see Retroactive Users Recognition
  • coordination β€” coordination service configuration. It is used in cluster Jitsu deployments. see Scaling Jitsu
  • notifications β€” notifier configuration. Server starts, system errors, synchronization statuses, and panics information will be sent to it. Currently, only Slack notifications are supported.
  • meta.storage - meta storage is the main application storage and it is required for some features. At present Jitsu supports only Redis version 5 and higher.
  • ui.base_url – base Configurator UI URL for generating links in notifications
  • node – node.js process pool size and max heap space in megabytes per process (node is used to execute JavaScript transformations and plugins).

Example:

server:
  name: instance1.domain.com
  port: 8081
  public_url: https://instances.domain.com
  log:
    path: /home/eventnative/data/logs/
  metrics.prometheus.enabled: true

geo.maxmind_path: /home/eventnative/data/config

log:
  path: /home/eventnative/data/logs/events
  rotation_min: 5

sql_debug_log:
  ddl:
  queries:

api_keys:
  - id: my_website_key
    client_secret: '193b6281-f211-47a9-b384-102cf4cd2d55'
    server_secret: 'a6880b16-73d0-476e-840a-804b74f7684d'

destinations:
  redshift:
  bigquery:

sources:
  facebook:
  google_analytics:

users_recognition:
  enabled: true

coordination:
  type: redis

notifications:
  slack:
    url: https://slack_web_hook_url

ui:
  base_url: https://your_configurator_ui_host:5555

meta:
  storage:
    redis:
      host: redis_host
      port: 6379
      password: secret_password
      database: 0

node:
  pool_size: 1 # default
  max_space: 100 # default

You can set ${env.OS_ENV_VAR_NAME} to any configuration parameter in YAML file. Jitsu will get the value from OS ENV (with name OS_ENV_VAR_NAME from the example).

Server

All fields from the server section are optional:

FieldTypeDescriptionDefault value
namestringUnique instance name. It is used in cluster deployments.unnamed-server
portintTCP port for the server to listen on.8001
public_urlstringService public URL. It is used on the welcome HTML page. Required in Heroku deployment.Will be got from Host request header
log.pathstringPath to application logs. If not set, app logs will be in stdout.-
log.rotation_minintLog files rotation minutes. If log.path is configured.-
api_keys_reload_secintIf an URL is set in api_keys section, authorization will be reloaded every api_keys_reload_sec seconds. see Authorization page.1
destinations_reload_secintIf an URL is set in destinations section, destinations will be reloaded every destinations_reload_sec seconds. see Destinations.1
sources_reload_secintIf an URL is set in sources section, sources will be reloaded every sources_reload_sec seconds. see Sources.1
admin_tokenstringsee Admin Endpoints page.-
metrics.prometheus.enabledbooleansee Application Metrics page.false
telemetry.disabled.usagebooleanFlag for disabling telemetry. Jitsu collects usage metrics about how you use it and how it is working. We don't collect any customer data.false
metrics.relay.disabledbooleanDisables extended telemetry metrics collection.false
metrics.relay.deployment_idstringAllows to provide deployment ID for extended telemetry collection.Cluster ID
disable_version_reminderbooleanFlag for disabling log reminder banner about new Jitsu versions availability.false
sync_tasks.store_logs.last_runsintLogs for how many task runs must be kept in meta storage. Controlled on Source's collection level. When number of task runs for Source collection exceed provided value – old records get removed from meta storage.-1 unlimited number of logs
event_enrichment.http_contextbooleanWhether the server should enrich incoming HTTP events with HTTP context (headers, etc.). Please note that when upgrading from Jitsu 1.41.6 you can switch this setting to true only separately from the upgrade itself, otherwise event data may get corrupted.false

Log

Jitsu Server supports destinations in streaming and batch modes. In the case of batch mode, all events are stored in JSON log files locally to path directory, and every rotation_min minutes they are processed and pushed to destinations. All fields from log section are optional:

FieldTypeDescriptionDefault value
pathstringEvents log files path./home/eventnative/data/logs/events
rotation_minintLog files rotation minutes.5
show_in_serverbooleanFlag for debugging. If true - all events JSON data is written in app logs.false