Destinations Configuration

Each destination is represented by YAML node of destinations section in the configuration file

destinations:
  destination_name1:
    type: postgres | snowflake | redshift | s3 | bigquery | clickhouse | mysql | google_analytics | facebook | amplitude | hubspot
    mode: stream | batch #Optional. Default value is 'batch'
    only_tokens: [] #Optinal. Default value is array with all authorization tokens
    staged: true | false #Optional. Default value is false
    data_layout: #Optional
      table_name_template: $.event_type #Optional. Default value is 'events'
      mappings: #Optional. See documentation link below
        ...
      primary_key_fields: [] #Optional. See documentation link below
    enrichment: #Optional. See below for details
      - rule1: #rule 1
      - rule2: #rule 1
    log: #Optional. See documentation link below
      ...
    users_recognition: #Optional. Overrides global configuration. See documentation link below
      ...

  destination_name2: ...

Except that, destination-specific parameters (usually, connection credentials) should be present. Please, see individual destination configuration pages (see full list below).

Please set up a destination name (destination_name1 in the example) carefully. This value will be treated as id in multiple places (monitoring counters, logs, folder names) and should not be changed

Following parameters are common for all destination types

Parameter (** required)	Description
type **	One of supported destination types
mode	`batch` or `stream`. Use batch for highloaded datasets (>10 events per second) and stream for others. Default value is `batch`
only_tokens	List of authorization tokens (secrets) ids. It is used for delimiting data from different tokens to different destinations. Please, see Authorization section. Please, put token id rather then client/server secret. Default value is all authorization tokens
data_layout.mappings	Optional parameter to configure the mapping. See Schema and Mappings
data_layout.primary_key_fields	Optional parameter to configure primary key (works for PostgresSQL and Redshift so far). See Primary keys configuration
data_layout.table_name_template	Name of the destination table. Can be either constant string (all events will be written in a single table) or expression in go template language . The subject of expression is the event JSON. Example: `data_{{.event_type}}`
enrichment	Data Enrichment rules configuration. See Enrichment Rules page
staged	If set to true, data won't be stored at the destination. Only dry run operation is supported for staged destinations

Configuring destinations via HTTP - endpoint#

If destinations configuration is generated by an external service, it is possible to externalize via HTTP end - point (or file) as follows:

destinations: "location"

The location can behttp(s):// of a local file (/path/to/file) location and should contain YAML or (JSON that is identical to YAML structure). If the location is an URL, the client will respect If-Modified-Since / Last-Modified caching.

Example of URL content:

{
  "destinations": { #json object where inner keys - destinations unique names
    "redshift_dab213ibda": { #destination config object
      "type": "redshift",
      ...
    },
    "clickhouse_in31o31": {
      "type": "clickhouse",
      ...
    }
  }
}