Click here to read our blog and company related news. Jitsu provides an open source data integration & event collection platform to record business data in one location.

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      h2: "h2",
      p: "p",
      strong: "strong",
      a: "a",
      ul: "ul",
      li: "li"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsx(_components.h2, {
        children: "Key-Value Storage for JS Transformations"
      }), "\n", _jsx(_components.p, {
        children: "Jitsu v1.43 allows accessing a persistent key-value storage from JavaScript transformations.\nDuring processing of one event you can store some data and access it later while processing another event."
      }), "\n", _jsx(_components.p, {
        children: "See an example of Jitsu managing users sessions in our JavaScript Debugger UI:"
      }), "\n", _jsx("img", {
        src: "/img/blog/1.43/keyvalue.png",
        width: "100%"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.strong, {
          children: _jsx(_components.a, {
            href: "/docs/other-features/javascript-transform/storage",
            children: "Read more about Key-Value Storage →"
          })
        })
      }), "\n", _jsx(_components.h2, {
        children: "Batch uploader: multiple threads support"
      }), "\n", _jsxs(_components.p, {
        children: ["Jitsu uploader service that process events files for destinations in batch mode runs in a single thread by default.\nThat is enough for the most typical use cases.\nBut in cases like having a lot of configured Api Keys, single thread may be not enough to process all batch files in time.\nThat is why we have added ", _jsx(_components.a, {
          href: "/docs/other-features/batches#general-parameters",
          children: "batch_uploader.threads_count"
        }), " setting to process events batches in parallel."]
      }), "\n", _jsx(_components.h2, {
        children: "Permissions for project Users"
      }), "\n", _jsx(_components.p, {
        children: "Previously we've made possible to invite additional users to Jitsu project.\nNow with help of Permissions editor you can limit access to \"view only\" for specific user."
      }), "\n", _jsx("img", {
        src: "/img/blog/1.43/permissions.png",
        width: "100%"
      }), "\n", _jsx(_components.h2, {
        children: "Other changes"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Improved bot detection in User-Agent parser"
        }), "\n", _jsx(_components.li, {
          children: "S3 destination UI: added option to provide custom S3 endpoint"
        }), "\n", _jsx(_components.li, {
          children: "Facebook Marketing source: API version updated to 15"
        }), "\n", _jsx(_components.li, {
          children: "Bugfixes"
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};


Jitsu v1.43: Permissions for project Users, Key-Value Storage for JS Transformations, and more

v1.43: Permissions for project Users, Key-Value Storage for JS Transformations, and more

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      h2: "h2",
      p: "p",
      a: "a",
      ul: "ul",
      li: "li",
      pre: "pre",
      code: "code",
      strong: "strong"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsx(_components.h2, {
        children: "SDK for development of Source Connectors"
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu-sdk",
          children: "Jitsu Extension SDK"
        }), " now support sources. Now possible for anyone to implement\nnew source connectors (in addition to ", _jsx(_components.a, {
          href: "http://localhost:3000/sources",
          children: "100+ connectors"
        }), " we support now). New version of Jitsu Servers fully supports\nloading of such plugins."]
      }), "\n", _jsx(_components.p, {
        children: "We've chosen TypeScript as a primary language because of several reasons:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Most of the typical connectors source code is gets data from HTTP and manipulates JSON objects. Typescript\nis a greate language for that"
        }), "\n", _jsx(_components.li, {
          children: "Tons of libraries are already available for TypeScript. If you decided to implement a connector to particular\nservices, chances that there is good library available is pretty high"
        }), "\n", _jsx(_components.li, {
          children: "Jitsu Transformations is already based on JavaScript, so it only makes sense to keep the tech stack consistent"
        }), "\n"]
      }), "\n", _jsx(_components.p, {
        children: "A completely new Source Connector can be developed in a few hours"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "#Creates Source plugin project boilerplat\nnpx jitsu-cli extension create --type source\n#Implement your plugin logic ...\n#Build and publish your plugin to npm repository\nnpm build\nnpm publish\n"
        })
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.strong, {
          children: _jsx(_components.a, {
            href: "/docs/extending/source-plugins",
            children: "Read more about source extension documentation →"
          })
        })
      }), "\n", _jsx(_components.h2, {
        children: "Jitsu now supports BigQuery as destinations"
      }), "\n", _jsx(_components.p, {
        children: "Before 1.41 Jitsu supported Google Big Query as a destination only for event streaming on batches. Now it's possible\nto synchronize data from Sources to BQ to (in addition to RedShift, Postgres, ClickHouse, MySQLm and Snowflake)."
      }), "\n", _jsx(_components.h2, {
        children: "New JavaScript Transformations UX"
      }), "\n", _jsx(_components.p, {
        children: "JavaScript Transformation has been always a core feature of Jitsu. In this release we significantly redesigned it and\nadded a few useful features"
      }), "\n", _jsx(_components.h2, {
        children: "Access HTTP headers in transformations"
      }), "\n", _jsx(_components.p, {
        children: "Not it's now possible to access HTTP headers of request, if the event has come from HTTP API. This\nis useful for filtering requests, for example we authorize users, or pass-through authorization to a\ndownstream destination:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-typescript",
          children: "return {\n        //Get auth value from HTTP header and pass it to destination in HTTP body\n        authToken: $context.header(\"X-Auth\"),\n        ...$\n    }\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Read more about this feature in ", _jsx(_components.a, {
          href: "/docs/other-features/javascript-transform",
          children: "Jitsu documentation →"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "Improved embedded JavaScript editor"
      }), "\n", _jsx("img", {
        src: "/img/blog/1.42/js2.gif",
        width: "100%"
      }), "\n", _jsxs(_components.p, {
        children: ["Now you can much easier debug your destination ", _jsx(_components.a, {
          href: "https://jitsu.com/docs/other-features/javascript-transform",
          children: "event transformation"
        }), " code."]
      }), "\n", _jsxs(_components.p, {
        children: ["Jitsu allows writing and debug JavaScript Transformations right in the browser. At 1.42 we completely redesign\nthe layout and implemented ", _jsx(_components.code, {
          children: "console.log"
        }), " output display, so you could use it for debugging"]
      }), "\n", _jsx(_components.h2, {
        children: "Revamped Statistics page"
      }), "\n", _jsx(_components.p, {
        children: "Statistics page shows how many records have flown through Jitsu. Now it's possible to break it down:\nby API Key, Destination or Source. Furthermore, you can select date range to see."
      }), "\n", _jsx(_components.p, {
        children: "That makes it much easier to understand how each connection performs."
      }), "\n", _jsx("img", {
        src: "/img/blog/1.42/stat.gif",
        width: "100%"
      }), "\n", _jsx(_components.h2, {
        children: "Redesigned geo-location configuration page"
      }), "\n", _jsxs(_components.p, {
        children: ["Jitsu supports ", _jsxs(_components.a, {
          href: "/docs/other-features/geo-data-resolution",
          children: [_jsx(_components.strong, {
            children: "Max Mind"
          }), " GEO IP databases"]
        }), " for enriching incoming events with geo-location data.\nYou can use the full-featured GeoIP2 database or GeoLite2 edition, which has a free license option.\nWe have improved the Geo Data resolver page design that makes it clear what features are covered with your license key."]
      }), "\n", _jsx("img", {
        src: "/img/blog/1.42/maxmind.png",
        width: "100%"
      }), "\n", _jsx(_components.h2, {
        children: "New connectors"
      }), "\n", _jsx("img", {
        src: "/img/blog/1.42/airtable_gcs.png",
        width: "80%"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "Airtable Source"
          }), " – the first one implemented with Jitsu Developers Platform in typescript language."]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "Google Cloud Storage Destination"
          }), " – works similarly to S3 destination – stores events in batch files."]
        }), "\n"]
      }), "\n", _jsx(_components.h2, {
        children: "Fixed JavaScript transformations memory leaks"
      }), "\n", _jsxs(_components.p, {
        children: ["In previous versions, in certain conditions JS transformation might have been causing memory leaks. It's not\na case anymore — we completely redesigned execution model. Now all JS are being executed in a pool of separate\nchild processes. Thanks to ", _jsx(_components.a, {
          href: "https://github.com/patriksimek/vm2",
          children: "vm2"
        }), " for the implementation of NodeJS Sandbox!"]
      })]
    });
  }
}
return {
  default: MDXContent
};


Jitsu v1.42: Source Connectors Plugin TypeScript SDK, New JS Transformations, and more

v1.42: Source Connectors Plugin TypeScript SDK, New JS Transformations, and more

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      h2: "h2",
      strong: "strong",
      a: "a",
      ul: "ul",
      li: "li"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsx(_components.p, {
        children: "We're pleased to announce Jitsu v1.41 with support for multiple Projects and Users per installation, SSO Authentication, HTML/Javascript Destination Tags and more"
      }), "\n", _jsx(_components.h2, {
        children: "Multiple Projects and Users"
      }), "\n", _jsxs(_components.p, {
        children: ["Users can now create and access multiple projects.\nEach project contains its own configuration: API keys, destinations, sources, etc.\nGo to your profile button in the top right corner → ", _jsx(_components.strong, {
          children: "Switch project"
        }), " or click on the project name in breadcrumbs at the top of the screen in order to switch or create a project:"]
      }), "\n", _jsx("img", {
        src: "/img/blog/1.41/projects.gif",
        width: "600"
      }), "\n", _jsxs(_components.p, {
        children: ["The new release also brings the project sharing feature which allows you to invite new or existing users for collaboration on the current project.\nNote that working SMTP configuration is required for inviting new users.\nGo to ", _jsx(_components.strong, {
          children: "Project Settings"
        }), " to manage user access to the project:"]
      }), "\n", _jsx("img", {
        src: "/img/blog/1.41/users.png",
        width: "660"
      }), "\n", _jsxs(_components.p, {
        children: ["User provisioning & project management API is ", _jsx(_components.a, {
          href: "/configurator-openapi.html",
          children: "also available"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "SSO Authentication"
      }), "\n", _jsxs(_components.p, {
        children: ["Added support of BoxyHQ SSO authentication for self-hosted clients.", _jsx("br", {}), "\nRead more about this feature ", _jsx(_components.a, {
          href: "/docs/configurator-configuration/sso-authentication",
          children: "here"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "Destination Tags"
      }), "\n", _jsxs(_components.p, {
        children: ["Destination Tag is a new type of destination that installs a configured HTML/Javascript tag to the website page on receiving an event from Javascript SDK.", _jsx("br", {}), "\nRead more about this feature ", _jsx(_components.a, {
          href: "/docs/other-features/destination-tags",
          children: "here"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "New version of @jitsu/sdk-js"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["We released a ", _jsx(_components.a, {
            href: "https://www.npmjs.com/package/@jitsu/sdk-js/v/2.5.0",
            children: "@jitsu/sdk-js v2.5.0"
          }), " with support of Destination Tags"]
        }), "\n", _jsxs(_components.li, {
          children: ["Also added new parameter ", _jsx(_components.a, {
            href: "/docs/sending-data/js-sdk/parameters-reference",
            children: "force_use_fetch"
          }), " for forcing browser to use fetch instead xmlhttprequest"]
        }), "\n"]
      }), "\n", _jsx(_components.h2, {
        children: "Other Changes"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Live Events performance and usability improvements"
        }), "\n", _jsx(_components.li, {
          children: "Optimization for synchronization of big Firebase databases"
        }), "\n", _jsx(_components.li, {
          children: "ALTER distributed tables in Clickhouse when applicable instead of DROP/CREATE"
        }), "\n", _jsx(_components.li, {
          children: "Fix Snowflake destination INSERT errors for different types in the same column"
        }), "\n", _jsx(_components.li, {
          children: "Bugfixes"
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};


Jitsu 1.41 is released. Multiple Projects and Users, SSO Authentication, Destination Tags, and more

Jitsu 1.41 is here! Multiple Projects and Users, SSO Authentication, Destination Tags, and more

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      a: "a",
      h2: "h2",
      ul: "ul",
      li: "li",
      code: "code"
    }, props.components), {Hint} = _components;
    if (!Hint) _missingMdxReference("Hint", true, "12:1-14:8");
    return _jsxs(_Fragment, {
      children: [_jsxs(_components.p, {
        children: ["We're pleased to announce Jitsu v1.40 with significant improvements for ", _jsx(_components.a, {
          href: "https://jitsu.com/docs/other-features/retroactive-user-recognition",
          children: "Retroactive User Recognition"
        }), " feature, server memory consumption,\nand new Slack notifications for synchronization tasks statuses."]
      }), "\n", _jsx(_components.h2, {
        children: "User Recognition overhaul"
      }), "\n", _jsxs(_components.p, {
        children: ["We have completely overhauled the ", _jsx(_components.a, {
          href: "https://jitsu.com/docs/other-features/retroactive-user-recognition",
          children: "Retroactive User Recognition"
        }), " feature. Now it applies to API token level.\nRecognized events repeat exactly the same pipeline as original events, as a result:\nUser Recognition now correctly works with destinations in batch mode.\nUser Recognition now correctly works with data mappings and transformations."]
      }), "\n", _jsx(Hint, {
        children: _jsx(_components.p, {
          children: "Starting from 1.40, User Recognition ID nodes customization are no longer supported on destination level. Please use global level settings"
        })
      }), "\n", _jsx(_components.h2, {
        children: "Slack Notification"
      }), "\n", _jsx(_components.p, {
        children: "Users can now add Slack notifications for synchronization task status per project or globally for the Jitsu server."
      }), "\n", _jsx("img", {
        src: "/img/blog/1.40/slack_notifications.gif",
        width: "600"
      }), "\n", _jsx(_components.h2, {
        children: "Memory Consumption improvements"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "User Recognition no longer causes indefinite growth of inmemory queue on setups with high events count."
        }), "\n", _jsx(_components.li, {
          children: "V8 engine heap size limited to 100mb preventing constant memory usage growth on setups using javascript transformation"
        }), "\n", _jsx(_components.li, {
          children: "Redis source now loads data in chunks - reducing overall memory consumption and preventing OOM crashes on huge volumes."
        }), "\n"]
      }), "\n", _jsx(_components.h2, {
        children: "New version of @jitsu/sdk-js"
      }), "\n", _jsxs(_components.p, {
        children: ["We released a ", _jsx(_components.a, {
          href: "https://www.npmjs.com/package/@jitsu/sdk-js/v/2.4.0",
          children: "new version of @jitsu/sdk-js"
        }), " that allows to add custom headers to tracking requests. Read more about JS SDK configuration ", _jsx(_components.a, {
          href: "https://jitsu.com/docs/sending-data/js-sdk/parameters-reference",
          children: "here"
        }), " (look for ", _jsx(_components.code, {
          children: "custom_headers"
        }), " parameter!)"]
      }), "\n", _jsx(_components.h2, {
        children: "Other Changes"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Clickhouse driver updated to the latest version"
        }), "\n", _jsx(_components.li, {
          children: "Facebook API version upgraded to v13.0"
        }), "\n", _jsx(_components.li, {
          children: "Improved responsiveness of Configurator UI by migration to objects v2 api"
        }), "\n", _jsx(_components.li, {
          children: "Bugfixes"
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};
function _missingMdxReference(id, component, place) {
  throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it." + (place ? "\nIt’s referenced in your code at `" + place + "`" : ""));
}


Jitsu 1.40 is released. Better User Recognition, Lower Memory usage, Sync statuses notifications

Jitsu 1.40 is here! Better User Recognition, Lower Memory usage and more

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      img: "img",
      em: "em",
      strong: "strong",
      a: "a",
      pre: "pre",
      code: "code",
      h2: "h2",
      h3: "h3",
      ul: "ul",
      li: "li"
    }, props.components), {BlogLead} = _components;
    if (!BlogLead) _missingMdxReference("BlogLead", true, "15:1-21:12");
    return _jsxs(_Fragment, {
      children: [_jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/1.39/cover.png",
          alt: ""
        })
      }), "\n", _jsxs(_components.p, {
        children: ["We're pleased to announce Jitsu v1.39  ", _jsx(_components.em, {
          children: "\"Foundation\""
        }), " 🌱! This is probably the most important release\nsince Jitsu inception."]
      }), "\n", _jsx(_components.p, {
        children: "Almost 10,000 developers has deployed Jitsu in 2021, and many of them asked for integrations we don't\nhave yet. We've tried to deliver as many integrations as possible, but as a lean team we have a pretty limited\nbandwidth."
      }), "\n", _jsx(_components.p, {
        children: "As an open-source project, we always welcomed an external contributions, but from practical\nsense it was pretty hard to implement a new source or destination for Jitsu. A developer would\nneed to learn our Go code-based, make a PR, wait for our team to merge it and release a new version."
      }), "\n", _jsxs(BlogLead, {
        children: [_jsxs(_components.p, {
          children: ["Today, we're introducing a ", _jsx(_components.strong, {
            children: "Jitsu Developer Platform"
          }), ". Our mission is to empower developers with tools to extend\nJitsu: add new destinations and sources"]
        }), _jsxs(_components.p, {
          children: ["As a first step, we're releasing beta version of ", _jsx(_components.a, {
            href: "https://github.com/jitsucom/jitsu-sdk",
            children: "Jitsu Extension SDK"
          }), ", a TypeScript based framework\nfor implementing custom destinations"]
        })]
      }), "\n", _jsxs(_components.p, {
        children: ["With new ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu-sdk",
          children: "SDK"
        }), ", custom destination can be developed\njust with a few simple steps:"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "npx jitsu-cli extension create --type destination\nnpm build\nnpm publish\n"
        })
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.strong, {
          children: _jsx(_components.a, {
            href: "/docs/extending",
            children: "Read more about extension development on documentation →"
          })
        })
      }), "\n", _jsxs(_components.p, {
        children: ["The first destination we implemented with new SDK is ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu-mixpanel",
          children: "Mixpanel"
        }), ". Thanks to\nMixpanel engineering team for help with navigating their API. The new release also brings support for custom properties\n– any non-standard property of an incoming event will be automatically sent to Mixpanel."]
      }), "\n", _jsx(_components.h2, {
        children: "Configuration API"
      }), "\n", _jsx(_components.p, {
        children: "The other frequent request from users was a configuration API. We had an API for a while,\nthat's how configuration user interface talks with backend. However, the API was not documented."
      }), "\n", _jsxs(BlogLead, {
        children: [_jsx(_components.p, {
          children: "Starting from 1.39, Jitsu configuration API is documented with Open-API spec."
        }), _jsx(_components.p, {
          children: _jsx(_components.a, {
            href: "https://jitsu.com/docs/other-features/jitsu-api",
            children: "Read a full API spec →"
          })
        })]
      }), "\n", _jsx(_components.h2, {
        children: "What's next"
      }), "\n", _jsxs(_components.p, {
        children: ["We are not going stop here. We are fully committed to make every aspect of Jitsu extensible. In next release, were\ngoing to expand Jitsu SDK to ", _jsx(_components.a, {
          href: "https://jitsu.com/sources/",
          children: "sources"
        }), ". Also, Open API is coming to Jitsu server: all\ncluster provisioning operations will be spec'ed too."]
      }), "\n", _jsx(_components.h2, {
        children: "Other features in 1.39"
      }), "\n", _jsx(_components.p, {
        children: "In addition to SDK and Open API, we released a few dozens of new features and improvements. The most\nnotable of them are:"
      }), "\n", _jsx(_components.h3, {
        children: "More convenient editor for existing sources"
      }), "\n", _jsx(_components.p, {
        children: "Edit any section of your source configuration in a few clicks without going through all the configuration steps."
      }), "\n", _jsx(_components.h3, {
        children: "Airbyte Sources version selector"
      }), "\n", _jsx(_components.p, {
        children: "Now you can easily migrate your Airbyte based source to the last or a certain version using the new selector on Jitsu UI."
      }), "\n", _jsx("img", {
        src: "/img/blog/1.39/airbyte_version_selector.png",
        width: "600"
      }), "\n", _jsx(_components.h2, {
        children: "Other Changes"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "All Jitsu Docker images now based on Debian image instead of Alpine due to v8 engine glibc requirement"
        }), "\n", _jsx(_components.li, {
          children: "An API keys user interface revamp"
        }), "\n", _jsx(_components.li, {
          children: "Authorization rework. Since 1.39, there's no need to set JWT secrets as Configurator env variables"
        }), "\n", _jsx(_components.li, {
          children: "Improvements of task parallelization. If the task is failed, it won't occupied a worker slot more than for few seconds\nafter a failure"
        }), "\n"]
      }), "\n", _jsx(_components.h2, {
        children: "Major bug fixes"
      }), "\n", _jsxs(_components.p, {
        children: ["Thanks to ", _jsx(_components.a, {
          href: "https://github.com/ezradiniz",
          children: "@ezradiniz"
        }), " for the following bug fixes:"]
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Fixed using 100% of one CPU core by Jitsu while idle"
        }), "\n", _jsx(_components.li, {
          children: "Fixed occasional error while Jitsu writes events JSON with lua script to Redis"
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};
function _missingMdxReference(id, component, place) {
  throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it." + (place ? "\nIt’s referenced in your code at `" + place + "`" : ""));
}


Jitsu 1.39 is released. New API, new SDK, destination plugins.

Announcing Jitsu Developers Platform and Jitsu v1.39

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      em: "em",
      h2: "h2",
      a: "a",
      ul: "ul",
      li: "li",
      code: "code",
      br: "br",
      strong: "strong"
    }, props.components), {Hint} = _components;
    if (!Hint) _missingMdxReference("Hint", true, "31:1-33:8");
    return _jsxs(_Fragment, {
      children: [_jsxs(_components.p, {
        children: ["We're pleased to announce Jitsu v1.38 ", _jsx(_components.em, {
          children: "\"DeLorean\""
        }), "! This release was mainly about performance optimizations."]
      }), "\n", _jsx("img", {
        src: "/img/blog/1.38/performance.png",
        width: "600"
      }), "\n", _jsx(_components.h2, {
        children: "Redis memory optimization"
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.a, {
          href: "/docs/other-features/retroactive-user-recognition",
          children: "User recognition"
        }), " could eat up a lot of RAM.\nWe implemented number of measures that makes Redis RAM consumption more manageable:"]
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Now it's possible a dedicated Redis instance for user recognition, and configure compression,\nauto eviction and so on"
        }), "\n", _jsx(_components.li, {
          children: "Jitsu can compress objects"
        }), "\n"]
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.a, {
          href: "/docs/other-features/retroactive-user-recognition/redis-optimization",
          children: "Read more about Redis optimization →"
        })
      }), "\n", _jsx(_components.h2, {
        children: "Performance Optimization"
      }), "\n", _jsxs(_components.p, {
        children: ["We made a number of refactorings that significantly improved how Jitsu works under a heavy load. Here's some benchmark numbers for\nAWS ", _jsx(_components.code, {
          children: "m5.xlarge"
        }), " (4 CPUs / 16 RAM):"]
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Stream mode: 16k RPS"
        }), "\n", _jsx(_components.li, {
          children: "Batch mode: 12k RPS"
        }), "\n"]
      }), "\n", _jsx(_components.h2, {
        children: "Improved Docker Images"
      }), "\n", _jsxs(_components.p, {
        children: ["Now ", _jsx(_components.a, {
          href: "/docs/deployment/deploy-with-docker/jitsu-server",
          children: "@jitsucom/server"
        }), " (if deployed standalone) can be configured with environment variables too.\nIt's very convenient for Kubernetes based deployments."]
      }), "\n", _jsx(Hint, {
        children: _jsxs(_components.p, {
          children: ["Learn about the difference between ", _jsx("a", {
            href: "/docs/deployment/deploy-with-docker",
            children: "various deployment methods here"
          })]
        })
      }), "\n", _jsx(_components.h2, {
        children: "Mixpanel destination and new destination SDK"
      }), "\n", _jsxs(_components.p, {
        children: ["We added ", _jsx(_components.a, {
          href: "/docs/destinations-configuration/mixpanel#user-profiles",
          children: "Mixpanel destination"
        }), ". This destination is the first one based on a new ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu-sdk",
          children: "Jitsu SDK"
        }), ".", _jsx(_components.br, {}), "\n", "With new Jitsu SDK anyone can implement a destination using Typescript (as npm package) and connect it to Jitsu. At the moment,\nwe released an ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu-sdk",
          children: "alpha version of SDK"
        }), ",\nand ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu-mixpanel",
          children: "Mixpanel destination"
        }), " based on the SDK. Our next steps are:"]
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "Finalize API spec"
          }), " (Jan 22). We will release a comprehensive documentation and step-by-step guide for implementing destination"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "Allow Jitsu users to plug in any npm packages"
          }), " (Feb 22). At the moment, Jitsu can work only with predefined set of destinations based on SDK. We will add an\nability to reference any npm package from Self-Hosted UI"]
        }), "\n"]
      }), "\n", _jsx(_components.h2, {
        children: "Bug Fixes"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["Airbyte based sources now properly works in AWS Elastic Beanstalk environment if deployed using ", _jsx(_components.a, {
            href: "https://github.com/jitsucom/jitsu/blob/master/Dockerrun.aws.json",
            children: "Dockerrun.aws.json"
          })]
        }), "\n", _jsx(_components.li, {
          children: "A lot of minor fixes"
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};
function _missingMdxReference(id, component, place) {
  throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it." + (place ? "\nIt’s referenced in your code at `" + place + "`" : ""));
}


Jitsu 1.38 is released. In this release we did a number of performance optimization

Jitsu 1.38

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      h2: "h2",
      p: "p",
      a: "a",
      em: "em",
      h3: "h3",
      code: "code",
      pre: "pre",
      ul: "ul",
      li: "li"
    }, props.components), {Hint} = _components;
    if (!Hint) _missingMdxReference("Hint", true, "52:1-54:8");
    return _jsxs(_Fragment, {
      children: [_jsx(_components.h2, {
        children: "Introduction"
      }), "\n", _jsxs(_components.p, {
        children: ["At Jitsu, we are big fans of Next.js. This website (jitsu.com) is built with it, and recently used Next.js to\nre-build ", _jsx(_components.a, {
          href: "https://cloud.jitsu.com",
          children: "Jitsu.Cloud"
        }), " billing server."]
      }), "\n", _jsxs(_components.p, {
        children: ["Few, we started another internal project and decided to experiment with Supabase and\n", _jsx(_components.a, {
          href: "https://nextjs.org/docs/middleware",
          children: "Next.js middleware"
        }), ". Here’s what we learned in the process"]
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.em, {
          children: "TL;DR"
        }), " - skip to ", _jsx(_components.a, {
          href: "#what-were-going-to-build",
          children: "What we’re going to build"
        }), " if you're familiar with Supabase and Next.js"]
      }), "\n", _jsx(_components.h3, {
        children: "What is Next.js"
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.a, {
          href: "https://nextjs.org/",
          children: "Next.js"
        }), " is an open-source a web-development framework built on React.\nUnlike with React, you can write server-side code with Next.js too. In fact, every Next.js application is mix of backend and frontend.\nNext.js is maintained by ", _jsx(_components.a, {
          href: "https://vercel.com",
          children: "Vercel"
        }), ". They make money out of paid Next.js hosting services."]
      }), "\n", _jsx(_components.h3, {
        children: "What is Next.js middleware?"
      }), "\n", _jsxs(_components.p, {
        children: ["Vercel team introduced middleware in recent v12 release. Middleware is just a file\n(", _jsx(_components.code, {
          children: "_middleware.js"
        }), " or ", _jsx(_components.code, {
          children: "middleware.ts"
        }), ") that you can place anywhere in ", _jsx(_components.code, {
          children: "pages/"
        }), " folder of your Next.js project.\nAll requests to pages in this folder (and subfolders) will go through ", _jsx(_components.code, {
          children: "middleware.ts"
        }), "). Middleware file can decided\nif request should go through, alter request and etc/ Think of middleware as filter for particular sub-tree of your pages."]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          children: "pages/\n\tfolder/\n\t\tmiddleware.ts\n\t\tpage.tsx\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["One of great use-cases for middleware is authorization. Example (", _jsx(_components.code, {
          children: "middleware.ts"
        }), "):"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-typescript",
          children: "export async function middleware(req: NextRequest, ev: NextFetchEvent) {\n  if (!isValidUser(req)) {\n    return NextResponse.redirect(`/login`)\n  } else {\n    //allow request to go through\n    return NextResponse.next()\n  }\n}\n"
        })
      }), "\n", _jsx(_components.h3, {
        children: "What is Supabase?"
      }), "\n", _jsx(_components.p, {
        children: "Supabase is an open-source Firebase alternative. It allows web developers to \"outsource\" backend logic - such as authorization,\nstructured data storage (SQL) and file storage."
      }), "\n", _jsx(_components.h2, {
        children: "What we’re going to build"
      }), "\n", _jsx(Hint, {
        children: _jsxs(_components.p, {
          children: ["The complete example can be found at ", _jsx("a", {
            href: "https://github.com/jitsucom/supabase-nextjs-middleware",
            children: "@jitsucom/supabase-nextjs-middleware"
          }), " GitHub repo"]
        })
      }), "\n", _jsxs(_components.p, {
        children: ["We’re going to build a simple Next.js app which consist of index page (", _jsx(_components.code, {
          children: "index.ts"
        }), ") and few pages in ", _jsx(_components.code, {
          children: "app/"
        }), " folder.\nIndex page will implement login/logout form and will be accessible for all users. We're going use\nGoogle as login provider, but you can configure any ", _jsx(_components.a, {
          href: "https://supabase.io/docs/guides/auth#third-party-logins",
          children: "supported provider"
        })]
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.code, {
          children: "app/"
        }), " pages will be hidden from unauthorized users. All unauthorized users will be redirected to login (index page)"]
      }), "\n", _jsx(_components.h2, {
        children: "Login page"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.a, {
          href: "https://github.com/jitsucom/supabase-nextjs-middleware/blob/main/pages/index.tsx",
          children: _jsx(_components.em, {
            children: "(See full source code here)"
          })
        })
      }), "\n", _jsx(_components.p, {
        children: "This page is fairly simple and standard. The authorization happens on a client side."
      }), "\n", _jsx(_components.p, {
        children: "The only thing that is not standard is snippet"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-typescript",
          children: "await fetch(\"/api/auth/set\", {\n  method: \"POST\",\n  headers: new Headers({ \"Content-Type\": \"application/json\" }),\n  credentials: \"same-origin\",\n  body: JSON.stringify({ event, session }),\n})\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["After successful login, we're calling ", _jsx(_components.code, {
          children: "/api/auth/set"
        }), " API that sets a server cookie ", _jsx(_components.code, {
          children: "sb:token"
        }), ". The cookie can be read\nby ", _jsx(_components.code, {
          children: "middleware.ts"
        }), " later."]
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.a, {
          href: "https://github.com/jitsucom/supabase-nextjs-middleware/blob/main/pages/api/auth/set.ts",
          children: _jsx(_components.code, {
            children: "pages/api/auth/set.ts"
          })
        }), " implements that API:"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-typescript",
          children: "export default async function handler(req, res) {\n    await supabase.auth.api.setAuthCookie(req, res)\n}\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "And we need to remove cookie when user logs out. We need to call API too:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          children: "await fetch(\"/api/auth/remove\", {\n  method: \"GET\",\n  credentials: \"same-origin\"\n})\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["See implementation in ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/supabase-nextjs-middleware/blob/main/pages/api/auth/remove.ts",
          children: _jsx(_components.code, {
            children: "pages/api/auth/remove.ts"
          })
        }), ".\nSupabase haven’t implemented ", _jsx(_components.code, {
          children: "removeAuthCookie()"
        }), " yet, so we have to do it manually"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-typescript",
          children: "export default async function handler(req, res) {\n  res.setHeader('Set-Cookie', 'sb:token=deleted; path=/; expires=Thu, 01 Jan 1970 00:00:00 GMT');\n  res.send({});\n}\n"
        })
      }), "\n", _jsx(_components.h2, {
        children: "Middleware"
      }), "\n", _jsxs(_components.p, {
        children: ["Here’s ", _jsx(_components.code, {
          children: "middleware.ts"
        }), ":"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-typescript",
          children: "export async function middleware(req: NextRequest, ev: NextFetchEvent) {\n  let authResult = await getUser(req)\n  if (authResult.error) {\n    console.log(\"Authorization error, redirecting to login page\", authResult.error)\n    return NextResponse.redirect(`/?ret=${encodeURIComponent(req.nextUrl.pathname)}`)\n  } else if (!authResult.user) {\n    console.log(\"No auth user, redirecting\")\n    return NextResponse.redirect(`/?ret=${encodeURIComponent(req.nextUrl.pathname)}`)\n  } else {\n    console.log(\"User is found\", authResult.user)\n    return NextResponse.next()\n  }\n}\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["All interesting staff happens in ", _jsx(_components.code, {
          children: "getUser"
        }), " method. Supabase has an API - ", _jsx(_components.code, {
          children: "supabase.auth.api.getUserByCookie"
        }), ", but unfortunately\n", _jsx(_components.a, {
          href: "https://github.com/supabase/supabase/issues/3783",
          children: "it doesn't work for Next.js"
        }), ". Good thing is that we can implement verify the auth manually by calling\n", _jsx(_components.code, {
          children: "/auth/v1/user"
        }), " Supabase end-point"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-typescript",
          children: "let token = req.cookies[\"sb:token\"]\nif (!token) {\n  return {\n    user: null,\n    data: null,\n    error: \"There is no supabase token in request cookies\",\n  }\n}\nlet authRequestResult = await fetch(`${process.env.NEXT_PUBLIC_SUPABASE_URL}/auth/v1/user`, {\n  headers: {\n    Authorization: `Bearer ${token}`,\n    APIKey: process.env.NEXT_PUBLIC_SUPABASE_KEY || \"\",\n  },\n})\n\nlet result = await authRequestResult.json()\nconsole.log(\"Supabase auth result\", result)\nif (authRequestResult.status != 200) {\n  return {\n    user: null,\n    data: null,\n    error: `Supabase auth returned ${authRequestResult.status}. See logs for details`,\n  }\n} else if (result.aud === \"authenticated\") {\n  return {\n    user: result,\n    data: result,\n    error: null,\n  }\n}\n"
        })
      }), "\n", _jsxs(_components.h2, {
        children: ["Accessing user from SSR environment ( ", _jsx(_components.code, {
          children: "getServerSideProps"
        }), ")"]
      }), "\n", _jsxs(_components.p, {
        children: ["Now, all authorized users will be able to access all pages inside ", _jsx(_components.code, {
          children: "app/"
        }), ". In the browser, Supabase user can be obtained through ", _jsx(_components.code, {
          children: "supabase.auth.user()"
        }), " call\n(don’t forget to wrap it in ", _jsx(_components.code, {
          children: "useEffect()"
        }), ", otherwise the code will be executed as SSR)."]
      }), "\n", _jsxs(_components.p, {
        children: ["But what if we want to access current user user information during SSR — inside ", _jsx(_components.code, {
          children: "getServerSideProps"
        }), "? Here’s what we can do:"]
      }), "\n", _jsxs(_components.p, {
        children: ["Supabase has an API for that — ", _jsx(_components.code, {
          children: "supabase.auth.api.getUser()"
        }), ". But this function queries Supabase HTTP API to validate the token.\nBut it's redundant since the token has been already validated in ", _jsx(_components.code, {
          children: "middleware.ts"
        }), " call.\nLuckily, Supabase token is a JWT token, meaning that all user information is encoded inside it.\nThe token also contains the signature that should be verified. But verification has been done in ", _jsx(_components.code, {
          children: "middleware.ts"
        }), " so we can\njust decode the token and skip verification."]
      }), "\n", _jsxs(_components.p, {
        children: ["See ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/supabase-nextjs-middleware/blob/main/pages/app/hidden-page.tsx",
          children: "pages/app/hidden-page.ts"
        })]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-typescript",
          children: "const Hidden: NextPage<{user: any}> = (props) => {\n  return <h1>Do not tell anybody about this page, {props.user.email}</h1>\n}\n\nexport async function getServerSideProps(context) {\n  let supabaseToken = context.req.cookies['sb:token']\n  if (!supabaseToken) {\n    throw new Error(\"It should not happen! Since this page is guarded by _middlware.ts the presense of supabase token cookie (sb:token) should be already checked\")\n  }\n  return {\n    props: {\n      //we do not need to verify JWT signature since it has been already done in _middlware.ts\n      user: jwt.decode(supabaseToken),\n    },\n  }\n}\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["(Don’t forget to ", _jsx(_components.code, {
          children: "yarn add jsonwebtoken"
        }), " or ", _jsx(_components.code, {
          children: "npm install jsonwebtoken"
        }), "!)"]
      }), "\n", _jsx(_components.h2, {
        children: "The complete example"
      }), "\n", _jsxs(_components.p, {
        children: ["The complete application example with all deployment and configuration example can be found [on our GitHub]\n(", _jsx(_components.a, {
          href: "https://github.com/jitsucom/supabase-nextjs-middleware/blob/main/pages/app/hidden-page.tsx",
          children: "https://github.com/jitsucom/supabase-nextjs-middleware/blob/main/pages/app/hidden-page.tsx"
        }), ")."]
      }), "\n", _jsx(_components.h2, {
        children: "Related articles"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: _jsx(_components.a, {
            href: "https://jitsu.com/blog/supabase",
            children: "Building Google Analytics alternative with Jitsu and Supabase"
          })
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};
function _missingMdxReference(id, component, place) {
  throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it." + (place ? "\nIt’s referenced in your code at `" + place + "`" : ""));
}


Supabase Auth with Next.js 12 middleware

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      h2: "h2",
      img: "img",
      a: "a"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsx(_components.p, {
        children: "We're pleased to announce Jitsu 1.37 with some breakthrough and quality of life improvements along with a bunch of minor fixes:"
      }), "\n", _jsx(_components.h2, {
        children: "JavaScript Transform"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/javascript-transform/code_debugger.png",
          alt: "Event Multiplexing with JavaScript Transform"
        })
      }), "\n", _jsx(_components.p, {
        children: "JavaScript Transform comes as a modern and flexible replacement of the Mapping feature.\nThis feature improves the usability and flexibility of data mapping\nand brings some excellent additions like Event Multiplexing."
      }), "\n", _jsxs(_components.p, {
        children: ["Read more in ", _jsx(_components.a, {
          href: "/blog/javascript-transform",
          children: "a separate blog post"
        }), "."]
      }), "\n", _jsxs(_components.p, {
        children: ["Related issues: ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/282",
          children: "#282"
        }), ", ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/437",
          children: "#437"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "New User interface"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/1.37/ui1_37.png",
          alt: "New cards layout"
        })
      }), "\n", _jsx(_components.p, {
        children: "We have completely revamped one of the main components of Jitsu UI - entities list.\nWe replaced it with beautiful card layouts. It looks incredible and gives users more context on what is going on with sources and destinations.\nThe new design also brings new possibilities like giving entities meaningful names."
      }), "\n", _jsxs(_components.p, {
        children: ["Related issues: ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/581",
          children: "#581"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "Source setup Wizard"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/1.37/wizard.gif",
          alt: "Source setup Wizard"
        })
      }), "\n", _jsx(_components.p, {
        children: "We have also added step by step wizard for Airbyte and Singer sources.\nIt is more streamlined and less error-prone."
      }), "\n", _jsxs(_components.p, {
        children: ["Related issues: ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/582",
          children: "#582"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "Stability improvements"
      }), "\n", _jsx(_components.p, {
        children: "We worked hard to improve Jitsu stability, and we have fixed one of the most annoying issues with the hanging Source sync tasks.\nNew Jitsu server constantly making sure that no hanging tasks blocking Source syncing pipeline.\nMoreover, it is now possible to Cancel any task that runs too long from the tasks log UI."
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/1.37/cancel.png",
          alt: "Canceling of Task"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Related issues: ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/546",
          children: "#546"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "Redis Sentinel automatic failover"
      }), "\n", _jsxs(_components.p, {
        children: ["Failover is the most important ability of the reliable system. Now you can configure Jitsu with ", _jsx(_components.a, {
          href: "https://redis.io/topics/sentinel",
          children: "Redis Sentinel"
        }), ".\nThis feature provides automatic switching between Redis instances from cluster and finding a healthy one in runtime.\nIt increases the system reliability. Thanks to our contributors ❤️!"]
      }), "\n", _jsxs(_components.p, {
        children: ["Related issues: ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/553",
          children: "#553"
        })]
      })]
    });
  }
}
return {
  default: MDXContent
};


Jitsu 1.37 is out!

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      ul: "ul",
      li: "li",
      strong: "strong",
      img: "img",
      pre: "pre",
      code: "code",
      h2: "h2",
      h3: "h3",
      a: "a"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsx(_components.p, {
        children: "We added JavaScript support in July'21 for destinations table name selection and webhooks body composition. But we didn't plan to stop there. As of today, we are happy to announce an expansion of JavaScript support in Jitsu: JavaScript Transform comes as a modern and flexible replacement of the Mapping feature."
      }), "\n", _jsx(_components.p, {
        children: "The Mapping feature with its predefined set of rules served well but has some shortcomings:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Complicated – each mapped field requires its UI block, making any rich mapping setup too bloated and very hard to manage."
        }), "\n", _jsxs(_components.li, {
          children: ["Not flexible – having just four rule types (", _jsx(_components.strong, {
            children: "move"
          }), ", ", _jsx(_components.strong, {
            children: "remove"
          }), ", ", _jsx(_components.strong, {
            children: "cast"
          }), ", and ", _jsx(_components.strong, {
            children: "constant"
          }), ") significantly limits what one can do with the data."]
        }), "\n", _jsx(_components.li, {
          children: "Not applied by default – some destination that relies on mapped data (e.g., Google Analytics or Facebook marketing) requires a user to manually apply the appropriate mapping template."
        }), "\n"]
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/javascript-transform/old_mapping.gif",
          alt: "Mapping UX problem"
        })
      }), "\n", _jsx(_components.p, {
        children: "JavaScript Transform solves all this and brings more to the table:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Even complicated mapping now can fit a single screen."
        }), "\n", _jsx(_components.li, {
          children: "The power and flexibility of the JavaScript language allow computing new objects or fields based on incoming event data."
        }), "\n", _jsx(_components.li, {
          children: "Given the nature of JavaScript object creation, Transform code may represent the resulting data structure in an obvious form:"
        }), "\n"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-javascript",
          children: "const context = $.eventn_ctx || $\nconst user = context.user || {}\nconst conversion = context.conversion || {}\n\nreturn {\n  event_name: $.event_type,\n  event_id: context.event_id || $.eventn_ctx_event_id,\n  event_source_url: context.url,\n  user_data: {\n    client_ip_address: $.source_ip,\n    em: user.email,\n    client_user_agent: context.user_agent,\n  },\n  custom_data: {\n    currency: conversion.currency || $.currency,\n    value: conversion.revenue || $.revenue,\n  },\n}\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "In the UI version of Jitsu new Google Analytics, Facebook, and Amplitude destination setups now have the appropriate Transform function initially enabled."
      }), "\n", _jsx(_components.h2, {
        children: "New to Javascript Transform"
      }), "\n", _jsx(_components.p, {
        children: "The following features weren't possible with the previous Mapping implementation:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Producing Multiple events based on data of a single incoming event."
        }), "\n", _jsx(_components.li, {
          children: "Selecting a destination table or skipping an event"
        }), "\n", _jsx(_components.li, {
          children: "Using predefined variables and functions"
        }), "\n"]
      }), "\n", _jsx(_components.h3, {
        children: "Event multiplexing"
      }), "\n", _jsxs(_components.p, {
        children: ["Let's say Jitsu receives a conversion event with multiple products added to the shopping cart. Now it is possible to break this event into a set of purchase events - one for each product.\n", _jsx(_components.img, {
          src: "/img/blog/javascript-transform/code_debugger.png",
          alt: "Event multiplexing in the shiny new Code Debugger"
        })]
      }), "\n", _jsx(_components.h3, {
        children: "Selecting a destination table or skipping an event"
      }), "\n", _jsx(_components.p, {
        children: "Select destination table with ease by setting one special property of resulting event:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-javascript",
          children: "return { ...$, JITSU_TABLE_NAME: \"new_events\" }\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["to skip event from processing, simply return ", _jsx(_components.code, {
          children: "null"
        })]
      }), "\n", _jsx(_components.h3, {
        children: "Using predefined variables and functions"
      }), "\n", _jsx(_components.p, {
        children: "Your transformation code is executed in a context where Jitsu puts some extra constants and functions that you may use:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "destinationId"
          }), " – constant containing id of current destination"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "destinationType"
          }), " – constant containing type of current destination (e.g., facebook, google_analytics)"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "toSegment"
          }), "(event) – the function that transforms event to make it compatible with segment data schema (part of ", _jsx(_components.a, {
            href: "https://jitsu.com/docs/other-features/segment-compatibility",
            children: "Segment Compatibility"
          }), ")"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "toGoogleAnalytics"
          }), "(event), ", _jsx(_components.strong, {
            children: "toFacebook"
          }), "(event), ", _jsx(_components.strong, {
            children: "toAmplitude"
          }), "(event) – builtin transform functions for corresponding destinations. In the UI version of Jitsu enabled by default for new destinations setup."]
        }), "\n"]
      }), "\n", _jsx(_components.h2, {
        children: "Improved Code Debugger"
      }), "\n", _jsx(_components.p, {
        children: "As you may already notice on the screenshot above, we have significantly improved our Code Debugger UX.\nIt now enables code completion based on loaded event data.\nIt is the only instrument you need to write and debug transformation code for your data."
      }), "\n", _jsx(_components.h2, {
        children: "Learn More"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["Read more in the ", _jsx(_components.a, {
            href: "https://jitsu.com/docs/other-features/javascript-transform",
            children: "documentation"
          }), "."]
        }), "\n", _jsxs(_components.li, {
          children: ["Follow us on ", _jsx(_components.a, {
            href: "https://twitter.com/jitsucom",
            children: "Twitter"
          }), " to get updates on further expansions of Javascript support."]
        }), "\n", _jsxs(_components.li, {
          children: ["Follow & star ", _jsx(_components.a, {
            href: "https://github.com/jitsucom/jitsu",
            children: "Jitsu on GitHub"
          })]
        }), "\n", _jsxs(_components.li, {
          children: ["Try a cloud version of ", _jsx(_components.a, {
            href: "https://cloud.jitsu.com",
            children: "Jitsu"
          }), ". It's free for up to 250,000 events per month"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.a, {
            href: "https://jitsu.com/slack",
            children: "Join our Slack"
          }), "! Our community will help you with any questions!"]
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};


JavaScript Transform comes as a modern and flexible replacement of the Mapping feature

JavaScript is fully in charge of Data Mappings.

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      a: "a",
      strong: "strong",
      h2: "h2",
      ul: "ul",
      li: "li",
      code: "code",
      img: "img",
      em: "em",
      pre: "pre"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsx("img", {
        src: "/img/blog/airbyte-integration/airbyte-integration.png",
        alt: "Jitsu & Airbyte",
        width: "90%"
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.a, {
          href: "https://airbyte.io/",
          children: "Airbyte"
        }), " is an open-source ETL platform that pipes data from services and databases to data warehouses.\nThe company is growing fast, as well as the number of connectors they support. At the moment, they support around 100 services.\nEach connector is a separate open-source package, similar to ", _jsx(_components.a, {
          href: "https://www.singer.io/#taps",
          children: "Singer Tap"
        }), ", but wrapped in a docker image.\nThe connector contains all logic for downloading data from a source and follows ", _jsx(_components.a, {
          href: "https://docs.airbyte.io/understanding-airbyte/airbyte-specification",
          children: "airbyte protocol"
        }), "."]
      }), "\n", _jsxs(_components.p, {
        children: ["All connectors are published on ", _jsx(_components.a, {
          href: "https://hub.docker.com/u/airbyte",
          children: "DockerHub"
        }), ", most of them are MIT licensed."]
      }), "\n", _jsxs(_components.p, {
        children: ["At Jitsu, we built a support ", _jsx(_components.a, {
          href: "/docs/sources-configuration/singer-taps",
          children: "Singer-based connectors"
        }), " a while ago. So we decided\nto support Airbyte protocol as well: ", _jsxs(_components.strong, {
          children: ["starting from ", _jsx(_components.a, {
            href: "/blog/jitsu-1.36",
            children: "Jitsu v1.36"
          }), " more than 80 connectors build by Airbyte team are available in Jitsu - both open-source version and ", _jsx(_components.a, {
            href: "https://cloud.jitsu.com/",
            children: "Jitsu.Cloud"
          })]
        })]
      }), "\n", _jsx("img", {
        src: "/img/blog/airbyte-integration/sources-list.gif",
        alt: "Jitsu Source Connectors",
        width: "90%"
      }), "\n", _jsx(_components.h2, {
        children: "Configuration"
      }), "\n", _jsxs(_components.p, {
        children: ["For ", _jsx(_components.a, {
          href: "https://cloud.jitsu.com",
          children: "Jitsu.Cloud"
        }), " users Airbyte connectors work without any additional configuration. If you're deploying Jitsu on-prem,\nadditional configuration required:"]
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["If you're deploying Jitsu with Docker, host machine should be exposed to an Jitsu docker image with ", _jsx(_components.code, {
            children: "-v /var/run/docker.sock:/var/run/docker.sock"
          }), ". Read ", _jsx(_components.a, {
            href: "/docs/deployment/deploy-with-docker",
            children: "docker deployment\nmanual for more information"
          })]
        }), "\n", _jsx(_components.li, {
          children: "Unfortunately, Airbyte connectors will not work for Heroku deployments. Heroku Docker environment won't allow you to expose host machine"
        }), "\n"]
      }), "\n", _jsx(_components.h2, {
        children: "How it works"
      }), "\n", _jsx(_components.p, {
        children: "Let's take a look how Airbyte connectors works inside Jitsu. As an example, we're going to pull data from Hubspot which\nwe use internally at Jitsu"
      }), "\n", _jsx(_components.h2, {
        children: "Step 1: search for Hubspot source"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/airbyte-integration/hubspot.gif",
          alt: ""
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Signup for ", _jsx(_components.a, {
          href: "https://cloud.jitsu.com",
          children: "Jitsu.Cloud"
        }), " or ", _jsx(_components.a, {
          href: "/docs/deployment",
          children: "Deploy Jitsu Open-Source"
        }), ". Select Sources and search for Hubspot source"]
      }), "\n", _jsx(_components.h2, {
        children: "Step 2: configure Hubspot connection"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/airbyte-integration/hubspot-props.png",
          alt: ""
        })
      }), "\n", _jsxs(_components.p, {
        children: ["We're going to use ", _jsx(_components.code, {
          children: "API Key Credentials"
        }), " authentication. This type of authentication requires a one developer\nkey which can be easily ", _jsx(_components.a, {
          href: "https://knowledge.hubspot.com/integrations/how-do-i-get-my-hubspot-api-key",
          children: "obtained from HubSpot"
        }), "."]
      }), "\n", _jsxs(_components.p, {
        children: ["Put any past date to ", _jsx(_components.code, {
          children: "Replication start"
        }), " field. We put ", _jsx(_components.code, {
          children: "2017-01-25T00:00:00Z"
        }), ", it means that data before 25-Jan-2017 will be synchronized"]
      }), "\n", _jsxs(_components.p, {
        children: ["Click ", _jsx(_components.strong, {
          children: "Test Connection"
        }), " button to verify that the connection works"]
      }), "\n", _jsx(_components.h2, {
        children: "Step 3: configure streams"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/airbyte-integration/hubspot-streams.png",
          alt: ""
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Go to ", _jsx(_components.strong, {
          children: "Streams"
        }), " tab and select entities you want to sync. We're going to sync only ", _jsx(_components.em, {
          children: "contacts"
        }), " and ", _jsx(_components.em, {
          children: "companies"
        }), ". ", _jsx(_components.strong, {
          children: "Stream"
        }), " is represents\na set ob objects from a source which will be mapped to a table in SQL database."]
      }), "\n", _jsx(_components.h2, {
        children: "Step 4: configure Postgres"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/airbyte-integration/postgres-connect.gif",
          alt: ""
        })
      }), "\n", _jsxs(_components.p, {
        children: ["We need to set up a destination. We're going to use Postgres SQL instance provided by ", _jsx(_components.a, {
          href: "https://aws.amazon.com/lightsail/",
          children: "AWS Lightsail"
        }), ".\nDon't forget to enable ", _jsx(_components.a, {
          href: "https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-configuring-database-public-mode",
          children: "Public Mode"
        }), " for your instance!\nNote: default database name for your instance is ", _jsx(_components.code, {
          children: "postgres"
        })]
      }), "\n", _jsxs(_components.p, {
        children: ["Once the destination is configured, connect the destination and the source at ", _jsx(_components.em, {
          children: "Linked Connectors & API Keys"
        }), " tab"]
      }), "\n", _jsxs(_components.p, {
        children: ["Ignore the field ", _jsx(_components.em, {
          children: "Table Name"
        }), ", this is relevant only for events coming through Push APIs (", _jsx(_components.a, {
          href: "https://jitsu.com/docs/sending-data/api",
          children: "JavaScript as an example"
        }), "). Hubspot connector will create their own\ntables"]
      }), "\n", _jsx(_components.h2, {
        children: "Step 5: synchronize"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/airbyte-integration/sync-now.png",
          alt: ""
        })
      }), "\n", _jsx(_components.p, {
        children: "By default, the data is synced on a daily basis. You can initiate the sync manually by going\nto Sources and clicking Sync Now."
      }), "\n", _jsx(_components.p, {
        children: "You'll be redirected to a list of tasks immediately. Be patient, the first run can take a while since the\ndata is synchronized for the first time. Subsequent runs will be faster"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/airbyte-integration/sync-logs.png",
          alt: ""
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Once the task is finished, you'll see a message in logs. The task will change the status from ", _jsx(_components.code, {
          children: "RUNNING"
        }), "\nto ", _jsx(_components.code, {
          children: "SUCCESS"
        }), " as well:"]
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/airbyte-integration/task-success.png",
          alt: ""
        })
      }), "\n", _jsx(_components.h2, {
        children: "Step 6: see the data"
      }), "\n", _jsx(_components.p, {
        children: "Connect to PostgresSQL database with your favourite tool (we're fans of DataGrip!):"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/airbyte-integration/table-list.png",
          alt: ""
        })
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-sql",
          children: "SELECT tablename FROM pg_catalog.pg_tables\nWHERE schemaname != 'pg_catalog' AND schemaname != 'information_schema';\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["This query will return list of all tables. Since we synchronized only 2 sources, only two tables will\nappear: ", _jsx(_components.code, {
          children: "airbyte_source_hubspot_companies"
        }), ", ", _jsx(_components.code, {
          children: "airbyte_source_hubspot_contacts"
        }), ". Let's see what inside\nwith ", _jsx(_components.code, {
          children: "select * from airbyte_source_hubspot_companies"
        })]
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/airbyte-integration/postgres-result.png",
          alt: ""
        })
      })]
    });
  }
}
return {
  default: MDXContent
};


140+ Source Connectors with Jitsu and Airbyte

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      h2: "h2",
      img: "img",
      a: "a",
      ul: "ul",
      li: "li"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsx(_components.p, {
        children: "We're pleased to announce Jitsu 1.36. We've been working on this realease for a while and here's what we've built:"
      }), "\n", _jsx(_components.h2, {
        children: "Airbyte Integration"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/1.36/airbyte.gif",
          alt: "Jitsu S3 Connector"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Starting from 1.36 Jitsu support ", _jsx(_components.a, {
          href: "https://airbyte.io/connectors",
          children: "Airbyte connectors"
        }), " as a backend for some\nof ", _jsx(_components.a, {
          href: "https://jitsu.com/sources",
          children: "Jitsu sources"
        }), ". Take a look at a separate announcement of this feature in\n", _jsx(_components.a, {
          href: "/blog/airbyte-integration",
          children: "a separate blog post"
        }), ". At this moment, Jitsu support 142 sources:"]
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "6 sources are native (the code is maintaned by Jitsu)"
        }), "\n", _jsx(_components.li, {
          children: "47 sources are using Singer connectors"
        }), "\n", _jsx(_components.li, {
          children: "89 sources are using Airbyte connectors"
        }), "\n"]
      }), "\n", _jsxs(_components.p, {
        children: ["Related issues: ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/476",
          children: "#476"
        }), ", ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/515",
          children: "#515"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "Google Ads connector"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/1.36/adwords.png",
          alt: "Google Ads Connector"
        })
      }), "\n", _jsx(_components.p, {
        children: "We significantly improved the quality of our Google Ads connector. Unlike Singer, and Airbyte, our\nconnector is fully configurable — any combination of fields can be pulled from the API."
      }), "\n", _jsxs(_components.p, {
        children: ["Please note, that for self-hosting you need to obtain your own\n", _jsx(_components.a, {
          href: "https://developers.google.com/google-ads/api/docs/first-call/dev-token",
          children: "Developer API token"
        }), ". ", _jsx(_components.a, {
          href: "https://cloud.jitsu.com",
          children: "Jitsu.Cloud"
        }), "\nworks with developer token owned by Jitsu"]
      }), "\n", _jsxs(_components.p, {
        children: ["Related issues: ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/387",
          children: "#387"
        }), ", ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/566",
          children: "#566"
        })]
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.a, {
          href: "https://jitsu.com/docs/sources/google-ads",
          children: "Read more about configuring AdWords source via configuration file"
        })
      }), "\n", _jsx(_components.h2, {
        children: "New Destination: S3"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/1.36/s3.png",
          alt: "Jitsu S3 Connector"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Now jitsu supports S3 as a destination. We can write data as CSV and JSON and\n", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/514",
          children: "Parquet support is coming"
        })]
      }), "\n", _jsxs(_components.p, {
        children: ["Related issues: ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/491",
          children: "#491"
        }), "."]
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.a, {
          href: "https://jitsu.com/docs/destinations-configuration/s3",
          children: "Read more about configuring S3 via configuration file"
        })
      }), "\n", _jsx(_components.h2, {
        children: "New Live Events Page"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/1.36/live-events.gif",
          alt: "Jitsu JavaScript functions debugger"
        })
      }), "\n", _jsx(_components.p, {
        children: "Live Events page has been completely rebuild. Now you can see if a particular event has been\nsent to a destination (or multiple destination):"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "For database destinations it shows an exact schema of destination table with type and value for each field"
        }), "\n", _jsx(_components.li, {
          children: "For webhook based destination the page displays HTTP payload"
        }), "\n"]
      }), "\n", _jsxs(_components.p, {
        children: ["Related issue: ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/issues/417",
          children: "417"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "New JavaScript functions debugger"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/1.36/debug.gif",
          alt: "Jitsu JavaScript functions debugger"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["We completely redesigned the UX of ", _jsx(_components.a, {
          href: "https://jitsu.com/blog/javascript-functions",
          children: "JavaScript functions debugger"
        }), ". This\nmakes easier to write and debug custom functions for WebHook destinations."]
      }), "\n", _jsx(_components.h2, {
        children: "iOS SDK"
      }), "\n", _jsxs(_components.p, {
        children: ["In addition to ", _jsx(_components.a, {
          href: "https://jitsu.com/docs/sending-data/api",
          children: "Event API"
        }), ",\nthat allows to send data to Jitsu from any platform, we're releasing ", _jsx(_components.a, {
          href: "https://jitsu.com/docs/sending-data/mobile-apps/ios-sdk",
          children: "iOS SDK"
        }), ". It's\na native iOS library (written in Swift, but works with Obj-C as well!)"]
      }), "\n", _jsx(_components.h2, {
        children: "New Docker deployment documentation"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/docs/jitsu-docker-images.png",
          alt: ""
        })
      }), "\n", _jsxs(_components.p, {
        children: ["We admit, our documentation about the Jitsu deployment was very messy. Based on users feedback we sat\ndown and rewrote it completely. The most confusing part was the difference between docker images,\nnow it's much better. Take a look onto new ", _jsx(_components.a, {
          href: "/deployment/deploy-with-docker",
          children: "documentation section"
        })]
      })]
    });
  }
}
return {
  default: MDXContent
};


Jitsu 1.36 is out!

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      h2: "h2",
      a: "a",
      img: "img",
      h3: "h3",
      code: "code",
      pre: "pre",
      ul: "ul",
      li: "li"
    }, props.components), {Hint} = _components;
    if (!Hint) _missingMdxReference("Hint", true, "28:1-31:8");
    return _jsxs(_Fragment, {
      children: [_jsx("br", {}), "\n", _jsx("img", {
        src: "/img/blog/supabase/cover.png",
        alt: "Supabase + Jitsu",
        width: "600"
      }), "\n", _jsx(_components.p, {
        children: "Supabase is the open-source Firebase alternative. Unlike Firebase which went with NoSQL key-value storage model, they use PostgresSQL\nas a storage layer. Key-value storages are great, but SQL gives users a more powerful way to aggregate data and build reports."
      }), "\n", _jsx(_components.p, {
        children: "However, Supabase lacks one important feature of Firebase - analytics. Firebase users can send events to linked Google Analytics account or to BigQuery table with JS API.\nSupabase do not have this yet. But luckily, we build Jitsu, an open-source data collection platform. Among other databases, Jitsu supports\nPostgres, which is a Supabase backbone."
      }), "\n", _jsx(_components.p, {
        children: "In this article we're going to show how Jitsu can be used along with Supabase to collect web-application events to PostgresSQL."
      }), "\n", _jsx(_components.h2, {
        children: "Setting up Supabase"
      }), "\n", _jsxs(_components.p, {
        children: ["You will need a Github account. Signup with it at ", _jsx(_components.a, {
          href: "https://app.supabase.io/",
          children: "app.supabase.com"
        }), " and create a new project"]
      }), "\n", _jsx("img", {
        src: "/img/blog/supabase/create-supabase-project.png",
        alt: "Destination Setup",
        width: "672"
      }), "\n", _jsx(_components.p, {
        children: "Make sure you saved a password somewhere. This password will be used later for Jitsu configuration."
      }), "\n", _jsx(_components.p, {
        children: "Go to Settings → Database → Connection Info, copy Postgres hostname and save it"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/supabase/copy-supabase-settings.png",
          alt: "Copy Hostname"
        })
      }), "\n", _jsx(_components.h2, {
        children: "Setting up Jitsu"
      }), "\n", _jsx(Hint, {
        children: _jsxs(_components.p, {
          children: ["For this tutorial, we're going to use a ", _jsx("a", {
            href: "https://jitsu.com",
            children: "Jitsu.Cloud"
          }), ", a hosted version Jitsu. However, the same\ntutorial can be applied to a ", _jsx("a", {
            href: "https://jitsu.com/docs/deployment",
            children: "self-hosted open-source Jitsu version"
          }), "."]
        })
      }), "\n", _jsx(_components.h3, {
        children: _jsx(_components.a, {
          href: "https://cloud.jitsu.com/register",
          children: "Signup"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Signup at ", _jsx(_components.a, {
          href: "https://cloud.jitsu.com/register",
          children: "cloud.jitsu.com/register"
        }), " We recommend using GitHub."]
      }), "\n", _jsx("img", {
        src: "/img/blog/supabase/jitsu-signup.png",
        alt: "Jitsu.Cloud Signup",
        width: "490"
      }), "\n", _jsx(_components.h3, {
        children: "Connect Jitsu to Supabase Postgres"
      }), "\n", _jsx(_components.p, {
        children: "Go through onboarding process and type in your name and company name. On Destination Setup step, select Postgres SQL"
      }), "\n", _jsx("img", {
        src: "/img/blog/supabase/jitsu-destination-setup.png",
        alt: "Jitsu Destination Setup",
        width: "605"
      }), "\n", _jsxs(_components.p, {
        children: ["Then set database host to the value copied from Supabase settings. Use saved Supabase password, use ", _jsx(_components.code, {
          children: "postgres"
        }), " as username\nand database name"]
      }), "\n", _jsx("img", {
        src: "/img/blog/supabase/jitsu-postgres-setup.png",
        alt: "Jitsu Postgres Setup",
        width: "605"
      }), "\n", _jsx(_components.h3, {
        children: "Add Jitsu SDK to your application"
      }), "\n", _jsx(_components.p, {
        children: "Use yarn:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-shell",
          children: "yarn add @jitsu/sdk-js\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "...or npm:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-shell",
          children: "npm install --save @jitsu/sdk-js\n"
        })
      }), "\n", _jsx(_components.h3, {
        children: "Start tracking events"
      }), "\n", _jsx(_components.p, {
        children: "You can use vanilla Javascript for sending events:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-javascript",
          children: "import { jitsuClient } from \"@jitsu/sdk-js\"\n//init\nconst jitsu = jitsuClient({\n  key: \"<JITSU KEY>\",\n  tracking_host: \"https://t.jitsu.com\",\n})\n//identify user\njitsu.id({\n  email: getEmail(),\n  internal_id: getId(),\n})\n//track page views\njitsu.track(\"app_page\")\n\n//track other events such as conversion\njitsu.track(\"conversion\")\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["To get a ", _jsx(_components.code, {
          children: "<JITSU KEY>"
        }), " go to Events API section and copy Client Secret"]
      }), "\n", _jsx("img", {
        src: "/img/blog/supabase/jitsu-api-key.png",
        alt: "Get Jitsu API Key",
        width: "1000"
      }), "\n", _jsx(Hint, {
        children: _jsxs(_components.p, {
          children: ["For extra security add your app domain (such as ", _jsx(_components.code, {
            children: "*.yourapp.com"
          }), ") by clicking Add Origin button"]
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Or get advantage of react hooks by implementing ", _jsx(_components.code, {
          children: "useJitsu()"
        }), ". Check out our ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu-react-example",
          children: "@jitsucom/jitsu-react-example"
        }), "\nrepository"]
      }), "\n", _jsx(_components.h3, {
        children: "Querying events"
      }), "\n", _jsxs(_components.p, {
        children: ["The data will be recorded to the table called ", _jsx(_components.code, {
          children: "events"
        }), " (you can change the name of the table in Jitsu settings, it's event possible\nto put events to different tables based on event content)."]
      }), "\n", _jsx(_components.p, {
        children: "Here's an example that displays the last tracked event with user location:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-sql",
          children: "select\n  _timestamp, page_title,\n  location_country, location_city, location_zip,\n  user_email\nfrom events order by _timestamp desc limit 1;\n"
        })
      }), "\n", _jsx("br", {}), "\n", _jsx("img", {
        src: "/img/blog/supabase/select-from-events.png",
        alt: "SQL Query",
        width: "605"
      }), "\n", _jsx(_components.h2, {
        children: "Next steps"
      }), "\n", _jsx(_components.p, {
        children: "Once the data is events are recorded into a database, we recommend using following tools to build\na full-stack open-source data pipeline:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: [_jsx(_components.a, {
            href: "https://www.getdbt.com/",
            children: "DBT"
          }), " is great for data modelling: building views and transforming. By the way, Jitsu is integrated with ", _jsx(_components.a, {
            href: "https://jitsu.com/blog/dbt-integration",
            children: "DBT Cloud"
          })]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.a, {
            href: "https://github.com/metabase/metabase",
            children: "Metabase"
          }), " is a great open-source visualization tool. It works smoothly with postgres & DBT. Another great open-source\nvisualization tools are ", _jsx(_components.a, {
            href: "https://github.com/apache/superset",
            children: "Apache Superset"
          }), " and ", _jsx(_components.a, {
            href: "https://github.com/getredash/redash",
            children: "Redash"
          })]
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};
function _missingMdxReference(id, component, place) {
  throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it." + (place ? "\nIt’s referenced in your code at `" + place + "`" : ""));
}


The step-by-step guide how to build an analytics collection layer on top of Supabase with Jitsu

Supabase analytics with Jitsu

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      a: "a",
      ul: "ul",
      li: "li",
      h2: "h2",
      img: "img"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsxs(_components.p, {
        children: ["You probably have heard of the words “Modern Data Stack.” There is a nice ", _jsx(_components.a, {
          href: "https://blog.getdbt.com/future-of-the-modern-data-stack/",
          children: "article"
        }), " that explains in great detail the idea, and I highly recommend reading it.\nLong story short, the last couple of years brought us a set of great tools that made it much easier and cheaper for a company to build a Data Analysis process.\nAnd one of the directions of further modern data stack evolution is making data processing real-time or getting close to it."]
      }), "\n", _jsx(_components.p, {
        children: "Jitsu is proud to be a part of the modern data stack. And we put much effort into delivering data to your destinations as fast as possible by supporting streaming events into warehouses and reducing batch processing intervals to just a few minutes."
      }), "\n", _jsxs(_components.p, {
        children: ["Today, we are announcing integration with dbt – a feature that allows you to reduce further the time it takes for data models to react to changes in a data flow.\n", _jsx(_components.a, {
          href: "https://getdbt.com/",
          children: "dbt"
        }), " – leading open-source data transformation platform and dbt Cloud is a cloud deployment of dbt tools with a friendly UI with IDE features like support for integrated documentation, version control, CI/DC, and API."]
      }), "\n", _jsx(_components.p, {
        children: "The integration consist of two parts:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Jitsu can trigger a dbt Cloud job after each Source synchronization or Destination batch run"
        }), "\n", _jsx(_components.li, {
          children: "Dbt package that makes it easier to build DBT models from Jitsu data"
        }), "\n"]
      }), "\n", _jsx(_components.h2, {
        children: "Triggering dbt Cloud Job"
      }), "\n", _jsxs(_components.p, {
        children: ["Now you can set up dbt Cloud job settings in Jitsu UI.\nWhen configured and enabled – each successful run of Source sync task or Destination in batch mode will trigger a dbt Cloud Job run.\n", _jsx(_components.img, {
          src: "/img/blog/dbt-integration/setup.png",
          alt: ""
        })]
      }), "\n", _jsxs(_components.p, {
        children: ["You can even use JavaScript to customize the ‘Cause’ message. That will help to understand what triggered sync.\n", _jsx(_components.img, {
          src: "/img/blog/dbt-integration/dbt-cloud.png",
          alt: ""
        })]
      }), "\n", _jsx(_components.h2, {
        children: "Jitsu dbt hub package"
      }), "\n", _jsxs(_components.p, {
        children: ["We have released our first package for dbt hub: ", _jsx(_components.a, {
          href: "https://hub.getdbt.com/jitsucom/dbt-jitsu/latest/",
          children: "dbt-jitsu"
        }), "\nts initial version implements the ‘Sessionization’ feature for Jitsu page views data similarly to dbt’s package for Segment."]
      }), "\n", _jsxs(_components.p, {
        children: ["Sessionization – is the process of aggregation of page view data into user sessions.\nThe result is a table with data like session duration, page views per session, referral source, and a total number of sessions per user – in other words, data required for proper user analysis.\n", _jsx(_components.img, {
          src: "/img/blog/dbt-integration/sessions.png",
          alt: ""
        })]
      }), "\n", _jsx(_components.p, {
        children: "A nice addition if you already process Jitsu data with dbt or just a good reason to start."
      }), "\n", _jsx(_components.p, {
        children: "We are going to expand our packages functions in the future by adding support for more kinds of source data, e.g., mobile application usage data\nand adding more data transformations for typical analytics demands."
      }), "\n", _jsx(_components.h2, {
        children: "Modern Data Stack with Jitsu and dbt"
      }), "\n", _jsx(_components.p, {
        children: "Bringing both parts together that is how Modern Data Stack may look like with Jitsu and dbt integrated:"
      }), "\n", _jsx("img", {
        src: "/img/blog/dbt-integration/modern-stack.png",
        alt: "",
        width: "700"
      }), "\n", _jsx(_components.p, {
        children: "In the most simple case, Jitsu allows getting new page views sessions data with retroactive user recognition in just a few minutes."
      }), "\n", _jsx(_components.p, {
        children: "But the fact that dbt Cloud integrated with Reverse ETL tools such as Hightouch and Census allows shortening data traveling time from an origin to the tools that bring value to your product."
      }), "\n", _jsx(_components.h2, {
        children: "Learning More"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["Follow & star ", _jsx(_components.a, {
            href: "https://github.com/jitsucom/jitsu",
            children: "Jitsu on GitHub"
          })]
        }), "\n", _jsxs(_components.li, {
          children: ["Try a cloud version of ", _jsx(_components.a, {
            href: "https://cloud.jitsu.com",
            children: "Jitsu"
          }), ". It's free for up to 250,000 events per month"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.a, {
            href: "https://jitsu.com/slack",
            children: "Join our Slack"
          }), "! Our community will help you with any questions!"]
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};


Modern Data Stack with Jitsu and dbt integrated

Jitsu and dbt integration

Learn how to use Jitsu in cookieless mode and comply with cookie laws

Cookies and privacy compliant web analytics

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      a: "a",
      img: "img",
      h2: "h2",
      ul: "ul",
      li: "li",
      strong: "strong",
      h3: "h3",
      pre: "pre",
      code: "code"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsxs(_components.p, {
        children: ["If you like to write scripts from time to time, or you are a seasoned software engineer – we have great news for you! We're rolling out ", _jsx(_components.a, {
          href: "https://jitsu.com/docs/other-features/javascript-transform",
          children: "JavaScript Functions"
        }), " as part of our efforts to make Jitsu a scriptable platform. We started with enabling functions for scripting table names and WebHook request bodies. Later, we're going to expand Functions to all aspects of event processing, including mapping and filtering."]
      }), "\n", _jsx(_components.p, {
        children: "If you are unfamiliar with programming languages – no worries. This may be a good place to start because our Code Debugger allows you to interactively practice writing JavaScript code on test data samples."
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/javascript-functions/code-debugger.png",
          alt: "Interactive Code Debugger"
        })
      }), "\n", _jsx(_components.h2, {
        children: "How to use"
      }), "\n", _jsx(_components.p, {
        children: "JavaScript Functions come in handy in cases when certain destination parameters need to be evaluated based on properties of an incoming event. These parameters are:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "Table name"
          }), " – to choose destination ", _jsx(_components.a, {
            href: "https://jitsu.com/docs/configuration/table-names-and-filters",
            children: "table name or skip"
          }), " event from processing"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "HTTP URL"
          }), " – to fill destination URL like ", _jsx(_components.a, {
            href: "https://jitsu.com//docs/destinations-configuration/webhook",
            children: "WebHook"
          }), " URL"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "JSON Body"
          }), " – to produce JSON payload that will be sent to destination"]
        }), "\n"]
      }), "\n", _jsx(_components.p, {
        children: "Jitsu UI marks fields where JavaScript Function can be applied with a green bug icon 🪲.\nClicking on that icon brings a Code Debugger window where you can perfect your function."
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/javascript-functions/webhook-form.png",
          alt: "Click the green bug"
        })
      }), "\n", _jsx(_components.h2, {
        children: "Technical Details"
      }), "\n", _jsx(_components.p, {
        children: "Fast delivery of new features for our clients is our primary goal in Jitsu."
      }), "\n", _jsxs(_components.p, {
        children: ["That is why the first implementation of the dynamic fields evaluation was based on the ", _jsx(_components.a, {
          href: "https://golang.org/pkg/text/template/",
          children: "GO text template"
        }), ".\nBecause we write Jitsu backend with Go language, the decision to choose GO text/template came naturally as it is the fastest tool to deliver that does the job.\nJitsu will keep supporting the GO text/template to maintain backward compatibility with all our current user setups. Language is detected automatically based on syntax of expression."]
      }), "\n", _jsxs(_components.p, {
        children: ["When we were deciding how to integrate JavaScript support into our Go application, we were committed to our goal.\nThat is why for our first implementation of JavaScript engine we preferred ", _jsx(_components.a, {
          href: "https://github.com/dop251/goja",
          children: "dop251/goja"
        }), " over a V8-based solution.\nGoja engine is written in pure Go. That allowed us to save a lot of time on not dealing with supporting different architectures of V8 dependency for our packages."]
      }), "\n", _jsx(_components.p, {
        children: "JavaScript Functions use Babel 7.14 internally to allow users to use some nice features of JavaScript language like Optional chaining or Template literals."
      }), "\n", _jsx(_components.p, {
        children: "We still like the idea of migrating to V8 in the future."
      }), "\n", _jsx(_components.h2, {
        children: "Examples"
      }), "\n", _jsx(_components.h3, {
        children: "Table name evaluation examples"
      }), "\n", _jsx(_components.p, {
        children: "Table name expression that skips events without user's email. Passed events go to table based on metric_type:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-javascript",
          children: "$.user?.email && $.metric_type\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Tables will have 'event_type_YYYY_MM' format based on 'event_type' and date. Golang's ", _jsx(_components.a, {
          href: "https://pkg.go.dev/time#Time.Format",
          children: "Format"
        }), " function is used for convenience (", _jsx("a", {
          href: "https://user-images.githubusercontent.com/18486255/98962749-fd5ae480-2517-11eb-9448-93aafdf6f97f.png",
          target: "blank",
          children: "Take a hint"
        }), ")."]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-javascript",
          children: ";`${_.event_type}_${_._timestamp.Format(\"2006_01\")}`\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "Keep only conversion events (skip other) in case a user isn't Google Analytics known user:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-javascript",
          children: "$.event_type == \"conversion\" && !$.ids?.ga && $.event_type\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "more readable form:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-javascript",
          children: "if ($.event_type == \"conversion\") {\n  //only for conversion events\n  if (!$.ids?.ga) {\n    //no ga id present\n    return $.event_type\n  }\n}\nreturn \"\"\n"
        })
      }), "\n", _jsx(_components.h3, {
        children: "Data Masking"
      }), "\n", _jsx(_components.p, {
        children: "Simple example that removes all emails from an event object before choosing a table name.\nObject modifications in JavaScript Function reflect on what actually will be sent to destination:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-javascript",
          children: "var emailRegexp =\n  /^([a-zA-Z0-9_\\-\\.]+)@((\\[[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.)|(([a-zA-Z0-9\\-]+\\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\\]?)$/\nfunction removeEmails(obj) {\n  for (const key in obj) {\n    if (typeof obj[key] === \"object\") {\n      removeEmails(obj[key])\n    } else if (typeof obj[key] === \"string\" && obj[key].match(emailRegexp)) {\n      delete obj[key]\n    }\n  }\n}\nremoveEmails($)\nreturn \"events_without_emails\"\n"
        })
      }), "\n", _jsx(_components.h3, {
        children: "WebHook with JSON Body example"
      }), "\n", _jsx(_components.p, {
        children: "Choose a Webhook URL based on event type:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-javascript",
          children: "let baseURL = \"https://hooks.slack.com/services/\"\nswitch ($.event_type) {\n  case \"conversion\":\n    return baseURL + \"123/abc/ABC\"\n  case \"newuser\":\n    return baseURL + \"987/xyz/XYZ\"\n}\nreturn \"\"\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "Fill the body of WebHook payload:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-javascript",
          children: "return {\n  blocks: [\n    {\n      type: \"header\",\n      text: {\n        type: \"plain_text\",\n        text: \"🤘\" + ` ${_.metric_type} from ${_.user?.email}`,\n        emoji: true,\n      },\n    },\n    {\n      type: \"section\",\n      text: {\n        type: \"mrkdwn\",\n        text: `User ${_.user?.email} has sent *${_.metric_type}* in <https://example.com?email=${encodeURIComponent(\n          _.user?.email\n        )}>`,\n      },\n    },\n    {\n      type: \"section\",\n      text: {\n        type: \"mrkdwn\",\n        text: \"*Data*:```\" + JSON.stringify(_, null, 4) + \"```\",\n      },\n    },\n  ],\n}\n"
        })
      }), "\n", _jsx(_components.h2, {
        children: "Learning More"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["Follow & star ", _jsx(_components.a, {
            href: "https://github.com/jitsucom/jitsu",
            children: "Jitsu on GitHub"
          })]
        }), "\n", _jsxs(_components.li, {
          children: ["Try a cloud version of ", _jsx(_components.a, {
            href: "https://cloud.jitsu.com",
            children: "Jitsu"
          }), ". It's free for up to 250,000 events per month"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.a, {
            href: "https://jitsu.com/slack",
            children: "Join our Slack"
          }), "! Our community will help you with any questions!"]
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};


Learn how the power of JavaScript language helps to achieve new level of flexibility in events processing

Javascript Functions (technical preview)

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      a: "a",
      h3: "h3",
      img: "img",
      h2: "h2",
      ol: "ol",
      li: "li",
      ul: "ul",
      code: "code",
      pre: "pre"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsxs(_components.p, {
        children: ["Recently we've released ", _jsx(_components.a, {
          href: "https://jitsu.com/docs/destinations-configuration/webhook",
          children: "WebHook destination"
        }), ".\nIt is a powerful feature that allows Jitsu to send events into a variety of services including ", _jsx(_components.a, {
          href: "http://slack.com/",
          children: "Slack"
        }), ".\nSlack notifications are suitable for staying up-to-date with significant events such as new user signups, payment plan updates, errors, and others.\nWe're big fans of ", _jsx(_components.a, {
          href: "https://en.wikipedia.org/wiki/Eating_your_own_dog_food",
          children: "dog-fooding"
        }), " and we use Jitsu for our ", _jsx(_components.a, {
          href: "https://jitsu.com",
          children: "jitsu.com"
        }), " and ", _jsx(_components.a, {
          href: "https://cloud.jitsu.com",
          children: "Jitsu.Cloud"
        }), ".\nEvery time a user signs up or changes their subscription plan we put a notification into a Slack-channel. Notifications about frontend errors go to the same channel as well."]
      }), "\n", _jsx(_components.h3, {
        children: "Stay up-to-date with new signups"
      }), "\n", _jsx(_components.p, {
        children: "On the early company stage every user signup is important. Once a user signs up, a customer success manager (or even a founder!) might want to take a look at user actions and settings (via logs) and make sure that the new customer experience is as smooth as possible."
      }), "\n", _jsxs(_components.p, {
        children: ["We receive Slack notifications about new customer signups on ", _jsx(_components.a, {
          href: "https://cloud.jitsu.com",
          children: "Jitsu.Cloud"
        }), ". Jitsu can include the source of an advertising campaign, and location geo data with\ncustomer city as in the example below:"]
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/slack-notifications/registration-highlighted.png",
          alt: "User has been signed up to notification"
        })
      }), "\n", _jsx(_components.h3, {
        children: "React to significant events"
      }), "\n", _jsx(_components.p, {
        children: "Another use-case is getting notifications within the app about significant events that have happened. Example: the customer just upgraded their payment plan.\nAt Jitsu we get notifications about plan upgrade and changes in sources/destinations configuration."
      }), "\n", _jsx(_components.h3, {
        children: "Error notification errors"
      }), "\n", _jsx(_components.p, {
        children: "While dedicated monitoring tools (such as Sentry) are better suited for advanced monitoring, Jitsu can do the basic error notification just as well. Once an error occurs the first time, we are immediately notified with the error description (stack trace context) along with all relevant details."
      }), "\n", _jsx(_components.h2, {
        children: "Configuration"
      }), "\n", _jsx(_components.p, {
        children: "See the detailed configuration we used below."
      }), "\n", _jsx(_components.h3, {
        children: "Slack App Configuration"
      }), "\n", _jsx(_components.p, {
        children: "Slack uses apps for accepting incoming requests and sending notifications to a certain channel. Configure Slack app with\nthe following steps:"
      }), "\n", _jsxs(_components.ol, {
        children: ["\n", _jsxs(_components.li, {
          children: ["Open ", _jsx(_components.a, {
            href: "https://api.slack.com/apps",
            children: "Slack Apps page"
          })]
        }), "\n", _jsx(_components.li, {
          children: "Create New App -> From scratch -> Select a clear name and Workspace"
        }), "\n"]
      }), "\n", _jsx("img", {
        src: "/img/blog/slack-notifications/create-slack-app.png",
        width: "800"
      }), "\n", _jsxs(_components.ol, {
        start: "3",
        children: ["\n", _jsx(_components.li, {
          children: "Application page is opened. Go to Features -> Incoming Webhooks menu -> Switch ON \"Activate Incoming Webhooks\" toggle"
        }), "\n"]
      }), "\n", _jsx("img", {
        src: "/img/blog/slack-notifications/enable-slack-app.png",
        width: "800"
      }), "\n", _jsxs(_components.ol, {
        start: "4",
        children: ["\n", _jsx(_components.li, {
          children: "Click on \"Add New Webhook to Workspace\" in the bottom of the page"
        }), "\n"]
      }), "\n", _jsx("img", {
        src: "/img/blog/slack-notifications/add-new-webhook-to-space.png",
        width: "600"
      }), "\n", _jsxs(_components.ol, {
        start: "5",
        children: ["\n", _jsx(_components.li, {
          children: "Select Slack channel for notifications and click \"Allow\""
        }), "\n"]
      }), "\n", _jsx("img", {
        src: "/img/blog/slack-notifications/allow-webhook-app.png",
        width: "400"
      }), "\n", _jsxs(_components.ol, {
        start: "6",
        children: ["\n", _jsx(_components.li, {
          children: "Copy Webhook URL and use it on Jitsu WebHook destination configuration page"
        }), "\n"]
      }), "\n", _jsx("img", {
        src: "/img/blog/slack-notifications/copy-webhook-url.png",
        width: "600"
      }), "\n", _jsx(_components.h3, {
        children: "Jitsu Destination Configuration"
      }), "\n", _jsx(_components.p, {
        children: "WebHook destination can build requests on the fly with dynamic structure based on input events. Configure Slack integration with\nthe following steps:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["Make sure you've added ", _jsx(_components.a, {
            href: "https://jitsu.com/docs/sending-data/js-sdk",
            children: "Jitsu JS tracking pixel"
          }), " and ", _jsx(_components.code, {
            children: "jitsu.track('{event_type}', {...event data})"
          }), " (where ", _jsx(_components.code, {
            children: "event_type"
          }), " might be ", _jsx(_components.code, {
            children: "registration"
          }), ", ", _jsx(_components.code, {
            children: "conversion"
          }), ", etc) tracking code on your website."]
        }), "\n", _jsxs(_components.li, {
          children: ["Open ", _jsx(_components.a, {
            href: "http://cloud.jitsu.com/",
            children: "Jitsu.Cloud"
          }), " or Jitsu self-hosted configurator UI"]
        }), "\n", _jsxs(_components.li, {
          children: ["Go to the Destinations page -> click ", _jsx(_components.code, {
            children: "Add Destination"
          }), " and select ", _jsx(_components.code, {
            children: "WebHook"
          })]
        }), "\n", _jsxs(_components.li, {
          children: ["Put Slack Webhook URL from ", _jsx(_components.a, {
            href: "#slack-app-configuration",
            children: "Slack App Configuration section"
          }), " into HTTP URL field, HTTP method - ", _jsx(_components.code, {
            children: "POST"
          })]
        }), "\n", _jsx(_components.li, {
          children: "Use Table Name field as a filter if you would like to send only registration events:"
        }), "\n"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          children: "{{if eq .event_type \"registration\"}}send{{else}}{{end}}\n"
        })
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["Fill HTTP JSON Body with your Slack notification template. It is a structured JSON with ", _jsx(_components.a, {
            href: "https://golang.org/pkg/text/template/",
            children: "GO text/template"
          }), " placeholders. For instance: ", _jsx(_components.code, {
            children: "{{.event_type}}"
          }), " extracts ", _jsx(_components.code, {
            children: "event_type"
          }), " field value from event JSON.\nCheck notification syntax on ", _jsx(_components.a, {
            href: "https://api.slack.com/docs/messages/builder?msg=%7B%22text%22%3A%22I%20am%20a%20test%20message%22%2C%22attachments%22%3A%5B%7B%22text%22%3A%22And%20here%E2%80%99s%20an%20attachment!%22%7D%5D%7D",
            children: "Slack page"
          }), ":"]
        }), "\n"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-json",
          children: "{\n  \"blocks\": [\n    {\n      \"type\": \"header\",\n      \"text\": {\n        \"type\": \"plain_text\",\n        \"text\": \"🤘 {{.event_type}} from {{.user.email}}\",\n        \"emoji\": true\n      }\n    },\n    {\n      \"type\": \"section\",\n      \"text\": {\n        \"type\": \"mrkdwn\",\n        \"text\": \"User {{.user.email}} has sent *{{.event_type}}*\"\n      }\n    },\n    {\n      \"type\": \"section\",\n      \"text\": {\n        \"type\": \"mrkdwn\",\n        \"text\": \"*Data*:```{{ json_indent_quote . }}```\"\n      }\n    }\n  ]\n}\n"
        })
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Get Jitsu notifications in your Slack!"
        }), "\n"]
      }), "\n", _jsx("img", {
        src: "/img/blog/slack-notifications/slack-notification.png",
        width: "800"
      }), "\n", _jsx(_components.h2, {
        children: "Learning More"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["Follow & star ", _jsx(_components.a, {
            href: "https://github.com/jitsucom/jitsu",
            children: "Jitsu on GitHub"
          })]
        }), "\n", _jsxs(_components.li, {
          children: ["Try a cloud version of ", _jsx(_components.a, {
            href: "https://cloud.jitsu.com",
            children: "Jitsu"
          }), ". It's free for up to 250,000 events per month"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.a, {
            href: "https://jitsu.com/slack",
            children: "Join our Slack"
          }), "! Our community will help you with any questions!"]
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};


Learn how to get Slack notifications about significant application events with Jitsu. No code required

No-code Jitsu: Flexible Slack Notifications

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      a: "a",
      h2: "h2",
      img: "img",
      strong: "strong",
      code: "code",
      em: "em",
      ul: "ul",
      li: "li"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsxs(_components.p, {
        children: ["We launched Jitsu on Product Hunt on ", _jsx(_components.a, {
          href: "https://www.producthunt.com/posts/jitsu-2",
          children: "June 2nd (Wednesday)"
        }), ".\nWhile Product Hunt is not a typical medium for launching products such as Jitsu (open-source platform for data engineers), we decided to do it anyway."]
      }), "\n", _jsx(_components.p, {
        children: "Our objective was pretty simple: to attract users. We didn't care much about getting a badge (becoming Product of the day). In our opinion, it’s a vanity metric rather than an objective measurement."
      }), "\n", _jsx(_components.p, {
        children: "We didn't expect much from this experiment, and the results were quite surprising."
      }), "\n", _jsx(_components.h2, {
        children: "Traffic"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/product-hunt-launch/product-hunt-traffic.svg",
          alt: "Product Hunt Daily Traffic",
          title: "Traffic by Day (PST)"
        })
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.strong, {
          children: "Excess Uniques"
        }), " is the number of unique users above our daily average.\nIt's a pretty accurate estimate of the effect of the launch. ", _jsx(_components.strong, {
          children: "Product Uniques"
        }), " is the number of users who can be explicitly linked with Product Hunt (through the referer or landing page URL which ends with ", _jsx(_components.code, {
          children: "?ref=producthunt"
        }), "."]
      }), "\n", _jsx(_components.p, {
        children: "As you may see, the difference is pretty substantial (from 70% to 47%, see green line on a plot).\nSome users clearly came to jitsu.com as a result of the launch, but not directly. We identified two major sources: github, Twitter. They account for 30% of excess traffic."
      }), "\n", _jsxs(_components.p, {
        children: ["It’s worth mentioning that, on a launch day, I’ve been approached by a few people on LinkedIn and have\nhave been offered the service of becoming ", _jsx(_components.em, {
          children: "Product of the Day"
        }), " at ProductHunt. It would assume that,\nif such services existed, people would tend to buy them from time to time. I can only guess how those service providers\nget the upvotes."]
      }), "\n", _jsx(_components.p, {
        children: "\"If you were beaten by another product, don’t worry! Somebody probably wasn't playing fair!\""
      }), "\n", _jsx(_components.h2, {
        children: "What about conversions?"
      }), "\n", _jsxs(_components.p, {
        children: ["Of course, website traffic wasn't the main objective of the launch. What we cared about, is conversions. Since Jitsu has both\n", _jsx(_components.a, {
          href: "/docs/deployment",
          children: "self-hosted"
        }), " and ", _jsx(_components.a, {
          href: "https://cloud.jitsu.com",
          children: "cloud"
        }), ", version conversion can happen in two ways:"]
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["User signed up for ", _jsx(_components.a, {
            href: "https://cloud.jitsu.com",
            children: "cloud.jitsu.com"
          }), " (", _jsx(_components.em, {
            children: "signups"
          }), ")"]
        }), "\n", _jsxs(_components.li, {
          children: ["User installed Jitsu on a server (", _jsx(_components.em, {
            children: "installs"
          }), ")"]
        }), "\n"]
      }), "\n", _jsxs(_components.p, {
        children: ["While signups' tracking is pretty straightforward, tracking ", _jsx(_components.em, {
          children: "installs"
        }), " and especially, attributing them to an acquisition channel is not an easy\ntask. Installs statistics will be covered later, let's take a look at signups."]
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/product-hunt-launch/product-hunt-conversions.svg",
          alt: "Product Hunt Conversions",
          title: "Product Hunt Conversions"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["The same as with traffic, not all signups can be directly linked to clicks from Product Hunt. We added\n", _jsx(_components.strong, {
          children: "Excess Signups"
        }), " (signups above our daily average)."]
      }), "\n", _jsx(_components.p, {
        children: "Surprisingly, the conversion rate went down from 5% to 2% after day one. We don't have a good explanation for that."
      }), "\n", _jsx(_components.h2, {
        children: "Open-Source Tracking (aka Telemetry)"
      }), "\n", _jsxs(_components.p, {
        children: ["In the open-source world, user tracking is called ", _jsx(_components.code, {
          children: "Telemetry"
        }), ". It's way less accurate than any SaaS tracking. Unlike SaaS, we:"]
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Collect only anonymized data (no emails). We can't identify users accurately"
        }), "\n", _jsxs(_components.li, {
          children: ["Allow users to opt out from tracking easily. If Jitsu is deployed with UI through ", _jsx(_components.a, {
            href: "https://jitsu.com/docs/deployment/deploy-with-docker#run-dockercompose",
            children: "Docker compose"
          }), "\nwe ask users if they are OK with anonymized usage collection. Opting out from ", _jsx(_components.a, {
            href: "https://jitsu.com/docs/deployment/deploy-with-docker#deploying-jitsu-server",
            children: "Server Only"
          }), " deploy is less obvious (it's done through config variable)."]
        }), "\n"]
      }), "\n", _jsx(_components.p, {
        children: "If the user deploys UI and switches off Telemetry, we send a last event about that fact. Surprisingly, more than half the users decided to opt out from telemetry collection"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/product-hunt-launch/telemetry-opt-out.svg",
          alt: "Telemetry Opt-Out stat",
          title: "Telemetry Opt-Out stat"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["However, it's not possible to link the user to the traffic source even if the telemetry is on. The only way to measure the result of the launch is to calculate ", _jsx(_components.strong, {
          children: "Excess Installs"
        }), " (installs above\nour averages)"]
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/product-hunt-launch/installs-vs-jitsu-cloud.svg",
          alt: "Installs vs Jitsu.Cloud",
          title: "Installs"
        })
      }), "\n", _jsx(_components.p, {
        children: "If we compare Installs with Signups, the latter will prevail. It's not surprising, since the Product Hunt audience is less technical and more product-oriented."
      }), "\n", _jsx(_components.h2, {
        children: "Time"
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/product-hunt-launch/hourly-traffic.svg",
          alt: "Hourly traffic",
          title: "Hourly traffic"
        })
      }), "\n", _jsx(_components.p, {
        children: "Another interesting topic is distribution of traffic over time. Product Hunt features products in daily batches where the day starts at 12:00 am PST. American\nusers are usually asleep at that time, while European users are just starting their day. As you may see, we got way more traffic at the beginning of the PST day. It's been coming mostly from the\nEU. It can be explained by two factors:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Historically, our current user base was mostly coming from the EU"
        }), "\n", _jsxs(_components.li, {
          children: ["It's been a long holiday for us before our launch (Wed, Jun 2nd), Monday was a ", _jsx(_components.a, {
            href: "https://en.wikipedia.org/wiki/Memorial_Day",
            children: "Memorial Day"
          })]
        }), "\n"]
      }), "\n", _jsx(_components.h2, {
        children: "Tools"
      }), "\n", _jsx(_components.p, {
        children: "We used those amazing tools to build the data above:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: [_jsx(_components.a, {
            href: "https://jitsu.com",
            children: "Jitsu"
          }), " 🙂 We're big fans of ", _jsx(_components.a, {
            href: "https://en.wikipedia.org/wiki/Eating_your_own_dog_food",
            children: "dog-fooding"
          }), "! ", _jsx(_components.a, {
            href: "https://jitsu.com/docs/other-features/retroactive-user-recognition",
            children: "Retroactive User Recognition"
          }), "\nhelped a lot with attributing signups to the traffic source"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.a, {
            href: "https://www.getdbt.com/",
            children: "DBT"
          }), " is another open-source tool. I would characterize them as ", _jsx(_components.em, {
            children: "\"SQL Views on Steroids\""
          }), ". We used DBT to post-process raw data collected by Jitsu"]
        }), "\n", _jsxs(_components.li, {
          children: ["Usually we do visualization in ", _jsx(_components.a, {
            href: "https://www.metabase.com/",
            children: "Metabase"
          }), " (as you may already guess, they are open-source too). However, this time, Google Sheets worked better, because we needed to match the plot colors with the website theme."]
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};


We launched Jitsu on ProductHunt a week ago. In this blog post, we're sharing our internal statistics: visitors, conversion rates and many more

Product Hunt launch Statistics

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      h2: "h2",
      p: "p",
      code: "code",
      a: "a",
      em: "em",
      strong: "strong",
      h3: "h3",
      pre: "pre",
      img: "img"
    }, props.components), {Hint} = _components;
    if (!Hint) _missingMdxReference("Hint", true, "15:1-18:8");
    return _jsxs(_Fragment, {
      children: [_jsx(_components.h2, {
        children: "Background"
      }), "\n", _jsxs(_components.p, {
        children: ["Apple released a new MacBooks based on a M1 chip a while ago. Unlike all previous Apple laptops (which were Intel based),\nM1 has a different architecture — ", _jsx(_components.code, {
          children: "arm64"
        }), " instead Intel's ", _jsx(_components.code, {
          children: "amd64"
        }), "."]
      }), "\n", _jsxs(_components.p, {
        children: ["New architecture means that the most software products should be repackaged: at least ones written in compiled-languages.\nIt applies to Jitsu which is written mostly in ", _jsx(_components.code, {
          children: "Go"
        }), ". However, it's not 100% true: Apple went to great lengths to make\nold compiled code compatible with new architecture. The piece of software that makes is possible is called\n", _jsx(_components.a, {
          href: "https://en.wikipedia.org/wiki/Rosetta_(software)",
          children: "Roseta 2"
        }), "."]
      }), "\n", _jsxs(_components.p, {
        children: ["Docker tried to catch up as well: ", _jsx(_components.a, {
          href: "https://docs.docker.com/docker-for-mac/apple-silicon/",
          children: "the M1 support came out a few weeks ago"
        }), ".\nHowever, things do not work quite as seamless as:"]
      }), "\n", _jsx(Hint, {
        children: _jsxs(_components.p, {
          children: ["Docker images built for ", _jsx("code", {
            inline: true,
            children: "amd64"
          }), " (Intel) will run on M1 Macs. But the stability isn't guaranteed. You should\nship a second version of your image for ", _jsx("code", {
            inline: true,
            children: "arm64"
          }), " (M1 chip)."]
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Docker relies on ", _jsx(_components.a, {
          href: "https://www.qemu.org/",
          children: "qemu"
        }), " to emulate Intel's architecture on M1 chips. Unfortunately,\nQEMU doesn't work quite well: ", _jsx(_components.em, {
          children: "However, attempts to run Intel-based containers on Apple Silicon machines can crash as QEMU sometimes fails to run the container. Filesystem change notification APIs (e.g. inotify) do not work under QEMU emulation [...]. Therefore, we\nrecommend that you run ARM64 containers on Apple Silicon machines. These containers are also\nfaster and use less memory than Intel-based containers."
        }), " (See docker ", _jsx(_components.a, {
          href: "https://docs.docker.com/docker-for-mac/apple-silicon/",
          children: "documentation for more details"
        }), ")"]
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.strong, {
          children: "Why care about M1 macs at all? Aren't all servers still run on Intel anyway?"
        }), " That's a great question! It's true,\nmost of the servers runs on Intel platform. However, at Jitsu ", _jsx(_components.strong, {
          children: "we committed to provide an excellent developer's experience"
        }), ".\nWe're sure that stable work at a developer's machine is as much important as server's stability.\nBesides, 1/2 of our dev team already got a new M1 Macs. So starting from ", _jsx(_components.a, {
          href: "/docs/changelog/1.29.0-19-mar-2021",
          children: "1.29.0"
        }), " we decided\nto ship all releases as dual-architecture images."]
      }), "\n", _jsx(_components.p, {
        children: "We're convinced that all other open-source project should do the same. Therefore, we decided to share our experience with multi-architecture\ndocker builds."
      }), "\n", _jsxs(_components.h2, {
        children: ["How to build an image for ", _jsx(_components.code, {
          children: "arm64"
        })]
      }), "\n", _jsxs(_components.p, {
        children: ["Luckily, ", _jsx(_components.a, {
          href: "https://www.docker.com/blog/multi-platform-docker-builds/",
          children: "Docker has announced"
        }), " the support of cross CPU architecture builds\na few weeks ago. ", _jsx(_components.strong, {
          children: "buildx"
        }), " - an experimental feature that allows building images for a certain architecture - arm64, amd64, etc."]
      }), "\n", _jsx(_components.h3, {
        children: "Step 1. Enable experimental features"
      }), "\n", _jsxs(_components.p, {
        children: ["If you’re using Docker Desktop for Mac you should just enable experimental features on the Docker CLI if you haven’t already – just edit\n", _jsx(_components.strong, {
          children: "~/.docker/config.json"
        }), " to include the following:"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-json",
          children: "{\n  ...\n  \"experimental\": \"enabled\"\n}\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "Restart your Docker Desktop after that"
      }), "\n", _jsx(_components.p, {
        children: "If you’re using Linux, just run the following command:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-shell",
          children: "docker run --privileged --rm docker/binfmt:a7996909642ee92942dcd6cff44b9b95f08dad64\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["And restart Docker with ", _jsx(_components.code, {
          children: "service docker restart"
        })]
      }), "\n", _jsx(_components.h3, {
        children: "Step 2. Create a new builder instance"
      }), "\n", _jsx(_components.p, {
        children: "Create a new builder instance:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-shell",
          children: "docker buildx create --use\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "It enables multiple builds in parallel. Make sure that new builder instance is created:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-shell",
          children: "docker buildx ls\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "Output:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-shell",
          children: "$ docker buildx ls\nNAME/NODE     DRIVER/ENDPOINT       STATUS  PLATFORMS\nmystifying_bell * docker-container\n  mystifying_bell0 unix:///var/run/docker.sock inactive\ndefault      docker\n  default     default           running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6\n"
        })
      }), "\n", _jsx(_components.h3, {
        children: "Step 3. Build multi-architecture image"
      }), "\n", _jsx(_components.p, {
        children: "Build your multi-architecture image:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-shell",
          children: "docker buildx build --platform linux/amd64,linux/arm64 -t company/image_name .\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.strong, {
          children: "platform"
        }), " flag specifies for which platforms Docker image will be built. Docker support 10 platforms,\nbut probably you shouldn use only ", _jsx(_components.code, {
          children: "linux/amd64"
        }), " (Intel) and ", _jsx(_components.code, {
          children: "linux/arm64"
        }), " (M1):"]
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/multi-platform-docker-builds/docker-hub-result.png",
          alt: "Docker Hub result",
          title: "Docker Hub result"
        })
      }), "\n", _jsx(_components.h3, {
        children: "Build multi-architecture images on CircleCI"
      }), "\n", _jsxs(_components.p, {
        children: ["At Jitsu we use CircleCI for building and pushing our Docker images to ", _jsx(_components.a, {
          href: "https://hub.docker.com/u/jitsucom",
          children: "Docker Hub"
        }), ".\nThere is an example of ", _jsx(_components.strong, {
          children: ".circleci/config.yaml"
        }), " with building multi-architecture Docker images on every git push to master branch:"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-yaml",
          children: "install_buildx: &install_buildx\n  run:\n    name: install docker buildx\n    command: |\n      mkdir -vp ~/.docker/cli-plugins/\n      curl --silent -L --output ~/.docker/cli-plugins/docker-buildx https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-amd64\n      chmod a+x ~/.docker/cli-plugins/docker-buildx\n      docker buildx version\n      docker context create jitsu\n      docker buildx create jitsu --use\n      docker run --privileged --rm tonistiigi/binfmt --install all\n\nbuild_jitsu: &build_jitsu\n  run:\n    name: Build and Push Jitsu Docker image\n    command: |\n      docker login -u $DOCKER_LOGIN -p $DOCKER_PWD\n      DOCKER_BUILDKIT=1 docker buildx build --platform linux/amd64,linux/arm64 --push -t $IMAGE_NAME:$IMAGE_VERSION -f $DOCKER_FILE .\n\nversion: 2.1\n\njobs:\n  build-latest-jitsu-docker:\n    environment:\n      IMAGE_NAME: jitsucom/server\n      IMAGE_VERSION: latest\n      DOCKER_FILE: server.Dockerfile\n    docker:\n      - image: cimg/go:1.14.15\n    steps:\n      - checkout\n      - setup_remote_docker:\n          version: 19.03.13\n      - <<: *install_buildx\n      - <<: *build_jitsu\n\nworkflows:\n  version: 2.1\n  build-docker:\n    jobs:\n      - build-latest-jitsu-docker:\n          context: jitsu\n          filters:\n            branches:\n              only: master\n"
        })
      })]
    });
  }
}
return {
  default: MDXContent
};
function _missingMdxReference(id, component, place) {
  throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it." + (place ? "\nIt’s referenced in your code at `" + place + "`" : ""));
}


Learn more about building multi-architecture Docker Images for both x86 and Apple Silicon (M1) locally and via CircleCI

How to build Docker Images for Apple Silicon (aka M1 Chip)

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      a: "a",
      img: "img",
      h1: "h1",
      h2: "h2",
      pre: "pre",
      code: "code",
      h4: "h4",
      h3: "h3",
      ul: "ul",
      li: "li"
    }, props.components), {CodeInTabs, CodeTab} = _components;
    if (!CodeInTabs) _missingMdxReference("CodeInTabs", true, "207:1-214:14");
    if (!CodeTab) _missingMdxReference("CodeTab", true, "208:1-210:11");
    return _jsxs(_Fragment, {
      children: [_jsxs(_components.p, {
        children: ["Data has become an invaluable asset that helps companies understand users, predict behavior,\nand identify trends. Jitsu is an open-source project designed to simplify event data collection.\n", _jsx(_components.a, {
          href: "https://github.com/jitsucom/jitsu/",
          children: "Jitsu"
        }), " supports a few data warehouses as storage backends, and ClickHouse\nis one of them."]
      }), "\n", _jsx(_components.p, {
        children: "ClickHouse is the first open source SQL data warehouse to match the performance, maturity, and\nscalability of proprietary databases like Vertica, and Snowflake."
      }), "\n", _jsx(_components.p, {
        children: "This article shows how to set up Jitsu with ClickHouse and gives operational advice on\nhow to achieve the best performance and reliability"
      }), "\n", _jsx(_components.p, {
        children: "Getting data to ClickHouse is not as easy a task as it seems. Streaming millions of events from\ndifferent applications where each event has its own structure can be very challenging. Things can\nbecome much more complicated when different versions of the same application are running in\nproduction (such as the different versions of iOS app)."
      }), "\n", _jsx(_components.p, {
        children: _jsx(_components.img, {
          src: "/img/blog/jitsu-clickhouse/architecture.png",
          alt: "Jitsu Architecture"
        })
      }), "\n", _jsx(_components.p, {
        children: "Jitsu’s architecture is very efficient and robust. It consists of a lightweight HTTP server that accepts an incoming event-stream (JSON objects)\nand buffers it to local-disk. A separate thread takes care of processing the buffer, mapping JSON to ClickHouse tables, adjusting the schema, and\nstoring the data."
      }), "\n", _jsx(_components.h1, {
        children: "ClickHouse and Jitsu quick-start"
      }), "\n", _jsx(_components.p, {
        children: "In this section we’ll configure a single node installation of ClickHouse and Jitsu using official Docker images."
      }), "\n", _jsx(_components.p, {
        children: "Note that this is a dev setup to get things going. In production scenarios you would want to deploy multiple Jitsu nodes and to enable ClickHouse\nreplicas to ensure availability of data as well as scale throughput."
      }), "\n", _jsx(_components.h2, {
        children: "1. Pull latest Docker images"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "docker pull jitsucom/server:latest && docker pull yandex/clickhouse-server:latest\n"
        })
      }), "\n", _jsx(_components.h2, {
        children: "2. Start ClickHouse"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "mkdir ./clickhouse_data && docker run -f --name clickhouse-test -p 8123:8123 -v $PWD/clickhouse_data:/var/lib/clickhouse yandex/clickhouse-server\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Make sure that clickhouse is running with ", _jsx(_components.code, {
          children: "docker ps | grep clickhouse-test"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "3. Configure Jitsu"
      }), "\n", _jsxs(_components.p, {
        children: ["Put the following content to ", _jsx(_components.code, {
          children: "eventnative.yaml"
        }), " (EventNative is a legacy name of Jitsu project, some configuration file still has an old name)"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-yaml",
          children: "server:\n  auth:\n    - server_secret: \"ia7i92rqp3mh\" # access token. We will need it later for sending events through HTTP API\ndestinations:\n  clickhouse:\n    mode: stream\n    clickhouse:\n      dsns:\n        - \"http://default:@host.docker.internal:8123?read_timeout=5m&timeout=5m\"\n      db: default\n    data_layout:\n      mappings:\n        fields:\n          - src: /field_1/sub_field_1\n            action: remove\n          - src: /field_2/sub_field_1\n            dst: /field_10/sub_field_1\n            action: move\n          - src: /field_3/sub_field_1/sub_sub_field_1\n            dst: /field_20\n            action: move\n            type: DateTime\n          - dst: /constant_field\n            action: constant\n            value: 1000\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["Create a directory for logs: ", _jsx(_components.code, {
          children: "mkdir ./jitsu-data && chmod -R 777 ./jitsu-data"
        })]
      }), "\n", _jsx(_components.h2, {
        children: "4. Start Jitsu"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "docker run -d -t --name jitsu-test -p 8001:8001 \\\n-v $PWD/eventnative.yaml:/home/eventnative/data/config/eventnative.yaml \\\n-v $PWD/jitsu-data:/home/eventnative/data/ jitsucom/server:latest\n"
        })
      }), "\n", _jsxs(_components.p, {
        children: ["On Linux, add ", _jsx(_components.code, {
          children: "--add-host=host.docker.internal:host-gateway"
        }), ":"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "docker run --add-host=host.docker.internal:host-gateway -d -t --name jitsu-test -p 8001:8001 \\\n-v $PWD/eventnative.yaml:/home/eventnative/data/config/eventnative.yaml \\\n-v $PWD/jitsu-data:/home/eventnative/data/ jitsucom/server:latest\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "y"
      }), "\n", _jsx(_components.h2, {
        children: "5. Send test event and check that it landed in ClickHouse"
      }), "\n", _jsxs(_components.p, {
        children: ["Put the following JSON to ", _jsx(_components.code, {
          children: "./api.json"
        }), ":"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-json",
          children: "{\n  \"eventn_ctx\": {\n    \"event_id\": \"19b9907d-e814-42d8-a16d-c5da51e01531\"\n  },\n  \"field_1\": {\n    \"sub_field_1\": \"text1\",\n    \"sub_field_2\": 100\n  },\n  \"field_2\": \"text2\",\n  \"field_3\": {\n    \"sub_field_1\": {\n      \"sub_sub_field_1\": \"2020-09-25T12:38:27\"\n    }\n  }\n}\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "Run the following command:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "curl -X POST -H \"Content-Type: application/json\" -d @./api.json 'http://localhost:8001/api/v1/s2s/event?token=ia7i92rqp3mh'\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "And then check events landed into database"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "echo 'SELECT * FROM events;' | curl 'http://localhost:8123/' --data-binary @-\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "You’ll see one event in the database. The test worked!"
      }), "\n", _jsx(_components.h2, {
        children: "6. Test event buffering"
      }), "\n", _jsx(_components.p, {
        children: "One of the core features of Jitsu is event buffering. Events are written to an internal queue\nwith disk persistence. If a destination (ClickHouse in our case) is down, data won’t be lost! It will be kept locally\nuntil ClickHouse is up again."
      }), "\n", _jsx(_components.p, {
        children: "Let’s test this feature!"
      }), "\n", _jsxs(_components.p, {
        children: ["Put the following JSON to ", _jsx(_components.code, {
          children: "./api2.json"
        }), ":"]
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-json",
          children: "{\n  \"eventn_ctx\": {\n    \"event_id\": \"4748c7bb-50d4-43a7-91b4-21a5bcccb12e\"\n  },\n  \"field_1\": {\n    \"sub_field_1\": \"text1\",\n    \"sub_field_2\": 100\n  },\n  \"field_2\": \"text2\",\n  \"field_3\": {\n    \"sub_field_1\": {\n      \"sub_sub_field_1\": \"2020-09-25T12:38:27\"\n    }\n  }\n}\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "Now let’s test buffering."
      }), "\n", _jsx(_components.h4, {
        children: "1. Shutdown ClickHouse:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "docker stop clickhouse-test\n"
        })
      }), "\n", _jsx(_components.h4, {
        children: "2. Send an event:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "curl -X POST -H \"Content-Type: application/json\" -d @./api2.json 'http://localhost:8001/api/v1/s2s/event?token=ia7i92rqp3mh'\n"
        })
      }), "\n", _jsx(_components.h4, {
        children: "3. Verify that ClickHouse is down:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "echo 'SELECT * FROM events;' | curl 'http://localhost:8123/' --data-binary @-\n"
        })
      }), "\n", _jsx(_components.h4, {
        children: "4. Start ClickHouse again:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "docker start clickhouse-test\n"
        })
      }), "\n", _jsx(_components.h4, {
        children: "5. Wait for 60 seconds, then verify that event hasn’t been lost:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-bash",
          children: "echo 'SELECT * FROM events;' | curl 'http://localhost:8123/' --data-binary @-\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "If you see the event on the last step, the test succeeded."
      }), "\n", _jsx(_components.h2, {
        children: "Schema management with Jitsu and ClickHouse"
      }), "\n", _jsx(_components.p, {
        children: "Jitsu is designed to be a schema-less component in your stack. This means you don’t have to\ncreate table schemas and maintain them in advance. Jitsu takes care of it automatically! Each\nincoming JSON field will be mapped to a SQL field. If the field is missing, it will be automatically created with ClickHouse."
      }), "\n", _jsx(_components.p, {
        children: "It’s particularly useful when one engineering team is in charge of event structure, and another\nteam operates ClickHouse. As an example: a frontend developer may start sending very simple data\nto track product page views (product_id and price), and add more sophisticated fields later (currency, images).\nIt’s nice to have."
      }), "\n", _jsx(_components.p, {
        children: "Example:"
      }), "\n", _jsxs(CodeInTabs, {
        children: [_jsx(CodeTab, {
          lang: "json",
          title: "Source",
          children: ` "product_id": "1e48fb70-ef12-4ea9-ab10-fd0b910c49ce", "product_price": 399.99, "price_currency": "USD", "product_type": "supplies", "product_release_start": "2020-09-25T12:38:27", "images": { "main": "picture1", "sub": "picture2" }`
        }), _jsx(CodeTab, {
          lang: "sql",
          title: "Result",
          children: `"product_id" String, "product_price" Float64, "price_currency" String, "product_type" String, "product_release_start" String, "images_main" String, "images_sub" String`
        })]
      }), "\n", _jsx(_components.h3, {
        children: "Mapping configuration details"
      }), "\n", _jsx(_components.p, {
        children: "Jitsu can be configured to apply particular transformations to incoming JSON objects such as:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Remove fields"
        }), "\n", _jsx(_components.li, {
          children: "Rename fields (including moving element to another node)"
        }), "\n", _jsx(_components.li, {
          children: "Explicitly defining the SQL type of the node"
        }), "\n", _jsx(_components.li, {
          children: "Setting a constant"
        }), "\n"]
      }), "\n", _jsx(_components.p, {
        children: "Rules:"
      }), "\n", _jsx(_components.pre, {
        children: _jsx(_components.code, {
          className: "language-yaml",
          children: " - src: /field_1/sub_field_1\n    action: remove\n  - src: /field_2/sub_field_1\n    dst: /field_10/sub_field_1\n    action: move\n  - src: /field_3/sub_field_1/sub_sub_field_1\n    dst: /field_20\n    action: move\n    type: DateTime\n  - dst: /constant_field\n    action: constant\n    value: 1000\n"
        })
      }), "\n", _jsx(_components.p, {
        children: "See how those rules are applied:"
      }), "\n", _jsxs(CodeInTabs, {
        children: [_jsx(CodeTab, {
          lang: "json",
          title: "Source",
          children: `
{
 "eventn_ctx": {
   "event_id": "19b9907d-e814-42d8-a16d-c5da51e01530"
   // this field indicates a unique id
 },
 "field_1":  {
   "sub_field_1": "text1",
   "sub_field_2": 100
 },
 "field_2": "text2",
 "field_3": {
   "sub_field_1": {
     "sub_sub_field_1": "2020-09-25T12:38:27"
   }
 }
}
`
        }), _jsx(CodeTab, {
          lang: "json",
          title: "Mapped and flattened JSON",
          children: `
{
 "eventn_ctx_eventn_id": "19b9907d-e814-42d8-a16d-c5da51e01530",
 "field_1_sub_field_1":  "text1",
 "field_1_sub_field_2":  100,
 "field_2": "text2",
 "field_3_sub_field_1_sub_sub_field_1": "2020-09-25T12:38:27.763000Z",
 "constant_field": 1000
}
`
        })]
      }), "\n", _jsxs(_components.p, {
        children: ["See a ", _jsx(_components.a, {
          href: "/docs/configuration/schema-and-mappings",
          children: "full description of this feature"
        }), " in the documentation."]
      }), "\n", _jsx(_components.h2, {
        children: "Performance Tips"
      }), "\n", _jsxs(_components.p, {
        children: [_jsx(_components.a, {
          href: "https://clickhouse.tech/docs/en/engines/table-engines/mergetree-family/replacingmergetree/",
          children: "ReplacingMergeTree"
        }), " (or ReplicatedReplacingMergeTree) is the best choice for data\nproduced by Jitsu. Here’s why:"]
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsx(_components.li, {
          children: "Usually, data produced by EvenNative is used in aggregated queries, such as the number of events per\nperiod satisfying filtering conditions. MergeTree engine family shows great performance for aggregation queries."
        }), "\n", _jsx(_components.li, {
          children: "ReplacingMergeTree (unlike ordinary MergeTree) has a nice side-effect of data deduplication. Often, mistakes are\nfound in data after it has been loaded. Sometimes, a replay is required. Since Jitsu can optionally\nkeep a copy of data locally for a while, it’s possible to write a script to fix data and send it to Jitsu\nonce again. ReplacingMergeTree will avoid data duplication provided each event has a unique id and the id is used as a key."
        }), "\n"]
      }), "\n", _jsxs(_components.p, {
        children: ["If the destination table is missing, Jitsu will create the table with ReplacingMergeTree or\nReplicatedReplacingMergeTree if cluster size is greater than 1. However, it’s possible to configure the\nengine manually. Please, read more about table creation ", _jsx(_components.a, {
          href: "/docs/destinations-configuration/clickhouse-destination",
          children: "in the documentation"
        }), "."]
      }), "\n", _jsx(_components.h2, {
        children: "Learning More"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["Follow & star ", _jsx(_components.a, {
            href: "https://github.com/jitsucom/jitsu",
            children: "Jitsu on GitHub"
          })]
        }), "\n", _jsxs(_components.li, {
          children: ["Try a cloud version of ", _jsx(_components.a, {
            href: "https://cloud.jitsu.com",
            children: "Jitsu"
          }), ". It's free for up to 1 million events per month and supports ClickHouse as well"]
        }), "\n", _jsxs(_components.li, {
          children: ["Check out ", _jsx(_components.a, {
            href: "https://altinity.com/",
            children: "Altinity"
          }), " — ClickHouse cloud provider"]
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};
function _missingMdxReference(id, component, place) {
  throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it." + (place ? "\nIt’s referenced in your code at `" + place + "`" : ""));
}


Learn more how Jitsu and ClickHouse works together to solve event streaming problem at large scale

Jitsu ❤️ ClickHouse — and easiest way to automate data collection on-prem

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, {})
  })) : _createMdxContent();
  function _createMdxContent() {
    const _components = Object.assign({
      p: "p",
      a: "a",
      h1: "h1",
      ol: "ol",
      li: "li",
      ul: "ul",
      strong: "strong"
    }, props.components);
    return _jsxs(_Fragment, {
      children: [_jsxs(_components.p, {
        children: ["After participating in ", _jsx(_components.a, {
          href: "https://www.ycombinator.com/companies?batch=S20",
          children: "YCombinator’s S2020"
        }), " batch,\nwe’re delighted to announce Jitsu’s $2m seed funding round led by Costanoa Ventures with participation of\nSignalFire, Altair Capital and few other investors. Tony from Costanoa Ventures wrote a very detailed blog post explaining\n", _jsx(_components.a, {
          href: "https://www.costanoavc.com/a-unified-open-core-platform-for-data-integration-our-investment-in-jitsu/",
          children: "why they decided to invest"
        }), "."]
      }), "\n", _jsxs(_components.p, {
        children: ["We released an initial version of ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/eventnative",
          children: "EventNative on GitHub"
        }), " last August and since then we’ve quickly\ngained traction, attracting more than a 1,000 stars and a few dozen developers set up our software.\nIt was clear a lot of companies needed a tool like this, which is why we decided to raise venture\ncapital to help us move faster towards our goal of solving the problem of consolidating valuable business data\ninto a data warehouse."]
      }), "\n", _jsx(_components.h1, {
        children: "The problem"
      }), "\n", _jsx(_components.p, {
        children: "Data is the new oil of the digital economy. To extract value from data, it needs\nto be stored and be easily accessible. The modern data architecture hinges on the\ndatabase as the single source of truth. Given the proliferation of SaaS applications, it’s surprising that\ncapturing data from these sources is not an easy task. Our research has shown that even a small company has\nvaluable data pieces scattered across at least 4 independent systems, for example, advertising platforms,\npayment processors, CRMs, etc. And event based user behavioral data — what your users do on your app or\nwebsite — is a critical proprietary asset!"
      }), "\n", _jsx(_components.p, {
        children: "Imagine you run an e-commerce website. It’s likely built on Shopify (source #1). You run advertisements on Facebook, Google,\nand Twitter (source #2, 3 & 4). Payments go through Stripe (source number #4). Customers complain about late delivery\ntime through Zendesk (source #5)... We can keep counting, but you get the picture - valuable customer data is split\nacross at least 5 sources."
      }), "\n", _jsx(_components.p, {
        children: "Let’s say you want an answer to simple questions about your users. What is the average revenue per user? What is the CAC?\nHow many support tickets are created per day? What pages do users visit before making a purchase decision? All those\nquestions can be easily answered with SQL queries. However, how do you bring all the necessary data into a single\ndatabase in the first place?"
      }), "\n", _jsx(_components.h1, {
        children: "The solution"
      }), "\n", _jsx(_components.p, {
        children: "There are several SaaS solutions helping businesses consolidate data in one place. However, they lack a few key features:\ntransparency, flexibility, and control."
      }), "\n", _jsxs(_components.ol, {
        children: ["\n", _jsx(_components.li, {
          children: "Businesses want to own their user data flow from start to finish"
        }), "\n", _jsx(_components.li, {
          children: "They don’t want to be locked into one cloud data warehouse provider."
        }), "\n", _jsx(_components.li, {
          children: "New regulations, such as GDPR and CCPA, show a trend towards internal user data management. Furthermore, technology\nsuch as AdBlock and 3rd party cookie restrictions make a shift to self-hosted deployment inevitable."
        }), "\n"]
      }), "\n", _jsxs(_components.p, {
        children: ["We built Jitsu with all these concerns in minds. Jitsu is built on an open-core called ", _jsx(_components.a, {
          href: "https://github.com/jitsucom/eventnative",
          children: "EventNative"
        }), "\nand you can host it by yourself. We also have a ", _jsx(_components.a, {
          href: "https://cloud.jitsu.com/",
          children: "hosted version"
        }), " that eliminates the necessity\nof deployment and maintenance process."]
      }), "\n", _jsxs(_components.p, {
        children: ["While developing EventNative we focused on simplicity and ease of use. It can be deployed with just a few clicks\nusing ", _jsx(_components.a, {
          href: "https://docs.eventnative.org/deployment/deploy-with-docker",
          children: "Docker"
        }), " (or ", _jsx(_components.a, {
          href: "https://docs.eventnative.org/deployment/deploy-with-heroku",
          children: "Heroku"
        }), ").\nYou can also be confident that your data is never shared with anyone and that you have a full control over it."]
      }), "\n", _jsx(_components.p, {
        children: "Jitsu and EventNative support both cloud (BigQuery, Redshift, Snowflake) and self-hosted warehouses (Clickhouse, PostgreSQL)"
      }), "\n", _jsx(_components.h1, {
        children: "Open-source"
      }), "\n", _jsx(_components.p, {
        children: "20 years ago SaaS wasn’t a thing. In the past, companies would buy on-prem enterprise software that was highly customized and required significant maintenance. Compared to traditional enterprise software, a website where you can just sign up and start using the product immediately was very refreshing and made for a much better user experience. SaaS was born because of that pain — a complexity of server software maintenance."
      }), "\n", _jsx(_components.p, {
        children: "A lot of things have changed since then: servers are commoditized (thanks to AWS, GCP and other cloud providers), Docker and surrounding tech revolutionized software packaging. Now setting up a server feels almost like installing a smartphone app."
      }), "\n", _jsx(_components.p, {
        children: "Our bet is that open source software will supplant SaaS. Maintaining server software shouldn’t bea job that only sophisticated organizations orlarge corporations can do. More and more teams prefer self-hosted alternatives over traditional SaaS (who could imagine 10 years ago that SaaS will be called “traditional”?). It feels refreshing because it gives you controland, by running your own instance of software, it guards users from privacy concerns."
      }), "\n", _jsx(_components.p, {
        children: "Ok, why open-source then? Why not just sell a pre-build closed-source packages?"
      }), "\n", _jsx(_components.p, {
        children: "The answer is trust and transparency. We want our users to be a part of our product journey. We want them to work with us to improve performance, design features and fix bugs. Open-source makes the process of running software a two-way street.‍"
      }), "\n", _jsx(_components.h1, {
        children: "Making money"
      }), "\n", _jsx(_components.p, {
        children: "How can open source software make money if it's free?Believe it or not, there are many ways to monetize! Let’s take a look at the most popular ones:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "Consulting"
          }), ". Help your users to configure and set up your product and charge for this work"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "Hosting (aka SaaS-version)"
          }), ". Charge for running and maintaining your software"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.strong, {
            children: "Open-Core"
          }), ". Charge a license fee for premium features which are not a part of open source.\nWe’ve chosen to monetize with the Hosting & Open-Core models. We don’t have any premium features yet, and\nopen-core monetization is a longer-term plan. However, you can use the ", _jsx(_components.a, {
            href: "https://cloud.jitsu.com/?utm_source=blog_seed",
            children: "hosted version of Jitsu"
          }), "\nright now if you don’t want to spend time on deployment or maintenance."]
        }), "\n"]
      }), "\n", _jsx(_components.h1, {
        children: "Join us"
      }), "\n", _jsx(_components.p, {
        children: "As an open-source project, we welcome contributors and developers! We can’t wait to see your patches and feature request."
      }), "\n", _jsx(_components.p, {
        children: "If you want to dig deeper:"
      }), "\n", _jsxs(_components.ul, {
        children: ["\n", _jsxs(_components.li, {
          children: ["Register at ", _jsx(_components.a, {
            href: "https://cloud.jitsu.com",
            children: "jitsu.com"
          }), " to try the product in 2 seconds!"]
        }), "\n", _jsxs(_components.li, {
          children: ["Or ", _jsx(_components.a, {
            href: "https://docs.eventnative.org/deployment?utm_source=blog_seed",
            children: "set up your own version of EventNative"
          }), " (our open source core)"]
        }), "\n", _jsxs(_components.li, {
          children: ["Check out our ", _jsx(_components.a, {
            href: "https://github.com/jitsucom/jitsu",
            children: "GitHub repo"
          }), " (and give us a star if you like what we do)"]
        }), "\n", _jsxs(_components.li, {
          children: [_jsx(_components.a, {
            href: "https://jitsu.com/slack",
            children: "Join our Slack"
          }), "!"]
        }), "\n", _jsxs(_components.li, {
          children: ["Drop us a line at ", _jsx(_components.a, {
            href: "mailto:hello@jitsu.com",
            children: "hello@jitsu.com"
          })]
        }), "\n"]
      })]
    });
  }
}
return {
  default: MDXContent
};


Jitsu announces $2m seed round led by Costanoa Ventures

Jitsu Blog

Ildar Nurislamov

More stories

v1.42: Source Connectors Plugin TypeScript SDK, New JS Transformations, and more

Ildar Nurislamov

Jitsu 1.41 is here! Multiple Projects and Users, SSO Authentication, Destination Tags, and more

Ildar Nurislamov

Jitsu 1.40 is here! Better User Recognition, Lower Memory usage and more

Ildar Nurislamov

Announcing Jitsu Developers Platform and Jitsu v1.39

Ildar Nurislamov

Jitsu 1.38

Vladimir Klimontovich

Supabase Auth with Next.js 12 middleware

Vladimir Klimontovich

Jitsu 1.37 is out!

Ildar Nurislamov

JavaScript is fully in charge of Data Mappings.

Ildar Nurislamov

140+ Source Connectors with Jitsu and Airbyte

Sergey Buryking

Jitsu 1.36 is out!

Vladimir Klimontovich

Supabase analytics with Jitsu

Vladimir Klimontovich

Jitsu and dbt integration

Ildar Nurislamov

Cookies and privacy compliant web analytics

Sergey Buryking

Javascript Functions (technical preview)

Ildar Nurislamov

No-code Jitsu: Flexible Slack Notifications

Sergey Buryking

Product Hunt launch Statistics

Vladimir Klimontovich

How to build Docker Images for Apple Silicon (aka M1 Chip)

Sergey Buryking

Jitsu ❤️ ClickHouse — and easiest way to automate data collection on-prem

Vladimir Klimontovich

Jitsu raises $2m seed to build open-source data integration platform

Vladimir Klimontovich

Product

Resources

Company