v1.42: Source Connectors Plugin TypeScript SDK, New JS Transformations, and more

SDK for development of Source Connectors#

Jitsu Extension SDK now support sources. Now possible for anyone to implement new source connectors (in addition to 100+ connectors we support now). New version of Jitsu Servers fully supports loading of such plugins.

We've chosen TypeScript as a primary language because of several reasons:

• Most of the typical connectors source code is gets data from HTTP and manipulates JSON objects. Typescript is a greate language for that
• Tons of libraries are already available for TypeScript. If you decided to implement a connector to particular services, chances that there is good library available is pretty high
• Jitsu Transformations is already based on JavaScript, so it only makes sense to keep the tech stack consistent

A completely new Source Connector can be developed in a few hours

#Creates Source plugin project boilerplat
npx jitsu-cli extension create --type source
#Implement your plugin logic ...
#Build and publish your plugin to npm repository
npm build
npm publish

Read more about source extension documentation →

Jitsu now supports BigQuery as destinations#

Before 1.41 Jitsu supported Google Big Query as a destination only for event streaming on batches. Now it's possible to synchronize data from Sources to BQ to (in addition to RedShift, Postgres, ClickHouse, MySQLm and Snowflake).

New JavaScript Transformations UX#

JavaScript Transformation has been always a core feature of Jitsu. In this release we significantly redesigned it and added a few useful features

Access HTTP headers in transformations#

Not it's now possible to access HTTP headers of request, if the event has come from HTTP API. This is useful for filtering requests, for example we authorize users, or pass-through authorization to a downstream destination:

return {
        //Get auth value from HTTP header and pass it to destination in HTTP body
        authToken: $context.header("X-Auth"),
        ...$
    }

Read more about this feature in Jitsu documentation →

Improved embedded JavaScript editor#

Now you can much easier debug your destination event transformation code.

Jitsu allows writing and debug JavaScript Transformations right in the browser. At 1.42 we completely redesign the layout and implemented console.log output display, so you could use it for debugging

Revamped Statistics page#

Statistics page shows how many records have flown through Jitsu. Now it's possible to break it down: by API Key, Destination or Source. Furthermore, you can select date range to see.

That makes it much easier to understand how each connection performs.

Redesigned geo-location configuration page#

Jitsu supports Max Mind GEO IP databases for enriching incoming events with geo-location data. You can use the full-featured GeoIP2 database or GeoLite2 edition, which has a free license option. We have improved the Geo Data resolver page design that makes it clear what features are covered with your license key.

New connectors#

• Airtable Source – the first one implemented with Jitsu Developers Platform in typescript language.
• Google Cloud Storage Destination – works similarly to S3 destination – stores events in batch files.

Fixed JavaScript transformations memory leaks#

In previous versions, in certain conditions JS transformation might have been causing memory leaks. It's not a case anymore — we completely redesigned execution model. Now all JS are being executed in a pool of separate child processes. Thanks to vm2 for the implementation of NodeJS Sandbox!