This is a Jitsu.Classic documentation. For the lastest version, please visit docs.jitsu.com. Read about differences here.

πŸ“œ Configuration

Configuration UI

πŸ‘©β€πŸ”¬ Extending Jitsu

Overview
Destination Extensions
Source Extensions
API Specs

Jitsu Internals

Jitsu Server

S3

Overview

The S3 source enables syncing of file-based tables with support for multiple files using glob-like pattern matching, and both Full Refresh and Incremental syncs, using the last_modified property of files to determine incremental batches. You can choose if this connector will read only the new/updated files, or all the matching files, every time a sync is run.

The source is using Airbyte docker image (@airbyte/source-s3). Learn more how Airbyte-based sources work

How to connect

Please read setup guide for more information.

Connection Parameters

ParameterDocumentation
dataset*
string
(required)
This source creates one table per connection, this field is the name of that table. This should include only letters, numbers, dash and underscores. Note that this may be altered according to destination.
path_pattern*
string
(required)
Add at least 1 pattern here to match filepaths against. Use | to separate multiple patterns. Airbyte uses these patterns to determine which files to pick up from the provider storage. See wcmatch.glob to understand pattern syntax (GLOBSTAR and SPLIT flags are enabled). Use pattern ** to pick up all files.
schema
string
(not required)
Optionally provide a schema to enforce, as a valid JSON string. Ensure this is a mapping of { "column" : "type" }, where types are valid JSON Schema datatypes. Leave as {} to auto-infer the schema.
format
object
(not required)
provider*
object
(required)