If you’re feeling overwhelmed or you’re unsure of what to look for, don’t worry. For a primer on data warehouses and setting the data strategy for your organization, check out our Data Strategy Guide.
When Stitch replicates your data, we’ll load it into the destination - or data warehouse - of your choosing. A data warehouse is a central repository of integrated data from disparate sources.
As Stitch currently only allows you to connect one destination to your account, we recommend asking yourself the questions below before making your selection. By fully assessing each choice first, you’ll decrease the likelihood of needing to switch destinations or re-replicate all of your data at a later date.
- Does it support all (or most of) your data sources?
- Will the structure of the data replicated by Stitch work with how you plan to use it?
- Do you need a fully-managed solution, or can you perform maintenance tasks on your own?
- Is the destination’s pricing structure suitable?
Side-by-side comparison
In the tabs below is a quick look at each of Stitch’s destinations and how they compare to each other.
The remaining sections of this guide expand on the information in these tabs.
Release Status | Fully Managed? | |
Amazon S3 | Released | No |
BigQuery | Released | Yes |
data.world | Released | Yes |
Azure SQL Data Warehouse | Released | Yes |
Panoply | Released | Yes |
PostgreSQL | Released | Depends |
Redshift | Released | Yes |
Snowflake | Released | Yes |
For more details, see the Pricing section.
Pricing Model | Free Option? | |
Amazon S3 | Storage | Yes (trial) |
BigQuery | Usage | No |
data.world | Varies | Yes |
Azure SQL Data Warehouse | Usage | Yes (plan & trial) |
Panoply | Monthly | Yes (trial) |
PostgreSQL | Varies | Depends |
Redshift | Hourly | Yes (plan & trial) |
Snowflake | Usage | No |
Supported Versions | Connection Methods | Incompatible Sources | |
Amazon S3 | n/a | n/a | Yes |
BigQuery | n/a | SSL | Yes |
data.world | n/a | Yes | |
Azure SQL Data Warehouse | n/a | SSL, SSH | Yes |
Panoply | n/a | SSL | Yes |
PostgreSQL | 9.3 or higher | SSL, SSH | Yes |
Redshift | n/a | SSL, SSH | Yes |
Snowflake | n/a | SSL | Yes |
Table Name Length |
Column Name Length |
Max Columns Per Table |
Case Type |
Reserved Words |
|
Amazon S3 | 1,024 bytes | n/a | n/a | n/a | None |
BigQuery | 1,024 | 128 | 10,000 | Insensitive | Full List |
data.world | n/a | n/a | n/a | n/a | None |
Azure SQL Data Warehouse | 112 | 128 | 1,024 | Insensitive | Full List |
Panoply | 127 | 115 | 1,600 | Insensitive | Full List |
PostgreSQL | 63 | 59 | 250-1,600 | Sensitive | Full List |
Redshift | 127 | 115 | 1,600 | Insensitive | Full List |
Snowflake | 255 | 251 | None | Insensitive | Full List |
For more details, see the Data Structure section.
Upsert support | Nested Structures | Max VARCHAR Width | |
Amazon S3 | Unsupported | Supported | n/a |
BigQuery | Unsupported | Supported | None |
data.world | Unsupported | Supported | n/a |
Azure SQL Data Warehouse | Supported | Unsupported | 4,000 bytes |
Panoply | Supported | Unsupported | 65K |
PostgreSQL | Supported | Unsupported | None |
Redshift | Supported | Unsupported | 65K |
Snowflake | Supported | Supported | 16 MB |
Getting started, now
If you simply want to try Stitch and Redshift, or if you don’t have the ability to spin up a Redshift cluster of your own in AWS, we recommend trying Panoply. With just a few clicks, you create your own fully-managed Redshift data warehouse and start replicating data in minutes.
Note: If you decide to switch destinations later, you’ll need to queue a full re-replication of your data to ensure historical data is present in the new destination.
Are your data sources compatible with your destination?
Some integrations may be partially or fully incompatible with some of the destinations offered by Stitch. For example: Some destinations don’t support storing multiple data types in the same column. If a SaaS integration sends over a column with mixed data types, some destinations may “reject” the data.
For integrations that allow you to control how data is structured, you may be able to fix the problem at the source and successfully replicate the data. If this is not possible, however, Stitch may never be able to successfully replicate the incompatible data.
Check Integration & Destination Compatibility
Will the structure of the data suit your needs?
While the majority of your data will look the same across our destinations, there are some key differences you should be aware of.
Nested data structures
Some destinations don’t natively support nested structures, meaning that before Stitch can load replicated data, these structures must be “de-nested”. During this process, Stitch will flatten nested structures into relational tables and subtables. As a result of creating subtables, a higher number of rows will be used.
If a destination does natively support nested structures, no de-nesting will occur and Stitch will store the records intact.
Check out the Handling of Nested Data & Row Count Impact for an in-depth look at what we mean by nested records, how Stitch handles nested data, and what those records will look like in your data warehouse.
Supports Nested Structures | No Nested Structure Support |
|
|
Updates to existing records
While all destinations support loading incrementally replicated data, how that data is stored in your destination will vary by destination.
Unlike other destinations, BigQuery and Amazon S3 store data in an Append-Only manner. This means that existing rows are never updated in the destination, but appended to the end of the table. In the case of Amazon S3, during each load a new file (CSV or JSON) will be created and added to the bucket.
This means that there can be many different rows in a table with the same Primary Key, each representing what the data was at that moment in time. These are not duplicate rows - they’re “snapshots” of the record at different points.
For more info, check out this detailed explanation on Append-Only Replication or our recommendations for querying append-only tables.
Redshift vs. PostgreSQL
If you’ve worked with PostgreSQL in the past and are considering Redshift as your data warehouse, you should note that Redshift implements some Postgres features differently. In addition, some features, data types, and functions aren’t supported at all.
What type of maintenance do you need?
With the exception of a self-hosted PostgreSQL instance, all the destinations offered by Stitch are cloud-hosted databases, meaning you won’t have to factor in server maintenance when making a decision.
Fully-managed solutions
Amazon Aurora PostgreSQL RDS, Azure SQL Data Warehouse, BigQuery, Heroku, Panoply, and Snowflake are fully-managed solutions that include routine maintenance and upgrades in their plans.
DIY solutions
Redshift, Amazon PostgreSQL-RDS, and self-hosted Postgres instances will require you to perform and schedule maintenance tasks on your own. Spinning up a Redshift and Amazon PostgreSQL-RDS instance will require technical knowledge and familiarity with the Amazon Web Services (AWS) console.
What's the destination's pricing structure?
Every destination offered by Stitch has its own pricing structure. Some providers charge by overall usage, others by an hourly rate or the amount of stored data. Depending on your needs, some pricing structures may fit better into your budget.
In the next section, you’ll find each destination’s pricing structure, including a link to detailed price info and whether a free option (trial or plan) is available. Here are a few things to keep in mind:
Amazon S3 |
Amazon S3 pricing is based on two factors: The amount of data stored in and location (region) of your Amazon S3 bucket. |
BigQuery |
BigQuery ‘s pricing isn’t based on a fixed rate, meaning your bill can vary over time. To learn more about how Stitch may impact your BigQuery costs, click here. |
data.world |
Data.world plans vary depending on the number of private projects/data sets, size limits per project/dataset, external integrations, and total number of team members that can belong to an account. All plans, however, include unlimited public projects/datasets, API access, joins, queries, activity alerts, and other standard features. |
Azure SQL Data Warehouse |
Azure SQL Data Warehouse bases their pricing on your compute and storage usage. Compute usage is charged using an hourly rate, meaning you’ll only be billed for the hours your data warehouse is active. Compute usage is billed in one hour increments. Storage charges include the size of your primary database and seven days of incremental snapshots. Microsoft Azure rounds charges to the nearest terabyte (TB). For example: If the data warehouse is 1.5 TB and you have 100 GB of snapshots, your bill will be for 2 TB of data. Refer to Microsoft’s documentation for more info and examples. |
Panoply |
Panoply charges based on the amount of data stored and offers several plan options for your needs. Refer to their pricing page for more information. |
PostgreSQL |
The software for hosting your own PostgreSQL instance is open-source, meaning it’s free. Heroku and Amazon RDS have a variety of plans to choose from. |
Redshift |
Currently, Redshift bases their pricing on an hourly rate that varies depending on the type and number of nodes in a cluster. The type and number of nodes you choose when creating a cluster is dependent on your needs and data set, but you can scale up or down over time should your requirements change. |
Snowflake |
Snowflake pricing is based on two factors: the volume or data stored in your Snowflake destination and the amount of compute usage (the time the server runs) in seconds. Snowflake offers two types of plans, each with varying levels of access and features. There are On Demand plans which are commitment-free and usage-based. The alternative is a Capacity option, which guarantees secure price discounts. Learn more about Snowflake plans and pricing here. |
Additional resources and setup tutorials
Ready to pick a destination and get started? Use the links below to check out a more in-depth look at each destination or move on to the setup and connection process.
-
Amazon S3
Overview | Setup
-
BigQuery
Overview | Setup
Data Loading -
data.world
Overview | Setup
-
Azure SQL Data Warehouse
Overview | Setup
-
Panoply
Overview | Setup
Data Loading -
PostgreSQL
Overview | Setup
Data Loading -
Redshift
Overview | Setup
Data Loading -
Snowflake
Overview | Setup
Data Loading
Related | Troubleshooting |
Questions? Feedback?
Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.