Need to change the location in which BigQuery stores your data? Using Stitch’s Destination Change feature, you can delete the current destination and connect a new one with the desired data storage location.

Considerations

Here’s what you need to know to ensure a smooth switch:

  • Your integrations will be paused. After the switch is complete, you’ll need to manually unpause the integrations you’d like to resume.

  • Some webhook data may be lost during this process. Due to their continuous, real-time nature, some webhook data may be lost during the switch.

  • Historical data from webhook-based integrations must be either manually backfilled or replayed. Some webhook providers - such as Segment - allow customers on certain plans to ‘replay’ their historical data. This feature varies from provider to provider and may not always be available.

Step 1: Select a historical data setting

If you need historical data in the new destination, you’ll need to either manually transfer your historical data or queue a historical replication job.

  • Manually transfer historical data: To accomplish this, you can use Google’s UI to import the existing datasets into the new BigQuery instance. Ensure that all dataset names remain the same during the transfer, or loading errors may occur.

    Complete the historical transfer before continuing.

  • Queue a historical replication job: Initializing a historical replication will re-replicate all historical data from your integrations. For SaaS integrations, Stitch will replicate data beginning with the Start Date currently in the integration’s settings.

Step 1.1: Define the setting

  1. From the Stitch Dashboard, click the Destination tab.
  2. At the bottom of the page, click the Change Destination button.
  3. In the Historical Data section, select how you want data to be replicated to the new destination:

Step 2: Delete the current destination

  1. Click Continue. A message will display asking you to confirm the removal of the current destination’s settings.

  2. To complete the switch, Stitch must delete your current destination configuration. Note: This will not delete data in the destination itself - it only clears this destination’s settings from Stitch.

    To continue with the switch, click OK to delete the current destination settings.

Step 3: Connect the new BigQuery destination

  1. On the next page in Stitch, click the BigQuery icon.
  2. Click Sign in with Google.
  3. If you aren’t already signed into your Google account, you’ll be prompted for your credentials.
  4. After you sign in, you’ll see a list of the permissions requested by Stitch:
    • Read/Write Access to Google Cloud Storage - Stitch requires Read/Write access to create and use a GCS bucket to load replicated data into BigQuery.
    • Full Access to BigQuery - Stitch requires full access to be able to create datasets and load data into BigQuery.
    • Read-Only Access to Projects - Stitch requires read-only access to projects to allow you to select a project to use during the BigQuery setup process.
    • Basic Profile Information - Stitch uses your basic profile info to retrieve your user ID.
    • Offline Access - To continuously load data, Stitch requires offline access. This allows the authorization token generated during setup process to be used for more than an hour after the initial authentication takes place.
  5. To grant access, click the Authorize button.
  6. After you sign into Google and grant Stitch access, you’ll be redirected back to Stitch. Fill in the fields that display:
    • Google Cloud Project: From the dropdown, select the project you want to connect to Stitch.
    • Google Cloud Storage Location: From the dropdown, select the new location you want to use.
  7. Click Finish Setup.

Step 4: Delete the existing Google Cloud Storage bucket

To ensure Stitch can load data into the new BigQuery instance, you’ll need to delete the Google Cloud Storage (GCS) bucket that’s attached to the project.

  1. In another browser tab, log into the Google console.
  2. Use Google’s instructions to locate the bucket in Google’s UI.
  3. Locate the Stitch bucket. If you’re unsure of which bucket belongs to Stitch, reach out to Stitch support.
  4. Delete the bucket.

Step 5: Unpause integrations

After you’ve successfully connected the new BigQuery destination and deleted the original Stitch GCS bucket, un-pause your integrations in Stitch.

Your data will begin replicating according to the historical data option selected in Step 1.

Related Troubleshooting
Questions? Feedback?

Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.