Snowflake table

Overview

t provides a single platform for data warehousing, data lakes, data engineering, data science, data application development, and secure sharing and consumption of real-time/shared data.

Snowflake enables data storage, processing, and analytic solutions.

Before you set up your data sync destination, make sure to configure your Source.

The Snowflake Table destination supports batch and real-time syncs.

Loading data into Snowflake

For batch syncs of 10 records or less, single Insert/Update/Delete statements are executed to perform operations against the target Snowflake table.

For batch syncs exceeding 10 records, the operations are performed in bulk.

The bulk operation process consists of:

  1. Generating a CSV containing a batch of records

  2. Creating a temporary table in Snowflake

  3. Copying the generated CSV into the temp table

  4. If needed, performing Insert operations against the target Snowflake table using the temp table

  5. If needed, performing Update operations against the target Snowflake table using the temp table

  6. If needed, performing Delete operations against the target Snowflake table using the temp table

  7. Dropping the temporary table

Real time sync volume size is based on a dynamic batch size up to configurable threshold.

Considerations

  • The temporary table generated in the bulk flow process for high volume scenarios transforms all columns of data type Number to be of type NUMBER(38, 18). This may cause precision loss if the number scale in the target table is higher

Destination tab

The following table outlines the mandatory and optional parameters you will find on the Destination tab (Image 1).

The following parameters will help to define your data sync destination and how it functions.

Next steps

Last updated