1 of 40

Data sync sources

Cinchy Event Broker

Cinchy Table

Cinchy Query

Binary file

A binary file is a computer file that's not a text file, and whose content is in a binary format consisting of a series of sequential bytes, each of which is eight bits in length.

You can use binary files from a Local upload, Amazon S3, or Azure Blob Storage in your data syncs.

Some benefits of using binary files include:

Better efficiency via compression
Better Security through the ability to create custom encoding standards.
Unmatched Speed, since the data is stored in a raw format, and isn't encoded using any character encoding standards, it's faster to read and store.

Copper

Delimited file

Dynamics 365

Dynamics 2015

DynamoDB

Excel

Fixed-width file

A fixed-width file is a file that has a specific format which allows for the saving of information in an organized fashion. The data is arranged in rows and columns, with one entry per row. Each column has a fixed-width, specified in characters, which determines the maximum amount of data it can contain. No delimiters are used to separate the fields in the file.

Advantages of using a fixed-width file include:

It's a compact representation of your data
It's fast to parse because every field is in the same place in every line

Kafka Topic

Publishes (writes) and subscribes to (reads) streams of events from sources like databases, cloud services, and software applications.
Stores these events durably and reliably for as long as you want.
Processes and reacts to the event streams in real-time and retrospectively.

Those events are organized and durably stored in topics. These topics are then partitioned over a number of buckets located on different Kafka brokers.

LDAP

Common uses of LDAP include when:

A single piece of data needs to be found and accessed regularly;
Your organization has a lot of smaller data entries;
Your organization wants all smaller pieces of data in one centralized location, and there doesn't need to be an extreme amount of organization between the data.

MongoDB collection

MongoDB collection (CDC triggered)

MS SQL Server (query and table)

ODBC query

ODBC is the database portion of the Microsoft Windows Open Services Architecture (WOSA), which is an interface that allows Windows-based desktop applications to connect to multiple computing environments without rewriting the application for each platform.

Oracle (query and table)

Parquet

Polling event

REST API

A REST API is an application programming interface that conforms to the constraints of REST (representational state transfer) architectural style and allows for interaction with RESTful web services.

REST APIs work by fielding requests for a resource and returning all relevant information about the resource, translated into a format that clients can easily interpret (this format is determined by the API receiving requests). Clients can also modify items on the server and even add new items to the server through a REST API.

Salesforce object (bulk API)

Salesforce objects are database tables that permit you to store data that's specific to an organization. Salesforce objects are of two types:

Standard Objects: Standard objects are the kind of objects that are provided by salesforce.com such as users, contracts, reports, dashboards, etc.
Custom Objects: Custom objects are those objects that are created by users. They supply information that's unique and essential to their organization. Custom objects are the heart of any application and provide a structure for sharing data.

Salesforce platform event

Salesforce Platform Events are secure and scalable messages that contain data. Publishers push out event messages that subscribers receive in real time.

Salesforce push topic

Push Topic events provide a secure and scalable way to receive notifications for changes to Salesforce data that match a SOQL query you define.

You can use PushTopic events to:

Receive notifications of Salesforce record changes, including create, update, delete, and undelete operations.
Capture changes for the fields and records that match a SOQL query.
Receive change notifications for only the records a user has access to based on sharing rules.
Limit the stream of events to only those events that match a subscription filter.

Snowflake

Snowflake enables data storage, processing, and analytic solutions.

SOAP 1.2 web service

SOAP (Simple Object Access Protocol) is an XML-based protocol for accessing web services over HTTP.

SOAP can communicate between different operating systems using different technologies and programming languages. You can use SOAP APIs to create, retrieve, update or delete records, such as passwords, accounts, leads, and custom objects, from a server.

SAP SuccessFactors

Cinchy Event Broker/CDC

Overview

The Cinchy Event Broker/CDC (Change Data Capture) source allows you to capture data changes on your table and use these events in your data syncs.

Use Case

To mitigate the labour and time costs of hosting information in a silo and remove the costly integration tax plaguing your IT teams, you want to connect your legacy systems into Cinchy to take advantage of the platform's sync capabilities.

To do this, you can set up a real-time sync between a Cinchy Table and Salesforce that updates Salesforce any time data is added, updated, or deleted on the Cinchy side. If you enable change notifications on your Cinchy table, you can set up a data sync and listener config with your source as the Cinchy Event Broker/CDC.

The Cinchy Event Broker/CDC supports both batch syncs and real-time syncs (most common).

Remember to set up your listener config if you are creating a real-time sync.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

CDC

Variables

Optional. Review our documentation on Variables here for more information about this field.

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

Cinchy Event Broker/CDC

Run Query

Path to Iterate

Optional. For the Cinchy Event Broker/CDC, the Path to Iterate function can be used to provide the JSON path to the array of items that you want to sync (provided that your event message contains JSON values).

To set up a real-time sync, you must configure your Listener values. You can do so through the Connections UI.

If there is more than one listener associated with your data sync, you will need to configure the addition listeners via the Listener Configuration table.

If you are creating a CDC listener config for a Cinchy Event Triggered REST API data source, pay in mind the following unique constraints:

Column names in the listener config shouldn't contain spaces. If they do, they will be automatically removed. For example, a column named First Name will become @FirstName.
The replacement variable names are case sensitive.
Column names in the listener config shouldn't be prefixes of other column names. For example, if you have a column called Name, you shouldn't have another called "Name2" as the value of @Name2 may end up being replaced by the value of @Name suffixed with a 2.

Reset behaviour

Parameter

Description

Example

Auto Offset Reset

Earliest, Latest or None. In the case where the listener is started and either there is no last message ID, or when the last message ID is invalid (due to it being deleted or it's just a new listener), it will use this column as a fallback to determine where to start reading events from.

None

Topic JSON

A Topic JSON is necessary for all real-time syncs. Enter your JSON parameters through the Connections UI, or edit them directly through the Listener Configuration table..

Topic JSON parameters

Parameter

Description

Example Value

tableGuid

Mandatory. GUID of the table you are reading from.

ef6710ca-6e59-4b4a-86d3-f6d24ed7658b

fields

Array of objects specifying columns to fetch.

See fields section below.

filter

WHERE clause for filtering records.

New.[Is Valid] = 1 AND (New.[Is Excluded] = 0 OR New.[Is Excluded] IS NULL)

messageKeyExpression

The messageKeyExpression parameter specifies a key that the listener application uses to route messages into specific topics within a Kafka broker. See below for more information.

value

next_cursor

The next_cursor parameter serves as an offset marker for paginated data retrieval in API requests. It helps in fetching large data sets chunk by chunk, making the process more manageable and efficient.

ABCS

batchSize

Number of records read per request.

1000

Fields Section

The following expands on the available parameters within the fields section above.

Field Property

Description

Example Value

column

Column name to fetch from the table.

Cinchy Id

alias

Alias for the column.

CinchyId

deserializeJsonValue

Converts text to JSON on read out.

true

Topic JSON example

The following is a topic JSON example:

{
  "tableGuid": "420c1851-31ed-4ada-a71b-31659bca6f92",
  "fields": [
    {
      "column": "Cinchy Id",
      "alias": "CinchyId"
    },
    {
      "column": "First Name",
      "alias": "Firstname"
    },
    {
      "column": "Last Name",
      "alias": "Lastname"
    }
  ],
  "messageKeyExpression": "CONCAT(New.[CinchyId], '-', New.[Name])",
  "filter": "New.[Lastname] IS not null OR (New.[CinchyId] is not null)",
  "batchSize": 100
}

Listener Configuration Parameters

messageKeyExpression

Each of your Event Listener message keys a message key. By default, this key is dictated by the Cinchy ID of the record being changed.

When the worker processes your Event Listener messages, it does so in batches, and for efficiency and to guarantee order, messages that contain the same key won't be processed in the same batch.

The messageKeyExpression property allows you to change the default message key to something else.

Use Case

Ensuring records with the same message key can be updated with the proper ordering to reflect an accurate collaboration log history.

Example Syntax

In this example, we want the message key to be based on the [Employee Id] and [Name] column of the table that CDC is enabled on.

{ "messageKeyExpression": "CONCAT(New.[Employee Id], '-', New.[Name])", … }

Old vs New Filter

The Cinchy Event Broker/CDC Stream Source has the unique capability to use "Old" and "New" parameters when filtering data. This filter can be a powerful tool for ensuring that you sync only the specific data that you want.

The "New" and "Old" parameters are based on updates to single records, not columns/rows.

"New" Example:

In the below filter, we only want to sync data where the [Approval State] of a record is newly Approved. For example, if a record was changed from Draft to Approved, the filter would sync the record.

Due to internal logic, newly created records will be tagged as both **New** and **Old**.

"filter": "New.[Approval State] = 'Approved'

"Old" Example:

In the below filter, we only want to sync data where the [Status] of a record was In Progress but has since been updated to any other [Status]. For example, if a record was changed from In Progress to Done, the filter would sync the record.

Due to internal logic, newly created records will be tagged as both **New** and **Old**.

"filter": "Old.[Status] = `In Progress`

Connection Attributes

You don`t need to provide Connections Attributes when using the Cinchy CDC Stream Source.

If you're inputting your configuration via the Listener Config table, you must insert the below text into the column:

{}

Schema section

The Schema section is where you define which source columns you want to sync in your connection. You have the option to add the following columns:

Standard
Calculated
Conditional
JavaScript

You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Advanced parameters

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Max Length

Optional if data type = text. You can input a numerical value in this field that represents the maximum length of the data that can be synced in your column. If the value is exceeded, the row will be rejected (you can find this error in the Execution Log).

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

                                                                                                                  |         |

String replacement

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination.
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
If more than one listener is needed for a real-time sync, configure it/them via the Listener Config table.
To run a real-time sync, enable your Listener from the Execution tab.

Appendix A - Source Parameters

The following sections outline more information about specific parameters you can find on this source.

Run Query

The Run Query parameter is available as an optional value for the Cinchy Event Broker/CDC connector. If set to true it executes a saved query; whichever record triggered the event becomes a parameter in that query. Thus the query now becomes the source instead of the table itself.

You are able to use any parameters defined in your listener config.

Example

The example below is a data sync using the Event Broker/CDC as a source. Our Listener Config has been set with the CinchyID attribute (Image 4).

We can enable the Run Query function to use the saved query "CDC Product Ticket Datestamps" as our source instead (Image 5). If we change the data from Record A in our source table to trigger our event, the Query Parameters below show that the Cinchy ID of Record A will be used in the query. This query is now our source.

It would appear in the data sync config XML as follows:

  <CinchyEventBrokerDataSource runQuery="true" domain="Product" name="CDC Product Ticket Datestamp" parameters="{&quot;@id&quot;: &quot;Cinchy Id&quot; }">

Cinchy Event Broker/CDC XML config example

Overview

This page highlights a few example XML configs that you can review when setting up your own Cinchy Event Broker/CDC data source.

You can review the source only example, the full example that shows both source and destination, and the listener config example.

Source example

The below example shows what the source parameters would look like in XML.

Full example

Use case

You want to set up a real-time sync between two Cinchy tables so that any time specific data is added, updated, or deleted from Table A it gets propagated to Table B. As long as you enable change notifications on your Cinchy table, you can do so by setting up a data sync and listener config with your source as the Cinchy Event Broker/CDC.

Listener config example

The following is the Cinchy CDC listener config Topic and Connection Attributes as it would be set for the above real time sync example to work.

Topic

Connection Attributes

Insert the text below for connection attributes:

Cinchy Table

Overview

Cinchy Tables are commonly used data sync sources.

Example use case

You want to set up batch sync between a Cinchy Table and HubSpot to sync important sales analytics information. You can do so by using the Cinchy Table as your source, and a REST API as your target.

The Cinchy Table source supports batch syncs. To do a real-time sync from a Cinchy Table, you would use the Cinchy Event Broker/CDC Source instead.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Select Show Advanced for more options for the Schema section.

You can choose to add in a Transformation > String Replacement by inputting the following:

Note that you can have more than one String Replacement

Next steps

Click Jobs > Start a Job to begin your sync.

Cinchy Table XML config example

Overview

This page has example XML configs that you can review when setting up your own Cinchy Table data source.

You can review the source only example or the full example that shows both source and destination.

Source example

The below example shows what the source parameters would look like in XML.

Full example

Example use case

You want to set up a batch sync, that you can run when needed, between a Cinchy Table and a MongoDB Collection. This sync will push out Client Name and Customer Number information.

Cinchy Query

Overview

Cinchy queries are commonly used data sync sources that leverage the platform's Saved Query functionality. For more on creating Saved Queries,.

Example use case

You want to set up batch sync between a Cinchy Query and a Cinchy Table. You query polls for any unapproved timesheets, out of office requests, or sick hours and, if found, adds them to an "Open Approval Tasks" table.

Info tab

You can review the parameters in the info tab below (Image 1).

Values

Parameter

Description

Example

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Select Show Advanced for more options for the Schema section.

Next steps

Click Jobs > Start a Job to begin your sync.

Cinchy Query XML config example

Overview

This page has example XML configs that you can review when setting up your own Cinchy Query data source.

You can review the source only example or the full example that shows both source and destination.

Source example

The below example shows what the source parameters would look like in XML.

Full example

Example use case

Copper

Overview

is a Customer Relationship Management (CRM) software. Copper is a tool focused on automation and simplicity, most known for its Google Workspace integration.

Example use case

You have customer information currently sitting in the Copper CRM software. You want to sync this data into Cinchy through a batch sync to liberate your data from the silo.

The Copper source supports batch syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Select Show Advanced for more options for the Schema section.

You can choose to add in a Transformation > String Replacement by inputting the following:

Note that you can have more than one String Replacement

Next steps

Click Jobs > Start a Job to begin your sync.

Cinchy Event Broker/CDC

Overview

The Cinchy Event Broker/CDC (Change Data Capture) source allows you to capture data changes on your table and use these events in your data syncs.

Use Case

The Cinchy Event Broker/CDC supports both batch syncs and real-time syncs (most common).

Remember to set up your listener config if you are creating a real-time sync.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

CDC

Variables

Optional. Review our documentation on Variables here for more information about this field.

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

Cinchy Event Broker/CDC

Run Query

Optional. If true, executes a saved query, using the Cinchy ID of the changed record as a parameter. These query results are then used as the sync source, rather than using the Cinchy table where the data change originated. Review for further details on this feature.

Path to Iterate

To set up a real-time sync, you must configure your Listener values. You can do so through the Connections UI.

If there is more than one listener associated with your data sync, you will need to configure the addition listeners via the Listener Configuration table.

If you are creating a CDC listener config for a Cinchy Event Triggered REST API data source, pay in mind the following unique constraints:

Column names in the listener config shouldn't contain spaces. If they do, they will be automatically removed. For example, a column named First Name will become @FirstName.
The replacement variable names are case sensitive.
Column names in the listener config shouldn't be prefixes of other column names. For example, if you have a column called Name, you shouldn't have another called "Name2" as the value of @Name2 may end up being replaced by the value of @Name suffixed with a 2.

Reset behaviour

Parameter

Description

Example

Auto Offset Reset

Earliest will start reading from the beginning on the queue (when the CDC was enabled on the table). This might be a suggested configuration if your use case is recoverable or re-runnable and if you need to reprocess all events to ensure accuracy. Latest will fetch the last value after whatever was last processed. This is the typical configuration. None won't read start reading any events. You are able to switch between Auto Offset Reset types after your initial configuration through the process outlined

None

Topic JSON

A Topic JSON is necessary for all real-time syncs. Enter your JSON parameters through the Connections UI, or edit them directly through the Listener Configuration table..

Topic JSON parameters

Parameter

Description

Example Value

tableGuid

Mandatory. GUID of the table you are reading from.

ef6710ca-6e59-4b4a-86d3-f6d24ed7658b

fields

Array of objects specifying columns to fetch.

See fields section below.

filter

WHERE clause for filtering records.

New.[Is Valid] = 1 AND (New.[Is Excluded] = 0 OR New.[Is Excluded] IS NULL)

messageKeyExpression

The messageKeyExpression parameter specifies a key that the listener application uses to route messages into specific topics within a Kafka broker. See below for more information.

value

next_cursor

ABCS

batchSize

Number of records read per request.

1000

Fields Section

The following expands on the available parameters within the fields section above.

Field Property

Description

Example Value

column

Column name to fetch from the table.

Cinchy Id

alias

Alias for the column.

CinchyId

deserializeJsonValue

Converts text to JSON on read out.

true

Topic JSON example

The following is a topic JSON example:

{
  "tableGuid": "420c1851-31ed-4ada-a71b-31659bca6f92",
  "fields": [
    {
      "column": "Cinchy Id",
      "alias": "CinchyId"
    },
    {
      "column": "First Name",
      "alias": "Firstname"
    },
    {
      "column": "Last Name",
      "alias": "Lastname"
    }
  ],
  "messageKeyExpression": "CONCAT(New.[CinchyId], '-', New.[Name])",
  "filter": "New.[Lastname] IS not null OR (New.[CinchyId] is not null)",
  "batchSize": 100
}

Listener Configuration Parameters

messageKeyExpression

Each of your Event Listener message keys a message key. By default, this key is dictated by the Cinchy ID of the record being changed.

When the worker processes your Event Listener messages, it does so in batches, and for efficiency and to guarantee order, messages that contain the same key won't be processed in the same batch.

The messageKeyExpression property allows you to change the default message key to something else.

Use Case

Ensuring records with the same message key can be updated with the proper ordering to reflect an accurate collaboration log history.

Example Syntax

In this example, we want the message key to be based on the [Employee Id] and [Name] column of the table that CDC is enabled on.

{ "messageKeyExpression": "CONCAT(New.[Employee Id], '-', New.[Name])", … }

Old vs New Filter

The "New" and "Old" parameters are based on updates to single records, not columns/rows.

"New" Example:

Due to internal logic, newly created records will be tagged as both **New** and **Old**.

"filter": "New.[Approval State] = 'Approved'

"Old" Example:

Due to internal logic, newly created records will be tagged as both **New** and **Old**.

"filter": "Old.[Status] = `In Progress`

Connection Attributes

You don`t need to provide Connections Attributes when using the Cinchy CDC Stream Source.

If you're inputting your configuration via the Listener Config table, you must insert the below text into the column:

{}

Schema section

The Schema section is where you define which source columns you want to sync in your connection. You have the option to add the following columns:

Standard
Calculated
Conditional
JavaScript

You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Advanced parameters

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Max Length

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

                                                                                                                  |         |

String replacement

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination.
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
If more than one listener is needed for a real-time sync, configure it/them via the Listener Config table.
To run a real-time sync, enable your Listener from the Execution tab.

Appendix A - Source Parameters

The following sections outline more information about specific parameters you can find on this source.

Run Query

You are able to use any parameters defined in your listener config.

Example

The example below is a data sync using the Event Broker/CDC as a source. Our Listener Config has been set with the CinchyID attribute (Image 4).

It would appear in the data sync config XML as follows:

  <CinchyEventBrokerDataSource runQuery="true" domain="Product" name="CDC Product Ticket Datestamp" parameters="{&quot;@id&quot;: &quot;Cinchy Id&quot; }">

Data sync sources

Cinchy Event Broker

Cinchy Table

Cinchy Query

Binary file

A binary file is a computer file that's not a text file, and whose content is in a binary format consisting of a series of sequential bytes, each of which is eight bits in length.

You can use binary files from a Local upload, Amazon S3, or Azure Blob Storage in your data syncs.

Some benefits of using binary files include:

Better efficiency via compression
Better Security through the ability to create custom encoding standards.
Unmatched Speed, since the data is stored in a raw format, and isn't encoded using any character encoding standards, it's faster to read and store.

Copper

is a Customer Relationship Management (CRM) software. Copper is a tool focused on automation and simplicity, most known for its Google Workspace integration.

Copper

Delimited file

Dynamics 365

functions as an interconnected CRM, ERP, and productivity suite that integrates processes, data, and business logic.

Dynamics

Dynamics 2015

Dynamics 2015 is a legacy CRM predecessor to Microsoft Dynamics 365. Mainstream end of life support finished in , with extended end of life support finishing in January 2025.

Dynamics 2015

DynamoDB

is a managed NoSQL database service that's offered by Amazon as part of the AWS portfolio.

DynamoDB

Excel

Fixed-width file

Advantages of using a fixed-width file include:

It's a compact representation of your data
It's fast to parse because every field is in the same place in every line

Fixed width file

Kafka Topic

is an end-to-end event streaming platform that:

Publishes (writes) and subscribes to (reads) streams of events from sources like databases, cloud services, and software applications.
Stores these events durably and reliably for as long as you want.
Processes and reacts to the event streams in real-time and retrospectively.

Those events are organized and durably stored in topics. These topics are then partitioned over a number of buckets located on different Kafka brokers.

Event streaming thus ensures a continuous flow and interpretation of data so that the right information is at the right place, at the right time

Kafka Topic

LDAP

(Lightweight Directory Access Protocol) is a mature, flexible, and well supported standards-based mechanism software protocol for enabling anyone to locate data whether on the public internet or on a corporate intranet.

Common uses of LDAP include when:

A single piece of data needs to be found and accessed regularly;
Your organization has a lot of smaller data entries;
Your organization wants all smaller pieces of data in one centralized location, and there doesn't need to be an extreme amount of organization between the data.

LDAP

MongoDB collection

MongoDB collection (CDC triggered)

is a scalable, flexible NoSQL document database platform known for its horizontal scaling and load balancing capabilities, which has given application developers an unprecedented level of flexibility and scalability. Data changes in Cinchy (CDC) can be used to trigger a data sync from a MongoDB data source to a specified target. The attributes of the CDC Event are available to use as parameters within the Data Source Definition to narrow the scope of the request. For example, a lookup.

MongoDB collection (Cinchy event)

MS SQL Server (query and table)

ODBC query

Open Database Connectivity () is a standard API for accessing database management systems (DBMS).

DynamoDB

Oracle (query and table)

is a relational database management system, commonly used for running online transaction processing, data warehousing and mixed database workloads. The system is built around a relational database framework in which data objects may be directly accessed by users (or an application front end) through structured query language (SQL).

Oracle (query and table)

Parquet

is an open source data file format built to handle flat columnar storage data formats. Parquet operates well with complex data in large volumes and is known for its both performant data compression and its ability to handle a wide variety of encoding types.

Parquet

Polling event

REST API

A REST API is an application programming interface that conforms to the constraints of REST (representational state transfer) architectural style and allows for interaction with RESTful web services.

REST API

Salesforce object (bulk API)

is a cloud-based CRM software designed for service, marketing, and sales.

Salesforce objects are database tables that permit you to store data that's specific to an organization. Salesforce objects are of two types:

Standard Objects: Standard objects are the kind of objects that are provided by salesforce.com such as users, contracts, reports, dashboards, etc.
Custom Objects: Custom objects are those objects that are created by users. They supply information that's unique and essential to their organization. Custom objects are the heart of any application and provide a structure for sharing data.

Salesforce Object (Bulk API)

Salesforce platform event

is a cloud-based CRM software designed for service, marketing, and sales.

Salesforce Platform Events are secure and scalable messages that contain data. Publishers push out event messages that subscribers receive in real time.

Salesforce platform event

Salesforce push topic

is a cloud-based CRM software designed for service, marketing, and sales.

Push Topic events provide a secure and scalable way to receive notifications for changes to Salesforce data that match a SOQL query you define.

You can use PushTopic events to:

Receive notifications of Salesforce record changes, including create, update, delete, and undelete operations.
Capture changes for the fields and records that match a SOQL query.
Receive change notifications for only the records a user has access to based on sharing rules.
Limit the stream of events to only those events that match a subscription filter.

Salesforce push topic

Snowflake

is a fully managed SaaS that provides a single platform for data warehousing, data lakes, data engineering, data science, data application development, and secure sharing and consumption of real-time/shared data.

Snowflake enables data storage, processing, and analytic solutions.

Snowflake

SOAP 1.2 web service

SOAP (Simple Object Access Protocol) is an XML-based protocol for accessing web services over HTTP.

SOAP 1.2 web service

SAP SuccessFactors

solutions are cloud-based HCM software applications that support core HR and payroll, talent management, HR analytics and workforce planning, and employee experience management.

SAP SuccessFactors

Binary file

Overview

A binary file is a computer file that's not a text file, and whose content is in a binary format consisting of a series of sequential bytes, each of which is eight bits in length.

You can use binary files from a Local upload, Amazon S3, or Azure Blob Storage in your data syncs.

Some benefits of using binary files include:

Better efficiency via compression
Better Security through the ability to create custom encoding standards.
Unmatched Speed, since the data is stored in a raw format, and isn't encoded using any character encoding standards, it's faster to read and store.

Example use case

You have a binary file that contains your Employee information. You want to use a batch sync to pull this info into a Cinchy table and liberate your data.

The Binary File source supports batch syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

Employee Sync

Variables

@Filepath

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

(Sync) Source

Mandatory. Select your source from the drop down menu.

Binary File

Source

The location of the source file. Either a Local upload, Amazon S3, or Azure Blob StorageThe following authentication methods are supported per source:Amazon S3: Access Key ID/Secret Access KeyAzure Blob Storage: Connection String

Local

Header Rows to Ignore

Mandatory. The number of records from the top of the file to ignore before the data starts (includes column header).

Footer Rows to Ignore

Mandatory. The number of records from the bottom of the file to ignore

Encoding

Optional. The encoding of the file. This default to UTF8, however also supports: UTF8_BOM, UTF16, ASCII.

Path

Mandatory. The path to the source file to load. To upload a local file, you must first insert a Variable in the Info tab of the connection (ex: filepath). Then, you would reference that same value in this location (Ex: @Filepath). This will then trigger a File Upload option to import your file.

@Filepath

AuthType

This field defines the authentication type for your data sync. Cinchy supports "Access Key" and "IAM" role. When selecting Access Key, you must provide the key and key secret. When selecting IAM role, a new field will appear for you to paste in the role's Amazon Resource Name (ARN). You also must ensure that:The role must be configured to have at least read access to the source. The Connections pods' role must have permission to assume the role specified in the data sync config.

Test Connection

You can use the "Test Connection" button to ensure that your credentials are properly configured to access your source. If configured correctly, a "Connection Successful" pop-up will appear. If configured incorrectly, a "Connection Failed" pop-up will appear along with a link to the applicable error logs to help you troubleshoot.

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Parse Content By (Only for Standard Columns)

Binary File sources have a unique, mandatory parameter for Standard Columns:

Parse Content By - Choose from the following three options to define how you want to parse your content:

Byte Length - The content length in number of bytes
Trailing Byte Sequence - the trailing sequence in base64 that indicates the end of the field
Succeeding Byte Sequence - the trailing sequence in base64 that indicates the start of the next field, and thus the end of this one.

Byte Length

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
Click Jobs > Start a Job to begin your sync.

DB2 (query and table)

Overview

DB2 (Formerly Db2 for LUW) is a relational database that delivers advanced data management and analytics capabilities for transactional workloads.

The DB2 source supports batch syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

DB2 to Cinchy

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

DB2

Connection String

Mandatory. The encrypted connection string used to access your DB2 database. The Connection UI will automatically encrypt this value for you.

Object

Mandatory. The type of object you want to use as your data sync. This will be either Table or Query.

Table

Appears when "Table" is selected as the Object Type. The name of your table as it appears in your DB2 database.

dbo.employees

Query

Appears when "Query" is selected as the Object Type. This should be a SELECT statement indicating the data you want to sync out of your DB2 database.

Select * from dbo.employees

Test Connection

You can use the "Test Connection" button to ensure that your credentials are properly configured to access your source.

If configured correctly, a "Connection Successful" pop-up will appear.

If configured incorrectly, a "Connection Failed" pop-up will appear along with a link to the applicable error logs to help you troubleshoot.

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source. This should be in all caps. EXCEPTION: If you chose "query" as your object and use double quotes around the column names, then this value should should match that casing.

NAME

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
Click Jobs > Start a Job to begin your sync.

Dynamics

Overview

Microsoft Dynamics 365 functions as an interconnected CRM, ERP, and productivity suite that integrates processes, data, and business logic.

Example use case

You have customer information currently sitting in the Dynamics CRM software. You want to sync this data into Cinchy through a batch sync to liberate your data from the silo.

The Dynamics source supports batch syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

Dynamics to Cinchy

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

Dynamics

Entity

Mandatory. The name of the entity you want to sync as it appears in your Dynamics CRM.

Companies

Service URL

Mandatory. The Web API URL of your instance.

https://org.api.crm.dynamics.com/api/data/v9.0/

Redirect URL

Mandatory. The Redirect URI from the Azure AD app registration

https://example.com/

Client ID

Mandatory. The encrypted Client ID found in your Azure AD app registration. The Connection UI will automatically encrypt this value for you.

Client Secret

Mandatory. The encrypted Client Secret found in your Azure AD app registration. The Connection UI will automatically encrypt this value for you.

Test Connection

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
Click Jobs > Start a Job to begin your sync.

DynamoDB

Overview

Amazon DynamoDB is a managed NoSQL database service that's offered by Amazon as part of the AWS portfolio.

Example use case

You currently use DynamoDB to store metrics on product use and growth, but being stuck in the DynamoDB silo means that you can't easily use this data across a range of business use cases or teams. You can use a batch sync to liberate your data into Cinchy.

The DynamoDB source supports batch syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

Product Metrics

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

DynamoDB

Entity

Mandatory. The name of the entity you want to sync as it appears in DynamoDB.

Metrics

AWS Access Key (Client ID)

Mandatory. The encrypted AWS Access Key (Client ID) used to access your DynamoDB.

AWS Secret (Client Secret)

Mandatory. The encrypted AWS Secret (Client Secret) used to access your DynamoDB.

AWS Region

Mandatory. The name of the region for your AWS instance.

US-East-1

Username

Mandatory. The name of a user with access to connect to your DynamoDB server.

Password

Mandatory. The password associated with the above user.

AuthType

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
Click Jobs > Start a Job to begin your sync.

MongoDB collection

Overview

MongoDB is a scalable, flexible NoSQL document database platform known for its horizontal scaling and load balancing capabilities, which has given application developers an unprecedented level of flexibility and scalability.

Considerations

Please review the following considerations before you set up your MongoDB Collection data sync source:

We currently only support SCRAM authentication (Mongo 4.0+).
Syncs are column based. This means that you must flatten the MongoDB source document prior to sync by using a projection (See section 2: Projection (JSON Object)).
The column names used in the source must match elements on the root object, with the exception of "$" which can be used to retrieve the full document.
By default, MongoDB batch size is 101.
By default, bulk operations size is 5000.
Due to a conversion of doubles to decimals that occurs during the sync process, minor data losses may occur.
The following data types aren't supported:
- Binary Data
- Regular Expression
- DBPointer
- JavaScript
- JavaScript code with scope
- Symbol
- Min Key
- Max Key
The following data types are supported with conversions:
- ObjectID is supported, but converted to string
- Object is supported, but converted to JSON
- Array is supported, but converted to JSON
- Timestamp is supported, but converted to 64-bit integers

The MongoDB Collection source supports batch syncs. (To enable real-time syncs with MongoDB, use the MongoDB Collection (Cinchy Event Triggered) or Mongo Event source instead.)

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

MongoDB Collection to Cinchy

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

MongoDB Collection

Connection String

Mandatory. This is the encrypted connection string. You can review MongoDB's Connection String guide and parameter descriptions here. Don't include the /[database] in your connection URL. By default services like MongoDB Atlas will automatically include it when copying the connection string. If authenticating against a database other than the admin db, please provide the name of the database associated with the user’s credentials using the authSource parameter.

Example (Default):mongodb+srv://:@Example (Against different database):mongodb+srv://:@?authSource=<authentication_db>

Database

Mandatory. The name of the MongoDB database that contains the collection listed in the "Collection" parameter.

Blog

Collection

Mandatory. The name of your MongoDB collection.

Article

Type

Mandatory. The method for retrieving your data. This will be either:- db.collection.find(): This method is used to select documents in a collection when there is no need to transform (flatten or aggregate) the data. It's used for basic queries where query and projection are sufficient.- db.collection.aggregate(): This method is used when there is a need to transform the data in a collection. It's used for more complex scenarios with single or multi-stages pipelines. In general, you will yield the quickest performance by using the find method, unless you need a specific aggregation operator.

Query (JSON Object)

A query for retrieving your data. This option appears if you have selected db.collection.find().

Example Query

Projection (JSON Object)

This option appears if you have selected db.collection.find().Syncs are column based. This means that you must flatten the MongoDB source document prior to sync using a projection.

Example Projection

Pipeline (JSON Array of Objects)

An aggregation pipeline consists of one or more stages that process documents. This option appears if you have selected db.collection.aggregate().

Use SSL

This checkbox can be used to define the use of x.509 certificate authentication for your sync. If checked, you will need to input the following values taken from your cert:- SSL Key PEM- SSL Certificate PEM- SSL CLA PEM

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source. This field is case sensitive and preserves spaces.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

Query example

// query: where "Price" is less than 10

blog> db.Articles.find({ "Price": { "$lt": 10 } })
[
  {
    _id: ObjectId("63d8137bd755fcdeed234403"),
    Name: 'Shirt',
    Price: 9.95,
    Details: { Color: 'White', Size: 'Small' },
    Stock: 61
  }
]

Projection example

// Flatten the document

blog> db.Articles.find({}, { Name: 1, Price: 1, Color: "Details.Color", Size: "Details.Size", Stock: 1 })
[
  {
    _id: ObjectId("63d812afd755fcdeed234402"),
    Name: 'Shirt',
    Price: 19.95,
    Stock: 12,
    Color: 'Details.Color',
    Size: 'Details.Size'
  },
  {
    _id: ObjectId("63d8137bd755fcdeed234403"),
    Name: 'Shirt',
    Price: 9.95,
    Stock: 61,
    Color: 'Details.Color',
    Size: 'Details.Size'
  }
]

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
To run a batch sync, select Jobs > Start Job.

Appendix A

Data types

The MongoDB Collection Data Source obtains BSON documents from MongoDB. BSON, short for Binary JSON, is a binary-encoded serialization of JSON-like documents. Like JSON, BSON supports the embedding of documents and arrays

within other documents and arrays. BSON also has extensions that allow representation of data types that aren't part of the JSON spec. For example, BSON makes a distinction between Int32 and Int64.

The following table shows how MongoDB data types are translated in Cinchy.

MongoDB

Cinchy

Notes

Double

Number

Supported

String

Text

Supported

Object

Text (JSON)

Supported

Array

Text (JSON)

Supported

Binary Data

Binary

Unsupported

ObjectId

Text

Supported

Boolean

Supported

Date

Supported

Null

Supported

RegEx

Unsupported

JavaScript

Unsupported

Timestamp

Number

Supported

32-bit Integer

Number

Supported

64-bit Integer

Number

Supported

Decimal28

Number

Supported

Min Key

Unsupported

Max Key

Unsupported

Geography

Unsupported

Geometry

Unsupported

Retry configuration

A retry configuration will automatically retry HTTP Requests on failure based on a defined set of conditions. This capability provides a mechanism to recover from transient errors such as network disruptions or temporary service outages.

Note: the maximum number of retries is capped at 10.

To set up a retry specification:

Select "Add Retry Configuration" from the Source tab.
Select your Delay Strategy.

Linear Backoff: Defines a delay of approximately n seconds where n = current retry attempt.
Exponential Backoff: A strategy where every new retry attempt is delayed exponentially by 2^n seconds, where n = current retry attempt.
- Example: you defined Max Attempts = 3. Your first retry is going to be in 2^1 = 2, second: 2^2 = 4, third: 2^3 = 8 sec.

3. Input your Max Attempts. The maximum number of retries allowed is 10.

4. Define your Retry Conditions. You must define the conditions under which a retry should be attempted. For the Retry to trigger, at least one of the "Retry Conditions" has to evaluate to true.

Retry conditions are only evaluated if the response code isn't 2xx Success.

Each Retry Condition contains one or more "Attribute Match" sections. This defines a Regex to evaluate against a section of the HTTP response. The following are the three areas of the HTTP response that can be inspected:

Response Code
Header
Body

If there are multiple "Attribute Match" blocks within a Retry Condition, all have to match for the retry condition to evaluate to true.

The Regex value should be entered as a regular expression. The Regex engine is .NET and expressions can be tested by using this online tool. In the below example, the Regex is designed to match any HTTP 5xx Server Error Codes, using a Regex value of 5[0-9][0-9]. For Headers, the format of the Header string which the Regex is applied against is {Header Name}={Header Value}. For example, Content-Type=application/json.

MongoDB collection (Cinchy event)

Overview

Data changes in the Cinchy Event Broker (CDC) can be used to trigger a data sync from a MongoDB data source to a specified target. The attributes of the CDC Event are available to use as parameters within the Data Source Definition to narrow the scope of the request. For example, a lookup.

The MongoDB Collection (Cinchy Event Triggered) Source supports real-time syncs.

Considerations

Please review the following considerations before you set up your MongoDB Collection data sync source:

We currently only support SCRAM authentication (Mongo 4.0+).
Syncs are column based. This means that you must flatten the MongoDB source document prior to sync by using a projection (See section 2: Projection (JSON Object)).
The column names used in the source must match elements on the root object, except for "$" which can be used to retrieve the full document.
By default, MongoDB batch size is 101.
By default, bulk operations size is 5000.
Due to a conversion of doubles to decimals that occurs during the sync process, minor data losses may occur.
The following data types aren't supported:
- Binary Data
- Regular Expression
- DBPointer
- JavaScript
- JavaScript code with scope
- Symbol
- Min Key
- Max Key
The following data types are supported with conversions:
- ObjectID is supported, but converted to string
- Object is supported, but converted to JSON
- Array is supported, but converted to JSON
- Timestamp is supported, but converted to 64-bit integers

Connection definitions

The following sections in the Source configuration of the Connections experience can reference attributes of the CDC Event as parameters:

Connection String
Database Name
Collection Name
Query
Projection
Pipeline

In Cinchy v5.6+, you can also reference attributes of the CDC Event in Calculated Columns.

Note that syncs making use of this must limit their batch size to 1.

Parameters use the column name or alias as defined in the CDC Event Listener Config prefixed with an @. For example, @CompanyName would be the parameter name for the following Cinchy CDC listener Topic configuration.

{
	"tableGuid": "420c1851-31ed-4ada-a71b-31659bca6f92",
	"fields": [
		{
			"column": "Cinchy Id",
                        "alias":"CinchyId"
                },
		{
			"column": "Company Name",
                        "alias":"CompanyName"
		}
	]
}

Parameter names are case sensitive when used in the Connection configuration. Parameter matching is performed using literal string replacements. Names shouldn't contain spaces (spaces are automatically removed), and should have differing prefixes.

The following set of parameters will be available on every event even if they're not present in the listener config

@Version
@DraftVersion
@CinchyRecordType
@ApprovalState
@ModifiedBy
@Modified
@Deleted

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

MongoDB Cinchy Event to Cinchy

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

MongoDB Collection (Cinchy Event Triggered)

Connection String

Example (Default):

mongodb+srv://:@

Example (Against different database):

mongodb+srv://:@?authSource=

Database

Blog

Collection

Article

Type

Query (JSON Object)

A query for retrieving your data. This option appears if you have selected db.collection.find().

Projection (JSON Object)

This option appears if you have selected db.collection.find(). Syncs are column based. This means that you must flatten the MongoDB source document prior to sync using a projection.

Pipeline (JSON Array of Objects)

Use SSL

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source. This field is case sensitive and preserves spaces.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

Query example

// query: where "Price" is less than 10

blog> db.Articles.find({ "Price": { "$lt": 10 } })
[
  {
    _id: ObjectId("63d8137bd755fcdeed234403"),
    Name: 'Shirt',
    Price: 9.95,
    Details: { Color: 'White', Size: 'Small' },
    Stock: 61
  }
]

Projection example

// Flatten the document

blog> db.Articles.find({}, { Name: 1, Price: 1, Color: "Details.Color", Size: "Details.Size", Stock: 1 })
[
  {
    _id: ObjectId("63d812afd755fcdeed234402"),
    Name: 'Shirt',
    Price: 19.95,
    Stock: 12,
    Color: 'Details.Color',
    Size: 'Details.Size'
  },
  {
    _id: ObjectId("63d8137bd755fcdeed234403"),
    Name: 'Shirt',
    Price: 9.95,
    Stock: 61,
    Color: 'Details.Color',
    Size: 'Details.Size'
  }
]

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
To run a real-time sync (using the Cinchy Event Triggered MongoDB Source), set up your Listener Config using the Cinchy Event Broker/CDC and enable it to begin your sync.

Appendix A

Data types

within other documents and arrays. BSON also contains extensions that allow representation of data types that aren't part of the JSON spec. For example BSON makes a distinction between Int32 and Int64.

The following table shows how MongoDB data types are translated in Cinchy.

MongoDB

Cinchy

Notes

Double

Number

Supported

String

Text

Supported

Object

Text (JSON)

Supported

Array

Text (JSON)

Supported

Binary Data

Binary

Unsupported

ObjectId

Text

Supported

Boolean

Bool

Supported

Date

Supported

Null

Supported

RegEx

Unsupported

JavaScript

Unsupported

Timestamp

Number

Supported

32-bit Integer

Number

Supported

64-bit Integer

Number

Supported

Decimal28

Number

Supported

Min Key

Unsupported

Max Key

Unsupported

Geography

Unsupported

Geometry

Unsupported

Listener Config

To configure a MongoDB Collection (Cinchy Event Triggered) connection, a listener must be configured via the Listener Config table with an Event Connector Type of Cinchy CDC.

Review the Cinchy Event Broker/CDC Listener Configuration values here, and then navigate to the Listener Config table to input a new row.

When setting up your listener configuration for your data sync, keeping the following constraints in mind:

Column names in the listener config shouldn't contain spaces. If they do, they will be automatically removed. For example, a column named Company Name will become the replacement parameter @CompanyName.
The replacement parameter names are case sensitive.
Column names in the listener config shouldn't be prefixes of other column names. For example, if you have a column called Name, you shouldn't have another called Name2 as the value of @Name2 may end up being replaced by the value of @Name suffixed with a 2._

Example Listener Configuration

{
  "tableGuid": "420c1851-31ed-4ada-a71b-31659bca6f92",
  "fields": [
    {
      "column": "Cinchy Id",
      "alias": "CinchyId"
    },
    {
      "column": "Company Name",
      "alias": "CompanyName"
    }
  ]
}

LDAP

Overview

LDAP (Lightweight Directory Access Protocol) is a mature, flexible, and well supported standards-based mechanism software protocol for enabling anyone to locate data whether on the public internet or on a corporate intranet.

Common uses of LDAP include when:

A single piece of data needs to be found and accessed regularly;
Your organization has a lot of smaller data entries;
Your organization wants all smaller pieces of data in one centralized location, and there doesn't need to be an extreme amount of organization between the data.

The LDAP source supports batch syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

LDAP to Cinchy

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

LDAP

Server

Mandatory. The name of your LDAP Server Directory

Company-1

Object Category

Internal-Metrics

Username

Mandatory. The name of a user who has access to the LDAP server.

Product

Password

Mandatory. The password for the above user. The Connections UI will encrypt this value.

CN (Common Name)

Optional.

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
Click Jobs > Start a Job to begin your sync.

Mongo event

Overview

The MongoDB Event stream source works similar to Cinchy's Change Data Capture functionality. The listener subscribes to monitor the change stream of a specific collection in the database of the MongoDB server. Any actions performed on document(s) inside of that collection are picked up by the listener and sent to the queue.

Limitations

To use change streams in MongoDB, there are a few requirements your environment must meet.

The database must be in a replica set or sharded cluster.
The database must use the WiredTiger storage engine.
The replica set or sharded cluster must use replica set protocol version 1.

The MongoDB Event source supports real-time syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

Mongo Event to Cinchy

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

MongoDB Event

To set up a real-time sync, you must configure your Listener values. You can do so through the Connections UI.

Note that If there is more than one listener associated with your data sync, you will need to configure the addition listeners via the Listener Configuration table.

Reset Behaviour

Parameter

Description

Example

Auto Offset Reset

None

Topic JSON

The below table can be used to help create your Topic JSON needed to set up a real-time sync.

Parameter

Description

Example

Database

Cinchy

Collection

Employee

Pipeline Stages

Optional. This parameter allows you to specify pipeline stages with filters.

Each stage performs an operation on the input documents. For example, a stage can filter documents, group documents, and calculate values.
The documents that are output from a stage are passed to the next stage.
An aggregation pipeline can return results for groups of documents. For example, return the total, average, maximum, and minimum values.

See the Example Topic JSON below. Our example config uses a filter to return documents with an ID between 0 and 10,000 AND documents with the location set to Montreal, OR where the operation type is 'delete'

Example Topic JSON

{
	"database": "cinchy",
	"collection": "employee",
        "changeStreamSettings": {
		"pipelineStages": [
			"{ $match: {
                                       $or: [ 
                                           { $and: [
                                                { 'fullDocument.id': { $gt: 0, $lt: 10000 } }, 
                                                { 'fullDocument.location': 'Montreal' } ] 
                                           }, 
                                           { 'operationType': 'delete' } 
                                      ]
                                } }"
		]
	}
}

Connection Attributes

The below table can be used to help create your Connection Attributes JSON needed to set up a real-time sync.

Parameter

Description

Example

connectionString

mongodb://localhost:9877

{
	"connectionString": "mongodb://localhost:9877"
}

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source. This field is case sensitive and preserves spaces.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
If more than one listener is needed for a real-time sync, configure it/them via the Listener Config table.
To run a real-time sync, enable your Listener from the Execution tab.

REST API

Overview

A REST API is an application programming interface that conforms to the constraints of REST (representational state transfer) architectural style and allows for interaction with RESTful web services.

The REST API source support batch syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Source tab

Mandatory and optional parameters for the Source tab are outlined below (Image 2).

Select Show Advanced for more options for the Schema section.

You can choose to add in a Transformation > String Replacement by inputting the following:

Note that you can have more than one String Replacement

More options are available to you under the "Add a Section" drop down.

Note that adding a Pagination Block is mandatory.

Best practices

Retrieve nested fields

To get fields in a nested array, you can either set the nested array as the root, or you can use Path to Iterate to expand the array.

Examples

Here is a sample JSON response:

Records Root JSONPath: $.users Schema:

$.userId for ID
$.name for Name

You can't reference "groupId" as it's one level above the specified root scope.

JSON array handling

Use $.data in Records Root JSONPath if the API returns a top-level JSON array.

Examples

Here is a sample JSON response:

Records Root JSONPath: $.data

Schema:

$.name for Name
$.age for Age

Path to Iterate

Use Path to Iterate to expand and target nested keys within the array. This only applies if the records within an array are objects.

If the record within the path to iterate is an array, each item within the array gets placed under an "item" key in a new JSON object.

Example

For example, here is a sample JSON response:

In this example, we want to iterate over the "transactions" array and capture the records for "transactionid" and assign them to the "Transaction ID" column, and then add the parent "name" key to a Name column .

Records Root JSONPath: $ Path to iterate: $.transactions

Schema:

$.name for Name
$.transactions.id for Transaction ID

Next steps

To run a batch sync, select Jobs > Start Job

Resources

Auth Request

Request Headers

Body

You are able to use this section to add body content.

Pagination

Retry Configuration

Retry Configuration automatically retries HTTP Requests on failure based on a defined set of conditions. This provides a mechanism to recover from transient errors, such as network disruptions or temporary service outages.

Note: the maximum number of retries is capped at 10.

To set up a retry specification:

Under the REST API source tab, select API Specification > Retry Configuration

Select your Delay Strategy.

Linear Backoff: Defines a delay of approximately n seconds where n = current retry attempt.
Exponential Backoff: A strategy where every new retry attempt is delayed exponentially by 2^n seconds, where n = current retry attempt.
- Example: you defined Max Attempts = 3. Your first retry is going to be in 2^1 = 2, second: 2^2 = 4, third: 2^3 = 8 sec.

3. Input your Max Attempts. The maximum number of retries allowed is 10.

Define your Retry Conditions. You must define the conditions under which a retry should be attempted. For the Retry to trigger, at least one of the "Retry Conditions" has to evaluate to true.

Retry conditions are only evaluated if the response code isn't 2xx Success.

Each Retry Condition contains one or more "Attribute Match" sections. This defines a regex to evaluate against a section of the HTTP response. The following are the three areas of the HTTP response that can be inspected:

Response Code
Header
Body

If there are multiple "Attribute Match" blocks within a Retry Condition, all have to match for the retry condition to evaluate to true.

;;

Salesforce push topic

Overview

Salesforce is a cloud-based CRM software designed for service, marketing, and sales.

Push Topic events provide a secure and scalable way to receive notifications for changes to Salesforce data that match a SOQL (Salesforce Object Query Language) query you define.

You can use Push Topic events to:

Receive notifications of Salesforce record changes, including create, update, delete, and undelete operations.
Capture changes for the fields and records that match a SOQL query.
Receive change notifications for only the records a user has access to based on sharing rules.
Limit the stream of events to only those events that match a subscription filter

The Salesforce Push Topic source supports real-time syncs.

Real-time sync scenarios

You can use a Push Topic already configured in Salesforce, or have Cinchy Event Listener create the Push Topic for you.

Scenario 1: Push Topic already exists in Salesforce.

Cinchy will compare the JSON with the properties on the push topic in Salesforce by name. If the attributes match, the listener will start listening on the push topic.

Scenario 2: Push Topic already exists in Salesforce and the configuration doesn't match.

Cinchy will compare the JSON with the properties on the push topic in Salesforce by name. If any of the attributes don't match, Cinchy will sync the push topic from Salesforce into Cinchy and disable the listener.

Scenario 3: Push Topic doesn't exist in Salesforce.

If the Push Topic name doesn't exist in Salesforce, Cinchy will attempt to create the Push Topic. If it's successful, it will sync in the Id from Salesforce and start listening on the push topic.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

Salesforce Push Topic

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

Salesforce Push Topic

To set up a real-time sync, you must configure your Listener values. You can do so through the Connections UI.

Note that If there is more than one listener associated with your data sync, you will need to configure the addition listeners via the Listener Configuration table.

Reset Behaviour

Auto Offset Reset table

Parameter

Description

Default Value

Auto Offset Reset

Determines the starting point for event reading. Options:

Earliest: Starts from the queue beginning. Useful for recoverable or re-runnable use cases.
Latest: Starts after the last processed event.
None: No events are read.

You can change this setting later.

None

Topic JSON

The below table can be used to help create your Topic JSON needed to set up a real-time sync.

Parameter

Description

Example

Name

Mandatory. Descriptive name of the PushTopic. Note that there is a 25 character limit on this field.

LeadsTopic

Query

Mandatory. The SOQL query statement that determines which record changes trigger events to be sent to the channel. This field has a 1,300 character limit.

SELECT Id, Name, Email FROM Lead

ApiVersion

Mandatory. The API version to use for executing the query specified in Query. It must be an API version greater than 20.0. If your query applies to a custom object from a package, this value must match the package's ApiVersion.

47.0

NotifyForOperationCreate

Set this to true if a create operation should generate a notification, otherwise, false. Defaults to true.

true

NotifyForOperationUpdate

Set this to true if an update operation should generate a notification, otherwise, false. Defaults to true.

true

NotifyForOperationUndelete

Set this to true if an undelete operation should generate a notification, otherwise, false. Defaults to true.

true

NotifyForOperationDelete

Set this to true if a delete operation should generate a notification, otherwise, false. Defaults to true.

true

NotifyForFields

Specifies which fields are evaluated to generate a notification. Possible values are:AllReferenced (default)SelectWhere

Referenced

Example Topic JSON

{
  "Id": "",
  "Name": "LeadsTopic",
  "Query": "SELECT Id, Name, Email FROM Lead",
  "ApiVersion": 47.0,
  "NotifyForOperationCreate": true,
  "NotifyForOperationUpdate": true,
  "NotifyForOperationUndelete": true,
  "NotifyForOperationDelete": true,
  "NotifyForFields": "Referenced"
}

Connection Attributes

The below table can be used to help create your Connection Attributes JSON needed to set up a real-time sync.

Parameter

Description

Example

ApiVersion

Mandatory. Your Salesforce API Version. Note that this needs to be an exact match; for instance 47.0 can't be written as simply 47.

47.0

GrantType

This value should be set to password.

password

ClientId

The encrypted Salesforce Client ID. You can encrypt this value using the Cinchy CLI.

Bn8UmtiLydmYQV6//qCL5dqfNUMhqchdk959hu0XXgauGMYAmYoyWN8FD+voGuMwGyJa7onrc60q1Hu6QFsQXHVA==

ClientSecret

The encrypted Salesforce Client Secret. You can encrypt this value using the Cinchy CLI.

DyU1hqde3cWwkPOwK97T6rzwqv6t3bgQeCGq/fUx+tKI=

Username

The encrypted Salesforce username. You can encrypt this value using the Cinchy CLI.

dXNlcm5hbWVAZW1haWwuY29t

Password

The encrypted Salesforce password You can encrypt this value using the Cinchy CLI.

cGFzc3dvcmRwYXNzd29yZA==

InstanceAuthUrl

The authorization URL of the Salesforce instance.

https://login.salesforce.com/services/oauth2/token

{
 "ApiVersion": 47.0,
 "GrantType": "password",
 "ClientId": "Bn8UmtiLydmYQV6//qCL5dqfNUMhqchdk959hu0XXgauGMYAmYoyWN8FD+voGuMwGyJa7onrc60q1Hu6QFsQXHVA==",
 "ClientSecret": "DyU1hqde3cWwkPOwK97T6rzwqv6t3bgQeCGq/fUx+tKI=",
 "UserName": "dXNlcm5hbWVAZW1haWwuY29t",
 "Password": "cGFzc3dvcmRwYXNzd29yZA=="
 "InstanceAuthUrl": "https://login.salesforce.com/services/oauth2/token"
}

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
If more than one listener is needed for a real-time sync, configure it/them via the Listener Config table.
To run a real-time sync, enable your Listener from the Execution tab.

Salesforce Object (Bulk API)

Overview

Salesforce is a cloud-based CRM software designed for service, marketing, and sales.

Salesforce objects are database tables that permit you to store data that's specific to an organization. Salesforce objects are of two types:

Standard Objects: Standard objects are the kind of objects that are provided by salesforce.com such as users, contracts, reports, dashboards, etc.
Custom Objects: Custom objects are those objects that are created by users. They supply information that's unique and essential to their organization. Custom objects are the heart of any application and provide a structure for sharing data.

The Salesforce Object (Bulk API) source supports batch syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

Salesforce Bulk API

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

Salesforce Object (Bulk API)

Object

Mandatory. The name of your Salesforce Object.

Auth URL

Mandatory. The URL that issues your Salesforce auth token.

Client ID

Mandatory. The encrypted Client ID to connect to your Salesforce Object. The Connections UI will automatically encrypt this value for you.

Client Secret

Mandatory. The encrypted Client Secret for the above Client ID. The Connections UI will automatically encrypt this value for yo.

Username

Mandatory. The encrypted Username of an account that can connect to your Salesforce Object. The Connections UI will automatically encrypt this value for you.

Password

Mandatory. The encrypted Password associated with the above account that can connect to your Salesforce Object. The Connections UI will automatically encrypt this value for you.

Test Connection

You can use the "Test Connection" button to ensure that your credentials are properly configured to access your source.

If configured correctly, a "Connection Successful" pop-up will appear.

If configured incorrectly, a "Connection Failed" pop-up will appear along with a link to the applicable error logs to help you troubleshoot.

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
Click Jobs > Start a Job to begin your sync.

Salesforce platform event

1. Overview

Salesforce is a cloud-based CRM software designed for service, marketing, and sales.

Salesforce Platform Events are secure and scalable messages that contain data. Publishers push out event messages that subscribers receive in real time.

The Salesforce Platform Event source supports real-time syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

Salesforce Platform Event

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

Salesforce Platform Event

To set up a real-time sync, you must configure your Listener values. You can do so through the Connections UI.

Note that If there is more than one listener associated with your data sync, you will need to configure the addition listeners via the Listener Configuration table.

Reset behaviour

Parameter

Description

Example

Auto Offset Reset

None

Topic JSON

The below table can be used to help create your Topic JSON needed to set up a real-time sync.

Parameter

Description

Example

Name

Mandatory. The name of the Platform Event, as it appears in Salesforce, that you want to subscribe to.

Notification__e

Example Topic JSON

{  
        "Name": "Notification__e"
}

Connection attributes

The below table can be used to help create your Connection Attributes JSON needed to set up a real-time sync.

Parameter

Description

Example

ApiVersion

Mandatory. Your Salesforce API Version. Note that this needs to be an exact match; for instance 47.0 can't be written as simply 47.

47.0

GrantType

This value should be set to password.

password

ClientId

The encrypted Salesforce Client ID. You can encrypt this value using the Cinchy CLI.

Bn8UmtiLydmYQV6//qCL5dqfNUMhqchdk959hu0XXgauGMYAmYoyWN8FD+voGuMwGyJa7onrc60q1Hu6QFsQXHVA==

ClientSecret

The encrypted Salesforce Client Secret. You can encrypt this value using the Cinchy CLI.

DyU1hqde3cWwkPOwK97T6rzwqv6t3bgQeCGq/fUx+tKI=

Username

The encrypted Salesforce username. You can encrypt this value using the Cinchy CLI.

dXNlcm5hbWVAZW1haWwuY29t

Password

The encrypted Salesforce password You can encrypt this value using the Cinchy CLI.

cGFzc3dvcmRwYXNzd29yZA==

InstanceAuthUrl

The authorization URL of the Salesforce instance.

{
 "ApiVersion": 47.0,
 "GrantType": "password",
 "ClientId": "Bn8UmtiLydmYQV6//qCL5dqfNUMhqchdk959hu0XXgauGMYAmYoyWN8FD+voGuMwGyJa7onrc60q1Hu6QFsQXHVA==",
 "ClientSecret": "DyU1hqde3cWwkPOwK97T6rzwqv6t3bgQeCGq/fUx+tKI=",
 "UserName": "dXNlcm5hbWVAZW1haWwuY29t",
 "Password": "cGFzc3dvcmRwYXNzd29yZA=="
 "InstanceAuthUrl": "https://login.salesforce.com/services/oauth2/token"
}

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

## Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
If more than one listener is needed for a real-time sync, configure it/them via the Listener Config table.
To run a real-time sync, enable your Listener from the Execution tab.

Snowflake

Overview

Snowflake is a fully managed SaaS that provides a single platform for data warehousing, data lakes, data engineering, data science, data application development, and secure sharing and consumption of real-time/shared data.

Snowflake enables data storage, processing, and analytic solutions.

The Snowflake source supports batch syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

Snowflake Sync

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

Snowflake

Connection String

Unencrypted example: account=wr38353.ca-central-1.aws;user=myuser;password=mypassword;db=CINCHY;schema=PUBLIC

Object

Mandatory. Select either Table or Query.

Table or Query.

Table

Appears if Object = Table. The name of the Table in Snowflake that you wish to sync.

Query

Appears if Object = Query. A SELECT statement query that will be used to fetch your data.

Test Connection

You can use the "Test Connection" button to ensure that your credentials are properly configured to access your source.

If configured correctly, a "Connection Successful" pop-up will appear.

If configured incorrectly, a "Connection Failed" pop-up will appear along with a link to the applicable error logs to help you troubleshoot.

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
Click Jobs > Start a Job to begin your sync.

SAP SuccessFactors

Overview

SAP SuccessFactors solutions are cloud-based HCM software applications that support core HR and payroll, talent management, HR analytics and workforce planning, and employee experience management.

The SAP SuccessFactors source support batch syncs.

Info tab

You can find the parameters in the Info tab below (Image 1).

Values

Parameter

Description

Example

Title

Mandatory. Input a name for your data sync

SAP SuccessFactors

Variables

Permissions

Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.

Source tab

The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).

The following parameters will help to define your data sync source and how it functions.

Parameter

Description

Example

Source

Mandatory. Select your source from the drop down menu.

SAP SuccessFactors

API Key

Mandatory. The encrypted API Key needed to connect to your SuccessFactors entity. The Connections UI will automatically encrypt this value for you.

User ID

Mandatory. A User ID with access to connect to your SuccessFactors entity.

Company ID

Mandatory. The Company ID associated with the above User ID and with access to connect to your SuccessFactors entity.

SAML Assertion URL

Mandatory. The URL used for SAML insertion.

OAUTH Token URL

Mandatory. The URL that issued your oauth token.

Private Key

Mandatory. The key to connect to your SuccessFactors entity. The Connections UI will automatically encrypt this value for you.

OData API URL

Mandatory. The URL of the OData API.

Entity

Mandatory. The name of your SuccessFactors entity.

The Schema section is where you define which source columns you want to sync in your connection. You can repeat the values for multiple columns.

Parameter

Description

Example

Name

Mandatory. The name of your column as it appears in the source.

Name

Alias

Optional. You may choose to use an alias on your column so that it has a different name in the data sync.

Data Type

Mandatory. The data type of the column values.

Text

Description

Optional. You may choose to add a description to your column.

Select Show Advanced for more options for the Schema section.

Parameter

Description

Example

Mandatory

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Mandatory is checked on a column, then all rows are synced with the execution log status of failed, and the source error of "Mandatory Rule Violation"

If just Validated is checked on a column, then all rows are synced.

Validate Data

If both Mandatory and Validated are checked on a column, then rows where the column is empty are rejected

If just Validated is checked on a column, then all rows are synced.

Trim Whitespace

Optional if data type = text. For Text data types, you can choose whether to trim the whitespace._

Max Length

You can choose to add in a Transformation > String Replacement by inputting the following:

Parameter

Description

Example

Pattern

Mandatory if using a Transformation. The pattern for your string replacement.

Replacement

What you want to replace your pattern with.

Note that you can have more than one String Replacement

You have the option to add a source filter to your data sync. Please review the documentation here for more information on source filters.

Next steps

Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
Click Jobs > Start a Job to begin your sync.