5️⃣
Cinchy Platform Documentation
Cinchy v5.8
Cinchy v5.8
  • Data Collaboration Overview
  • Release notes
    • Release notes
      • 5.9 release notes
      • 5.8 Release Notes
      • 5.7 Release Notes
      • 5.6 Release Notes
      • 5.5 Release Notes
      • 5.4 Release Notes
      • 5.3 Release Notes
      • 5.2 Release Notes
      • 5.1 Release Notes
      • 5.0 Release Notes
  • Support
  • Glossary
  • FAQ
  • Deployment guide
    • Deploying Cinchy
      • Plan your deployment
        • Deployment architecture
          • Kubernetes architecture
          • IIS architecture
        • Deployment prerequisites
          • Single Sign-On (SSO) integration
            • Enable TLS 1.2
            • Configure ADFS
            • AD Group Integration
      • Kubernetes
        • Disable your Kubernetes applications
        • Change your file storage configuration
        • Configure AWS IAM for Connections
        • Use Self-Signed SSL Certs (Kubernetes)
        • Deploy the CLI (Kubernetes)
      • IIS
  • Upgrade guide
    • Upgrade Cinchy
      • Cinchy Upgrade Utility
      • Kubernetes upgrades
        • v5.1 (Kubernetes)
        • v5.2 (Kubernetes)
        • v5.3 (Kubernetes)
        • v5.4 (Kubernetes)
        • v5.5 (Kubernetes)
        • v5.6 (Kubernetes)
        • v5.7 (Kubernetes)
        • v5.8 (Kubernetes)
        • Upgrade AWS EKS Kubernetes version
        • Update the Kubernetes Image Registry
        • Upgrade Azure Kubernetes Service (AKS)
      • IIS upgrades
        • v4.21 (IIS)
        • v4.x to v5.x (IIS)
        • v5.1 (IIS)
        • v5.2 (IIS)
        • v5.3 (IIS)
        • v5.4 (IIS)
        • v5.5 (IIS)
        • v5.6 (IIS)
        • v5.7 (IIS)
        • v5.8 (IIS)
      • Upgrading from v4 to v5
  • Guides for using Cinchy
    • User Guide
      • Data Browser overview
      • The Admin panel
      • User preferences
        • Personal access tokens
      • Table features
      • Data management
      • Queries
      • Version management
        • Versioning best practices
      • Commentary
    • Builder Guide
      • Best practices
      • Create tables
        • Attach files
        • Columns
        • Data controls
          • Data entitlements
          • Data erasure
          • Data compression
        • Formatting rules
        • Indexing & partitioning
        • Linking data
        • Table and column GUIDs
        • System tables
      • Delete tables
        • Restore tables, columns, and rows
      • Saved queries
      • CinchyDXD
        • Overview
        • DXD workflow
        • Package the data experience
        • Install the data experience
        • Release package
        • Changelog
        • References
          • Cinchy DXD CLI reference
          • Data Experience Definitions table
          • Data Experience Reference table
      • Multilingual support
      • Integration guides
    • Administrator Guide
    • Additional guides
      • Monitor and Log on Kubernetes
        • Grafana
        • OpenSearch dashboards
          • Set up Alerts
        • Monitor via ArgoCD
      • Maintenance
      • Cinchy Secrets Manager
      • GraphQL (Beta)
      • System properties
      • Enable Data At Rest Encryption (DARE)
      • Application experiences
        • Network map
          • Custom node results
          • Custom results in the Network Map
        • Set up experiences
  • API Guide
    • API overview
      • API authentication
      • API saved queries
      • ExecuteCQL
      • Webhook ingestion
  • CQL
    • Overview
      • CQL examples
      • CQL statements overview
        • Cinchy DML statements
        • Cinchy DDL statements
      • Cinchy supported functions
        • Cinchy functions
        • Cinchy system values
        • Cinchy User Defined Functions (UDFs)
          • Table-valued functions
          • Scalar-valued functions
        • Conversion functions
        • Date and Time types and functions
          • Return System Date and Time values
          • Return Date and Time parts
          • Return Date and Time values from their parts
          • Return Date and Time difference values
          • Modify Date and Time values
          • Validate Date and Time values
        • Logical functions
        • Math functions
        • String functions
        • Geometry and Geography data type and functions
          • OGC methods on Geometry & Geography instances
          • Extended methods on Geometry & Geography instances
        • Full Text Search functions
        • Connections functions
        • JSON functions
    • CQL functions reference list
  • Meta-Forms
    • Introduction
    • Install Meta-Forms
      • Deploy Meta-Forms (Kubernetes)
      • Deploy Meta-Forms (IIS)
    • Forms data types
    • Meta-Forms Builder Guide
      • Create a dynamic meta-form with tables
      • Create a dynamic meta-form example with Form Designer
      • Add links to a form
      • Rich text editing in forms
  • Data syncs
    • Get started with data syncs
    • IIS installation
      • Install Connections
      • Install the Worker/Listener
      • Install the Connections CLI
    • Build data syncs
      • Data sync types
      • Design patterns
      • Sync actions
      • Columns and mappings
        • Calculated column examples
      • Advanced settings
        • Filters
        • Variables
        • Auth requests
        • Request headers
        • Post sync scripts
        • Pagination
      • Batch data sync example
      • Real-time sync example
      • Schedule a data sync
      • Connection functions
    • Data sync sources
      • Cinchy Event Broker/CDC
        • Cinchy Event Broker/CDC XML config example
      • Cinchy Table
        • Cinchy Table XML config example
      • Cinchy Query
        • Cinchy Query XML config example
      • Copper
      • DB2 (query and table)
      • Dynamics 2015
      • Dynamics
      • DynamoDB
      • File-based sources
        • Binary file
        • Delimited file
        • Excel
        • Fixed width file
        • Parquet
      • Kafka Topic
        • Kafka Topic example config
        • Apache AVRO data format
      • LDAP
      • MongoDB collection
        • MongoDB collection source example
      • Mongo event
      • MongoDB collection (Cinchy event)
      • MS SQL Server (query and table)
      • ODBC Query
      • Oracle (query and table)
      • Polling event
        • Polling event example config
      • REST API
      • REST API (Cinchy event)
      • SAP SuccessFactors
      • Salesforce Object (Bulk API)
      • Salesforce platform event
      • Salesforce push topic
      • Snowflake
        • Snowflake source example config
      • SOAP 1.2 web service
      • SOAP 1.2 web service (Cinchy Event Triggered)
    • Data sync destinations
      • Cinchy Table
      • DB2 table
      • Dynamics
      • Kafka Topic
      • MongoDB collection
      • MS SQL Server table
      • Oracle table
      • REST API
      • Salesforce
      • Snowflake table
      • SOAP 1.2 web service
    • Real-time sync stream sources
      • The Listener Config table
      • Cinchy Event Broker/CDC
      • Data Polling
      • Kafka Topic
      • MongoDB
      • Salesforce Push Topic
      • Salesforce Platform Event
    • CLI commands list
    • Troubleshooting
  • Other Resources
    • Angular SDK
    • JavaScript SQK
Powered by GitBook
On this page
  • Overview
  • Full file sync
  • Sync key
  • Sync Record Behaviour
  • Delta Sync
  • Appendix A
  • Full File Sync - Conditional Changed Record Behaviour
  1. Data syncs
  2. Build data syncs

Sync actions

PreviousDesign patternsNextColumns and mappings

Last updated 1 year ago

Overview

When configuring a data sync you must set your sync behaviour. You have two options for this: Full File or Delta.

Full File syncs intake both the source and the destination data and reconcile the records by matching up the sync key. This determines any differences and allows it to perform updates, inserts, ignores, or deletes at the destination.

Delta syncs skip the reconciliation process. In batch syncs, it simply grabs records from the source and inserts it into the destination. In real-time syncs, it may act differently depending on the event type. For example, when using the Cinchy Event Broker/CDC with an insert event, a delta sync will insert the data into the destination, an update event will update, etc.

Delta syncs also have the option to provide an "Action Type Column" for REST API destinations. This reads the value of the source record from a specified column. If the value is INSERT, then it inserts the record, UPDATE, then it updates, DELETE, then it deletes.

Full file sync

When using the Full File synchronization strategy there are four distinct sections that must be configured: the Sync Key and the Sync Record Behaviours, which include actions for New, Dropped, and Changed records. (Image 1).

Sync key

The sync key is a unique key reference you use for data syncs from the data source into your destination. You can use it to match the data between the source and the target for updates on changed records.

To set this using a config XML, use the following guide:

  • Elements: <SyncKeyColumnReference>

    • This is used in the <SyncKey> element when specifying which columns in the Target Table to be utilized as a unique key for the syncing process.

  • Contained-In: <SyncKey>

  • Attributes: name. The name of a column in the destination that you are syncing data into.

<SyncKeyColumnReference
    name:"string">
</SyncKeyColumnReference>

Sync Record Behaviour

The Sync Record Behaviour divides into three subsections which defines what action to take on certain records (Image 2).

Values in the attributes section of the config XML for record behaviour are case sensitive.

New Record Behaviour

New Record Behaviour defines what action to take when a new record is found in the sync source. This can be either INSERT or IGNORE.

To set this using a config XML, use the following guide:

Attribute
Description
Values

type

The type defines the action upon the new record.

It can either be INSERT or IGNORE.

  • INSERT will insert the new record.

  • IGNORE will do nothing to the record.

<NewRecordBehaviour type="INSERT" />

Dropped Record Behaviour

Dropped Record Behaviour defines what action to take when a new record isn't found in the sync source, but exists in the target. This can be either DELETE, IGNORE, or EXPIRE.

To set this using a config XML, use the following guide:

Attribute
Description
Values

type

The type defines the action upon the dropped record.

It can either be IGNORE, EXPIRE, or DELETE.

  • IGNORE will do nothing to the record.

  • EXPIRE will populate a specified expiration timestamp field as the current time. An expirationTimestampField must be provided. This is a reference to a date/time column in the target that should be updated with the execution timestamp if the record is dropped.

  • DELETE will delete dropped records in the target data set.

expirationTimestampField

This attribute is only applicable if the type is equal to EXPIRE.

The expirationTimestampField is the name of an existing date field to be filled with the current time.

<DroppedRecordBehaviour
    type="EXPIRE"
    expirationTimestampField="string">
    ...
</DroppedRecordBehaviour>

2.2.3 Changed Record Behaviour

Changed Record Behaviour defines what action to take when a new record with a sync key is found in the sync source and also exists in the target. This can be either Update or Ignore.

To set this using a config XML, use the following guide:

Attribute
Description
Value

type

The type defines the action upon the new record.

It can either be UPDATE, IGNORE, or CONDITIONAL.

  • "IGNORE" will do nothing to the record.

  • "UPDATE" will update the record.

<ChangedRecordBehaviour type="UPDATE" />

Delta Sync

When using the Delta synchronization strategy there is one optional configuration that you can expose when running a sync with a REST API destination (Image 3).

The Action Type Column reads the value of the source record from a specified column. If the value is INSERT, then it inserts the record, UPDATE, then it updates, DELETE, then it deletes.

Appendix A

Full File Sync - Conditional Changed Record Behaviour

Added in Cinchy v5.6, the Changed Record Behaviour - Conditional feature allows you to define specific conditions upon which to update your records (Image 4).

  • You can add multiple Conditions to a single data sync by using the AND/OR and +Rule buttons.

  • You are able to group your Rules into a rule set by using the +Ruleset button.

  • If your Condition evaluates to true then it will update your records

  • Use the The left-most drop down to select either a source or a target column as defined in your Source and Destination tabs

  • Use The centre drop-down to select from the following options:

    • \=

    • !=

    • Contains

    • Is Null

    • Is Not Null

  • Use the The right-most drop-down to:

    • Add a plain value (ex: text, numerical, etc.) This will adjust based on the column data type picked in the left-most drop down. For example, if in the source schema the column is a date, then it renders a date picker.

    • Select either a source or a target column as defined in your Source and Destination tabs (when used in conjunction with the Use Columns checkbox)

For example, the below condition would only update records where the target column "Name" is null (Image 5).

"CONDITIONAL" will open a new UI section allowing you to define Conditions upon which to update your records. for more information on the Conditional behaviour.

Review Appendix A
Image 1: Full File Syncs
Image 2: Sync Record Behaviour
Image 3: Delta Syncs
Image 4: Conditional record behaviours
Image 5: Conditional Example