What’s the purpose of Dataware and Data Collaboration?

Dataware is the emergence of a common group of technologies that solve data related problems across many business use cases. One of the most exciting new categories within this group is Data Collaboration.

Data Collaboration solves the costly, time-consuming, and ineffective integration processes born from silo-ing your data in a traditional app-centric environment. Instead of your data serving your applications, data collaboration refocuses and pivots to a model where your data is at the forefront and is being served by your apps (Image 1).

Harnessing this power, Cinchy connects unlimited data sources within a networked architecture, offering persistent delivery of real-time solutions, without complex integrations. And the more data you connect into your data network, the more powerful your processes can be.

The root causes of IT delay and frustration

When a new IT project is green-lit, you often pay a hefty fine called the integration tax where you're continuously building new integrations between applications, just to reuse data that is already available in your systems.

Over time, this never-ending cycle of copying data between fragmented apps gets more complex, resulting in delayed launches, budget overruns, and “shadow IT” projects.

This process of making copies now consumes up to 50% of the resources on large IT projects, and it's the reason that delivery often takes months, sometimes years, and costs millions (Image 2).

How can this be fixed?

Data Collaboration: Using Dataware for accelerated solutions delivery

With data collaboration, you shift your approach from integration for data sharing, to access for data collaboration (Image 3).

For every app you build where you leverage data collaboration, you’re able to access the network to reuse information for future apps. Your IT team will find that previous projects have already connected many of the data sources they need to the network.

Cinchy's data collaboration platform does for application development what the power grid does for individual buildings. In the same way that buildings no longer need to generate their own power thanks to the power grid, with a data collaboration platform applications no longer need to manage, integrate, and protect their own data (Image 4).

Organizations can build data collaboration applications in half the time, while enabling effortless and copyless data sharing across applications. Using Cinchy is unique in that it eliminates the "integration tax" that today consumes half of most enterprise IT budgets -- that is to say, data collaboration makes integration obsolete (Image 5).

Now, instead of connecting apps to gain access to data through costly point-to-point integration, your apps serve your data by leveraging and connecting it all together via Cinchy. In this way, you can still gain the best usage out of your apps through zero-copy integration while avoiding the disadvantages of data silo-ing. You have both full access to and full control of your data (Image 6).

Not just connected, but autonomous.

​Simply putting pipes between data silos, and centralizing a few housekeeping tasks, is not data collaboration. What that's actually doing is leading you down a path of managing endless copies. True data collaboration not only connects your data but upgrades it as part of an interconnected, autonomous network.

Autonomous data exists independently of any application. It is self-controlled, self-protected, and self-describing. This creates a number of benefits compared to traditional app-dependent data, including the ability to simplify cross-application usage and reporting. And when you use autonomous data in an interconnected network, wherein individual contributors maintain their roles and priorities as they apply their unique skills and leadership autonomy in a problem solving process, you get: Collaborative Autonomy.

Collaborative Autonomy is thus the principle underpinning Collaborative Intelligence, the entire basis of Dataware and Cinchy.

Individuals are not homogenized, as in consensus-driven processes, nor equalized through quantitative data processing, as in collective intelligence. Consensus is not required. Problem resolution is achieved through systematic convergence toward coherent results. Collaborative intelligence relies on the principle of collaborative autonomy to overcome “the consensus barrier” and succeed where other methods have failed.

Universal access controls, automated governance

One of the most significant advantages of dataware is the ease with which data owners can set universal data access controls at the cellular level, and automate data quality (Data Governance) with a “golden record” of data.

In effect, it is removing the need to maintain access controls within individual apps and centralizing these functions in an incredibly efficient way.

Compare this with designing and maintaining controls within thousands of apps and systems independently. It’s not only incredibly challenging and costly, but virtually impossible to maintain consistency (Image 7).

Game Changer: Network Effects for IT delivery

Dataware is a game changer for IT delivery: it produces network effects, where each new solution actually speeds up delivery times and reduces costs

Network-based designs scale beautifully and become more efficient as they expand. Consider the human brain; its neuroplasticity helps it learn more as it grows. The more interconnected it gets, the better. The neural pathways are reorganizing themselves such that the fewer connections, the higher the intelligence, because information is more efficient to operationalize.

It's the same with dataware. The more you connect your data, the harder and better it works. It's the ability to have the platform take care of your whole data journey and transformation. You don't have to manage the changes of all your applications, regression testing, the QA you have to go through, etc.

And it's also your time machine - you can have applications based on different points in time of your data, and it's all done through network-based design.

Now that you know how efficient and secure things can be there is no going back.

Let's build the connected future, together.

Table of Contents

New Connector

Kafka

We are continuing to improve our Connections offerings, and we now support Kafka as a data sync target in Connections.

Apache Kafka is an end-to-end event streaming platform that:

• Publishes (writes) and subscribes to (reads) streams of events from sources like databases, cloud services, and software applications.

• Stores these events durably and reliably for as long as you want.

• Processes and reacts to the event streams in real-time and retrospectively.

Event streaming thus ensures a continuous flow and interpretation of data so that the right information is at the right place, at the right time for your key use cases.

For information on setting up data syncs with Kafka as a target, please review the documentation here.

New Inbound Data Format for Connections

Apache AVRO

We have also added support for Apache AVRO (inbound) as a data format and added integration with the Kafka Schema Registry, which helps enforce data governance within a Kafka architecture.

Avro is an open source data serialization system that helps with data exchange between systems, programming languages, and processing frameworks. Avro stores both the data definition and the data together in one message or file. Avro stores the data definition in JSON format making it easy to read and interpret; the data itself is stored in binary format making it compact and efficient.

Some of the benefits for using AVRO as a data format are:

• It's compact;

• It has a direct mapping to/from JSON;

• It's fast;

• It has bindings for a wide variety of programming languages.

For more about AVRO and Kafka, read the documentation here.

For information on configuring AVRO in your platform, review the documentation here.

Custom Results in the Network Map

Focus the results of your Network Map to show only the data that you really want to see with our new URL parameters.

You can now add Target Node, Depth Level, and Max Depth Level Parameters, if you choose.

Example: <base url>/apps/datanetworkvisualizer?targetNode=&maxDepth=&depthLevel=

• Target Node: Using the Target Node parameter defines which of your nodes will be the central node from which all connections branch from.

  • Target Node uses the TableID number, which you can find in the URL of any table.

  • Example: <base url>/apps/datanetworkvisualizer?targetNode=8 will show TableID 8 as the central node

• Max Depths: This parameter defines how many levels of network hierarchy you want to display.

  • Example: <base url>/apps/datanetworkvisualizer?maxDepth=2 will only show you two levels of connections.

• Depth Level: Depth Level is a UI parameter that will highlight/focus on a certain depth of connections.

  • Example: <base url>/apps/datanetworkvisualizer?DepthLevel=1 will highlight all first level network connections, while the rest will appear muted.

The below example visualizer uses the following URL: <base url>/apps/datanetworkvisualizer?targetNode=8&maxDepth=2&depthLevel=1

• It shows Table ID 8 ("Groups") as the central node.

• It only displays the Max Depth of 2 connections from the central node.

• It highlights the nodes that have a Depth Level of 1 from the central node.

Enhancements

• We have increased the length of the [Parameters] field in the [Cinchy].[Execution Log] to 100,000 characters.

• Two new parameters are now available to use in real time syncs that have a Cinchy Table as the target. @InsertedRecordIds() and @UpdatedRecordIds() can be used in post sync scripts to insert and update Record IDs respectively, with the desired value input as comma separated list format.

Bug Fixes

• We have fixed a bug that was preventing some new SSO users belonging to existing active directory groups from seeing tables that they should have access to.

• We have fixed a bug where a Webhook would return a 400 error if a JSON body was provided, and the key was in the query parameter of the HTTP request.

GraphQL

• We continue to optimize our GraphQL beta capabilities by improving memory utilization and performance.

Contacting Support

If at any point you require help or clarification on this documentation or your Cinchy platform, you can reach out to our Support team:

• Via email: support@cinchy.com

• Via phone: 1-888-792-6051

• Through the support portal: Support Portal

Table of Contents

Can I get a record count from a delimited file before running the CLI?

You can use PowerShell to count the lines in a delimited file and based on the result decide if you will run the CLI.

There is currently no way for you to find out how many records will be inserted/updated/deleted if you run a CLI without performing the sync.

Can I restore my deleted data?

If the record is still in the table, but has been overwritten by mistake, access your Collaboration Log for the row, and restore back to the correct version.

If your row has been deleted by mistake, access your Recycling Bin, locate the row and restore it.

Can I send multiple comma-delimited values to a query parameter? [i.e. 234,233,365 to be used in WHERE [Id] IN (@param)]

For example: 4,10,15 to be used in WHERE [Id] IN (@param)

This can be done by using parameters in {}, such as {0},{1},...

These will be replaced with the exact text when running the query.

For example: ​ SELECT * FROM [HR].[Employees] WHERE [Deleted] IS NULL AND [Employee ID] IN ({0}) (Image 1).

Cinchy dates are being saved as a 1900-01-01 when updating using a variable

When updating a date field using a variable, and no value is entered for that variable, the date field will be 1900-01-01. To avoid this, use a case statement to replace the empty string with NULL, as shown in the following example:

Does a row filter restrict access for a Cinchy administrator?

Currently Cinchy Administrators have access to view/edit/approve all data in the platform. There is no way currently to restrict access for Cinchy administrators.

A workaround is to create a separate "administrators" group which has edit access to all Cinchy system tables, and just leave the "admin" user account or superadmins as "Cinchy administrators."

How can I automatically check if a CLI data sync was successful or failed?

You can check if a data sync was successful by its exit code. Below is sample code in Powershell to check for the exit code and what they mean.

From the command prompt the following will also return the error code:

How can I check for platform errors?

In Version 4:

First check <base URL>/elmah, which stores web-related induced errors.

Then check the logs, which can be accessed from <base URL>/admin/index.

Cinchy logs will contain all exceptions thrown by the Cinchy Web application. This includes failed queries, stack overflows and much more.

CinchySSO logs will contain IDP errors.

How can I enter a new line into a field in Manage Data?

You can add line breaks in a cell on the UI, the same way as in Excel, by typing Alt+Enter. If you use the expanded row heights option, or manually expand the row, it will show the line breaks.

How can I prevent wrong data loading in from external applications?

You can do the following to preview your changes:

• Create staging tables to validate the data first.

• Use formatting rules in Cinchy, to highlight data that is not valid.

• Configure a CLI using a Cinchy Query source to move the data from the staging tables to the permanent tables.

How can I see who has modified my data?

Right click on the row you want additional information and select the Collaboration Log.

You can also add the "Modified By" and "Modified" columns into the current view/to your query if you want to see it for multiple rows at once.

How do I create a Cinchy user with a set password?

One Time setup:

1. Open the Users table

2. For the password of this user, copy the admin user's password and paste it into the Password field of "defaultuser".

3. Set the Password Expiration Timestamp to today

4. In an Incognito browser, navigate to the Cinchy website

5. Sign in as defaultuser with the admin user password

6. Cinchy will ask you to change the password for defaultuser, change it to a default password you will give out every time you create an account.

7. In the original session window, refresh the Users table and remove the Password Expiration Timestamp for defaultuser.

Each time, for new users:

1. Open the Users table

2. Create the new user, for example "sandip"

3. For the password of this user, copy the "defaultuser" password and paste it into the Password field of "sandip".

4. Set the Password Expiration Timestamp to today

5. Give the user their username and friendly password created in step 7 above. They will be asked to change their password on first sign in.

How do I get the change history through CQL?

You write the query for the records for which you want the change history, including system columns like [Version], [Created], ... and the columns for which you like to see the changes.

You can add an ORDER BY [Version] (either ASC or DESC)

Then you change the query return type to "Query Results (Including Version History )"

The following query will show when the Cinchy instances were upgraded.

How do I insert, update, and delete links in a multi-select field using CQL?

Removing and updating a multi-select a link, is the same as setting the link field. The field needs to be updated with the list of values.

The value is a concatenated string of '[Cinchy Id],[Version],[Cinchy Id],[Version],[Cinchy Id],[Version]' from the lookup values

In this example it would set [Multi-Link Field Name] to values with [Cinchy Id] 1, 2, and 3. The version after each Cinchy Id should be 1.

How do I know which version of Cinchy I am running?

Navigate to <baseURL>/healthcheck

(ex. if your current URL is https://cinchy.mycompany.com/Tables/123?viewId=0 then you would navigate to https://cinchy.mycompany.com/healthcheck)

The response looks this:

In this case your Cinchy version is 4.14.0.0

If you would like to use the health check link for monitoring of the Cinchy application you can add "return503OnFailure=true" to the URL

How do I map a parameter's value to one of my target columns?

Use the model loader to load it back in the system (/apps/modelloader).

You create a calculated column in the source and give it the value of the parameter.

For each table, export and import the data via the UI.

Then map the calculated source column to the target. The order of the columns in the source is important. If your source is a file, put the calculated columns at the end in the source, after all the actual columns in the file.

How do I parse a pipe delimited file using the CLI?

Set the delimiter to "|".

How do I remove the leading 0 from an incoming field using the CLI?

This can be done by using Transformations in the sync configuration of a column. Here is an example:

The pattern contains a regular expression:

^ - anchor for the beginning of the string

0 - the string to replace

* - quantifier to be applied to 0 or more occurrences

How do I clone a Cinchy table?

1. Export the Model to XML from the Design Table info tab

2. Open the exported model in an editor and change the name of the model

3. Change the name of the table

4. remove the guids from the table in the model and save the file

5. Use the modelloader at <cinchy base URL>/apps/modelloader to upload the modified model

6. Export the data from the Manage Data screen of the initial table and import it in the new table

How do I clone a domain?

Table only instructions:

1. Create a dummy Data Experience and add all your tables from your domain to it (Image 2).

2. Hit the following endpoint with the GUID in your row:

<CinchyURL>/api/createdxversion?guid=<GUID>

I am unable to use COALESCE with a link column in a calculated column

If [Person 1] and [Person 2] are Link columns and [Member] is a Text column, a calculated column with the following expression will fail to save:

COALESCE([Person 1],[Person 2],[Member])

Please cast the link columns to VARCHAR:

COALESCE(CAST([Person 1] AS VARCHAR(50)),CAST([Person 2] AS VARCHAR(50)),[Member])

I cannot disable Change Approvals

This is caused by records in Draft status. To retrieve these records, run a query with return type Query Results (Including Draft Data).

After approving these records, you will be able to disable change approval.

I cannot find the [Cinchy].[Table Access Control] table

The [Cinchy].[Table Access Control] table does not show in the Market Place, but you can query for the data in the table.

I cannot find the column I want to link to even though the column is present in the table

In this example it would set [Multi-Link Field Name] to values with [Cinchy Id] 1, 2, and 3. The version after each Cinchy Id should be 1."

Columns do not "Allow Linking" by default. Check the properties of the column in the original table and make sure that in “Show Advanced” the “Allow Linking” checkmark box is selected. If you do not have Design Table access to that table, you will need to ask someone who does to do it.

I have access to a table but I cannot see any rows

There are a couple reasons why you may not be able to see any rows:

• View Filter

• Data access controls

• Error with the View or Table

View Filter

Check the All Data view and see if there is data there, if that is the case but a particular view has no rows, there could be a filter on the view. For example, if there is a "Due Soon" or "My Actions" view, it could just be that there are no records assigned to you that require actioning.

Data Access Controls

Access controls set on the table could cause you to have access to 0 records. Since you are able to set row level filters in Cinchy, it may be the case that the permissions of the table has not changed, but the data has changed such that you no longer have permission or vice versa.

Error

There may be an error on the view. If the bottom of the page does not show 0 records then there may be an error on the page (Image 3).

Is it possible to correct/replace a table or column's GUID?

It can be done. It is very unlikely that the GUID you want to change to is already allocated, but you should still check. Filter the [Cinchy].[Table Columns] for the new GUID. You should not find it. Then replace it in two places:

• the json field in [Cinchy].[Tables] - replace it in the column definition

• the GUID field in [Cinchy].[Table Columns]

To replace the table GUID, replace it in the json in [Cinchy].[Tables] and in the GUID field in [Cinchy].[Tables].

When you are done, restart the Cinchy UI. ​

My Insert/Update statement is making multiple changes instead of just one

A query like the following will cause multiple inserts instead of one if your result type is set to Query Results instead of # of Rows Affected.

The same applies to UPDATE statements.

If you need to perform inserts and updates in a query and want to return data at the end, another option is to use the "Single value (First Column of First Row)" return type, which will only be able to return a single value.

My query parameter is not working

When I pass a value to the following query, the result is empty.

The query works without the DECLARE statement. When the DECLARE statement is present, the input variable is ignored, and needs to be SET. In order to still get the variable from the input, a second variable is needed.

Null values are not updating correctly in Salesforce using the data sync

When performing a data sync with a Salesforce target, you need to replace nulls with '#N/A' in the source. You can use ISNULL([Column],'#N/A') in the source query. The following is a link to the Salesforce documentation related to this topic:

Passing Parameters to a query called With Exec

Declare and set the parameters before invoking the query:

[HR].[Employees and Departments] is:

Some of the columns have been rearranged under the default "All Data" view

The default All Data view displays the columns in the same order as in Design Table. But you can create a view and change the columns displayed and their order.

The multi-select option on the Link Column is disabled

Once link column is added to a table and saved, the multi-select checkbox should be disabled. If you need to change the option, you need to rename the column and create a new link column with the correct option.

What permissions are needed for a user to be able to create and edit views?

The user needs to have "Design Table" permissions granted for the table where he/she will create or edit views and also needs to have the "Can Design Tables" checked in the [Cinchy].[Users] table.

Table of Contents

New Connector

• A new connector has been added to the Cinchy Connections experience: you can now use as a source and target connector when performing data syncs. You can review the documentation on connecting as a source and as a target .

GraphQL API Beta Release

• Our GraphQL API provides a complete and understandable description of your data and gives you the power to ask for exactly what data you need and nothing more all in a single request, while leveraging the existing ecosystem of GraphQL developer tools.

• This is a beta release that offers read-only queries. Future releases will include more query features and mutation support (writes).

Enhancements

Connections

• Performing Data Sync from Cinchy to Salesforce no longer requires write access to the sync key column. This means that you can maintain your Salesforce environment and security protocols without needing to either modify them or create additional attributes, for your sync to work.

Data Browser

• We have added WCAG 2.1 AA Accessibility fixes to improve screen-reader performance and keyboard navigation accessibility.

• We’ve implemented a new loading screen for when Cinchy is installing and initializing.

Meta Forms

• We have improved the performance of Meta Forms by reducing the rendering time and adding visual guides to help you see which form sections have completed loading.

• Date fields with custom display formats will now render correctly, as opposed to showing up in mm/dd/yyyy format by default.

• In forms that have a 1:1 parent/child hierarchy, we have added the option to render the child form as a flattened form, instead of in a table grid.

• To improve UI consistency across Forms, the record selection drop down will now appear even if no records exist in the destination table.

CQL Updates

Bug Fixes

• We have fixed an error where scrolling in a table with a file column in certain situations prevents the UI from rendering all the data.

• We have fixed an issue where sorting by columns with a ‘%’ in the column name caused the rows not to sort correctly in the UI.

• We have fixed a bug in Meta Forms that prevented queries in child form filters from working as expected when using OR conditions.

Table of Contents

Another Move Toward Infinite Scalability

We have increased the number of possible Cinchy IDs that can be generated. This in turn allows the creation of more records within one table, so that you can create and manage larger data sets.

Previous Limit: 2,147,483,647 (2^31-1) Cinchy IDs per table

Updated Limit: 9,223,372,036,854,775,807 (2^63-1) Cinchy IDs per table

For backward compatibility with your database, you will need to manually run the below script against your TSQL or PGSQL databases. For instructions on how to run this upgrade,

Connections Enhancements

• We have added new optimizations and quicker processing of Kafka messages for real time data syncs in Connections.

• For added security, any logged password or sensitive parameters from the request details of a SOAP connector data sync is now redacted in the logs.

• Dead messages in the event listener are now written out to the execution errors table for easy collection and querying.

• The Connections experience now supports sourcing file based data sources from Azure Blob Storage and Amazon S3.

General Enhancements

• We have made improvements to the Files API to avoid cache build up and optimize the API.

• General security fixes and updates.

• We have added anonymous API access to GraphQL, no token required.

• We have added write operations to our GraphQL beta, meaning that you can now insert and modify data.

Bug Fixes

• We have fixed a UI instability bug that resulted in the inability to resize view panes in the query designer and difficulty in selecting any cell in the first row of a table. This bug was affecting Chromium users (Google Chrome, Brave, Microsoft Edge, etc.) who had recently updated their internet browsers.

• We have fixed a bug that caused the “Created” column to incorrectly display as the last approved date of the column instead of the column created date.

• You can now add/remove a column from a table that has a columnar index without needing to remove said index entirely. We have also fixed a bug that prevented users from reapplying their columnar index to a table once it had been removed.

• We have fixed an issue where an “Unsupported Function Call” error was raised in certain situations when using the REPLACE function in conditional calculated columns in Connections.

• We have fixed a bug that caused unnecessary updates to the Users table when a user’s Language and Region was not set.

• We have fixed a bug that was causing some GUID calculated columns to appear as blank, such as in the Integrated Clients table. If you are experiencing this bug, a manual update on the affected rows, either through the UI or through an UPDATE query, will resolve it.

• We have fixed an issue where the '&' in links was sometimes showing up as 'amp&' in the table view instead. This fix will only appear for customers on Postgres/Microsoft SQL servers 2017 or higher.

• We have fixed an issue where certain Update statements on multi-select link columns were failing to properly update with the link values specified. This bug was affecting statement with long strings done via the API.

• We have fixed a bug that was causing the Event Listener to pick up and process messages from deleted configs from the LIstener Configs table.

• We have fixed a bug that was causing an InvalidOperationException when executing a POST request to a Saved Query API.

• We have fixed a bug that was throwing errors on reconciliation when Data Syncs compared Text Conditional Calculated Columns to Links (PGSQL).

• FOR JSON PATH now works as expected in Postgres deployments.

Introduction

This section of Cinchy's documentation will guide you through the deployment process for Cinchy version 5: from planning all the way through to installation and upgrades.

• If you are looking to deploy Cinchy v5, please start here and read through all the sub-pages: Deployment Planning Overview and Checklist

  • Once you have familiarized yourself with the above documentation, you may move on to either of the below guides, depending on your preference:

    • Kubernetes Deployment Installation

    • IIS Deployment Installation

• If you are a customer currently on v4 and want to upgrade to v5, start here: Upgrading from v4 to v5

If you have any questions about the processes outlined in this section, please reach out to our Support team:

• Via email: support@cinchy.com

• Via phone: 1-888-792-6051

• Through the support portal: Support Portal

1. Overview

There are various things to consider before deploying Cinchy v5.

The pages in this Deployment Planning section will guide you through the considerations you should think about and the prerequisites that you should implement before deploying version 5 of the Cinchy platform.

The pages in this section include:

• Deployment Architecture Overview: This page explores your two high-level options for deploying Cinchy, on Kubernetes or on VM, and why we recommend a Kubernetes deployment. It also walks you through the important decision of selecting a database to run your deployment on, as well as some sizing considerations.

  • Kubernetes Deployment Architecture: This page provides Infrastructure (for both Azure and AWS), Cluster, and Platform component overviews for Kubernetes deployments. It also guides you through considerations concerning your cluster configuration.

  • IIS Deployment Architecture: This page provides Infrastructure and Platform component overviews for IIS (VM) deployments.

• Deployment Prerequisites: This page details important prerequisites for deploying Cinchy v5.

2. Deployment Planning Checklist

We have provided the following checklist for you to use when planning for your Cinchy v5 deployment. Each item is linked to the appropriate documentation page to provide more insight and clarity.

• Will you run on Kubernetes or a VM?

The main differences between a Kubernetes based deployment and an IIS deployment are:

• Kubernetes offers the ability to elastically scale.

• IIS limits certain components to running single instances.

• As all caching is in memory in an IIS deployment, in the event that multiple instances are stood up for redundancy there is point to point communication between them (http requests on the server IPs) required to maintain the cache.

• Performance is better on Kubernetes because of Kafka/Redis

• Prometheus/Grafana and Opensearch are not available in an IIS deployment

• The Maintenance CLI runs as a cronjob in Kubernetes while this needs to be orchestrated using a scheduler for an IIS deployment.

• Upgrades are simpler with the container images on Kubernetes.

If you will be running on Kubernetes, please review the following checklist:

• Which database will you use?

• Define your sizing.

• Define your cluster configuration.

• How many clusters will you need?

  • Will you be sharing from an existing cluster?

  • Will you be running multiple environments on a single cluster?

  • Will you be using Cinchy's recommended cluster components or your own?

• Define your object storage requirements.

  • Create an S3 compatible bucket.

• Create your SSL Certs (With the option to use Self-Signed).

• Define your Secrets Management, if desired.

• Define whether you will use Cinchy's Docker Images or your own.

  • If using Cinchy’s, pull the images.

• Access the deployment repositories and copy them into your own repo (Github or similar).

If you will be running on IIS, please review the following checklist:

• Ensure that you have an instance of SQL Server 2017+

• Ensure that you have a Windows Server 2012+ machine with IIS 7.5+ installed

• For v5.4+:

  • Install .net core Hosting bundle Version 6.0

    • Specifically, install: ASP.NET Core/.NET Core Runtime & Hosting Bundle

• For v5.0-5.3:

  • Install .net core Hosting bundle Version 3.1 - found here​

    • Specifically, install: ASP.NET Core/.NET Core: Runtime & Hosting Bundle​

• Ensure that you review the minimum web server hardware recommendations

• Ensure that you review the minimum database server hardware recommendations

• Define your application storage requirements.

• Ensure you have access to the release binary.

New Rich Text Editing Capabilities in Forms

Customize the appearance of your Form text with our new rich text editing capabilities. Enabling this on your text columns will give you access to exciting new formatting options previously unavailable in Forms such as:

• Bold, Italic, Underlined text

• Checklists

• Headers

• Hyperlinks

• etc.

For more information on how to make a visual impact with our new rich text editing capabilities, please review the documentation here.

Editable GUIDs

A GUID is a globally unique identifier, formatted as a 128-bit text string, that represents a unique ID. All Cinchy Tables and Columns have a GUID.

You now have the ability to display and edit table and column GUIDs within the Design Table screen.

This feature is particularly useful when deploying between Cinchy instances.

For example, in a model deployment, you must have matching GUIDs on your columns in order for them to properly load between environment A and environment B. There might be times when these GUIDs don’t automatically match, however, such as if you manually added a new column to environment B and also manually added it to environment A.

In this case, the two columns would have different GUIDs, and the model deployment would fail. With this new capability, however, you can match up conflicting GUIDs to properly load your model.

Polling Event Data Sync

Version 5.4 of the Cinchy platform introduces data polling, which uses the Cinchy Event Listener to continuously monitor and sync data entries from your SQLServer or DB2 server into your Cinchy table. This capability makes data polling a much easier, effective, and streamlined process and avoids implementing the complex orchestration logic that was previous necessary to capture frequently changing data.

You can read more about setting up Data Polling here.

BigInt Upgrade Utility

A mandatory database upgrade script was introduced in v5.2 that increased the number of possible Cinchy IDs that can be generated (read the releases note here). To streamline this process further, we have created a utility to deploy the changes. This should save you valuable time and resources when performing the upgrade, even on large databases.

For more information on the utility, please review the documentation here.

New Platform Default Timeout

For new environments (or if your setting was previously left blank), we have changed the Cinchy default session timeout from 30 minutes to 7 days. This will keep you logged in and working without interruptions. You can further change or revert this session timeout value in your appsettings.

In an IIS deployment, you can find the value in your CinchySSO > appsettings.json

“ConfigSettings”: {
	“AppSettings”: {
		"CinchyAccessTokenLifetime": "7.00:00:00",

In a Kubernetes deployment, you can find the value in your deployment.json file.

// The Cinchy session timeout
"cinchy_session_timeout" : "7.00:00:00",

Enhancements

• We have upgraded our application components to .NET 6.0 to ensure official Microsoft support for another 2 years. See the documentation here for more on .NET 6.

• We have added a silent refresh to the Connections experience to keep your session active while you're on the UI and to keep you working without interruptions.

• Real time data sync will now continue to retry if an "Out of Memory Exception" is thrown, avoiding unnecessary downtime.

• You now have the ability to choose between Debian or Alpine based Docker images when using a Kubernetes deployment of the Cinchy platform to be able to connect to a DB2 data source in Connections.

  • When either installing or upgrading your platform, you can use the following Docker image tags for the listener, worker, and connections:

    • Alpine: "5.x.x"

    • Debian: "5.x.x-debian"

• You now have the option to update the default passwords for Grafana and Opensearch in a Kubernetes deployment by configuring your deployment.json file. See here for instructions on updating Grafana and here for Opensearch.

• We have increased the average throughput for CDC subscriptions returning the Cinchy ID, so that it will now be able to process a greater number of events per second. Being able to reliably exceed 1000 events per second, based on the average use case, means that you can leverage the CDC capability for more demanding use cases.

• Prior to this release, the Files API could only handle files up to 100mb. We have now upped the maximum default file size to 1GB and have added a configurable property to allow you to set your own upload size should you wish.

  • In an IIS deployment, you can find the value in your Cinchy > appsettings.json

“ConfigSettings”: {
	“AppSettings”: {
		“MaxRequestBodySize”: 1073741824 // 1g

• In a Kubernetes deployment, you can find the value in your deployment.json file.

// this field defaulted at 1gb, dictates the max file upload size set in the web appsettings.json
"web_max_request_body_size" : 1073741824,

Bug Fixes

• We have fixed an error that occurred when attempting a data sync with conflicting target and source data types in link columns, where the error message would read: Value must be specified from the available options

• We have fixed an issue that was preventing new Connection jobs from starting when a previous job got stuck.

• We have fixed an issue where data syncs would fail if your sync key used a Target Column with a Link Column property that is different from the Primary Linked Column in the table definition.

• We have fixed a bug that was impacting write performance to tables on PostgreSQL with Data Change Notifications enabled.

• We have fixed a "cell entitlements failed" error on Forms that would occur if a Form column contained a single quote in the column name.

• We have fixed an issue on Forms where adding a [Created By] or [Modified By] field would return an error.

• The /healthcheck no longer redirects to the initialization screen during a Cinchy startup, allowing you to properly hit the endpoint.

Table of Contents

New Capabilities

• Cinchy on Kubernetes: We have added the ability for deploying Cinchy v5 on the Kubernetes system. Kubernetes is an open-source system that manages and automates the full lifecycle of container-based applications. You now have the ability to deploy Cinchy v5 on Kubernetes, and with it comes a myriad of features that help to simplify your deployment and enhance your scaling. Kubernetes can maximize your container capacity and easily scale up/down with your current operations.

  • Fluentbit, Opensearch: Available to those who deploy on Kubernetes, Fluentbit collects logs which are then displayed through the Opensearch visual dashboard, for all pods that write to stdout. This streamlines your search for information by putting the control into your hands and compiling your logs in one easy to access place — you can now easily write a query against all of your logs, in all of your environments. You will have access to a default configuration out of the box, but you can also customize your dashboards as well.

  • Grafana and Prometheus: With the Kubernetes addition, you now have access to Prometheus to collect your metrics. Prometheus records real-time metrics in a time series database used for event monitoring and alerting. You can then create custom dashboards through Grafana to display your data in an easy to use visual that makes reporting on your metrics easy, and even set up push alerts based on your custom needs.

  • PostgreSQL: You now have the option to deploy Cinchy on PostgreSQL, an open source alternative to the Microsoft SQL Server that can save you the cost of licensing fees. It is standards-compliant, reliable, highly programmable, and allows for concurrency. Utilizing this deployment makes Cinchy more affordable and scalable. We recommend Amazon Aurora for AWS users.

  • Kafka: Kafka is an open-source event streaming platform. This is designed to act as the middleware that allows for messaging between components through a queuing mechanism.

  • Redis: Redis is currently being used to facilitate a distributed lock using RedLock, which guarantees lock synchronizations across Cinchy instances. It is also a storage location for the execution output when running batch data syncs.

• All components have been transitioned from Log4Net to Serilog.

• The BuildIdentifier property from the appsettings.json will now appear in the healthcheck endpoint at the root level of the json payload, with a key of “buildidentifier”.

Enhancements

• Elmah was removed from the platform.

• Refactored hidden passwords in initialization.

• Added the ability to ingest S3 Datasources for delimited or parquet files.

• Improved performance of the Connection UI when there is a large (250+) number of columns/mappings.

• Optimization of bulk upsert performance.

• Added the following UI optimizations for handling large tables: default views set to collapsed, page size limited to 1k records, and added a button for getting the row count. Added the ability for Connections to read/write error files to S3 when an S3 bucket is specified.

• Added accessibility fixes.

• We have added support for PUT, PATCH, and DELETE in UDF Extensions in addition to GET/POST.

• You are able to override a Kafka topic within the appsettings for Connections, the Worker, and the Event Listener.

CQL Updates

• We have added support for the INSERT INTO SELECT statement, which copies data from one table and inserts it into another table. Click here for more information on INSERT INTO SELECT.

• When using Postgres, the SELECT @cinchy_row_id function will fail in queries. Instead, use the OUTPUT clause with INSERT, UPDATE, DELETE. Click here for more information on OUTPUT.

• We have added support for the TRUNCATE TABLE statement, which removes all rows from a table, without logging the individual row deletions. Click here for more information on TRUNCATE TABLE. (Please note that we do not support the "With Partitions" argument.)

Bug Fixes

• Fixed an issue where two users approving data could cause the row to become corrupted.

• Addressed a memory leak in query translation issue through the creation of a background task that removes expired objects from the cache.

• Fixed an issue where updating the formula of a non-cached calculated column wouldn’t reflect properly in the Table Columns table.

• Addressed an issue where changing a field in a table with multi-select links results in the removal of the field value from the version history.

• Fixed an API issue when updating UDF columns.

• Fixed an issue where numeric calculated columns that resolved off of a link column's numeric display column wouldn't work.

• Enablement of Content-Type headers to be added for REST API data syncs during GET requests.

Security Fixes

• Added frame-ancestors to the UI to prevent UI redress attacks.

• Implemented HSTS headers for when HTTPS is enabled on Cinchy.

Table of Contents

1. Infrastructure Configuration (On Cluster)

The below diagram shows a high level overview of a possible Infrastructure diagram with components on the cluster, however your specific configuration may vary (Image 1).

2. AWS Infrastructure Configuration (Outside Cluster)

When deploying Cinchy version 5 on Kubernetes, you may deploy via Amazon Web Services (AWS). The below diagram shows a high level overview of a possible AWS Infrastructure with components outside the cluster, however your specific configuration may vary (Image 2).

2.1 Infrastructure Component Overview

3. Azure Infrastructure Configuration (Outside Cluster)

When deploying Cinchy version 5 on Kubernetes, you may deploy via Microsoft Azure. The below diagram shows a high level overview of possible Azure Infrastructure with components outside the cluster, however your specific configuration may vary (Image 3).

3.1 Infrastructure Component Overview

4. Cluster Level Component Overview

The following highlighted area provides a high-level overview of cluster level components used when deploying Cinchy on Kubernetes, as well as what versions they are running.

These are created once per cluster. Clients may choose to run these components outside of the cluster or replace with their own comparable components. This diagram shows them in the cluster (Image 4).

Cluster Level Components

These are created once per cluster. Clients may choose to run these components outside of the cluster or replace with their own comparable components.

• Service Mesh - Istio: All inbound traffic to your Cinchy instance is routed and handled through this component, keeping it secure and managed.

• Monitoring/Alerting - Prometheus & Grafana: Prometheus consumes metrics from the running components in your environment, which can then be visualized into user friendly graphs and dashboards by Grafana. Prometheus can also connect to third party services to provide alerting capabilities. Both Prometheus and Grafana use persistent storage.

• Logging - Opensearch and Fluentbit: All logs are captured and indexed by Opensearch in a single, easily accessible location. These logs can be queried, searched, and filtered, and Correlation IDs mean that they can also be traced across various components. These logging components take advantage of persistent storage.

• Caching - Redis: Redis is currently being used to facilitate a distributed lock using RedLock, which guarantees lock synchronizations across Cinchy instances. It is also a storage location for the execution output when running batch data syncs.

• Event Processing - Kafka: This is designed to act as the middleware that allows for messaging between components through a queuing mechanism. Kafka features persistent storage.

4.1 Cluster Configuration

There are a few things to consider about your cluster configuration before you deploy Cinchy on Kubernetes:

• How many clusters will you need?

• Will you be sharing from an existing cluster?

• Will you be running multiple environments on a single cluster?

5. Instance Component Overview

Each instance of Cinchy has the following components, which are used to either provide an experience to users/applications or connect data in/out of Cinchy. Since multiple Cinchy instances can be deployed per cluster, these components will repeat for each environment.

The following highlighted area provides a high-level overview of instance level components used when running Cinchy on Kubernetes (Image 5).

• Meta Experiences: Cinchy offers pre-packaged experiences that you can import into your Cinchy environment and use on your data network. This includes experiences like Meta-Forms and Meta-Reports.

• Connections: The Cinchy Connections experience is used to help easily create data syncs in/out of the platform. It features persistent storage.

• Data Browser: Cinchy’s Dataware platform features a Universal Data Browser that allows users to view, change, analyze, and otherwise interact with all data on the network. The Data Browser even enables non-technical business users to manage and update data, build models, and set controls, all through an easy and intuitive UI.

• Identity Provider: An Identity Provider (IdP) creates and manages user credentials and associated identity attributes. IdPs authentication services are used in Cinchy to authenticate end-users.

• Event Listener: The Event Listener is used to picks up events from connected sources during a data sync. Review Cinchy's Data Sync documentation for further information on the Event Listener. The Event Listener uses persistent storage.

• Event Stream Worker: The Event Stream Worker is used to process data picked up by the Event Listener during data syncs. Review Cinchy's Data Sync documentation for further information on the Event Stream Worker. The Event Worker uses persistent storage.

• Maintenance (Batch Jobs): Cinchy performs maintenance tasks through the CLI. This currently includes the data erasure and data compression deletions.

6. GitOps

ArgoCD is a declarative, GitOps continuous delivery tool for Kubernetes that simplifies the application deployment and lifecycle management. ArgoCD is highly recommended for deploying Cinchy, however you may also choose to use another tool.

Once you configurations are set, ArgoCD automates the deployment of the desired application states into your specified target environments. Implemented as a Kubernetes controller, it continuously monitors running applications and compares the current, live state against the desired target state (as specified in your repositories).

Table of Contents

1. Kubernetes vs IIS

When choosing to deploy Cinchy version 5, you must decide whether to deploy via Kubernetes or on a VM (IIS).

Kubernetes is an open-source system that manages and automates the full lifecycle of container-based applications. You now have the ability to deploy Cinchy v5 on Kubernetes, and with it comes a myriad of features that help to simplify your deployment and enhance your scaling. Kubernetes can maximize your container capacity and easily scale up/down with your current operations.

• Grafana, Opensearch, Opensearch Dashboard: Working together, these three applications provide a visual logging dashboard for all of the information coming in from your database pods. This streamlines your search for information by putting the control into your hands and compiling your logs in one easy to access place — you can now easily write a query against all of your logs, in all of your environments. You will have access to a default configuration out of the box, but you can also customize your dashboards as well.

• Prometheus: With the Kubernetes addition, you now have access to Prometheus for your metrics dashboard. Prometheus records real-time metrics in a time series database used for event monitoring and alerting. You can then create custom dashboards to display your data in an easy to use visual that makes reporting on your metrics easy, and even set up push alerts based on your custom needs.

You also have the option to run Cinchy on Microsoft IIS, which was the traditional deployment method prior to Cinchy v5. Internet Information Services (IIS) for Windows Server is a flexible, secure and manageable Web server for hosting anything on the Web.

We recommend using Kubernetes to deploy Cinchy v5, because of the robust features that you can leverage, such as improved logging and metrics. Using Kubernetes allows for a greater ability to scale your Cinchy instances as well as the ability to lower your costs by using PostgreSQL.

2. Choosing a Database

Before deploying Cinchy v5, you must select which database you want to use.

The following list outlines which databases we support for Kubernetes Deployments.

For IIS Deployments please review the architecture requirements here.

2.1 Microsoft SQL Server

2.1.1 MS SQL Server

• Microsoft SQL Server is a relational database management system. As a database server, it is a software product with the primary function of storing and retrieving data as requested by other software applications—which may run either on the same computer or on another computer across a network.

• Resource limits

2.1.2 Azure SQL Database

• Microsoft Azure SQL Database is a managed cloud database provided as part of Microsoft Azure. It runs on a cloud computing platform, and access to it is provided as a service. Managed database services take care of scalability, backup, and high availability of the database.

• Differences between Azure SQL and Azure Managed SQL

• Resource Limits

2.1.3 Azure Managed SQL Instance

• SQL Managed Instance is a managed, always up-to-date SQL instance in the cloud that combines broad SQL Server engine compatibility with the benefits of a fully managed PaaS.

• Differences between Azure SQL and Azure Managed SQL

• Resource Limits

2.2 Amazon Aurora

Amazon Aurora

• Amazon Aurora (Aurora) is a fully managed relational database engine that's compatible with MySQL and PostgreSQL. It combines the speed and reliability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases. Aurora is part of the managed database service Amazon Relational Database Service (Amazon RDS). Amazon RDS is a web service that makes it easier to set up, operate, and scale a relational database in the cloud.

• Resource Limits

2.3 PostgreSQL

2.3.1 PostgreSQL

• PostgreSQL is a free and open-source relational database management system emphasizing extensibility and SQL compliance. PostgreSQL comes with features aimed to help developers build applications, administrators to protect data integrity and build fault-tolerant environments, and help you manage your data no matter how big or small the dataset.

2.3.2 AWS RDS for PostgreSQL

• Amazon RDS makes it easy to set up, operate, and scale PostgreSQL deployments in the cloud. With Amazon RDS, you can deploy scalable PostgreSQL deployments with cost-efficient and resizable hardware capacity. Amazon RDS manages complex and time-consuming administrative tasks such as PostgreSQL software installation and upgrades; storage management; replication for high availability and read throughput; and backups for disaster recovery. Amazon RDS for PostgreSQL gives you access to the capabilities of the familiar PostgreSQL database engine.

• Resource Limits

2.3.3 Azure PostgreSQL Database

• This is a fully managed and intelligent Azure Database for PostgreSQL. Enjoy high availability with a service-level agreement (SLA) up to 99.99 percent, AI-powered performance optimization, and advanced security. A fully managed database that automates maintenance, patching, and updates. Provision in minutes and independently scale compute or storage in seconds.

• Resource Limits

3. Sizing Considerations and Requirements

Prior to deploying Cinchy version 5, you need to define your sizing requirements.

3.1 Sizing

3.1.1 Kubernetes Sizing

Cluster sizing recommendations vary and are dependant on a myriad of deployment factors. We have provided the following very general sizing recommendations, but encourage you to explore more personalized options.

CPU: 8 Cores

Memory: 32GB Ram

Number of Servers: 3

AWS: m5.2xlarge

Azure: D8 v3

3.1.2 IIS Sizing

For sizing recommendations and prerequisites concerning an IIS deployment, please review the documentation found here.

3.2 Application Storage Requirements

If you are choosing to deploy Cinchy v5 on IIS, then you need to ensure that your VM disks have enough application storage to run your clusters.

3.3 Object Storage Requirements

Cinchy supports both Amazon S3 and Azure Blob Storage.

If you are using Terraform for your Kubernetes deployment, you will need to set up a new S3 compatible bucket to manually to store your state file. You will also need a bucket for Connections, to store error files created during data syncs.

You will create your two S3 compatible buckets using either Amazon or Azure. Ensure that you use the following convention when naming your buckets so that the automation script runs correctly: <org>-<component>-<cluster>. These bucket names will be referenced in your configuration files when you deploy Cinchy on Kubernetes.

Example Terraform Bucket: cinchy-terraform-state

Example Connection Bucket: cinchy-connections-cinchy-nonprod

There is no sizing definitions, as S3 provides unlimited scalability and it charges only for what you use/how much you store on it.

• How to set up an Amazon S3 bucket

• How to set up Azure Blob Storage

Cinchy Upgrade Utility

The Cinchy Upgrade Utility was previously introduced in v5.2 in order to facilitate a mandatory INT to BigInt upgrade. This tool will continue to be used in subsequent releases as an easy way to deploy necessary changes to your Cinchy platform.

For version 5.5, you must run the Upgrade Utility in order to fix a row-breaking issue that could be triggered on cells with over 4000 characters, where you are unable to update any column in your record.

Please review the Utility Guide or Upgrade Guide for further details.

New: Personal Access Tokens

You now have the option to use personal access tokens (PATs) in Cinchy, which are alternatives to using passwords for authentication. Similar to Cinchy Bearer Tokens, you can use a Cinchy PAT to call the Cinchy API as your current user, meaning your associated access controls will be honoured as well. Cinchy PATs, however, have an expiration date of up to 1 year. A single user can have up to 5 PATs active at one time.

For information on setting up, configuring, and managing PATs, please review the documentation here.

MongoDB

We have added MongoDB to our Connections offering as both a source and target connector.

MongoDB is a scalable, flexible NoSQL document database platform known for its horizontal scaling and load balancing capabilities, which has given application developers an unprecedented level of flexibility and scalability.

Review the following documentation to utilize this new capability in Cinchy:

• Setting up a MongoDB Collection (Event Triggered) as a source connector

• Setting up a MongoDB Collection as a source connector

• Setting up MongoDB as a target connector

• Setting up a MongoDB change stream

Enhancements

• We are continuing to improve our text editor functionality for forms. You can now embed tables and images into your text. We have also made various styling and usability quality of life updates, including the addition of checkbox style lists.

• We have added support for ephemeral volumes in Connections on a Kubernetes deployment. Unlike persistent volumes, ephemeral storage is unstructured and the space is shared between all pods running on a node; it allows pods to be started and stopped without being limited to the location of persistent volume. Running more than one pod for Connections per availability zone enables you to effectively leverage auto-scaling functionality.

• We have updated the Connections experience to enable more use cases. You can now use CDC parameters in Calculated Columns and use the CinchyID in the sync key in real-time syncs.

• Kafka supports cluster encryption and authentication, which can encrypt data-in-transit between your applications and Kafka. We have added the ability to include this encryption/authentication in the Listener Config when setting up real-time syncs using Kafka.

  • Using this parameter will specify which protocol will be used for communication between client and server. Cinchy currently supports the following options: Plaintext, SaslPlaintext, or SaslSsl.

    • Paintext: Unauthenticated, non-encrypted.

    • SaslPlaintext: SASL-based authentication, non-encrypted.

    • SaslSSL: SASL-based authentication, TLS-based encryption.

  • Review the Listener Config documentation on Kafka here.

• We have improved the implementation of tooltips such that linked columns display the tables that they link to. Hovering over the i symbol on a linked column will show the linked domain and table in the following format: Domain - Table; ex: HR - Employees. You can now also see them in the grid view.

• We have introduced a Retry Configuration for REST API sources and targets. This will automatically retry HTTP Requests on failure based on a defined set of conditions. This capability provides a mechanism to recover from transient errors such as network disruptions or temporary service outages.

  • For more information on using this configuration, refer to the documentation here for REST sources and here for REST targets.

• We have increased the default retention for Prometheus from 5GB to 50GB to allow you to store more metric data at a time.

  • This change is automatically reflected in new v5.5 deployments. Customers on previous v5 versions wishing to implement the change are able to rerun the automation script and deploy the new template to reflect the update.

• To make the Forms experience more responsive and process quicker, we have introduced lazy loading of records while searching. Instead of loading and rendering every form record in the search box, which can be a slow process for use cases with millions of records, lazy loading will initially retrieve a limited number of records. These results can then be further optimized by inputting your Lookup Filter Conditions.

• We have added the ability to pass parameters from a REST response into post sync scripts during both real-time and batch data syncs, allowing you to do more with your REST API data.

  • For an example and instructions on this capability, please refer to the documentation here.

• Data changes in Cinchy (CDC) can now be used to trigger a data sync from a REST API or MongoDB data source to a specified target. This works as an alternative to RunQuery.

  • For more information, please review the documentation here (REST API) and here (MongoDB).

• We have added two new functions, JSON_ESCAPE and URL_ESCAPE, which can be used in Connections to escape parameter values when used in constructing the body of a REST API Request or in the URL

• We have added an Authorization header type for REST API data syncs in Connections. An authorization request header can be used to provide credentials that authenticate a user with a server, allowing access to a protected resource.Selecting this header defines the Header Value as a password field.

Bug Fixes

• We have solved an issue that was causing Connections to get stuck behind long running jobs despite there being capacity to execute. This fix enables predictable execution behavior without stoppage.

• We have fixed an issue in the MatchEngine where execution was failing in versions of Cinchy above 5.2.

• File sourced data syncs will no longer fail, allowing you to run Connection jobs with uploaded files without the risk of a file not found error when auto-scaling is enabled.

• In order to prevent needlessly exhausting Cinchy IDs, the platform will no longer continuously retry to update records that have failed to save. This can sometimes occur when a value causes a calculated field to violate a uniqueness constraint. If the below error appears, you will have to manually update the cell to retry the save.

• We have fixed a bug that was causing the Connections UI to crash if you attempted to run a job while there was an empty parameter in the Info tab (ex: no name or formula)

• We have fixed a bug that would causes images in Forms to sometimes appear with a label above them, using the image's URL as the label's value.

• We have fixed a bug that was forcibly terminating authenticated sessions in Grafana, now allowing you to work without interruptions.

• We have solved an issue where using a form as a child form with file links wouldn't render the link thumbnail correctly in the "edit record" view.

• We have fixed a bug that prevented record updates when multiple users attempted to update a row in too quick of a succession.

• We have fixed an issue where doing delta batch syncs with a REST API target wouldn’t replace the @COLUMN parameter correctly.

• We have fixed a bug in Connections where an Oracle sync target would have the wrong tag in the Config XML.

• We have fixed a bug that was causing a “Listener is running” message to erroneously appear when the status of the listener was actually set to Disabled.

• We have fixed a bug that was preventing REST API real-time sync execution errors from being inserted into the execution errors table.

Table of Contents

1. Component Diagram

The below diagram shows a high level overview of Cinchy's Infrastructure components when deploying on IIS.

2. Component Overview

Table of Contents

1. ADFS Configuration

1. On your ADFS Server, Open AD FS Management.

2. Right-click on Relying Party Trusts and select Add Relying Party Trust to launch the Add Relying Party Trust Wizard (Image 1).

3. In the ADFS Wizard, select Claims Aware > Start > Select Data Source

4. Select Enter Data About the Relying Part Manually > Next

5. Under Specify Display Name, enter a Display Name of your choice

6. Under Configure Certificates, do not choose any certificates

7. Under Configure URL, Select Enable support for the SAML 2.0 SSO Web SSO protocol.

8. Enter your Login URL in the below format:

https://<cinchy-sso-URL>/Saml2/Acs

9. Under Configure Identifiers, choose an Identifier

10. Select Next until the process finishes.

2. Claim Issuance Policy

1. To begin configuring you Claim Issuance policy, Right-click on the Relying Party Trust you just created (look for the Display Name) and click Edit Claim Issuance Policy.

2. Click on Add Rule > Claim Rule > Send LDAP Attributes as Claims.

3. Add your Claim Rule Name

4. Under Attribute Store, choose Active Directory. Map the LDAP attribute to the following outgoing claim types:

4. Click Finish.

5. Click on Edit Rule.

6. Click on View Rule Language and copy out the Claim URLs for the claims defined. This information will be needed in a later step to configure your Cinchy appsettings.json. This will look something like this:

c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname", Issuer == "AD AUTHORITY"]
  => issue(store = "Active Directory",
           types = ("http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier",
                    "sub",
                    "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname",
                    "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname",
                    "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",
                    "http://schemas.microsoft.com/ws/2008/06/identity/claims/role"),
           query = ";userPrincipalName,sAMAccountName,givenName,sn,mail,memberOf;{0}",
           param = c.Value);

7. Click OK to save the rule.

8. Right-click on Relying Party Trust > Properties.

9. Go to the Advanced tab and set the secure hash algorithm to SHA-256 (Image 3).

3. Cinchy Configuration

1. Open https://<your.AD.server>/FederationMetadata/2007-06/FederationMetadata.xml in a browser and save the XML file in the cinchysso folder.

2. Open IIS Manager and create an HTTPS binding on the Cinchy site (if necessary).

3. Go to sso site and bind HTTPS with it. Make sure to use the same port as the login URL above if specified.

Cinchy appsettings.json

AppSettings Section

External Identity Claim Section

You will need the Rule Language URLs you copied out from the ADFS Configuration above. Using the same example as above, we would get the following (replace with your own URLs).

{
  "AppSettings": {
    ...
    },
  "ExternalIdentityClaimSection": {
    "FirstName": {
      "ExternalClaimName": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname"
    },
    "LastName": {
      "ExternalClaimName": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname"
    },
    "Email": {
      "ExternalClaimName": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress"
    },
    "MemberOf": {
      "ExternalClaimName": "http://schemas.microsoft.com/ws/2008/06/identity/claims/role"
    }
  }
}

Web.config

Add the 3 following lines to your web.config within the appSettings section:

<appSettings>
  ...
  <add key="UseHttps" value="true" />
  <add key="StsAuthorityUri" value="https://<your.cinchy.url>" />
  <add key="StsRedirectUri" value="https://<your.cinchysso.url>/Account/LoginRedirect" />
  ...
</appSettings>

1. Overview

Cinchy supports integration with any Identity Provider that issues SAML tokens (e.g. Active Directory Federation Services) for authenticating users.

It follows an SP Initiated SSO pattern where the SP will Redirect to the IdP and the IdP must submit the SAML Response via an HTTP Post to the SP Assertion Consumer Service.

Below is a diagram outlining the flow when a non-authenticated user attempt to access a Cinchy resource (Image 1).

2. Configure SAML Authentication - IIS Deployments

Cinchy must be registered with the Identity Provider. As part of that process you'll supply the Assertion Consumer Service URL, choose a client identifier for the Cinchy application, and generate a metadata XML file.

The Assertion Consumer Service URL for Cinchy is the base URL for the CinchySSO application followed by "{AcsURLModule}/Acs"

e.g. https:///<CinchySSO URL>/Saml2/Acs

e.g. https://myCinchyServer/Saml2/Acs

To enable SAML authentication within Cinchy, follow the below steps:

1. You can find the necessary metadata XML from the applicable identity provider you're using the login against. Place the metadata file in the deployment directory of the CinchySSO web application.

2. Update the values of the below app settings in the CinchySSO appsettings.json file.

• SAMLClientEntityId - The client identifier chosen when registering with the Identity Provider

• SAMLIDPEntityId - The entityID from the Identity Provider metadata XML

• SAMLMetadataXmlPath - The full path to the metadata XML file

• AcsURLModule - This parameter is needs to be configured as per your SAML ACS URL. For example, if your ACS URL looks like this "https:///<CinchySSO URL>/Saml2/Acs", then the value of this parameter should be "/Saml2"

Once SSO is enabled, the next time a user arrives at the Cinchy login screen they will see an additional button for "Single Sign-On".

3. Configure SAML Authentication - Kubernetes Deployments

1. Retrieve your metadata.xml file from the identity provider you're using the login against.

2. Navigate to your cinchy.kubernetes\environment_kustomizations_template\instance_template\idp\kustomization.yaml file.

3. Add your metadata.xml patch into your secrets where specified below as <<metadata.xml>>

- target:
    version: v1
    kind: Secret
    name: idp-secret-appsettings
  patch: |-
    - op: replace
      path: /data/appsettings.json
      value: <<idp_appsettings_json>>
    - op: add
      path: /data/metadata.xml
      value: <<metadata.xml>>

4. Navigate to your devops.automation > deployment.json in your Cinchy instance.

5. Add the following fields into the .json and update them below using the metadata.xml.

"sso": {
    "SAMLClientEntityId": "cinchy-dev", // Cinchy instance name
    "SAMLMetadataXmlPath": "/usr/share/appsettings/metadata.xml",
    // All below values cam be taken from the metadata.xml retrieved in step 1.
    "SAMLIDPEntityId": "",
    "SAMLSSOServiceURL": "", // this value is taken from the 'Location' attribute in the metadata.xml
    "AcsURLModule": "",
    "FirstNameExternalClaimName": "",
    "LastNameExternalClaimName": "",
    "EmailExternalClaimName": "",
    "MemberOfExternalClaimName": "",
    "metadata.xml": "BASE64_ENCODED_METADATA.XML"
}

6. Navigate to your kubernetes\environment_kustomizations_template\instance_template_encoded_vars\idp_appsettings_json.

7. Update the below code with your proper AppSettings and ExternalIdentityClaimSection details.

{
    "ConfigSettings": {
        "AppSettings": {
            "SAMLClientEntityId": "<<SAMLClientEntityId>>",
            "SAMLIDPEntityId": "<<SAMLIDPEntityId>>",
            "SAMLMetadataXmlPath": "<<SAMLMetadataXmlPath>>",
            "SAMLSSOServiceURL": "<<SAMLSSOServiceURL>>",
            "AcsURLModule": "<<AcsURLModule>>",
        },
        "ExternalIdentityClaimSection": {
            "FirstName": {
                "ExternalClaimName": "<<FirstNameExternalClaimName>>"
            },
            "LastName": {
                "ExternalClaimName": "<<LastNameExternalClaimName>>"
            },
            "Email": {
                "ExternalClaimName": "<<EmailExternalClaimName>>"
            },
            "MemberOf": {
                "ExternalClaimName": "<<MemberOfExternalClaimName>>"
            }
        }
    }
}

8. Run devops automation script which will populate the updated outputs into the cinchy.kubernetes repo.

9. Commit your changes and push to your source control system.

10. Navigate to your ArgoCD dashboard and refresh the idp-app to pick up your changes. Your pods will be restarted and will pick up the latest secrets.

11. Once the pods are healthy, you can verify the changes by looking for the SSO Tab on your Cinchy login page.

4. User Management

Before a user is able to login through the SSO flow, the user must be set up in Cinchy with the appropriate authentication configuration.

Users in Cinchy are maintained within the Users table in the Cinchy domain. Each user in the system is configured with 1 of 3 Authentication Methods:

• Cinchy User Account - These are users that are created and managed directly in the Cinchy application. They log into Cinchy by entering their username and password on the login screen.

• Non Interactive - These accounts are intended for application use.

• Single Sign-On - These users authenticate through the SSO Identity Provider (configured using the steps above). They log into Cinchy by clicking the "Login with Single Sign-On" link on the login screen.

4.1 How to define a new SSO User

Create a new record within the Users table with the Authentication Method set to "Single Sign-On".

4.2 How to convert an existing user to SSO User

Change the Authentication Method of the existing user to "Single Sign-On".

4.3 Logging in with SSO

Once a user is configured for SSO, they can then click the "Login with Single Sign-On" link on the login page and that will then take them through the Identity Provider's authentication flow and bring them into Cinchy.

5. Automatic User Creation - IIS Deployments

Once SSO has been enabled on your instance of Cinchy, by default, any user that does not exist in the Cinchy Users table will not be able to login regardless if they are authenticated by the Identity Provider.

Enabling Automatic User Creation means that upon login, if the Identity Provider authorizes the user, an entry for this user will automatically be created in the Cinchy Users table if one does not already exist. This means that any SSO authenticated user is guaranteed to be able to access the platform.

In addition to creating a user record, if AD Groups are configured within Cinchy, then the authenticated user will automatically be added to any Cinchy mapped AD Groups where they are a member. See AD Group Integration for additional information on how to define AD Groups in Cinchy.

See below for details on how to enable Automatic User Creation.

5.1 Prerequisites for Automatic User Creation

The Identity Provider configuration must include the following claims in addition to the base configuration in the SAML token response:

• First Name

• Last Name

• Email

In order to enable automatic group assignment for newly created users (which is applicable if you plan on using AD Groups), then you must also include an attribute that captures the groups that this user is a member of (e.g. memberOf field in AD)

5.2 Configuration Setup

Enabling automatic user creation requires the following changes. For IIS Deployments this will be done to the appsettings.json file in the CinchySSO web application.

1. Add ExternalClaimName attribute values under "ExternalIdentityClaimSection" in appsettings.json file. Do not add the value for "MemberOf" if you don't want to enable automatic group assignment .

2. The ExternalClaimName value must be updated to create a mapping between the attribute name in the SAML response and the required field. (e.g. http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname is the name in the SAML response for the FirstName field)

"ExternalIdentityClaimSection": {
			"FirstName": {
				"ExternalClaimName": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname"
			},
			"LastName": {
				"ExternalClaimName": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname"
			},
			"Email": {
				"ExternalClaimName": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress"
			},
			"MemberOf": {
				"ExternalClaimName": "http://schemas.microsoft.com/ws/2008/06/identity/claims/role"
			}
		}

6. Further Reading

• Configuring ADFS

• AD Group Integration

Table of Contents

1. Group Management

This section defines how to manage Groups.

1.1 Cinchy Groups

Cinchy Groups are containers that have Users and other Groups within them as members. They are used to provision access controls throughout the platform. Cinchy Groups enable centralized administration for access controls.

Groups are defined in the "Groups" table within the Cinchy domain. By default this table can only be managed by members of the Cinchy Administrators group. Each group has the following attributes:

1.2 How to Define a New AD Group

1. To define a new AD Group, create a new record within the Groups Table with the same name as the AD Group (using the cn attribute).

2. Set the Group Type to "AD Group".

1.3 How to Convert an Existing Group to Sync with AD

1. To convert an existing group, update the Name attribute of the existing group record to match the AD Group (using the cn attribute).

2. Set the Group Type to "AD Group".

2. Group Membership Sync

AD Groups defined in Cinchy have their members sync-ed from AD through a batch process that leverages the Cinchy Command-line-interface (CLI).

2.1 Execution Flow

The sync operation performs the following high-level steps:

1. Fetches all Cinchy registered AD Groups using a Saved Query.

2. Retrieves the usernames of all members for each AD Group. The default attribute for username that is retrieved is "userPrincipalName", but configurable as part of the sync process.

3. For each AD Group, it loads the users that are both a member in AD and exist in the Cinchy Users table (matched on the Username) into the "Users" attribute of the Cinchy Groups table.

2.2 Dependencies

1. The Cinchy CLI Model must be installed in your instance of Cinchy.

2. An instance of the Cinchy CLI must be available to execute the sync.

3. A task scheduler is required to perform the sync on a regular basis (e.g. Autosys).

2.3 Configuration Steps

2.3.1 Create a Saved Query to retrieve AD Groups from Cinchy

1. Create a new query within Cinchy with the below CQL to fetch all AD Groups from the Groups table. The domain & name assigned to the query will be referenced in the subsequent step.

SELECT [Name]
FROM [Cinchy].[Cinchy].[Groups]
WHERE [Group Type] = 'AD Group' 
AND [Deleted] IS NULL

2.3.2 Create the Sync Config

1. Copy the below XML into a text editor of your choice and update the attributes listed in the table below the XML to align to your environment specific settings.

2. Once complete, create an entry with the config in your Data Sync Configurations table (part of the Cinchy CLI model).

<?xml version="1.0" encoding="utf-16" ?>
<BatchDataSyncConfig name="AD_GROUP_SYNC" version="1.0.0" xmlns="http://www.cinchy.co">
  <Parameters />
  <LDAPDataSource objectCategory="group" ldapserver="LDAP:\\activedirectoryserver.domain.com" username="encryptedUsername" password="encryptedPassword" >
    <Schema>
      <Column name="cn" ordinal="1" dataType="Text" maxLength="5000" isMandatory="false" validateData="false" trimWhitespace="true" description=""/>
      <Column name="member.userPrincipalName" ordinal="2" dataType="Text" maxLength="200" isMandatory="false" validateData="false" trimWhitespace="true" description=""/>
    </Schema>
    <Filter>
      lookup('Domain Name','Query Name')
    </Filter>
  </LDAPDataSource>
  <CinchyTableTarget model="" domain="Cinchy" table="Groups" suppressDuplicateErrors="false">
    <ColumnMappings>
      <ColumnMapping sourceColumn="cn" targetColumn="name" />
      <ColumnMapping sourceColumn="member.userPrincipalName" targetColumn="Users" linkColumn="Username" />
    </ColumnMappings>
    <SyncKey>
      <SyncKeyColumnReference name="name" />
    </SyncKey>
	<ChangedRecordBehaviour type="UPDATE" />
    <DroppedRecordBehaviour type="IGNORE" />
  </CinchyTableTarget>
</BatchDataSyncConfig>

2.3.3 Sync Execution & Scheduling

1. The below CLI command (see here for additional information on the syncdata command) should be used to execute the sync.

2. Update the command parameters (described in the table below) with your environment specific settings.

3. Execution of this command can be scheduled at your desired frequency using your scheduler of choice.

dotnet Cinchy.CLI.dll syncdata -s cinchyAppServer -u username -p "encryptedPassword" -m "Model" -f "AD_GROUP_SYNC" -d "TempDirectory" 

3. High Number of Groups in ADFS

If you are syncing someone with a lot of ADFS groups, the server may reject the request for the header being too large. If you are able to login as a user with a few groups in ADFS but run into an error with users with a lot of ADFS groups (regardless of if those ADFS groups are in Cinchy), you will need to make the following changes:

3.1 Update the Server Max Request Header Size

1. Follow the instructions outlined in this document.

3.2 CinchySSO App Settings

1. In your CinchySSO app settings, you will also need to increase the max size of the request, as follows:

"AppSettings": {
  ...
  "MaxRequestHeadersTotalSize": {max size in bytes},
  "MaxRequestRequestBufferSize": {max size in bytes, use same as above},
  "MaxRequestBodySize": -1
}

1. Overview

In Cinchy v5.2, we implemented the ability to free up database space by using S3 compatible or Azure Blob Storage for file storage. This configuration is set in the deployment.json of a Kubernetes installation, or the appsettings.json of an IIS installation.

2. Kubernetes File Storage

1. If you are using a Kubernetes deployment, you will change your file storage config in the deployment.json.

2. Navigate to the object storage section, where you will see either S3 or Azure Blob storage, depending on your deployment structure.

Azure Example

AWS Example

3. To utilize Blob Storage or S3, update each line with your own parameters.

4. To utilize Local storage, leave each line blank with the exception of the Connections_Storage_Type, which should be set to Local:

5. Run the deployment script by using the following command in the root directory of your devops.automations repo:

6. Commit and push your changes.

3. IIS File Storage

1. If you are using an IIS deployment, you will change your file storage config in the Cinchy Web appsettings file.

2. Locate the StorageType section of the file and set it to either "Local", "AzureBlobStorage" or "S3".

3. If you selected "AzureBlobStorage", fill out the following lines in the same file:

4. If your selected "S3", fill out the following lines in the same file:

1. How to Enable TLS 1.2

1. Navigate to the CinchySSO Folder > appsettings.json file.

2. Find the following line:

3. Replace the above line with the following:

4. Navigate to the Cinchy Folder > web.config file.

5. Find the following line:

6. Replace the above line with the following:

7. You may need to restart the application pools in IIS for the changes to take effect.

Table of Contents

1. General Kubernetes Deployment Prerequisites

The following is a list of steps that are required prior to deploying Cinchy v5 on Kubernetes.

1.1 Download your Tools

The following tools should be installed on the machine where the deployment will run:

• (v1.23.0+)

• ( may be used on Windows)

1.2 Creating your Domains

All of your Cinchy environments will need a domain for each of the following:

• ArgoCD

• Opensearch

• Grafana

1.3 SSL Certs

You will need to have valid SSL Certs ready when you deploy Cinchy v5. This should be a wildcard certificate if ArgoCD will be exposed via a subdomain (preferred). Without the wildcard certificate, accessing ArgoCD's portal requires a port forward to be created using kubectl on demand.

1.4 Secrets Management

Secrets management is optional, however Cinchy recommends it for securely storing and accessing secrets that will be used in the deployment process. Cinchy currently supports:

1.5 Single Sign-On

1.6 Docker Images

You have the option to either use Cinchy's Docker images or your own. If you would like to use Cinchy's, please follow the section below to access them.

1.6.1 Accessing Cinchy's Docker images

You will pull Docker images from Cinchy's AWS Elastic Container Registry (ECR).

To gain access to Cinchy's Docker images, you will need login credentials to the ECR. You can gain these by contacting Cinchy Support.

1.7 Create your Repositories

• cinchy.terraform: This repo contains all Terraform configurations.

• cinchy.argocd: This repo contains all ArgoCD configurations.

• cinchy.kubernetes: This repo contains cluster and application component deployment manifests.

• cinchy.devops.automations: This repo contains the single configuration file and binary utility that maintains the contents of the above three repositories.

1.8 Access to Cinchy Artifacts

You will need to access and download the Cinchy artifacts prior to deployment.

To access the Kubernetes artifacts:

2. Navigate to the release you wish to deploy

3. Download the .zip file(s) listed under the Kubernetes Artifacts column

2. Kubernetes Deployments (Azure Specific Requirements)

The following prerequisites are required if you are deployment Cinchy v5 on Azure.

• Terraform Backend Requirements:

  • A resource group that will contain the Azure Blob Storage with the terraform state.

  • A storage account and container (Azure Blob Storage) for persisting terraform state.

• The deployment template has the option of either leveraging an existing resource group or creating a new one:

  • If an existing resource group is preferred, the prerequisite requires the following be provisioned in advance of the deployment:

    • The resource group.

    • A VNet within the resource group.

    • A single subnet. It's important that the address range be sufficient for all executing processes within the cluster, e.g. a CIDR ending with /22 to provide a range of 1024 IPs.

  • If a new resource group is preferred, all resources will be automatically provisioned.

• The quota limit of the Total Regional vCPUs and the Standard DSv3 Family vCPUs (or equivalent) must provide sufficient availability for the required number of vCPUs (minimum of 24).

• An AAD user account to connect to Azure, which has the necessary privileges to create resources in any existing resource groups and the ability to create a resource group (if required).

3. Kubernetes Deployments (AWS Specific Requirements)

The following prerequisites are required if you are deployment Cinchy v5 on AWS.

• Terraform Backend Requirements:

  • An S3 bucket that will contain the terraform state.

• The template has the option of either leveraging an existing VPC or creating a new one:

  • If an existing VPC is preferred, the prerequisite requires the following be provisioned in advance of the deployment:

    • The VPC. It's important that the address range be sufficient for all executing processes within the cluster, e.g. a CIDR ending with /21 to provide a range of 2048 IPs.

    • 3 Subnets (one per AZ). It's important that the address range be sufficient for all executing processes within the cluster, e.g. a CIDR ending with /23 to provide a range of 512 IPs.

    • If the subnets are private, a NAT Gateway is required to enable node group registration with the EKS cluster.

  • If a new VPC is preferred, all resources will be automatically provisioned.

• The limit of the Running On-Demand All Standard vCPUs must provide sufficient availability for the required number of vCPUs (minimum of 24).

• An IAM user account to connect to AWS which has the necessary privileges to create resources in any existing VPC and the ability to create a VPC (if required).

• The SSL certificate must be imported into AWS Certificate Manager (or a new certificate can be requested via AWS Certificate Manager).

4. IIS Deployment Prerequisites

The following is a list of steps that are required prior to deploying Cinchy v5 on IIS

4.1 Access the Artifacts

You will need to access and download the Cinchy binary prior to deployment:

• Navigate to the release you wish to deploy

• Download the files listed under the Component Artifacts column. This should include zip files for:

  • Cinchy Platform

4.2 General Requirements

1. An instance of SQL Server 2017+

2. A Windows Server 2012+ machine with IIS 7.5+ installed

3. • Specifically, install: ASP.NET Core/.NET Core Runtime & Hosting Bundle

4.3 System Requirements

Minimum Web Server Hardware Recommendations

• 2 x 2 GHz Processor

• 8 GB RAM

• 4 GB Hard Disk storage available

Minimum Database Server Hardware Recommendations

• 4 x 2 GHz Processor

• 12 GB RAM

• Hard disk storage dependent upon use case. Note that Cinchy maintains historical versions of data and performs soft deletes which will add to the overall storage requirements.

4.4 Clustering

Clustering considerations are applicable to both the Web and Database tiers in the Cinchy deployment architecture.

The web tier can be clustered by introducing a load balancer and scaling web server instances horizontally. Each node within Cinchy uses an in-memory cache of metadata information, and expiration of cached elements is triggered upon data changes that would impact that metadata. Data changes processed by one node wouldn't immediately be known to other nodes without establishing connectivity between them. For this reason the nodes must be able to communicate over either http or https through an IP based binding on the IIS server that allows cache expiration messages to be broadcast. The port used for this communication is different from the standard port that is used by the application when a domain name is involved. Often for customers this means that a firewall port must be opened on these servers.

The database tier relies on standard MS SQL Server failover clustering capabilities.

4.5 Scaling Considerations

The web application is responsible for all interactions with Cinchy be it through the UI or connectivity from an application. It interprets/routes incoming requests, handles serialization/deserialization of data, data validation, enforcement of access controls, and the query engine to transform Cinchy queries into the physical representation for the database. The memory footprint for the application is fairly low as caching is limited to metadata, but the CPU utilization grows with request volume and complexity (e.g. insert / update operations are more complex than select operations). As the user population grows or request volume increases from batch processes / upstream system API calls there may be a need to add nodes.

The database tier relies on a persistence platform that scales vertically. As the user population grows and request volume increases from batch processes / upstream system API calls the system may require additional CPU / Memory. Starting off in an environment that allows flexibility (e.g. a VM) would be advised until the real world load can be profiled and a configuration established. On the storage side, Cinchy maintains historical versions of records when changes are made and performs soft deletes of data which will add to the storage requirements. The volume of updates occurring to records should be considered when estimating the storage size.

4.6 Backups

Outside of log files there is no other data generated & stored on the web servers by the application, which means backups are generally centered around the database. Since the underlying persistence platform is a MS SQL Server, this relies on standard procedures for this platform.

1. Disabling your Applications

There may be times when you want to temporarily disable your Kubernetes pods in order to perform maintenance or certain upgrades. You can do so through the following steps:

1. Access your ArgoCD

2. Navigate to the application directory for the namespace you wish to disable, in this case development-cinchy (Image 1). You should see your cluster component applications.

3. Click on the main application (i.e. development-cinchy) (Image 2).

4. Navigate to Summary > Sync Policy > Automated. Click on Disable Auto-Sync > OK (Image 3).

5. For each of the cluster applications that you wish to disable, click on the "..." > Delete (Image 5).

6. Your apps should all appear as "out of sync" (Image 6).

2. Re-enabling your Applications

1. To re-enable your applications, return to the application directory for your disabled namespace (Image 7).

2. Click on the main application (i.e. development-cinchy) (Image 8).

3. Navigate to Summary > Sync Policy. Click on Enable Auto-Sync > OK (Image 9).

1. Generate the self-signed certificate by executing the following commands in any folder:

2. Create a yaml file located at cinchy.kubernetes/platform_components/base/self-signed-ssl-root-ca.yaml.

3. Add the following to the yaml file:

4. Add the self signed root CA cert file to the cinchy.kubernetes/environment_kustomizations/cinchy_nonprod/base folder.

5. Add the yaml code snippet to the cinchy.kubernetes/environment_kustomizations/cinchy_nonprod/base/kustomization.yaml file, changing the below files key value as per your root ca cert file name:

6. Add the following line to the cinchy.kubernetes/platform_components/base/kustomization.yaml file

7. Add the below Deployment patchesJson6902 to each of your cinchy.kubernetes/environment_kustomizations/cinchy_nonprod/ENV_NAME/PLATFORM_COMPONENT_NAME/kustomization.yaml files, except "base".

• Ensure that the rootCA.crt file name is matched with ConfigMap data, configMapGenerator files, and the patch subpath.

8. Once the changes are deployed, verify the root CA cert is available on the pod under /etc/ssl/certs with below command, inputing your own POD_NAME and NAMESPACE where noted:

Table of Contents

1. Introduction

This page details the instructions for deployment of Cinchy v5 on Kubernetes. We recommend, and have documented below, that this is done via Terraform and ArgoCD. This setup involves a utility to centralize and streamline your configurations.

The Terraform scripts and instructions provided enable deployment on Azure and AWS cloud environments.

2. Deployment Prerequisites

The following lists are required prerequisites for installing Cinchy v5 on Kubernetes.

Note that some prerequisites will depend on whether you are deploying on Azure or on AWS.

2.1 All Platforms

These prerequisites apply whether you are installing on Azure or on AWS.

• • cinchy.terraform: This repo contains all Terraform configurations.

  • cinchy.argocd: This repo contains all ArgoCD configurations.

  • cinchy.kubernetes: This repo contains cluster and application component deployment manifests.

  • cinchy.devops.automations: This repo contains the single configuration file and binary utility that maintains the contents of the above three repositories.

• You must have a service account with read/write permissions to the git repos created above.

• The following tools should be installed on the machine where the deployment will run:

• You will need a single domain for accessing ArgoCD, Grafana, Opensearch Dashboard, and any deployed Cinchy instances. There are two routing options for accessing these applications - path based or subdomains. See below for an example with multiple cinchy instances:

2.2 Azure Deployment

The following prerequisites are required if you are deployment Cinchy v5 on Azure.

• Terraform Backend Requirements:

  • A resource group that will contain the Azure Blob Storage with the terraform state.

  • A storage account and container (Azure Blob Storage) for persisting terraform state.

• The deployment template has the option of either leveraging an existing resource group or creating a new one:

  • If an existing resource group is preferred, the prerequisite requires the following be provisioned in advance of the deployment:

    • The resource group.

    • A VNet within the resource group.

    • A single subnet. It's important that the address range be sufficient for all executing processes within the cluster, e.g. a CIDR ending with /22 to provide a range of 1024 IPs.

  • If a new resource group is preferred, all resources will be automatically provisioned.

• The quota limit of the Total Regional vCPUs and the Standard DSv3 Family vCPUs (or equivalent) must provide sufficient availability for the required number of vCPUs (minimum of 24).

• An AAD user account to connect to Azure, which has the necessary privileges to create resources in any existing resource groups and the ability to create a resource group (if required).

2.3 AWS Deployment

The following prerequisites are required if you are deployment Cinchy v5 on AWS.

• Terraform Backend Requirements:
• The template has the option of either leveraging an existing VPC or creating a new one:

  • If an existing VPC is preferred, the prerequisite requires the following be provisioned in advance of the deployment:

    • The VPC. It's important that the address range be sufficient for all executing processes within the cluster, e.g. a CIDR ending with /21 to provide a range of 2048 IPs.

    • 3 Subnets (one per AZ). It's important that the address range be sufficient for all executing processes within the cluster, e.g. a CIDR ending with /23 to provide a range of 512 IPs.

    • If the subnets are private, a NAT Gateway is required to enable node group registration with the EKS cluster.

  • If a new VPC is preferred, all resources will be automatically provisioned.

• The limit of the Running On-Demand All Standard vCPUs must provide sufficient availability for the required number of vCPUs (minimum of 24).

• An IAM user account to connect to AWS which has the necessary privileges to create resources in any existing VPC and the ability to create a VPC (if required).

3. Initial Configuration

The following steps detail the instructions for setting up the initial configurations.

3.1 Configure the Deployment.json

1. Navigate to your cinchy.devops.automations repository where you will see an aws.json and azure.json.

2. Depending on the cloud platform that you are deploying to, select the appropriate file and copy it into a new file named deployment.json (or <cluster name>.json) within the same directory.

3. This file will contain the configuration for the infrastructure resources and the Cinchy instances to deploy. Each property within the configuration file has comments in-line describing its purpose along with instructions on how to populate it.

4. Follow the guidance within the file to configure the properties.

5. Commit and push your changes.

3.2 Execute cinchy.devops.automations

This utility updates the configurations in the cinchy.terraform, cinchy.argocd, and cinchy.kubernetes repositories.

1. From a shell/terminal, navigate to the cinchy.devops.automations directory location and execute the following command:

3. The console output should terminate with a "Completed successfully".

4. Terraform Deployment

The following steps detail how to deploy Terraform.

Cinchy.terraform Repo Structure - AWS

If deploying on AWS: Within the Terraform > AWS directory, a new folder named eks_cluster is created. Nested within that is a subdirectory with the same name as the newly created cluster.

To perform terraform operations, the cluster directory must be the working directory during execution. This applies to everything within step 4 of this guide.

Cinchy.terraform Repo Structure - Azure

If deploying on Azure: Within the Terraform > Azure directory, a new folder named aks_cluster is created. Nested within that is a subdirectory with the same name as the newly created cluster.

To perform terraform operations, the cluster directory must be the working directory during execution.

4.1 Cloud Provider Authentication

1. Launch a shell/terminal with the working directory set to the cluster directory within the cinchy.terraform repo.

2. If you are using AWS, run the following commands to authenticate the session:

3. If using Azure, run the following command and follow the on screen instructions to authenticate the session:

4.2 Deploy the Infrastructure

1. Execute the following command to create the cluster:

2. Type yes when prompted to apply the terraform changes.

The resource creation process can take approx. 15-20 minutes. At the end of the execution there will be a section with the following header

======= Output Variables =======

If deploying on AWS, this section will contain 2 values: Aurora RDS Server Host and Aurora RDS Password

If deploying on Azure, this section will contain a single value: Azure SQL Database Password

These variable values are required to update the connection string within the deployment.json file (or equivalent) in the cinchy.devops.automations repo.

4.3 Retrieve the SSH Keys

The following section breaks down how to retrieve your SSH keys for both AWS and Azure deployments.

4.3.1 AWS SSH Keys

1. The SSH key to connect to the Kubernetes nodes is maintained within the terraform state and can be retrieved by executing the following command:

4.3.2 Azure SSH Keys

1. The SSH key is output to the directory containing the cluster terraform configurations.

5. Update the Deployment.json

The following section pertains to updating the Deployment.json file.

5.1 Update the Database Connection String

2. Each object within represents an instance that will be deployed on the cluster. Each instance configuration contains a database_connection_string property. This has placeholders for the host name and password that must be updated using output variables from the previous section.

5.2 Create the IAM User for S3 (AWS Only)

The terraform script will create an S3 bucket for the cluster that must be accessible to the Cinchy application components.

To access this programmatically, an IAM user that has read/write permissions to the new S3 bucket is required. This can be an existing user.

The Access Key and Secret Access Key for the IAM user must be specified under the object_storage section of the deployment.json

5.3 Update Blob Storage Connection Details (Azure Only)

1. Within the deployment.json, the azure_blob_storage_conn_str must be set.

2. The in-line comments outline the commands required to source this value from the Azure CLI.

5.3.1 Enabling Azure Key Vault Secrets

If you have the key_vault_secrets_provider_enabled=true value in the azure.json then the below secrets files would have been created during the execution of step 3.2:

You will need to add the following secrets to your azure key vault:

• worker-secret-appsettings-<cinchy_instance_name>

• web-secret-appsettings-<cinchy_instance_name>

• maintenance-cli-secret-appsettings-<cinchy_instance_name>

• idp-secret-appsettings-<cinchy_instance_name>

• forms-secret-config-<cinchy_instance_name>

• event-listener-secret-appsettings-<cinchy_instance_name>

• connections-secret-config-<cinchy_instance_name>

• connections-secret-appsettings-<cinchy_instance_name>

To create your new secrets:

1. Navigate to your key vault in the Azure portal.

2. Open your Key Vault Settings and select Secrets.

3. Select Generate/Import.

4. On the Create a Secret screen, choose the following values:

   1. Upload options: Manual.

   2. Name: Choose the secret name from the above list. They will all follow the format of: <app>-secret-appsettings-<cinchy_instance_name> or <app>-secret-config-<cinchy_instance_name>

   3. Value: The value for the secret will be the content of each app JSON located in the cinchy.kubernetes\environment_kustomizations\nonprod<cinchy_instance_name>\secrets folder, and as shown in above image.

   4. Content type: JSON

5. Leave the other values to their defaults.

6. Select Create.

Once you receive the message that the first secret has been successfully created, you may proceed to create the other secrets. There are a total of 8 secrets to create as shown in the above list of secret names.

5.4 Execute cinchy.devops.automations

This utility updates the configurations in the cinchy.terraform, cinchy.argocd, and cinchy.kubernetes repositories.

1. From a shell/terminal, navigate to the cinchy.devops.automations directory and execute the following command:

3. The console output should terminate with a "Completed successfully".

4. The updates must be committed to Git before proceeding to the next step.

6. Connect with kubectl

6.1 Update the Kubeconfig

6.1.1 AWS

1. From a shell/terminal run the following command, replacing <region> and <cluster_name> with the accurate values for those placeholders:

6.1.2 Azure

1. From a shell/terminal run the following commands, replacing <subscription_id>, <deployment_resource_group>, and <cluster_name> with the accurate values for those placeholders.

6.2 Verify the Connection

1. Verify that the connection has been established and the context is the correct cluster by running the following command:

7. Deploy and Access ArgoCD

In this step, we will deploy and access ArgoCD.

7.1 Deploy ArgoCD

1. Launch a shell/terminal with the working directory set to the root of the cinchy.argocd repository.

2. Execute the following command to deploy ArgoCD:

3. Monitor the pods within the argocd namespace by running the following command every 30 seconds until they all move into a healthy state:

7.2 Access ArgoCD

1. Launch a new shell/terminal with the working directory set to the root of the cinchy.argocd repository.

2. Execute the following command to access ArgoCD: