Grafana Overview

Grafana is an open source analytics and interactive visualization web application. When connected to your Cinchy platform, it provides charts, graphs, and alerting capabilities (Image 1).

Grafana, and its paired application Prometheus (which consumes metrics from the running components in your environment) is the recommended visualization application for Cinchy v5 on Kubernetes.

Get started with Grafana

Grafana has a robust library of documentation of tutorials designed to help you learn the fundamentals of the application. We've listed some notable ones below:

• Exploring your Metrics

• Best Practices for Creating Dashboards

• Building a Dashboard

• Guide to Dashboard Types and Capabilities

• Creating a Managed Alert

• All Documentation

Access your saved dashboards

Cinchy comes with some saved dashboards that come out of the box. These dashboards will provide a great jumping off point for your metrics monitoring, and you can always customize, manage, and add further dashboards at your leisure.

1. Navigate to the left navigation pane, select the Dashboards icon > Manage (Image 2).

2. You will see a list of all of the Dashboards available to you (Image 3). Clicking on any of them will take you to a full metrics view (Image 4).

3. You can favourite any of your commonly used or most important dashboards by clicking on the star (Image 5).

4. Once you favorite a dashboard, you can easily find it by navigating to the left navigation pane, select the Dashboards icon > Home. This will open the Dashboards Home. You can see both your favourite and your recent dashboards in this view (Image 6)

Recommended Dashboards

Your Cinchy v5 deployment comes with some out-of-the-box dashboards already made. You are able to customize these to suit your specifications. The following are a few notable ones:

Kubernetes/Compute Resources/Cluster

Purpose: This dashboard provides a general overview of your entire cluster including all of your environments and pods (Image 7).

Metrics:

The following are some example metrics that you could expect to see from this dashboard:

• CPU Usage

• CPU OTA

• Memory Use

• Memory Requests

• Current Network Usage

• Bandwidth (Transmitted and Received)

• Average Container Bandwidth by Namespace

• Rate of Packets

• Rate of Packets Dropped

• Storage IO & Distribution

Kubernetes/Compute Resources/Namespace (Workloads)

Purpose: This dashboard is useful for looking at environment specific details (Image 8). You can use the namespace drop down menu to select which environment you want to visualize (Image 9). This can be particularly helpful during load testing. You are also able to drill down to a specific workload by clicking on its name.

Metrics:

The following are some example metrics that you could expect to see from this dashboard**:**

• CPU Usage

• CPU OTA

• Memory Use

• Memory Quota

• Current Network Usage

• Bandwidth (Transmitted and Received)

• Average Container Bandwidth by Workload

• Rate of Packets

• Rate of Packets Dropped

Set up alerts

Grafana lets you to set up push alerts against your dashboards and queries. Once you have created your dashboard, you can follow the steps below to set up your alert.

Set up your notifications channel

Your notifications channel refers to who will be receiving your alert. To set one up:

1. Click on the Alert icon on the left navigation tab (Image 10), and locate "Notifications Channel"

1. Click the "Add a Channel" button.

2. Add in the following parameters, including any optional checkboxes you wish to use (Image 11):

Name: The name of this channel.

Type: You have several options here, but email is the most common.

Addresses: Input all the email addresses you want to be notified of this alert, separated by a comma.

1. Click Test to send out a test email, if desired.

2. Save your Notification Channel.

Set up your alert

The following details how to set up alerts on your dashboards. You can also set up alerts upon creation of your dashboard from the same window.

1. Navigate to the dashboard and dashboard panel that you want to set up an alert for. This example, sets up an alert for CPU usage on our cluster.

2. Click on the dashboard name > Edit

3. Click on the Alert tab (Image 12).

1. Input the following parameters to set up your alert (Image 13):

• Alert Name: A title for your alert

• Alert Timing: Choose how often to evaluate and for how long. In this example it's evaluated every minute for five minutes.

• Conditions: Here you can set your threshold conditions for when an alert will be sent out. In this example, it's sent when the average of query A is above 75.

• Set what happens if there's no data, or an error in your data

• Add in your notification channel (who will be sent this notification)

• Add a message to accompany the alert.

• Click Apply > Save to finalize your alert.

Recommended alerts

Below are a few alerts we recommend setting up on your Grafana.

CPU usage

Set up this alert to notify you when the CPU Usage on your nodes exceeds a specified limit.

Dashboard Query

You can use the following example queries to set up a dashboard that will capture CPU Usage by Node (Image 14).

avg by (node_name) (100 - ((avg by (cpu,node_name) (irate(node_cpu_seconds_total{mode="idle"}[1m]))) * 100))

100 - ((avg by (cpu,node_name) (irate(node_cpu_seconds_total{mode="idle"}[1m]))) * 100)

Alert:

Set up your alert. This example uses a threshold limit of 75 (Image 15).

Memory Usage

Set up this alert to notify you when the Memory Usage on your nodes exceeds a specified limit.

Dashboard Query

You can use the following example queries to set up a dashboard that will capture CPU Usage by Node (Image 16)

((node_memory_MemTotal_bytes-node_memory_MemAvailable_bytes) / (node_memory_MemTotal_bytes))*100

Alert:

Set up your alert. This example uses a threshold limit of 85 (Image 17).

Disk Usage

Set up this alert to notify you when the Disk Usage on your nodes exceeds a specified limit.

Dashboard Query

You can use the following example queries to set up a dashboard that will capture Disk Usage by Node (Image 18)

(sum((node_filesystem_size_bytes))by(node_name) - sum((node_filesystem_free_bytes))by(node_name)) *100/(sum((node_filesystem_avail_bytes))by(node_name)+(sum((node_filesystem_size_bytes))by(node_name) - sum((node_filesystem_free_bytes))by(node_name)))

Alert

Set up your alert. This example uses a threshold limit of 80 (Image 17).

I/O wait

Set up this alert to check the amount of iowait from the CPU. A high value usually indicates a slow/overloaded HDD or Network.

Dashboard Query

You can use the following example queries to set up a dashboard that will capture the CPU I/O wait (Image 19).

(sum(irate(node_cpu_seconds_total{mode="iowait"}[1m]))by(node_name) * 100 / 4)

Alert

Set up your alert. This example uses a threshold limit of 60 (Image 19).

Update your Grafana Password

Your Grafana password can be updated in your deployment.json file (you may have renamed this during your original deployment).

1. Navigate to cluster_component_config > grafana.

2. The default password is set to prom-operator; update this with your preferred new password, written in clear text.

3. Run the below command in the root directory of your devops.automations repository to update your configurations. If you have changed the name of your deployment.json file, make sure to update the command accordingly.

dotnet Cinchy.DevOps.Automations.dll "deployment.json"

1. Commit and push your changes.

2. If your environment isn't set-up to automatically apply upon configuration,navigate to the ArgoCD portal and refresh your component(s). If that doesn't work, re-sync.

Introduction

Cinchy v5 on Kubernetes offers and recommends a robust set of open-source tools and applications for monitoring, logging, and visualizing the data on your Cinchy instances.

Click on the respective tool below to learn more about it:

• Grafana

• OpenSearch

• ArgoCD

Overview

When deploying Cinchy v5 on Kubernetes, Cinchy recommends using OpenSearch Dashboards for your logging. OpenSearch is a community-driven fork of Elasticsearch created by Amazon, and it captures and indexes all your logs into a single, accessible dashboard location. These logs can be queried, searched, and filtered, and Correlation IDs mean that they can also be traced across various components. These logging components take advantage of persistent storage.

You can view OpenSearch documentation here:

• Introducing OpenSearch

• General OpenSearch Documentation

• Using DQL (Dashboards Query Language)

• Troubleshooting and Common Errors

• Alerts

  • Anomaly Detection

Get started with OpenSearch Dashboards

These sections guide you through setting up your first Index, Visualization, Dashboard, and Alert.

Define your log level

1. Navigate to your cinchy.kubernetes/environment_kustomizations/instance_template/worker/kustomization.yaml file.

2. In the below code, copy the Base64 encoded string in the value parameter.

patch: |-
  - op: replace
    path: /data/appsettings.json
    value: wcxJItEmCWQJQPZidpLUuV6Ll79ZUr8BimlMJysLwcxJItEmCWQJQPZidpLUuV6Ll79ZUr8BimlMJysL

1. Decode the value to retrieve your AppSettings.

2. Navigate to the below Serilog section of the code and update the "Default" parameter as needed to set your log level. The options are:

"Serilog": {
    "MinimumLevel": {
      "Default": "Debug",

1. Ensure that you commit your changes.

2. Navigate to ArgoCD > Worker Application and refresh.

Common log search patterns

The following are some common search patterns when looking through your OpenSearch Logs.

• If an HTTP request to Cinchy Web/IDP fails, check the page's requests and the relevant response headers to find the "x-correlation-id" header. That header value can be used to search and find all logs associated with the HTTP request.

• When debugging batch syncs, filter the "ExecutionId" field in the logs for your batch sync execution ID to narrow down your search.

• When debugging real time syncs, search for your data sync config name in the Event Listener or Workers logs to find all the associated logging information.

Set up an index

The first step to utilizing the power of OpenSearch Dashboards is to set up an index to pull data from your sources. An Index Pattern identifies which indices you want to explore. An index pattern can point to a specific index, for example, your log data from yesterday, or all indices that contain your log data.

1. Login to OpenSearch. You would have configured the access point during your deployment installation; traditionally it will be found at <baseurl>/dashboard.

1. Navigate to the Stack Management tab in the left navigation menu (Image 1).

1. From the left navigation, click on Index Patterns (Image 2).

1. Click on the Create Index Pattern button.

2. To set up your index pattern, you must define the source. OpenSearch will list the sources available to you on the screen below. Input your desired source(s) in the text box (Image 3).

1. Configure your index pattern settings (Image 4).

• Time field: Select a primary time field to use with the global time filter

• Custom index pattern ID: By default, OpenSearch gives a unique identifier to each index pattern. You can use this field to optional override the default ID with a custom one.

1. Once created, you can review your Index Patterns from the Index Patterns page (Image 5).

1. Click on your Index Pattern to review your fields (Image 6).

Create a visualization

You can pull out any data from your index sources and view them in a variety of visualizations.

1. From the left navigation pane, click Visualize (Image 7).

1. If you have any Visualizations, they will appear on this page. To create a new one, click the Create Visualization button (Image 8).

1. Select your visualization type from the populated list (Image 9).

1. Choose your source (Image 10). If the source you want to pull data from isn't listed, you will need to set it up as an index first.

1. Configure the data parameters that appear in the right hand pane of the Create screen. These options will vary depending on what type of visualization you choose in step 3. The following example uses a pie chart visualization (Image 11):

• Metrics

  • Aggregation: Choose how you want your data aggregated. This example uses Count.

  • Custom Label: You can use this optional field for custom labelling.

• Buckets

  • Aggregation: Choose how you want your data aggregated. This example uses Split Slices > Terms.

  • Field: This drop down is populated based on the index source your chose. Select which field you want to use in your visualization. This example uses machine.os.keyword.

  • Order By: Define how you want your data to be ordered. This example uses Metric: Count, in descending order of size 10.

  • Choose whether to group other values in a separate bucket. If you toggle this on, you will need to label the new bucket.

  • Choose whether to show missing values.

• Advanced

  • You can optionally choose a JSON input. These will be merged with the OpenSearch aggregation definition.

• Options

  • The variables in the options tab can be used to configure the UI of the visualization itself.

1. You can also further focus your visualization:

• Use DQL to search your index data (Image 12). You can also save any queries you write for easy access by clicking on the save icon.

• Add a filter on any of your fields (Image 13).

• Update your date filter (Image 14).

1. Click save when finished with your visualization.

Create a dashboard

Once you have created your visualizations, you can combine them together on one Dashboard for easy access.

1. From the left navigation pane, click on Dashboards (Image 15).

1. If you have any Dashboards, they will appear on this page. To create a new one, click the Create Dashboard button (Image 16).

1. The "Editing New Dashboard" screen will appear. Click on Add an Existing object (Image 17).

1. Select any of the visualizations you created and it will automatically add to your Dashboard (Image 18). Repeat this step for as many visualizations as you'd like to appear.

1. Click Save to finish (Image 19).

Update your OpenSearch password

Your OpenSearch password can be updated in your deployment.json file (you may have renamed this during your original deployment).

1. Navigate to "cluster_component_config > OpenSearch.

2. OpenSearch has two users that you can configure the passwords for: Admin and Kibana Server. Kibana Server is used for communication between the opensearch dashboard and the opensearch server. The default password for both is set to "password";. To update this, you will need to use a machine with docker available.

3. Update your Admin password:

   1. Your password must be hashed. You can do so by running the following command on a machine with docker available, inputting your new password where noted:

   Copy

   docker run -it opensearchproject/opensearch /usr/share/opensearch/plugins/opensearch-security/tools/hash.sh -p <<newpassword>>

   1. Navigate to "opensearch_admin_user_hashed_password" and input your hashed password.

   2. You must also provide your password in a base64 encoded format; input your cleartext password here to receive your new encoded password.

   3. Navigate to "opensearch_admin_user_password_base64" and input your encoded password.

4. Update your Kibana Server password:

   1. Your password must be hashed. You can do so by running the following command on a machine with docker available, inputting your new password where noted:

   Copy

   docker run -it opensearchproject/opensearch /usr/share/opensearch/plugins/opensearch-security/tools/hash.sh -p <<newpassword>>

   1. Navigate to "opensearch_kibanaserver_user_hashed_password" and input your hashed password.

   2. You must also provide your new password in cleartext. Navigate to "opensearch_kibanaserver_user_password" and input your cleartext password.

5. Run the below command in the root directory of your devops.automations repo to update your configurations. If you have changed the name of your deployment.json file, make sure to update the command accordingly.

   Copy

   dotnet Cinchy.DevOps.Automations.dll "deployment.json"

6. Commit and push your changes.

7. If your environment isn't set-up to automatically apply upon configuration,navigate to the ArgoCD portal and refresh your component(s). If that doesn't work, re-sync.

ArgoCD Overview

ArgoCD is implemented as a Kubernetes controller which continuously monitors running applications and compares the current, live state against the desired target state (as specified in your Git repository).

You can use ArgoCD's dashboard (Image 1) to visually monitor your namespaces and pods, and to quickly visualize deployment issues. It can easily show you what your cluster or pods are doing, and if they're healthy.

Get started with ArgoCD

ArgoCD has a robust set of documentation that can help you to get started with the application. We recommend the following two pages:

• A General Overview

• ArgoCD Metrics

ArgoCD dashboard

Your ArgoCD dashboard has a lot of important information about how your Cinchy instance is behaving.

Application tiles view

The application tiles view is the default view when logging into ArgoCD. You can also access it through the grid widget in the upper right hand corner of the screen.

These application tiles each represent a pod in one of your Cinchy environments. By default, you should have an application tile for each of the following pods per namespace (Image 2):

• Base

• Connections

• Event Listener

• IDP

• Maintenance CLI

• Meta Forms

• Web

• Worker

Each application tile shows you some important information about the pod, including its health and sync status. You can use the Sync, Refresh, and Delete buttons to manage your pods.

You can also click the star button on any application title to favourite the associated pod.

Pie chart view

The pie chart view (Image 3) shows an easy visualization of the health of your applications. Access this view by clicking on the pie chart widget located in the upper right hand corner of the screen.

Filtering

You can filter your view so that it only shows certain data. You can find the various filter options from the Filter column on the left hand side of any of the data views.

• Favorites Only: Filter by only favorite tiles (Image 4).

• Sync Status: You can easily view what's out of sync using this filter (Image 5).

• Health Status: Check on the health of your applications by filtering using these parameters (Image 6).

• Labels: Use these labels to filter by specific instances/environments (Image 7).

• Projects: If you choose to group your applications into projects, you can filter them using this tile (Image 8).

• Clusters: If you have more than one cluster, you can filter for it using the Clusters tile (Image 9).

• Namespace: Lastly, you can filter by Namespace. The below example shows a filter based on the dev-aurora-1 namespace (Image 10).

Detailed view

To bring up a more detailed view of your applications, click on the application tile. This view will show you all components, their health and their sync status (Image 11). You can use the top navigational buttons perform actions such as syncing, rolling back, or refreshing.

This view can be useful for load testing, since you can see each individual pod spinning up and down.

Health and sync status

View the status of your apps by looking at the health and sync status along the top of the page (Image 12).

Summary information

Clicking on any individual pod or component tile in this view will bring up its information, including a Summary, a list of Events, your Manifest, and Parameters (Image 13).

Sync policies

From the detailed tile summary you can also set your sync policies, such as automation, resource pruning, and self healing (Image 14).

Logs

You can click on the Logs tab in your detailed summary page to view the applicable logs for your selected pod (Image 15). You can filter, follow, snooze, copy, or download any logs.

Monitoring and alerting

OpenSearch comes with the ability to set up alerts based on any number of monitors. You can then push these alerts via email, should you desire.

Definitions:

Create your destination

Your destination will be where you want your alerts to be pushed to. OpenSearch supports various options, but this guide focuses on email.

1. From the left navigation pane, click Alerting (Image 1).

1. Click on the Destinations Tab > Add Destination

2. Add a name to label your destination and select Email as type (Image 2)

1. You will need to assign a Sender. This is the email address that the alert will send from when you specify this specific destination. To add a new Sender, click Manage Senders (Image 3).

1. Click Add Sender

2. Add in the following information (Image 4):

• Sender Name

• Email Address

• Host (this is the host address for the email provider)

• Port (this is the Port of the email provider)

• Encryption

1. You will need to assign your Recipients. This is the email address(es) that will receive the alert when you specify this specific destination. To add a new Recipient, you can either type their email(s) into the box, or click Manage Senders to create an email group (Image 5).

1. Click Update to finish your Destination.

Authenticate your sender

You will need to authenticate your sender for emails to come through. Please contact Cinchy Customer Support to help you with this step.

• Via email: support@cinchy.com

• Via phone: 1-888-792-6051

• Through the support portal: Support Portal

Create your monitor

Your monitor is a job that runs on a defined schedule and queries OpenSearch indices. The results of these queries are then used as input for one or more triggers.

1. From the Alerting dashboard, select Monitors > Create Monitor (Image 6).

1. Under Monitor Details, add in the following information (Image 7).

• Monitor Name

• Monitor Type (This example uses Per Bucket)

  • Whereas query-level monitors run your specified query and then check whether the query’s results triggers any alerts, bucket-level monitors let you select fields to create buckets and categorize your results into those buckets.

  • The alerting plugin runs each bucket’s unique results against a script you define later, so you have finer control over which results should trigger alerts. Each of those buckets can trigger an alert, but query-level monitors can only trigger one alert at a time.

• Monitor Defining Method: the way you want to define your query and triggers. (This example uses Visual Editor)

  • Visual definition works well for monitors that you can define as “some value is above or below some threshold for some amount of time.”

  • Query definition gives you flexibility in terms of what you query for (using the OpenSearch query DSL) and how you evaluate the results of that query (Painless scripting).

• Schedule: Choose a frequency and time zone for your monitor.

1. Under Data Source add in the following information (Image 8):

• Index: Define the index you want to use as a source for this monitor

• Time Field: Select the time field that will be used for the x-axis of your monitor

1. The Query section will appear differently depending on the Monitor Defining Method selected in step 2 (Image 9). This example is using the visual editor.

To define a monitor visually, select an aggregation (for example, count() or average()), a data filter if you want to monitor a subset of your source index, and a group-by field if you want to include an aggregation field in your query. At least one group-by field is required if you’re defining a bucket-level monitor. Visual definition works well for most monitors.

Add a trigger

A trigger is a condition that, if met, will generate an alert.

1. To add a trigger, click the Add a Trigger button (Image 10).

1. Under New Trigger, define your trigger name and severity level (with 1 being the highest) (Image 11).

1. Under Trigger Conditions, you will specify the thresholds for the query metrics you set up previously (Image 12). In the below example, our trigger will be met if our COUNT threshold goes ABOVE 10000.

You can also use keyword filters to drill down into a more specific subset of data from your data source.

1. In the Action section you will define what happens if the trigger condition is met (Image 13). Enter the following information to set up your Action:

• Action Name

• Destination

• Message Subject: In the case of an email alert, this will be the email subject line.

• Message: In the case of an email alert, this will be the email body.

• Perform Action: If you’re using a bucket-level monitor, decide whether the action is performed per execution or per alert.

• Throttling: Enable action throttling if you wish. Use action throttling to limit the number of notifications you receive within a given span of time.

1. Click Send Test Message, if you want to test that the alert functions correctly.

2. Click Save.

Example alerts

• Monitor Cluster Metrics

Alerting on Stream Errors

This example pushes an alert based on errors. We will monitor our Connections stream for any instance of 'error', and push out an alert when our trigger threshold is hit.

1. First we create our Monitor by defining the following (Image 14):

• Index: This example looks at Connections.

• Time Field

• Time Range: Define how far back you want to monitor

• Data Filter: We want to monitor specifically whenever the Stream field of our index is stderr (standard error).

This is how our example monitor will appear; it shows when in the last 15 days our Connections app had errors in the log (Image 15).

1. Once our monitor is created, we need to define a trigger condition. When this condition is met, the alert will be pushed out to our defined Recipient(s). In this example we want to be alerted when there is more than one stderr in our Connections stream (Image 16). Input the following:

• Trigger Name

• Severity Level

• Trigger Condition: In this example, we use IS ABOVE and the threshold of 1.

The trigger threshold will be visible on your monitoring graph as a red line.

Alerting on Kubernetes restarts

This example pushes an alert based on the kubectl.kubernetes.io/restartedAt annotation, which updates whenever your pod restarts. We will monitor this annotation across our entire product-mssql instance, and push out an alert when our trigger threshold is hit.

1. First we create our Monitor by defining the following (Image 17):

• Index: This example looks at the entire product-mssql instance.

• Time Field

• Query: This example is using the total count of the kubectl.kubernetes.io/restartedAt annotation.

• Time Range: Define how far back you want to monitor. This example goes back 30 days.

This is how our example monitor will appear; it shows when in the last 30 days our instance had restarts (Image 18).

2. Once our monitor is created, we need to define a trigger condition. When this condition is met, the alert will be pushed out to our defined Recipient(s). In this example we want to be alerted when there is more than 100 restarts across our instance (Image 19). Input the following:

• Trigger Name

• Severity Level

• Trigger Condition: In this example, we use IS ABOVE and the threshold of 100.

The trigger threshold will be visible on your monitoring graph as a red line.

Alerting on status codes

This example pushes an alert based on status codes. We will monitor our entire instance for 400 status codes and push out an alert when our trigger threshold is hit.

1. First we create our Monitor by defining the following (Image 20):

• Index: This example looks across the entire product-mssql-1 instance.

• Time Field

• Time Range: Define how far back you want to monitor. The time range for this example is the past day.

• Data Filter: We want to monitor specifically whenever the Status Code is 400 (bad request).

This is how our example monitor will appear (note that there are no instances of a 400 status code in this graph) (Image 21).

1. Once our monitor is created, we need to define a trigger condition. When this condition is met, the alert will be pushed out to the defined Recipient(s). In this example we want to be alerted when there is at least one 400 status code across out instance (Image 22). Input the following:

• Trigger Name

• Severity Level

• Trigger Condition: In this example, we use IS ABOVE and the threshold of 0.

The trigger threshold will be visible on your monitoring graph as a red line.

Prerequisites

• Requires CLI v4.7+

Maintenance

Cinchy performs maintenance tasks through the CLI. This currently includes the data erasure and data compression deletions.

Cinchy.Maintenance.CLI.exe maintenance -s "cinchyBaseURL" -u username -p "encryptedPassword" -t 180

Overview

The Cinchy platform provides a built-in solution for securely storing secrets known as the Cinchy Secrets Table. Built with adherence to Cinchy’s Universal Access Controls, this table functions as a key vault similar to services like Azure Key Vault or AWS Secrets Manager. It allows you to store sensitive data that's accessible only to specific user groups with authorized access.

Within the Connections UI, you can use variables stored in this table, which then resolve as secrets. This approach ensures careful handling of confidential information. Some common use cases include:

• Including them in a connection string.

• Using them in REST Headers, URLs, or the request body.

• Configuring the Listener via the Listener Config table.

Cinchy has also introduced a new API endpoint for retrieving your stored secrets.

Creating a secret

To create a secret in Cinchy:

1. Navigate to the [Cinchy].[Secrets] table on your platform (see Image 1).

2. Provide the following details for your secret:

Call a secret via API

Cinchy has a new API endpoint designed for retrieving secrets. By utilizing the endpoint provided below, you can specify the <base-url>, <secret-name>, and <domain-name> to retrieve the desired secret.

This endpoint functions seamlessly with Cinchy’s Personal Access Token capability, along with Access Tokens obtained from your Identity Provider (IDP).

Blank Example:

<base-url>/api/v1.0/secrets-manager/secret?secretName=<secret-name>&domain=<domain-name>

Populated Example:

The example below uses ExampleSecret as a secretName and Sandbox as the domain:

Cinchy.net/api/v1.0/secrets-manager/secret?secretName=<ExampleSecret>&domain=<Sandbox>

The API response will be in the following format:

{
    "secretValue": "password123"
}

Use a secret as a connections variable

You can use secrets stored in the Cinchy Secrets table as variables for your data syncs, wherever you use a variable. For instance, you can incorporate them within a connection string, an access key ID, or within a REST Source or Destination in the Header.

To use a Secret within Connections:

1. In the Connections UI, navigate to Info > Variables.

2. Under the Variables section, select Secret.

3. Enter the name of your variable.

4. Under the Value dropdown, select the secret you want to assign from the Secrets table.

Use a secret in real-time syncs

You can also use your Cinchy Secrets when configuring your Listener for real-time syncs.

To use a secret in real-time syncs:

1. When configuring your sync, navigate to the Info Tab > Variables.

2. Under the Variables section, choose Secret.

3. Input the name of your variable.

4. Under the Value dropdown, choose the secret you intend to assign from the Secrets table.

5. Go to the Source tab.

6. Within the Listener section, input the secret variables as values for the relevant property in your Topic or Connection Attribute fields.

   For example:

   Copy

   {
     "InstanceAuthUrl": "@Url",
     "ApiVersion": 41.0,
     "GrantType": "@GrantType",
     "ClientId": "@ClientId",
     "UserName": "@Username",
     "Password": "@Password"
   }

Use a secret in the Listener Config table

You can also add a secret that's attached to a variable to the Topic or Connection Attributes in the Listener Config table.

1. Open the Listener Config table.

2. Select the row that corresponds to your data sync.

3. Select the Topic or Connection Attribute cell you want to change.

4. Replace the value for a property with the variable assigned to a secret.

For example, in the JSON code below, the Connection Attribute property connectionString is replaced with the @connectionString variable defined in the data sync.

{
  "connectionString": "@connectionString",
  "retryConfiguration": {
    "retryMaxAttempts": "2",
    "retryDelayStrategy": "Linear"
  }
}

Listener Config parameters

The following table provides an overview of which parameters you can use as secrets for each event connector type.

Introduction

and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives you the power to ask for exactly what you need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.

In a Cinchy context, the primary audience for the GraphQL betta **are developers writing apps on top of Cinchy.**It's a simple, efficient, new way to retrieve and manage data and build apps on via API.

GraphQL was introduced to Cinchy as a supplement to the and the functions. With GraphQL, not only are your app building capabilities streamlined and more powerful, but there is no switching between CQL and code; all code changes can be done within the GraphQL user interface. It also adheres to your defined access controls, included anonymous level access, ensuring that your data remains secure.

Use GraphQL

You can access GraphQL at the following URL: <baseurl>/graphqlplayground.

GraphQL has a robust set of documentation, videos, and training resources that will help you realize its full capabilities. Here are some to get started:

Notes

• When writing your query, you can bring up an auto complete menu of fields by hitting the ctrl+space keys when your mouse in inside a {. The fields brought up will be related to the specific level you are on.

• You can use the # symbol for in-line commenting.

Errors

The following section contains common errors and their solutions.

Server can't be reached

• Problem: If your Cinchy instance times out and prompts you to re-enter your credentials/SSO authentication, you might get the above error when trying to hit the GraphQL endpoint again.

• Solution: Log out and log back in to Cinchy. Hit the GraphQL endpoint again and refresh to remove the error.

Query examples

The following are some query examples to help you get a feel for using GraphQL in a Cinchy context.

Example 1: Return data from a single source

This example returns data from a single source, the Videos table in the CinchyTV domain. It specifically requests all data in the Title column of the table.

As a reminder, queries must adhere to the data structure in Cinchy. You must first hit the domain (in this example: Cinchy TV) and then the table (in this example: Videos)

Example query

Example response

Example 2: Return data from multiple sources

This example returns data from two sources, the Applets table and the Users table, both in the same Cinchy domain.

It requests the following information:

• The Name, Description, Application URL, Version, and Domain Name of all the applets on the environment

• The Email Address of every user on the environment.

Example query

Example results

Example 3: Use a data filter

This example uses a filter to return a specific subset of data. This example returns all results from the Videos table in the CinchyTV domain that contains the search term "Apps" in the Title field/column.

The query will give us the Title and Description of all matching data rows.

Example query

Example Results

Introduction

Cinchy 2.0 has added the feature to encrypt data at rest. This means that you can encrypt data in the database such that users with access to view data in the database will see in those columns. All users with authorized access to the data via Cinchy will see the data as plain text. To use this feature, your database administrator will be need to create a database master key (see below for instructions).

Create master Key in database

1. The first step is to create a master key in the database. Do so by connecting directly whichever database your Cinchy instance is running on.

2. Run the below query to create your master key:

3. You can now encrypt data via the user interface (Image 1):

Backup master key

After you have created your master key you can create a backup file of that key in case any data corruption occurs in future.

You will need the password you used to create your master key to complete this operation.

1. Run the following command:

Restore master key

In the use case where you require to restore your master key due to data corruption, you may use the following steps.

You will need the password you used to create you master key to complete this operation.

1. Run the following command:

Rather than traditional code-centric applications which creates data silos, you can build meta-data driven application experiences directly on the Cinchy platform. These look and feel like regular applications, but persists its data on the data network autonomously, rather than managing its own persistence.

These experiences automatically adapt as your data evolves (Image 1).

Once you deploy your UI, API and logic, you will need to create an integrated client to leverage the data fabric for persistence and controls. If you would like a link to your experience from the data fabric, you will need to create an Experience in the Applets table. See to see how to set up both. The Cinchy Platform also comes with a built-in Experience called My Data Network, this is a tool to help you visualize your data through its connections. You can read more about the or create your own on your own data.

Overview

Metadata Quality Exceptions (MDQE) can send out notifications based on a set of rules and the “exceptions” that break them. This powerful tool can be used to send notifications for exceptions such as:

• Health checks returning a critical status

• Upcoming Project Due Dates/Timelines

• Client Risk Ratings reaching a high threshold

• Tracking Ticket Urgency or Status markers

• Unfulfilled and Pending Tasks/Deliverables

• Etc.

MDQE monitors for specific changes in data, and then pushes out notifications when that change occurs.

Prerequisites

• You will need to have the

• Request the installation package from the

Install MDQE

To install MDQE in your Cinchy environment, follow the below steps:

1. Download the MDQE Installation package.

2. Unzip the file.

3. Open an instance of PowerShell as an Administrator and navigate to the path where you extracted the MDQE package in step 2 > Metadata Quality Exceptions V.x > Metadata Quality Exceptions.

4. Run the following command to install all MDQE components in your environment, using the table below as a parameter guide.

1. Within the MDQE file package, navigate to the PowerShell - DQE Orchestration folder.

2. Extract the contents.

3. Navigate to the _config.json file and update the parameter values using the below as a guide. Make sure to save when finished.

Create MDQE rules

1. In the environment where you installed MDQE, search for and open the [Cinchy MDQE].[Rules] table.

2. Using the “Create Rule” view and the following data, create your Rule:

View all exceptions

All exceptions can be viewed in the [Cinchy MDQE].[Data Quality Exceptions] table

• The Default view only displays exceptions assigned to the currently logged in user.

• The All Data view displays all exceptions. This is only visible with admin privileges.

Debugging

Ways to debug your rules:

• If your PowerShell scripts aren't running: Run the script files in the PowerShell - DQE Orchestration folder using an IDE to make sure that all the configurations are correct.

• Check to see if your bugged Rule is part of the “Invalid Rules” view.

• If you have admin privileges, check to see if an equivalent SQL statement has been created in the [Rules CQL] table.

• Check if there is a row for the Rule’s Signature value in the [Cinchy].[Formatting Rules] table.

Schedule MDQE jobs

You can use the Windows Task Scheduler to run MDQE jobs at regular intervals.

1. Navigate to your MDQE installation package > Windows Task Scheduler Jobs folder.

2. Import the files into the Windows Task Scheduler, updating the parameters accordingly.

Network Map

Cinchy comes out of the box with a system applet called Network Map (Image 1), which is a visualization of your data on the platform and how everything interconnects.

My Data Network is another way to view and navigate the data you have access to within Cinchy.

Each node represents a table you have access to within Cinchy, and each edge is one link between two tables. The size of the table is determined by the number of links referencing that table. The timeline on the bottom allows you to check out your data network at a point in the past and look at the evolution of your network.

It uses the user's entitlements for viewable tables and linked columns.

When you click on a node, you will see its description in the top right hand corner. You can click the Open button to navigate to the table (Image 2).

You will find the Network Map data experience on the Homepage (Image 3).

Custom network visualizer

You can also set up a custom network visualizer as follows:

Nodes

The nodes query defines the nodes in the network (Image 4 and 5).

Edges

The edges query defines the relationships between the nodes (Image 6).

Node groups

Node groups are an optional query you can provide to group your nodes (Image 7 and 8).

Timeline

If no start or end date is specified, the data network is just shown as is. If there's a start or end date, the other CQL queries must have a @date parameter and that will be used to render the data network at a point in time.

You can use @date between [Modified] and [Replaced] with a version history query to see data at a specific time. You can also simply use @date > [Created] if it's an additive system.

Timeline start date

This CQL should return a date value as startDate.

Timeline end date

This CQL should return a date value as endDate.

Slicers

To use slicers, you must define the slicers in the Slicers column and add the additional attributes to the nodes query.

Attribute is the column name from the nodes query, displayName is what shows up in the visualizer (Image 9 and 10).

System tables

All the information above is entered into the [Cinchy].[Networks] table. To access the network, go to:

<Cinchy URL>/Cinchy/apps/datanetworkvisualizer?network=<NAME>

Alternatively you can go to My Data Network and then add ?network=<NAME> to the end of it.

It's highly recommended to add a new applet for each custom data network visualizer for ease of access.

Optional URL parameters

Cinchy version 5.2 added the ability to include new parameters on the URL path for your network visualizer to focus your node view. You can now add Target Node, Depth Level, and Max Depth Level Parameters, if you choose.

Example: <base url>/apps/datanetworkvisualizer?targetNode=&maxDepth=&depthLevel=

• Target Node: Using the Target Node parameter defines which of your nodes will be the central node from which all connections branch from.

  • Target Node uses the TableID number, which you can find in the URL of any table.

  • Example: <base url>/apps/datanetworkvisualizer?targetNode=8 will show TableID 8 as the central node

• Max Depths: This parameter defines how many levels of network hierarchy you want to display.

  • Example: <base url>/apps/datanetworkvisualizer?maxDepth=2 will only show you two levels of connections.

• Depth Level: Depth Level is a UI parameter that will highlight/focus on a certain depth of connections.

  • Example: <base url>/apps/datanetworkvisualizer?DepthLevel=1 will highlight all first level network connections, while the rest will appear muted.

The below example visualizer uses the following URL: <base url>/apps/datanetworkvisualizer?targetNode=8&maxDepth=2&depthLevel=1

• It shows Table ID 8 ("Groups") as the central node.

• It only displays the Max Depth of 2 connections from the central node.

• It highlights the nodes that have a Depth Level of 1 from the central node.

Example network

The following is an example of a network map (Image 11).

For ease of testing, save the following as saved queries and then in the Networks table, add exec [Domain].[Saved Query Name] as the CQL queries.

Node groups CQL

Nodes CQL

Edges CQL

System Properties is a table within Cinchy for managing system properties, such as default time zones, system lockout durations, password expiration, password properties, password attempts allowed etc.

Setup

The Default of the Systems Properties table is set up as follows:

Configure the table values

The System Properties requirements can be changed by an admin user by editing the 'Value' columns where applicable:

Default time zone

Users can set their own time zones in their user profile. The default time zone values are entered manually and must correspond with the one of the values in the Default Time Zone value list located below. For changes to take effect, you must either clear the application cache or restart the instance.

Minimum password length

The minimum password length is 8 characters. The length will always default to 8 if an invalid value is provided, or if you attempt to set it to less than 8. This number can be changed (made higher than 8) in the Value column to require users to have longer passwords.

Password requires symbols

This property specifies whether symbols are required in a user's password. The 'Value' 0 means symbols aren't required and 1 means they're required.

Password requires numbers

This property specifies whether numbers are required in a user's password. The 'Value' 0 means numbers aren't required and 1 means they're required.

Password expiration (days)

This property specifies how many days until a password expires. This defaults to 90 but can be set to be shorter or longer by changing the number in the 'Value' column.

Password attempts allowed

This property specifies how many bad password attempts a user can make before they're locked out of the system. The default is 3 but this can be set to be more or less attempts by changing the number in the 'Value' column.

System lockout duration (minutes)

This property specifies how long a user is locked out of the system once they've run out of bad password attempts. The default is 15 minutes but this can be set to be shorter or longer by changing the number in the 'Value' column.

Maintenance enabled

This property, defaulted to 0, shows this warning when a data owner is setting up Data Erasure or Data Compression on a table (Image 2). It's the administrator's responsibility to set up a scheduled maintenance job for performing compression and erasure, and then to change the property to 1 so that the warning no longer appears.

Forbidden Passwords Table

You can add more blocked passwords to this list as well, and users won't be able to set their password to it (this can be used to add your company's name, or to import other blocked password lists). The check against the list is case insensitive.

Like other password policies, this check occurs when your password changes, so to enforce this you will need to set all passwords to be expired.

References

Default Time Zone value list