Introduction

Cinchy v5 on Kubernetes offers and recommends a robust set of open-source tools and applications for monitoring, logging, and visualizing the data on your Cinchy instances.

Click on the respective tool below to learn more about it:

• Grafana

• Opensearch

• ArgoCD

Table of Contents

1. Grafana Overview

Grafana is an open source analytics and interactive visualization web application. When connected to your Cinchy platform, it provides charts, graphs, and alerting capabilities (Image 1).

Grafana, and its paired application Prometheus (which consumes metrics from the running components in your environment) is the recommended visualization application for Cinchy v5 on Kubernetes.

2. Getting Started with Grafana

Grafana has a robust library of documentation of tutorials designed to help you learn the fundamentals of the application. We have listed a few notable ones below:

• Exploring your Metrics

• Best Practices for Creating Dashboards

• Building a Dashboard

• Guide to Dashboard Types and Capabilities

• Creating a Managed Alert

• All Documentation

2.1 Accessing your Saved Dashboards:

There are some saved dashboards that come out of the box with your Cinchy deployment. These dashboards will provide a great jumping off point for your metrics monitoring, and you can always customize, manage, and add further dashboards at your leisure.

1. Navigate to the left navigation pane, select the Dashboards icon > Manage (Image 2).

2. You will see a list of all of the Dashboards available to you (Image 3). Clicking on any of them will take you to a full metrics view (Image 4).

3. You can favourite any of your commonly used or most important dashboards by clicking on the star (Image 5).

4. Once you have favourited a dashboard, you can easily find it by navigating to the left navigation pane, select the Dashboards icon > Home. This will open the Dashboards Home. You can see both your favourite and your recent dashboards in this view (Image 6)

3. Recommended Dashboards

Your Cinchy v5 deployment comes with some out-of-the-box dashboards premade for you. You are able to customize these to suit your specifications. The following are a few notable ones:

3.1 Kubernetes/Compute Resources/Cluster

Purpose: This dashboard provides a general overview of your entire cluster including all of your environments and pods (Image 7).

Metrics:

The following are some example metrics that you could expect to see from this dashboard:

• CPU Usage

• CPU Quota

• Memory Usage

• Memory Requests

• Current Network Usage

• Bandwidth (Transmitted and Received)

• Average Container Bandwidth by Namespace

• Rate of Packets

• Rate of Packets Dropped

• Storage IO & Distribution

3.2 Kubernetes/Compute Resources/Namespace (Workloads)

Purpose: This dashboard is useful for looking at environment specific details (Image 8). You can use the namespace drop down menu to select which environment you want to visualize (Image 9). This can be particularly helpful during load testing. You are also able to drill down to a specific workload by clicking on its name.

Metrics:

The following are some example metrics that you could expect to see from this dashboard:

• CPU Usage

• CPU Quota

• Memory Usage

• Memory Quota

• Current Network Usage

• Bandwidth (Transmitted and Received)

• Average Container Bandwidth by Workload

• Rate of Packets

• Rate of Packets Dropped

4. Setting Up Alerts

Grafana allows you to set up push alerts against your dashboards and queries. Once you have created your dashboard, you can follow the steps below to set up your alert.

4.1 Set Up Your Notifications Channel

Your notifications channel refers to who will be receiving your alert. To set one up:

1. Click on the Alert icon on the left navigation tab (Image 10), and locate "Notifications Channel"

2. Click the "Add a Channel" button

3. Add in the following parameters, including any optional checkboxes you wish to use (Image 11):

Name: The name of this channel

Type: You have several options here, but email is the most common

Addresses: Input all the email addresses you want to be notified of this alert, separated by a comma

4. Click Test to send out a test email, if desired.

5. Save your Notification Channel

4.2 Setting up your Alert

The following details how to set up alerts on your dashboards. You can also set up alerts upon creation of your dashboard from the same window.

1. Navigate to the dashboard and dashboard panel that you want to set up an alert for. In this example, we are setting up an alert for CPU usage on our cluster.

2. Click on the dashboard name > Edit

3. Click on the Alert tab (Image 12).

4. Input the following parameters to set up your alert (Image 13):

• Alert Name: A title for your alert

• Alert Timing: Choose how often to evaluate and for how long. In this example it is evaluated every minute for five minutes.

• Conditions: Here you can set your threshold conditions for when an alert will be sent out. In this example, it is sent when the average of query A is above 75.

• Set what happens if there's no data, or an error in your data

• Add in your notification channel (i.e., who will be sent this notification)

• Add a message to accompany the alert.

• Click Apply > Save to finalize your alert.

5. Recommended Alerts

Below are a few alerts we recommend setting up on your Grafana.

5.1 CPU Usage

Set up this alert to notify you when the CPU Usage on your nodes exceeds a specified limit.

Dashboard Query:

You can use the following example queries to set up a dashboard that will capture CPU Usage by Node (Image 14).

avg by (node_name) (100 - ((avg by (cpu,node_name) (irate(node_cpu_seconds_total{mode="idle"}[1m]))) * 100))

100 - ((avg by (cpu,node_name) (irate(node_cpu_seconds_total{mode="idle"}[1m]))) * 100)

Alert:

Set up your alert. This example uses a threshold limit of 75 (Image 15).

5.2 Memory Usage

Set up this alert to notify you when the Memory Usage on your nodes exceeds a specified limit.

Dashboard Query:

You can use the following example queries to set up a dashboard that will capture CPU Usage by Node (Image 16)

((node_memory_MemTotal_bytes-node_memory_MemAvailable_bytes) / (node_memory_MemTotal_bytes))*100

Alert:

Set up your alert. This example uses a threshold limit of 85 (Image 17).

5.3 Disk Usage

Set up this alert to notify you when the Disk Usage on your nodes exceeds a specified limit.

Dashboard Query:

You can use the following example queries to set up a dashboard that will capture Disk Usage by Node (Image 18)

(sum((node_filesystem_size_bytes))by(node_name) - sum((node_filesystem_free_bytes))by(node_name)) *100/(sum((node_filesystem_avail_bytes))by(node_name)+(sum((node_filesystem_size_bytes))by(node_name) - sum((node_filesystem_free_bytes))by(node_name)))

Alert:

Set up your alert. This example uses a threshold limit of 80 (Image 17).

5.4 Iowait

Set up this alert to check the amount of iowait from the CPU. A high value usually indicates a slow/overloaded HDD or Network.

Dashboard Query:

You can use the following example queries to set up a dashboard that will capture the CPU Iowait (Image 19).

(sum(irate(node_cpu_seconds_total{mode="iowait"}[1m]))by(node_name) * 100 / 4)

Alert:

Set up your alert. This example uses a threshold limit of 60 (Image 19).

6. Updating your Grafana Password

Your Grafana password can be updated in your deployment.json file (you may have renamed this during your original deployment).

1. Navigate to "cluster_component_config" > "grafana".

2. The default password is set to "prom-operator"; update this with your preferred new password, written in clear text.

3. Run the below command in the root directory of your devops.automations repo to update your configurations. If you have changed the name of your deployment.json file, make sure to update the command accordingly.

dotnet Cinchy.DevOps.Automations.dll "deployment.json"

4. Commit and push your changes.

5. If your environment is not set-up to automatically apply upon configuration,navigate to the ArgoCD portal and refresh your component(s). If that does not work, re-sync.

Table of Contents

1. ArgoCD Overview

ArgoCD is implemented as a kubernetes controller which continuously monitors running applications and compares the current, live state against the desired target state (as specified in your Git repo).

You can use ArgoCD's dashboard (Image 1) to visually monitor your namespaces and pods, and to quickly visualize deployment issues. It can easily show you what your cluster or pods are doing, and if they are healthy.

2. Getting Started with ArgoCD

ArgoCD has a robust set of documentation that can help you to get started with the application. We recommend the following two pages:

3. Your ArgoCD Dashboard

Your ArgoCD dashboard contains a lot of important information about how your Cinchy instance is behaving.

3.1 Application Tiles View

The application tiles view is the default view when logging into ArgoCD. You can also access it through the grid widget in the upper right hand corner of the screen.

These application tiles each represent a pod in one of your Cinchy environments. By default, you should have an application tile for each of the following pods per namespace (Image 2):

• Base

• Connections

• Event Listener

• IDP

• Maintenance CLI

• Meta Forms

• Web

• Worker

Each application tile shows you some important information about the pod, including its health and sync status. You can easily use the Sync, Refresh, and Delete buttons to manage your pods.

You can also click the star button on any application title to favourite the associated pod.

3.2 Pie Chart View

The pie chart view (Image 3) shows an easy visualization of the health of your applications. Access this view by clicking on the pie chart widget located in the upper right hand corner of the screen.

4. Filtering

You can filter your view so that it only shows certain data. You can find the various filter options from the Filter column on the left hand side of any of the data views.

• Favorites Only: You can filter by only the tiles that you have favourited (Image 4).

• Sync Status: You can easily view what is out of sync using this filter (Image 5).

• Health Status: Check on the health of your applications by filtering using these parameters (Image 6).

• Labels: Use these labels to filter by specific instances/environments (Image 7).

• Projects: If you choose to group your applications into projects, you can filter them using this tile (Image 8).

• Clusters: If you have more than one cluster, you can filter for it using the Clusters tile (Image 9).

• Namespace: Lastly, you can filter by Namespace. The below example shows a filter based on the dev-aurora-1 namespace (Image 10).

5. Detailed View

To bring up a more detailed view of your applications, click on the application tile. This view will show you all of components, their health and their sync status (Image 11). You can use the top navigational buttons perform actions such as syncing, rolling back, or refreshing.

This view can be particularly useful for load testing, snice you can see each individual pod spinning up and down.

5.1 Health and Sync Status

Quickly view the status of your apps by looking at the health and sync status along the top of the page (Image 12).

5.2 Summary Information

Clicking on any individual pod or component tile in this view will bring up its information, including a Summary, a list of Events, your Manifest, and Parameters (Image 13).

5.3 Sync Policies

From the detailed tile summary you can also set your sync policies, such as automation, resource pruning, and self healing (Image 14).

5.4 Logs

You can click on the "Logs" tab in your detailed summary page to view the applicable logs for your selected pod (Image 15). You can filter, follow, snooze, copy, or download any logs.

1. Monitoring and Alerting

Opensearch comes with the ability to set up alerts based on any number of monitors. You can then push these alerts via email, should you desire.

Definitions:

1.1 Create your Destination

Your destination will be where you want your alerts to be pushed to. Opensearch supports various options, however this guide will focus on email.

1. From the left navigation pane, click Alerting (Image 1).

2. Click on the Destinations Tab > Add Destination

3. Add a name to label your destination and select Email as type (Image 2)

4. You will need to assign a Sender. This is the email address that the alert will send from when you specify this specific destination. To add a new Sender, click Manage Senders (Image 3).

5. Click Add Sender

6. Add in the following information (Image 4):

• Sender Name

• Email Address

• Host (this is the host address for the email provider)

• Port (this is the Port of the email provider)

• Encryption

7. You will need to assign your Recipients. This is the email address(es) that will receive the alert when you specify this specific destination. To add a new Recipient, you can either type their email(s) into the box, or click Manage Senders to create an email group (Image 5).

8. Click Update to finalize your Destination.

1.2 Authenticate your Sender

You will need to authenticate your sender to allow emails to come through. Please contact Cinchy Customer Support to help you with this step.

• Via email: support@cinchy.com

• Via phone: 1-888-792-6051

1.3 Create your Monitor

Your monitor is a job that runs on a defined schedule and queries OpenSearch indices. The results of these queries are then used as input for one or more triggers.

1. From the Alerting dashboard, select Monitors > Create Monitor (Image 6).

2. Under Monitor Details, add in the following information (Image 7).

• Monitor Name

• Monitor Type (This example uses Per Bucket)

  • Whereas query-level monitors run your specified query and then check whether the query’s results triggers any alerts, bucket-level monitors let you select fields to create buckets and categorize your results into those buckets.

  • The alerting plugin runs each bucket’s unique results against a script you define later, so you have finer control over which results should trigger alerts. Each of those buckets can trigger an alert, but query-level monitors can only trigger one alert at a time.

• Monitor Defining Method: the way you want to define your query and triggers. (This example uses Visual Editor)

  • Visual definition works well for monitors that you can define as “some value is above or below some threshold for some amount of time.”
• Schedule: Choose a frequency and time zone for your monitor.

3. Under Data Source add in the following information (Image 8):

• Index: Define the index you want to use as a source for this monitor

• Time Field: Select the time field that will be used for the x-axis of your monitor

4. The Query section will appear differently depending on the Monitor Defining Method selected in step 2 (Image 9). This example is using the visual editor.

To define a monitor visually, select an aggregation (for example, count() or average()), a data filter if you want to monitor a subset of your source index, and a group-by field if you want to include an aggregation field in your query. At least one group-by field is required if you’re defining a bucket-level monitor. Visual definition works well for most monitors.

1.4 Add a Trigger

A trigger is a condition that, if met, will generate an alert.

1. To add a trigger, click the Add a Trigger button (Image 10).

2. Under New Trigger, define your trigger name and severity level (with 1 being the highest) (Image 11).

3. Under Trigger Conditions, you will specify the thresholds for the query metrics you set up previously (Image 12). In the below example, our trigger will be met if our COUNT threshold goes ABOVE 10000.

You can also use keyword filters to drill down into a more specific subset of data from your data source.

4. In the Action section you will define what happens if the trigger condition is met (Image 13). Enter the following information to set up your Action:

• Action Name

• Message Subject: In the case of an email alert, this will be the email subject line.

• Message: In the case of an email alert, this will be the email body.

• Perform Action: If you’re using a bucket-level monitor, decide whether the action is performed per execution or per alert.

• Throttling: Enable action throttling if you wish. Use action throttling to limit the number of notifications you receive within a given span of time.

5. Click Send Test Message, if you want to test that the alert functions correctly.

6. Click Save.

Example Alerts

1. Alerting on Stream Errors

In this example, we are pushing an alert based on errors. We will monitor our Connections stream for any instance of 'error', and push out an alert when our trigger threshold is hit.

• Index: In this example we are looking specifically at Connections.

• Time Field

• Time Range: Define how far back you want to monitor

• Data Filter: We want to monitor specifically whenever the Stream field of our index is stderr (standard error).

This is how our example monitor will appear; it shows when in the last 15 days our Connections app had errors in the log (Image 15).

• Trigger Name

• Severity Level

• Trigger Condition: In this example, we use IS ABOVE and the threshold of 1.

The trigger threshold will be visible on your monitoring graph as a red line.

2. Alerting on Kubernetes Restarts

In this example, we are pushing an alert based on the kubectl.kubernetes.io/restartedAt annotation, which updates whenever your pod restarts. We will monitor this annotation across our entire product-mssql instance, and push out an alert when our trigger threshold is hit.

• Index: In this example we are looking at our entire product-mssql instance.

• Time Field

• Query: This example is using the total count of the kubectl.kubernetes.io/restartedAt annotattion

• Time Range: Define how far back you want to monitor. This example goes back 30 days.

This is how our example monitor will appear; it shows when in the last 30 days our instance had restarts (Image 18).

• Trigger Name

• Severity Level

• Trigger Condition: In this example, we use IS ABOVE and the threshold of 100.

The trigger threshold will be visible on your monitoring graph as a red line.

3. Alerting on Status Codes

In this example, we are pushing an alert based on status codes. We will monitor our entire instance for 400 status codes and push out an alert when our trigger threshold is hit.

• Index: In this example we are looking across out entire product-mssql-1 instance.

• Time Field

• Time Range: Define how far back you want to monitor. In this example we are looking at the past day.

• Data Filter: We want to monitor specifically whenever the Status Code is 400 (bad request).

This is how our example monitor will appear (note that there are no instances of a 400 status code in this graph) (Image 21).

• Trigger Name

• Severity Level

• Trigger Condition: In this example, we use IS ABOVE and the threshold of 0.

The trigger threshold will be visible on your monitoring graph as a red line.

Table of Contents

1. Opensearch Dashboards Overview

When deploying Cinchy v5 on Kubernetes, we recommend using Opensearch Dashboards for your logging. Opensearch is a community-driven fork of Elasticsearch created by Amazon, and it captures and indexes all of your logs into a single, easily accessible dashboard location. These logs can be queried, searched, and filtered, and Correlation IDs mean that they can also be traced across various components. These logging components take advantage of persistent storage.

Review some of the Opensearch documentation here:

2. Getting Started with Opensearch Dashboards

The below sections will guide you through setting up your first Index, Visualization, Dashboard, and Alert.

2.1 Defining your Log Level

1. Navigate to your cinchy.kubernetes/environment_kustomizations/instance_template/worker/kustomization.yaml file.

2. In the below code, copy the Base64 encoded string in the value parameter.

2. Navigate to the below Serilog section of the code and update the "Default" parameter as needed to set your log level. The options are:

1. Ensure that you commit your changes.

2. Navigate to ArgoCD > Worker Application and refresh.

2.1.1 Common Log Search Patterns

The following are some common search patterns when looking through your Opensearch Logs.

• If an HTTP request to Cinchy Web/IDP fails, check the page's requests and the relevant response headers to find the "x-correlation-id" header. That header value can be used to search and find all logs associated with the HTTP request.

• When debugging batch syncs, filter the "ExecutionId" field in the logs for your batch sync execution ID to narrow down your search.

• When debugging real time syncs, search for your data sync config name in the Event Listener or Workers logs to find all the associated logging information.

2.2 Setting up an Index

The first step to utilizing the power of Opensearch Dashboards is to set up an index to pull data from your sources. An Index Pattern identifies which indices you want to explore. An index pattern can point to a specific index, for example, your log data from yesterday, or all indices that contain your log data.

2. Navigate to the Stack Management tab in the left navigation menu (Image 1).

3. From the left navigation, click on Index Patterns (Image 2).

4. Click on the Create Index Pattern button.

5. To set up your index pattern, you must define the source. Opensearch will list the sources available to you on the screen below. Input your desired source(s) in the text box (Image 3).

6. Configure your index pattern settings (Image 4).

• Time field: Select a primary time field to use with the global time filter

• Custom index pattern ID: By default, Opensearch gives a unique identifier to each index pattern. You can use this field to optional override the default ID with a custom one.

7. Once created, you can review your Index Patterns from the Index Patterns page (Image 5).

8. Click on your Index Pattern to review your fields (Image 6).

2.3 Create a Visualization

You can easily pull out any data from your index sources and view them in a variety of visualizations.

1. From the left navigation pane, click Visualize (Image 7).

2. If you have any Visualizations, they will appear on this page. To create a new one, click the Create Visualization button (Image 8).

3. Select your visualization type from the populated list (Image 9).

5. Configure the data parameters that appear in the right hand pane of the Create screen. These options will vary depending on what type of visualization you choose in step 3. The following example uses a pie chart visualization (Image 11):

• Metrics

  • Aggregation: Choose how you want your data aggregated. This example uses Count.

  • Custom Label: You can use this optional field for custom labelling.

• Buckets

  • Aggregation: Choose how you want your data aggregated. This example uses Split Slices > Terms.

  • Field: This drop down is populated based on the index source your chose. Select which field you want to use in your visualization. This example uses machine.os.keyword.

  • Order By: Define how you want your data to be ordered. This example uses Metric: Count, in descending order of size 10.

  • Choose wheteher to group other values in a seperate bucket. If you toggle this on, you will need to label the new bucket.

  • Choose whether to show missing values.

• Advanced

  • You can optionally choose a JSON input. These will be merged with the opensearch aggregation definition.

• Options

  • The variables in the options tab can be used to configure the UI of the visualization itself.

6. There are a few ways to further focus your visualization:

• Add a filter on any of your fields (Image 13).

• Update your date filter (Image 14).

7. Click save when finished with your visualization.

2.4 Create a Dashboard

Once you have created your visualizations, you can combine them together on one Dashboard for easy access.

1. From the left navigation pane, click on Dashboards (Image 15).

2. If you have any Dashboards, they will appear on this page. To create a new one, click the Create Dashboard button (Image 16).

3. The "Editing New Dashboard" screen will appear. Click on Add an Existing object (Image 17).

4. Select any of the visualizations you created and it will automatically add to your Dashboard (Image 18). Repeat this step for as many visualizations as you'd like to appear.

5. Click Save to finish (Image 19).

3. Updating your Opensearch Password

Your Opensearch password can be updated in your deployment.json file (you may have renamed this during your original deployment).

1. Navigate to "cluster_component_config" > "opensearch".

2. There are two users that you can configure the passwords for: Admin and Kibana Server. Kibana Server is used for communication between the opensearch dashboard and the opensearch server. The default password for both is set to "password";. To update this, you will need to use a machine with docker available.

3. Update your Admin password:

   1. Your password must be hashed. You can do so by running the following command on a machine with docker available, inputting your new password where noted:

2. Navigate to "opensearch_admin_user_hashed_password" and input your hashed password.

4. Navigate to "opensearch_admin_user_password_base64" and input your encoded password.

4. Update your Kibana Server password:

1. Your password must be hashed. You can do so by running the following command on a machine with docker available, inputting your new password where noted:

2. Navigate to "opensearch_kibanaserver_user_hashed_password" and input your hashed password.

3. You must also provide your new password in cleartext. Navigate to "opensearch_kibanaserver_user_password" and input your cleartext password.

5. Run the below command in the root directory of your devops.automations repo to update your configurations. If you have changed the name of your deployment.json file, make sure to update the command accordingly.

4. Commit and push your changes.

5. If your environment is not set-up to automatically apply upon configuration,navigate to the ArgoCD portal and refresh your component(s). If that does not work, re-sync.

1. Introduction

and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives you the power to ask for exactly what you need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.

In a Cinchy context, the primary audience for the GraphQL betta are developers writing apps on top of Cinchy. It is a simple, efficient, new way to retrieve and manage data and build apps on via API.

GraphQL was introduced to Cinchy as a supplement to the and the functions. With GraphQL, not only are your app building capabilities streamlined and more powerful, but there is no switching between CQL and code; all code changes can be done within the GraphQL user interface. It also adheres to your defined access controls, included anonymous level access, ensuring that your data remains secure.

2. Using GraphQL

GraphQL can be accessed at the following URL: <baseurl>/graphqlplayground.

GraphQL has a robust set of documentation, videos, and training resources that will help you realize its full capabilities. Here are some to get started:

Other things to note:

• When writing your query, you can bring up an auto-complete menu of fields by hitting the ctrl+space keys when your mouse in inside a {. The fields brought up will be related to the specific level you are on.

• You can use the # symbol for in-line commenting.

Errors

The following section contains common errors and their solutions.

Server Cannot be Reached

• Problem: If your Cinchy instance times out and prompts you to re-enter your credentials/SSO authentication, you might get the above error when trying to hit the GraphQL endpoint again.

• Solution: Log out and log back in to Cinchy. Hit the GraphQL endpoint again and refresh to remove the error.

3. Sample Queries

The following are some sample queries to help you get a feel for using GraphQL in a Cinchy context.

Example 1: Returning Data from a Single Source

This example returns data from a single source, the Videos table in the CinchyTV domain. We are specifically requested all data in the Title column of the table.

As a reminder, queries must adhere to the data structure in Cinchy. You must first hit the domain (in this example: Cinchy TV) and then the table (in this example: Videos)

Example Query

Example Response

Example 2: Returning Data from Multiple Sources

This example returns data from two sources, the Applets table and the Users table, both in the same Cinchy domain.

We are requesting the following information:

• The Name, Description, Application URL, Version, and Domain Name of all the applets on the environment

• The Email Address of every user on the environment.

Example Query

Example Results

Example 3: Using a Data Filter

This example uses a filter to return a specific subset of data. In this example, we are returning all results from the Videos table in the CinchyTV domain that contains our search term "Apps" in the Title field/column.

The query will give us the Title and Description of all matching data rows.

Example Query

Example Results

System Properties is a table within Cinchy for managing system properties, such as default time zones, system lockout durations, password expirations, password properties, password attempts allowed etc.

Table of Contents

1. Set up

The Default of the Systems Properties table is set up as follows (Image 1):

2. Configuring the Table Values

The System Properties requirements can be changed by an admin user simply by editing the 'Value' columns where applicable:

Default Time Zone

Users can set their own time zones in their user profile. If a user does not set one up, the system default time zone will take effect. If this property does not exist or is invalid, the default time zone will default to UTC.

Minimum Password Length

The minimum password length is 8 characters. The length will always default to 8 if an invalid value is provided, or if you attempt to set it to less than 8. This number can be changed (i.e. made higher than 8) in the 'Value' column to require users to have longer passwords.

Password Requires Symbols

This property specifies whether symbols are required in a user's password. The 'Value' 0 means symbols are not required and 1 means they are required.

Password Requires Numbers

This property specifies whether numbers are required in a user's password. The 'Value' 0 means numbers are not required and 1 means they are required.

Password Expiration (Days)

This property specifies how many days until a password expires. This defaults to 90 but can be set to be shorter or longer by changing the number in the 'Value' column.

Password Attempts Allowed

This property specifies how many bad password attempts a user can make before they are locked out of the system. The default is 3 but this can be set to be more or less attempts by changing the number in the 'Value' column.

System Lockout Duration (minutes)

This property specifies how long a user is locked out of the system once they've run out of bad password attempts. The default is 15 minutes but this can be set to be shorter or longer by changing the number in the 'Value' column.

Maintenance Enabled

3. Forbidden Passwords

You can add more blocked passwords to this list as well, and users will not be able to set their password to it (this can be used to add your company's name, or to import other blocked password lists). The check against the list is case insensitive.

Like other password policies, this check occurs when your password changes, so to enforce this you will need to set all passwords to be expired.

Custom Results in the Network Map

Focus the results of your Network Map to show only the data that you really want to see with our new URL parameters.

You can now add Target Node, Depth Level, and Max Depth Level Parameters, if you choose.

Example: <base url>/apps/datanetworkvisualizer?targetNode=&maxDepth=&depthLevel=

• Target Node: Using the Target Node parameter defines which of your nodes will be the central node from which all connections branch from.

  • Target Node uses the TableID number, which you can find in the URL of any table.

  • Example: <base url>/apps/datanetworkvisualizer?targetNode=8 will show TableID 8 as the central node

• Max Depths: This parameter defines how many levels of network hierarchy you want to display.

  • Example: <base url>/apps/datanetworkvisualizer?maxDepth=2 will only show you two levels of connections.

• Depth Level: Depth Level is a UI parameter that will highlight/focus on a certain depth of connections.

  • Example: <base url>/apps/datanetworkvisualizer?DepthLevel=1 will highlight all first level network connections, while the rest will appear muted.

The below example visualizer uses the following URL: <base url>/apps/datanetworkvisualizer?targetNode=8&maxDepth=2&depthLevel=1

• It shows Table ID 8 ("Groups") as the central node.

• It only displays the Max Depth of 2 connections from the central node.

• It highlights the nodes that have a Depth Level of 1 from the central node.

1. Overview

MDQE, which stands for Metadata Quality Exceptions, can send out notifications based on a set of rules and the “exceptions” that break them. This powerful tool can be used to send notifications for exceptions such as:

• Healthchecks returning a critical status

• Upcoming Project Due Dates/Timelines

• Client Risk Ratings reaching a high threshold

• Tracking Ticket Urgency or Status markers

• Unfulfilled and Pending Tasks/Deliverables

• Etc.

In a nutshell, MDQE monitors for specific changes in data, and then pushes out notifications when that change occurs.

2. Prerequisites

• You will need to have the Cinchy CLI installed.

• Request the installation package from the Cinchy Support team.

3. Installing MDQE

To install MDQE in your Cinchy environment, follow the below steps:

1. Download the MDQE Installation package.

2. Unzip the file.

3. Open an instance of Powershell as an Administrator and navigate to the path where you extracted the MDQE package in step 2 > Metadata Quality Exceptions V.x > Metadata Quality Exceptions.

4. Run the following command to install all MDQE components in your environment, using the table below as a parameter guide.

.\CinchyDXD.ps1 install `
-s "url.com/Cinchy" `
-sso "url.com/CinchySSO" `
-u "CinchyDQE" `
-p "cinchy" `
-c "C:\Cinchy CLI\Cinchy CLI v4.12.0.564" `
-d "C:\Cinchy CLI\Cinchy CLI Error Logs" `
-y

5. Within the MDQE file package, navigate to the Powershell - DQE Orchestration folder.

6. Extract the contents.

7. Navigate to the _config.json file and update the parameter values using the below as a guide. Make sure to save when finished.

4. Creating MDQE Rules

1. In the environment where you installed MDQE, search for and open the [Cinchy MDQE].[Rules] table.

2. Using the “Create Rule” view and the following data, create your Rule:

5. Viewing All Exceptions

All exceptions can be viewed in the [Cinchy MDQE].[Data Quality Exceptions] table

• The Default view only displays exceptions assigned to the currently logged in user.

• The All Data view displays all exceptions. This is only visible with admin privileges.

6. Debugging

Ways to debug your rules:

• If your Powershell scripts aren't running: Run the script files in the Powershell - DQE Orchestration folder using an IDE to make sure that all the configurations are correct.

• Check to see if your bugged Rule is part of the “Invalid Rules” view.

• If you have admin privileges, check to see if an equivalent SQL statement has been created in the [Rules CQL] table.

• Check if there is a row for the Rule’s Signature value in the [Cinchy].[Formatting Rules] table.

7. Scheduling MDQE Jobs

You can use the Windows Task Scheduler to run MDQE jobs at regular intervals.

1. Navigate to your MDQE installation package > Windows Task Scheduler Jobs folder.

2. Import the files into the Windows Task Scheduler, updating the parameters accordingly.

Table of Contents

Introduction

Cinchy 2.0 has added the feature to encrypt data at rest. This means that you can encrypt data in the database such that users with access to view data in the database will see ciphertext in those columns. However, all users with authorized access to the data via Cinchy will see the data as plain text.In order to use this feature, your database administrator will be need to create a database master key (see below for instructions).

1. Create Master Key in Database

1. The first step is to create a master key in the database. Do so by connecting directly whichever database your Cinchy instance is running on.

2. Run the below query to create your master key:

CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password';

3. You can now encrypt data via the user interface (Image 1):

2. Backup Master Key

After you have created your master key you can create a backup file of that key in case any data corruption occurs in future.

You will need the password you used to create your master key in order to complete this operation.

1. Run the following command:

BACKUP MASTER KEY TO FILE = 'path_to_file'   
    ENCRYPTION BY PASSWORD = 'password' 

Further documentation on creating a backup master key can be found here.

3. Restore Master Key

In the use case where you require to restore your master key due to data corruption, you may use the following steps.

You will need the password you used to create you master key in order to complete this operation.

1. Run the following command:

RESTORE MASTER KEY FROM FILE = 'path_to_file'   
    DECRYPTION BY PASSWORD = 'password'  
    ENCRYPTION BY PASSWORD = 'password'  
    [ FORCE ] 

Further documentation on restoring the master key can be found here.

Application Experiences

Rather than traditional code-centric applications which creates data silos, you can build meta-data driven application experiences directly on the Cinchy platform. These look and feel like regular applications, but persists its data on the data network autonomously, rather than managing its own persistence.

These experiences automatically adapt as your data evolves (Image 1).

Once you deploy your UI, API and logic, you will need to create an integrated client to leverage the data fabric for persistence and controls. If you would like a link to your experience from the data fabric, you will need to create an Experience in the Applets table. See Setting Up Experiences to see how to set up both.The Cinchy Platform also comes with a built-in Experience called My Data Network, this is a tool to help you visualize your data through its connections. You can read more about the Network Map, or create your own custom data network visualizer on your own data.

Prerequisites

• Requires CLI v4.7+

Maintenance

Cinchy performs maintenance tasks through the CLI. This currently includes the data erasure and data compression deletions.

Cinchy.Maintenance.CLI.exe maintenance -s "cinchyBaseURL" -u username -p "encryptedPassword" -t 180

Integrated Client Setup

Applet Setup

Custom Results in the Network Map

Focus the results of your Network Map to show only the data that you really want to see with our new URL parameters.

You can now add Target Node, Depth Level, and Max Depth Level Parameters, if you choose.

Example: <base url>/apps/datanetworkvisualizer?targetNode=&maxDepth=&depthLevel=

• Target Node: Using the Target Node parameter defines which of your nodes will be the central node from which all connections branch from.

  • Target Node uses the TableID number, which you can find in the URL of any table.

  • Example: <base url>/apps/datanetworkvisualizer?targetNode=8 will show TableID 8 as the central node

• Max Depths: This parameter defines how many levels of network hierarchy you want to display.

  • Example: <base url>/apps/datanetworkvisualizer?maxDepth=2 will only show you two levels of connections.

• Depth Level: Depth Level is a UI parameter that will highlight/focus on a certain depth of connections.

  • Example: <base url>/apps/datanetworkvisualizer?DepthLevel=1 will highlight all first level network connections, while the rest will appear muted.

The below example visualizer uses the following URL: <base url>/apps/datanetworkvisualizer?targetNode=8&maxDepth=2&depthLevel=1

• It shows Table ID 8 ("Groups") as the central node.

• It only displays the Max Depth of 2 connections from the central node.

• It highlights the nodes that have a Depth Level of 1 from the central node.

Enhancements

Table of Contents

1. Network Map

Cinchy comes out of the box with a system applet called "Network Map" (Image 1), which is a visualization of your data on the platform and how everything interconnects.

My Data Network is another way to view and navigate the data you have access to within Cinchy.

Each node represents a table you have access to within Cinchy, and each edge is one link between two tables. The size of the table is determined by the number of links referencing that table. The timeline on the bottom allows you to check out your data network at a point in the past and look at the evolution of your network.

It uses the user's entitlements for viewable tables and linked columns.

When you click on a node, you will see its description in the top right hand corner. You can click the Open button to navigate to the table (Image 2).

You will find the Network Map data experience on the Homepage (Image 3).

2. Custom Network Visualizer

You can also set up a custom network visualizer as follows:

2.1 Nodes

The nodes query defines the nodes in the network (Image 4 and 5).

2.2 Edges

The edges query defines the relationships between the nodes (Image 6).

2.3 Node Groups

Node groups are an optional query you can provide to group your nodes (Image 7 and 8).

2.4 Timeline

If no start or end date is specified, the data network is just shown as is. If there's a start or end date, the other CQLs need to have a @date parameter and that will be used to render the data network at a point in time.

You can use @date between [Modified] and [Replaced] with a version history query to see data at a point in time. You can also simply use @date > [Created] if it's an additive system.

Timeline Start Date

This CQL should return a date value as 'startDate'.

Timeline End Date

This CQL should return a date value as 'endDate'.

2.5 Slicers

To use slicers, you need to define the slicers in the [Slicers] column and add the additional attributes to the nodes query.

[
    {
        "attribute": "slice1",
        "displayName": "First Slicer"
    },
    {
        "attribute": "slice2",
        "displayName": "Second Slicer"
    }
]

Attribute is the column name from the nodes query, displayName is what shows up in the visualizer (Image 9 and 10).

2.6 System Tables

All the information above is entered into the [Cinchy].[Networks] table. To access the network, go to

<Cinchy URL>/Cinchy/apps/datanetworkvisualizer?network=<NAME>

Alternatively you can go to My Data Network and then add ?network=<NAME> to the end of it.

It is highly recommended to add a new applet for each custom data network visualizer for ease of access.

2.7 Optional URL Parameters

Cinchy version 5.2 added the ability to include new parameters on the URL path for your network visualizer in order to focus your node view. You can now add Target Node, Depth Level, and Max Depth Level Parameters, if you choose.

Example: <base url>/apps/datanetworkvisualizer?targetNode=&maxDepth=&depthLevel=

• Target Node: Using the Target Node parameter defines which of your nodes will be the central node from which all connections branch from.

  • Target Node uses the TableID number, which you can find in the URL of any table.

  • Example: <base url>/apps/datanetworkvisualizer?targetNode=8 will show TableID 8 as the central node

• Max Depths: This parameter defines how many levels of network hierarchy you want to display.

  • Example: <base url>/apps/datanetworkvisualizer?maxDepth=2 will only show you two levels of connections.

• Depth Level: Depth Level is a UI parameter that will highlight/focus on a certain depth of connections.

  • Example: <base url>/apps/datanetworkvisualizer?DepthLevel=1 will highlight all first level network connections, while the rest will appear muted.

The below example visualizer uses the following URL: <base url>/apps/datanetworkvisualizer?targetNode=8&maxDepth=2&depthLevel=1

• It shows Table ID 8 ("Groups") as the central node.

• It only displays the Max Depth of 2 connections from the central node.

• It highlights the nodes that have a Depth Level of 1 from the central node.

3. Example Network

The following is an example of a network map (Image 11).

For ease of testing, save the following as saved queries and then in the Networks table simply add exec [Domain].[Saved Query Name] as the CQLs.

Node Groups CQL

-- Sample Data
SELECT
  [Sub Network] = 'Blue Network'
, [Color] = '#03a9f4'
INTO #TEMP
UNION SELECT
  [Sub Network] = 'Green Network'
, [Color] = '#8bc34a'
-- Node Groups CQL
SELECT
  t.[Sub Network] as 'sub network'
, t.[Color] as 'color'
FROM #TEMP t

Nodes CQL

-- Sample Data
SELECT
   [Id] = 'N01'
,  [Title] = 'Hover text on the node.'
,  [Label] = 'Purple Node 1 (Ax)'
,  [Value] = 5
,  [Mass] = 5
-- optional
,  [Group] = 'Green Network'
,  [Color] = '#ab47bc'
,  [Description] = 'This description shows up in the top right hand corner when you select a node.'
,  [Node URL] = 'https://www.cinchy.com'
,  [Slicer 1] = 'A'
,  [Slicer 2] = 'x'
INTO #TEMP
UNION SELECT
   [Id] = 'N02'
,  [Title] = 'Hover text on node 2.'
,  [Label] = 'Node 2 (Ay)'
,  [Value] = 10
,  [Mass] = 10
-- optional
,  [Group] = 'Green Network'
,  [Color] = ''
,  [Description] = 'This description shows up in the top right hand corner when you select a node.'
,  [Node URL] = ''
,  [Slicer 1] = 'A'
,  [Slicer 2] = 'y'
UNION SELECT
   [Id] = 'N03'
,  [Title] = 'Hover text on node 3.'
,  [Label] = 'Node 3 (Bx)'
,  [Value] = 1
,  [Mass] = 1
-- optional
,  [Group] = 'Blue Network'
,  [Color] = ''
,  [Description] = ''
,  [Node URL] = ''
,  [Slicer 1] = 'B'
,  [Slicer 2] = 'x'
UNION SELECT
   [Id] = 'N04'
,  [Title] = 'Hover text on node 4.'
,  [Label] = 'Minimum Node 4 (y)'
,  [Value] = 3
,  [Mass] = 3
-- optional
,  [Group] = ''
,  [Color] = ''
,  [Description] = ''
,  [Node URL] = ''
,  [Slicer 1] = ''
,  [Slicer 2] = 'y'
UNION SELECT
   [Id] = 'N05'
,  [Title] = 'Hover text on node 4.'
,  [Label] = 'Orphan Node 5 (By)'
,  [Value] = 2
,  [Mass] = 2
-- optional
,  [Group] = ''
,  [Color] = ''
,  [Description] = ''
,  [Node URL] = ''
,  [Slicer 1] = 'B'
,  [Slicer 2] = 'y'
-- Nodes CQL
SELECT
   t.[Id] as 'id'
,  t.[Title] as 'title'
,  t.[Label] as 'label'
,  t.[Value] as 'value'
,  t.[Mass] as 'mass'
,  t.[Group] as 'group'
,  t.[Color] as 'color'
,  t.[Description] as 'description'
,  t.[Node URL] as 'nodeURL'
,  t.[Slicer 1] as 'slice1'
,  t.[Slicer 2] as 'slice2'
FROM #TEMP t

Edges CQL

-- Sample Data
SELECT
  [Id] = 'E01'
, [Label] = 'Node 1 to Node 2, Double Arrows'
, [From] = 'N01'
, [To] = 'N02'
-- optional
, [Show Arrow To] = 'True'
, [Show Arrow From] = 'True'
INTO #TEMP
UNION SELECT
  [Id] = 'E02'
, [Label] = 'Node 2 to Node 3, To Arrow'
, [From] = 'N02'
, [To] = 'N03'
-- optional
, [Show Arrow To] = 'True'
, [Show Arrow From] = 'False'
UNION SELECT
  [Id] = 'E03'
, [Label] = 'Node 1 to Node 4, From Arrow'
, [From] = 'N01'
, [To] = 'N04'
-- optional
, [Show Arrow To] = ''
, [Show Arrow From] = 'True'
UNION SELECT
  [Id] = 'E04'
, [Label] = 'Node 3 to Node 3, No Arrow'
, [From] = 'N03'
, [To] = 'N03'
-- optional
, [Show Arrow To] = ''
, [Show Arrow From] = ''

-- Edges CQL
SELECT
   t.[Id] as 'id'
,  t.[Label] as 'label'
,  t.[From] as 'from'
,  t.[To] as 'to'
,  t.[Show Arrow To] as 'showArrowTo'
,  t.[Show Arrow From] as 'showArrowFrom'
FROM #TEMP t