Cinchy Platform Documentation
Cinchy v5.0 - v5.5
Cinchy v5.0 - v5.5
  • Data Collaboration Overview
  • Other Wiki Spaces
    • Cinchy Data Sync
    • Angular SDK
    • JavaScript SQK
  • Release Notes
    • Release Notes
      • 5.0 Release Notes
      • 5.1 Release Notes
      • 5.2 Release Notes
      • 5.3 Release Notes
      • 5.4 Release Notes
      • 5.5 Release Notes
      • 5.6 Release Notes
  • Getting Help
  • Frequently Asked Questions
  • Deployment Guide
    • Deployment Installation Guides
      • Deployment Planning Overview and Checklist
        • Deployment Architecture Overview
          • Kubernetes Deployment Architecture
          • IIS Deployment Architecture
        • Deployment Prerequisites
          • Single Sign-On (SSO) Integration
            • Enabling TLS 1.2
            • Configuring ADFS
            • AD Group Integration
      • Kubernetes Deployment Installation
        • Disabling your Kubernetes Applications
        • Changing your File Storage Configuration
        • Using Self-Signed SSL Certs (Kubernetes Deployments)
        • Deploying the CLI (Kubernetes)
      • IIS Deployment Platform Installation
        • Deploying Connections and the CLI (IIS)
        • Deploying the Event Listener/Worker (IIS)
    • Upgrade Guides
      • Upgrading Cinchy Versions
        • Cinchy Upgrade Utility
        • Kubernetes Upgrades
          • v5.1 (Kubernetes)
          • v5.2 (Kubernetes)
          • v5.3 (Kubernetes)
          • v5.4 (Kubernetes)
          • v5.5 (Kubernetes)
          • v5.6 (Kubernetes)
          • Updating the Kubernetes Image Registry
          • Upgrading AWS EKS Kubernetes Version
          • Upgrading AKS (Azure Kubernetes Service)
        • IIS Upgrades
          • v4.21 (IIS)
          • v4.x to v5.x (IIS)
          • v5.1 (IIS)
          • v5.2 (IIS)
          • v5.3 (IIS)
          • v5.4 (IIS)
          • v5.5 (IIS)
          • v5.6 (IIS)
      • Upgrading from v4 to v5
  • Guides for Using Cinchy
    • User Guides
      • Overview of the Data Browser
      • The Admin Panel
      • User Preferences
        • Personal Access Tokens
      • Table Features
      • Data Management
      • Queries
      • Version Management
        • Versioning Best Practices
      • Commentary
    • Builder Guides
      • Best Practices
      • Creating Tables
        • Attaching Files
        • Columns
        • Data Controls
          • Data Entitlements
          • Data Erasure
          • Data Compression
        • Restoring Tables, Columns, and Rows
        • Formatting Rules
        • Indexing and Partitioning
        • Linking Data
        • Table and Column GUIDs
        • System Tables
      • Saved Queries
      • CinchyDXD Utility
        • Building the Data Experience (CinchyDXD)
        • Packaging the Data Experience (CinchyDXD)
        • Installing the Data Experience (CinchyDXD)
        • Updating the Data Experience (CinchyDXD)
        • Repackaging the Data Experience (CinchyDXD)
        • Reinstalling the Data Experience (CinchyDXD)
      • Multi-Lingual Support
      • Integration Guides
    • Administrator Guide
    • Additional Guides
      • Monitoring and Logging on Kubernetes
        • Grafana
        • Opensearch Dashboards
          • Setting up Alerts
        • Monitoring via ArgoCD
      • Maintenance
      • GraphQL (Beta)
      • System Properties
      • Enable Data At Rest Encryption
      • MDQE
      • Application Experiences
        • Network Map
          • Custom Node Results
          • Custom Results in the Network Map
        • Setting Up Experiences
  • API Guide
    • API Overview
      • API Authentication
      • API Saved Queries
      • ExecuteCQL
      • Webhook Ingestion
  • CQL
    • The Basics of CQL
      • CQL Examples
      • CQL Functions Master List
      • CQL Statements Overview
        • Cinchy DML Statements
        • Cinchy DDL Statements
      • Cinchy Supported Functions
        • Cinchy Functions
        • Cinchy System Values
        • Cinchy User Defined Functions
          • Table-Valued Functions
          • Scalar-Valued Functions
        • Conversion Functions
        • Date and Time Types and Functions
          • Return System Date and Time Values
          • Return Date and Time Parts
          • Return Date and Time Values From Their Parts
          • Return Date and Time Difference Values
          • Modify Date and Time Values
          • Validate Date and Time Values
        • Logical Functions
        • Mathematical Functions
        • String Functions
        • Geometry and Geography Data Type and Functions
          • OGC Methods on Geometry & Geography Instances
          • Extended Methods on Geometry & Geography Instances
        • Full Text Search Functions
        • Connections Functions
        • JSON Functions
  • Meta Forms
    • Introduction to Meta-Forms
    • Meta-Forms Deployment Installation Guide
      • Deploying Meta-Forms (Kubernetes)
      • Deploying Meta-Forms (IIS)
    • Creating a Dynamic Meta-Form (Using Tables)
    • Creating a Dynamic Meta-Form Example (Using Form Designer)
    • Forms Data Types
    • Adding Links to a Form
    • Rich Text Editing in Forms
Powered by GitBook
On this page
  • 1. Monitoring and Alerting
  • 1.1 Create your Destination
  • 1.2 Authenticate your Sender
  • 1.3 Create your Monitor
  • 1.4 Add a Trigger
  • Example Alerts
  • 1. Alerting on Stream Errors
  • 2. Alerting on Kubernetes Restarts
  • 3. Alerting on Status Codes

Was this helpful?

Export as PDF
  1. Guides for Using Cinchy
  2. Additional Guides
  3. Monitoring and Logging on Kubernetes
  4. Opensearch Dashboards

Setting up Alerts

PreviousOpensearch DashboardsNextMonitoring via ArgoCD

Last updated 2 years ago

Was this helpful?

1. Monitoring and Alerting

Opensearch comes with the ability to set up alerts based on any number of monitors. You can then push these alerts via email, should you desire.

Prior to setting up a monitor or alert, ensure that you have .

Definitions:

Monitor

A job that runs on a defined schedule and queries OpenSearch indices. The results of these queries are then used as input for one or more triggers.

Trigger

Conditions that, if met, generate alerts.

Alert

An event associated with a trigger. When an alert is created, the trigger performs actions, which can include sending a notification.

Action

The information that you want the monitor to send out after being triggered. Actions have a destination, a message subject, and a message body.

Destination

A reusable location for an action. Supported locations are Amazon Chime, Email, Slack, or custom webhook.

1.1 Create your Destination

Your destination will be where you want your alerts to be pushed to. Opensearch supports various options, however this guide will focus on email.

  1. From the left navigation pane, click Alerting (Image 1).

2. Click on the Destinations Tab > Add Destination

3. Add a name to label your destination and select Email as type (Image 2)

4. You will need to assign a Sender. This is the email address that the alert will send from when you specify this specific destination. To add a new Sender, click Manage Senders (Image 3).

5. Click Add Sender

6. Add in the following information (Image 4):

  • Sender Name

  • Email Address

  • Host (this is the host address for the email provider)

  • Port (this is the Port of the email provider)

  • Encryption

7. You will need to assign your Recipients. This is the email address(es) that will receive the alert when you specify this specific destination. To add a new Recipient, you can either type their email(s) into the box, or click Manage Senders to create an email group (Image 5).

8. Click Update to finalize your Destination.

1.2 Authenticate your Sender

You will need to authenticate your sender to allow emails to come through. Please contact Cinchy Customer Support to help you with this step.

  • Via email: support@cinchy.com

  • Via phone: 1-888-792-6051

1.3 Create your Monitor

Your monitor is a job that runs on a defined schedule and queries OpenSearch indices. The results of these queries are then used as input for one or more triggers.

  1. From the Alerting dashboard, select Monitors > Create Monitor (Image 6).

2. Under Monitor Details, add in the following information (Image 7).

  • Monitor Name

  • Monitor Type (This example uses Per Bucket)

    • Whereas query-level monitors run your specified query and then check whether the query’s results triggers any alerts, bucket-level monitors let you select fields to create buckets and categorize your results into those buckets.

    • The alerting plugin runs each bucket’s unique results against a script you define later, so you have finer control over which results should trigger alerts. Each of those buckets can trigger an alert, but query-level monitors can only trigger one alert at a time.

  • Monitor Defining Method: the way you want to define your query and triggers. (This example uses Visual Editor)

    • Visual definition works well for monitors that you can define as “some value is above or below some threshold for some amount of time.”

  • Schedule: Choose a frequency and time zone for your monitor.

3. Under Data Source add in the following information (Image 8):

  • Index: Define the index you want to use as a source for this monitor

  • Time Field: Select the time field that will be used for the x-axis of your monitor

4. The Query section will appear differently depending on the Monitor Defining Method selected in step 2 (Image 9). This example is using the visual editor.

To define a monitor visually, select an aggregation (for example, count() or average()), a data filter if you want to monitor a subset of your source index, and a group-by field if you want to include an aggregation field in your query. At least one group-by field is required if you’re defining a bucket-level monitor. Visual definition works well for most monitors.

1.4 Add a Trigger

A trigger is a condition that, if met, will generate an alert.

  1. To add a trigger, click the Add a Trigger button (Image 10).

2. Under New Trigger, define your trigger name and severity level (with 1 being the highest) (Image 11).

3. Under Trigger Conditions, you will specify the thresholds for the query metrics you set up previously (Image 12). In the below example, our trigger will be met if our COUNT threshold goes ABOVE 10000.

You can also use keyword filters to drill down into a more specific subset of data from your data source.

4. In the Action section you will define what happens if the trigger condition is met (Image 13). Enter the following information to set up your Action:

  • Action Name

  • Message Subject: In the case of an email alert, this will be the email subject line.

  • Message: In the case of an email alert, this will be the email body.

  • Perform Action: If you’re using a bucket-level monitor, decide whether the action is performed per execution or per alert.

  • Throttling: Enable action throttling if you wish. Use action throttling to limit the number of notifications you receive within a given span of time.

5. Click Send Test Message, if you want to test that the alert functions correctly.

6. Click Save.

Example Alerts

1. Alerting on Stream Errors

In this example, we are pushing an alert based on errors. We will monitor our Connections stream for any instance of 'error', and push out an alert when our trigger threshold is hit.

  • Index: In this example we are looking specifically at Connections.

  • Time Field

  • Time Range: Define how far back you want to monitor

  • Data Filter: We want to monitor specifically whenever the Stream field of our index is stderr (standard error).

This is how our example monitor will appear; it shows when in the last 15 days our Connections app had errors in the log (Image 15).

  • Trigger Name

  • Severity Level

  • Trigger Condition: In this example, we use IS ABOVE and the threshold of 1.

The trigger threshold will be visible on your monitoring graph as a red line.

2. Alerting on Kubernetes Restarts

In this example, we are pushing an alert based on the kubectl.kubernetes.io/restartedAt annotation, which updates whenever your pod restarts. We will monitor this annotation across our entire product-mssql instance, and push out an alert when our trigger threshold is hit.

  • Index: In this example we are looking at our entire product-mssql instance.

  • Time Field

  • Query: This example is using the total count of the kubectl.kubernetes.io/restartedAt annotattion

  • Time Range: Define how far back you want to monitor. This example goes back 30 days.

This is how our example monitor will appear; it shows when in the last 30 days our instance had restarts (Image 18).

  • Trigger Name

  • Severity Level

  • Trigger Condition: In this example, we use IS ABOVE and the threshold of 100.

The trigger threshold will be visible on your monitoring graph as a red line.

3. Alerting on Status Codes

In this example, we are pushing an alert based on status codes. We will monitor our entire instance for 400 status codes and push out an alert when our trigger threshold is hit.

  • Index: In this example we are looking across out entire product-mssql-1 instance.

  • Time Field

  • Time Range: Define how far back you want to monitor. In this example we are looking at the past day.

  • Data Filter: We want to monitor specifically whenever the Status Code is 400 (bad request).

This is how our example monitor will appear (note that there are no instances of a 400 status code in this graph) (Image 21).

  • Trigger Name

  • Severity Level

  • Trigger Condition: In this example, we use IS ABOVE and the threshold of 0.

The trigger threshold will be visible on your monitoring graph as a red line.

Ensure that you, else your alert will not work.

Through the support portal:

Query definition gives you flexibility in terms of what you query for (using ) and how you evaluate the results of that query (Painless scripting).

First we create our by defining the following (Image 14):

2. Once our monitor is created, we need to define a . When this condition is met, the alert will be pushed out to our defined In this example we want to be alerted when there is more than one stderr in our Connections stream (Image 16). Input the following:

First we create our by defining the following (Image 17):

2. Once our monitor is created, we need to define a . When this condition is met, the alert will be pushed out to our defined In this example we want to be alerted when there is more than 100 restarts across our instance (Image 19). Input the following:

First we create our by defining the following (Image 20):

2. Once our monitor is created, we need to define a . When this condition is met, the alert will be pushed out to our defined In this example we want to be alerted when there is at least one 400 status code across out instance (Image 22). Input the following:

Support Portal
the OpenSearch query DSL
Monitor Cluster Metrics
Endpoint Monitoring with Blackbox Exporter
authenticate the Sender
Destination
Monitor
trigger condition
Recipient(s).
Monitor
trigger condition
Recipient(s).
Monitor
trigger condition
Recipient(s).
added your data source as an index pattern
Image 1: Click on Alerting
Image 2: Update your destination
Image 3: Manage your Senders
Image 4: Configure your Sender
Image 5: Configure your Recipients
Image 6: Create your Monitor
Image 7: Define your Monitor details
Image 8: Configure your Data Source
Image 9: Define your Query
Image 10: Add a Trigger
Image 11: Define your Trigger.
Image 12: Trigger Conditions
Image 13: Define your Actions
Image 14: Define your Data Source and Query
Image 15: Example monitor
Image 16: Example Trigger
Image 17: Define your Query and Data Source
Image 18: Example Monitor
Image 19: Trigger Conditions
Image 20: Define your Query and Data Source
Image 21: Example Monitor
Image 22: Trigger Conditions