Notification Configuration

Configure notifications #

Replicant can send you important notifications regarding various replication status. For example, you can easily choose the type of notifications, notification recipients, and so on. Having up to date information always at your disposal can significantly help you monitor your data pipelines more effectively.

Overview #

You can choose to receive notifications through either emails, or use a custom script. See the following sections for more information on each.

The notification configuration file #

The notification configuration file notifications.yaml specifies the types of notifications Replicant sends. You can find a sample notification configuration file inside the conf/notification directory of your Arcion self-hosted download. To know about the available configuration parameters in the notification configuration file, see the following sections.

Configure mail notifications #

If you choose to receive notifications by mail, you can configure them under the mail-alert section of the configuration file. For more information, see the sample at the end of this section.

The following parameters are available to configure mail notifications:

enable #

true or false.

Whether to enable mail notifications.

smtp-host #

The SMTP hostname—for example, smtp.gmail.com.

smtp-port #

The SMTP port number—for example, 587.

authentication #

To configure authentication. For authentication, the following parameters are available:

enable
true or false.

Whether to enable authentication. You must enable this for Gmail, Yahoo, or other authenticated services.

protocol #

The authentication protocol—for example, TLS and SSL. Note that port for TLS and SSL are different.

sender-username [20.07.16.4] #

Optional. Set this parameter if username is different from sender-email.

sender-email #

The email ID that sends the mail.

sender-password #

The password associated with sender-email. You must set this parameter if you’ve enabled authentication.

receiver-email #

The email IDs of the recipients enclosed in square brackets. You can specify multiple email addresses—for example, ['sara@example.com', 'alex@example.com]`.

channels #

The channels that you want to subscribe to for notifications, enclosed in square brackets. The following channels are available:

  • ALL
  • GENERAL
  • LAG
  • WARNING
  • RETRY_FAILURE
  • SNAPSHOT_COMPLETE
  • SNAPSHOT_SUMMARY

Use the ALL channel to subscribe to all channels. Otherwise, specify one or more channels of your choosing—for example, [WARNING, RETRY_FAILURE].

subject-prefix [20.07.16.5] #

The prefix string to add to the email subject—for example, PRODUCTION.

The following is a sample notification configuration:

mail-alert:
  enable: true
  smtp-host: 'smtp.example.com'
  smtp-port: 587
  authentication:
    enable: true
    protocol: 'TLS'
  sender-username: 'user' 
  sender-email: 'sara@example.com'
  sender-password: '*********'
  receiver-email: ['sara@example.com']
  channels: [ALL]
  subject-prefix: "PROD"

Specify multiple mail-alert configurations #

You can specify multiple mail-alert configurations as a list under the mail-alerts field. This is useful if you don’t want the same notification configuration for all the receipients. The following is a sample:

mail-alerts:
- enable: true
  receiver-email: ['sara@example.com']
  channels: [ALL]
- enable: true
  receiver-email: ['alex@example.com']
  channels: [LAG]

Configure script notifications #

If you choose to use a script to receive the notifications, you can configure them under the script-alert section of the configuration file. For more information, see the sample at the end of this section.

The following parameters are available to configure script alerts:

enable #

true or false.

Whether to enable script notifications.

script #

The full path to your script file.

output-file #

The full path to the file where error output of script is saved.

channels #

The channels that you want to subscribe to, enclosed in square brackets. The following channels are available:

  • ALL
  • GENERAL
  • LAG
  • WARNING
  • RETRY_FAILURE
  • SNAPSHOT_COMPLETE
  • SNAPSHOT_SUMMARY

alert-repetitively #

true or false.

Whether you want to get the same notification repetitively.

The following is a sample script notification configuration:

script-alert:
  enable: true
  script: "/home/ubuntu/script.sh"
  output-file: "/home/ubuntu/script.out"
  channels: [LAG]
  alert-repetitively: false

Specify multiple script-alert configurations #

You can specify multiple script-alert configurations as a list. This is useful if you don’t want the same script alert configuration for all the receipients. The following is a sample:

script-alert:
- enable: true
  script: "/home/ubuntu/script.sh"
  output-file: "/home/ubuntu/script.out"
  channels: [LAG]
  alert-repetitively: false
- enable: true
  script: "/home/ubuntu/script1.sh"
  output-file: "/home/ubuntu/script1.out"
  channels: [RETRY_FAILURE]
  alert-repetitively: true
- enable: true
  script: "/home/ubuntu/script1.sh"
  output-file: "/home/ubuntu/script1.out"
  channels: [SNAPSHOT_COMPLETE]
  alert-repetitively: false

Configure lag notifications #

The notification configuration file lets you specifically configure notifications about replication lags. For example, you can configure Replicant to send you notification when replication is lagging more than a threshold value.

You need to specify your configurations under the lag-notification section of the notification configuration file. For more information, see the sample at the end of this section. To understand how lag notifications work, see How lag notifications work.

The following parameters are available to configure lag notifications:

enable #

true or false.

Whether to enable lag notifications.

threshold-ms #

The threshold value in milliseconds.

stable-time-out-ms #

A timeout value in milliseconds within which we expect the replication to recover from lag and stabilize below threshold-ms.

check-interval-s #

The time period in seconds after which Replicant calculates the global lag every time. In case of distributed replication, Replicant calculates the lag across all Replicant nodes.

The following is a sample lag notification configuration:

lag-notification:
  enable: true
  notify-above-threshold: false
  threshold-ms: 10_000
  stable-time-out-ms: 60_000

How lag notifications work #

You can configure lag notifications in two ways:

  • Get notification when replication lag is below the threshold value threshold-ms. You can choose this by setting notify-above-threshold to false.
  • Get notification when replication lag is above the threshold value threshold-ms. You can choose this by setting notify-above-threshold to true.

Replicant sends a lag notification only when replication lag is continously out of range of threshold value threshold-ms for stable-time-out-ms period of time. After sending a notification, Replicant won’t send any further notification untill the replication lag is in the range of threshold-ms. After lag enters in the range of threshold-ms, if replication lag again goes out of range of threshold-ms for stable-time-out-ms period of time, Replicant will send a new notification. Replicant will continue operating in this manner.

Note:

  1. If you use Gmail SMTP server, you must configure the Gmail account to allow “Less secure apps”. Otherwise, leave the default settings intact and change only the receiver-email.

  2. Since version 20.07.16.5, for mail-alert, only receiver-email & channels are mandatory. Use the rest of the fields only for using a custom mail server.

  3. Prior to version 20.07.16.5, the receiver-email parameter supports only a single value, without the square brackets.