New Relic

Send replication statistics and logs to New Relic #

Enterprises can gain valuable business metrics from a centralized, all-in-one log management platform such as New Relic. With that in mind, Arcion has added support for New Relic from version 22.10.28.2.

To send your replication statistics and logs to New Relic, follow the steps below.

Configure logging details #

Replicant can take a YAML configuration file as inuput when using New Relic as cloud logger. In this file, you can configure logging details to suit your requirements. The following options are available:

type #

The type of cloud logger. In this case it’s NEW_RELIC.

license-key #

The 40-character New Relic license key associated with your New Relic account. For more information, see New Relic License Key.

region #

Region where New Relic is hosted. Available regions are US and EU.

Default value: US.

metrics-logging #

Option to configure replication statistics that you want to send to New Relic. It has the following parameters:

enable
true or false.

Enable sending replication statistics to New Relic.

Default value: true.
interval-s
The interval between each successive transfer in seconds.
prefix
The prefix to add to each metric. Default prefix is arcion.core.
custom-attributes
To add user-defined attribute (key, val) in each metric for grouping. For example, you may add the pipeline information by adding Source and Target attributes like below: 
custom-attributes:
  source: Oracle
  target: Databricks

trace-logging #

Option to configure replication logs that you want to send to New Relic. It has the following parameters:

enable
true or false.

Enable sending replication logs to New Relic.

Default value: false
interval-s
The interval between each successive transfer in seconds.
custom-attributes
To add user-defined attribute (key, val) in each log for grouping. For example, you may add the pipeline information by adding Source and Target attributes like below: 
custom-attributes:
  source: Oracle
  target: Databricks

monitor-activity #

true or false.

If true, metric and trace logger threads will dump their activity in the trace.log file.

max-retries #

Number of times Replicant will try again a failed operation.

Default value: 20.

retry-wait-duration-ms #

Duration in milliseconds Replicant should wait before performing then next retry of a failed operation.

Default value: 1000.

Below is a sample configuration file:

type: NEW_RELIC

license-key: "LICENSE_KEY"

metrics-logging:
  enable: true
  interval-s: 5
  custom-attributes:
    host: localhost
    pipeline: Oracle->Databricks

trace-logging:
  enable: false
  interval-s: 5
  custom-attributes:
    replicationMode: FULL
    pipeline: Oracle->Databricks
  
monitor-activity: true

Replace LICENSE_KEY with your New Relic license key.

Run Replicant #

As the final step, run Replicant with the --cloud-logger argument, providing it the path to the configuration file. For example:

./bin/replicant full conf/conn/oracle_src.yaml conf/conn/databricks_aws_uc.yaml \
--filter filter/oracle_filter.yaml \
--cloud-logger conf/cloudlogger/newrelic.yaml