Arcion Self-hosted Quickstart #

I. Host Machine Prerequisites #

Please ensure your host machine that will run Replicant meets the following minimum hardware and software prerequisites.

Minimum Hardware Requirements #

16 CPU cores and 32GB of memory
100GB-200GB of SSD/NVMe storage space

Minimum System Requirements #

Linux (CentOS/Ubuntu/Redhat)
JDK 8 either from a JRE or JDK installation

Note: The storage requirements may increase depending on your source-target pair and method of replication.

II. Download Replicant and Create a Home Repository #

Download the latest version of Replicant from Arcion Self-hosted.
Unzip the downloaded archive:
```
unzip replicant-cli-<version>.zip
```

This will create a directory named replicant-cli that will serve as REPLICANT_HOME. For the proceeding steps, position yourself in REPLICANT_HOME.

III. Licensing #

Download the license file for Replicant and rename it to replicant.lic
The license file must be named replicant.lic
Copy replicant.lic into REPLICANT_HOME
You must copy the replicant.lic file into Replicant’s home directory, not in the licenses folder of Replicant.

IV. Setup Source Database Configuration #

From REPLICANT_HOME navigate to the sample connection configuration file of the source database:

vi conf/conn/<database-name>.yaml

Make the necessary changes as shown below:

type: <database-name>

host: <host> #Enter the hostname or IP address to connect to the source
             #database
port: <port> #Enter the port on which the source database server is running

username: <database-username> #Enter the source database username
password: <user-password>  #Enter the source database password

max-connections: 30

max-retries: 10
retry-wait-duration-ms: 1000

Please note that certain databases might have additional configuration parameters. For example, Oracle has the additional parameter service-name.

For further database specific examples, please refer to one of our Source database setup pages.

From REPLICANT_HOME, navigate to the sample filter file of the source database:

vi filter/<source-database-name>_filter.yaml

Make the necessary changes as shown below:

allow:
- catalog: "catalog_name" #Enter your database name if applicable
  schema : "schema_name" #Enter your schema name if applicable
  types: [TABLE,VIEW] #Database object types to replicate
  allow:
    "Table_Name1": #Enter the name of your table
    "Table_Name2": #Enter the name of your table

V. Target Database Setup #

From REPLICANT_HOME navigate to the sample connection configuration file of the target database:

vi conf/conn/<database-name>.yaml

Make the necessary changes as shown below:

type: <database-name>

host: <host> #Enter the hostname or IP address to connect to the target
             #database
port: <port> #Enter the port on which the target database server is running

username: <database-username> #Enter the target database username
password: <user-password>  #Enter the target database password

max-connections: 30

max-retries: 10
retry-wait-duration-ms: 1000

For database specific examples, please refer to one of our Target database setup pages.

VI. Run Replicant Snapshot #

Replicant is now ready to run in snapshot mode. The snapshot will only transfer existing data from the source database to the target database. If you would like to transfer real-time changes in addition to the snapshot, skip step VI and proceed to steps VII and VIII to run Replicant in full mode.

Execute the following command from REPLICANT_HOME to run Replicant in snapshot mode:

./bin/replicant snapshot conf/conn/<source-database-name>.yaml \
conf/conn/<destination-database-name>.yaml \
--extractor conf/src/<source-database-name>.yaml \
--applier conf/dst/<destination-database-name>.yaml \
--filter filter/<source-database-name>_filter.yaml

The proceeding steps are only required if you intend to run Replicant in real-time mode.

VII. Heartbeat table setup #

Create a heartbeat table in the catalog/schema you are going to replicate with the following DDL:

CREATE TABLE <catalog>.<schema>.replicate_io_cdc_heartbeat( \
timestamp <data_type_equivalent_to_long>)

Grant INSERT, UPDATE, and DELETE privileges to the user configured for Replicant.
From REPLICANT_HOME, navigate to the heartbeat table’s configuration.
```
vi conf/src/<source-database-name>.yaml
```

Under the Realtime Section, make the necessary changes as follows

heartbeat:
  enable: true
  catalog: <catalog_name> #if the source database supports catalog, change the catalogue name accordingly
  schema: <schema_name> #if the source database supports schema, change the schema name accordingly
  interval-ms: 10000

VIII. Run Replicant in full mode #

From REPLICANT_HOME, enter the following to run Replicant in full mode:

./bin/replicant full conf/conn/<source-database-name>.yaml \
conf/conn/<destination-database-name>.yaml \
--extractor conf/src/<source-database-name>.yaml \
--applier conf/dst/<destination-database-name>.yaml \
--filter filter/<source-database-name>_filter.yaml \

Upgrade Arcion Replicant #

We recommend that you always use the latest version of Replicant. That way, you can enjoy the latest features and various quality improvmements.

You can grab the latest version of Replicant from our Arcion Self-hosted page.

To get the most out of your upgrades, follow the tips below:

Keep the configuration files isolated from the replicant-cli directory tree. During version upgrades, they might get overwritten.
After upgrade, copy your replicant.lic license file to the new REPLICANT_HOME.
Copy the contents of your older $REPLICANT_HOME/lib directory to the new version lib directory $REPLICANT_HOME/lib.

Database Specific Setup Overview #

Different source and target databases may have slightly more specific and different setup instructions than the general guidelines provided in this Quickstart Guide. Follow the six steps below for a pipeline specific setup for Replicant:

Source Database Setup
Target Database Setup
Running Replicant
Performance Enhancing and Troubleshoot