Arcion self-hosted quickstart #
I. Host machine requirements #
Please ensure your host machine that runs Replicant meets the following minimum hardware and software requirements.
Minimum hardware requirements #
- 16 CPU cores and 32GB of memory
- 100GB-200GB of SSD/NVMe storage space
Minimum system requirements #
- Linux (CentOS/Ubuntu/Redhat)
- JDK 8 either from a JRE or JDK installation
Note: The storage requirements may increase depending on your source-target pair and method of replication.
Internet access #
Arcion self-hosted doesn’t require internet access except in the following two scenarios:
- When you provide an online license for your Arcion server (the default), Arcion must connect to the internet to validate the license.
- The Arcion on-premises Docker container connects to the internet to access the documentation on this website. This documentation seamlessly appears when you set up connections using the on-premises Docker container.
- Arcion requires internet access to remotely update the SQL Server CDC Agent in SQL Server replication.
II. Get Arcion self-hosted #
To find out how to download Arcion self-hosted, send us a request and someone from our team will contact you. After getting Arcion self-hosted, unzip the archive for the Replicant CLI:
unzip replicant-cli-<version>.zip
This creates a directory replicant-cli
that serves as REPLICANT_HOME
. For the following steps, position yourself in the REPLICANT_HOME
directory.
III. Licensing #
- Download the license file for Replicant and rename it to
replicant.lic
The license file must be named replicant.lic
- Copy r
eplicant.lic
intoREPLICANT_HOME
You must copy the
replicant.lic
file into Replicant’s home directory, not in thelicenses
folder of Replicant.
IV. Set up source database configuration #
-
From
REPLICANT_HOME
navigate to the sample connection configuration file of the source database:vi conf/conn/<database-name>.yaml
Make the necessary changes as shown below:
type: <database-name> host: <host> #Enter the hostname or IP address to connect to the source #database port: <port> #Enter the port on which the source database server is running username: <database-username> #Enter the source database username password: <user-password> #Enter the source database password max-connections: 30 max-retries: 10 retry-wait-duration-ms: 1000
Please note that certain databases might have additional configuration parameters. For example, Oracle has the additional parameter
service-name
.For further database specific examples, please refer to one of our Source database setup pages.
-
From
REPLICANT_HOME
, navigate to the sample filter file of the source database:vi filter/<source-database-name>_filter.yaml
Make the necessary changes as shown below:
allow: - catalog: "catalog_name" #Enter your database name if applicable schema : "schema_name" #Enter your schema name if applicable types: [TABLE,VIEW] #Database object types to replicate allow: "Table_Name1": #Enter the name of your table "Table_Name2": #Enter the name of your table
V. Set up target database #
-
From
REPLICANT_HOME
navigate to the sample connection configuration file of the target database:vi conf/conn/<database-name>.yaml
Make the necessary changes as shown below:
type: <database-name> host: <host> #Enter the hostname or IP address to connect to the target #database port: <port> #Enter the port on which the target database server is running username: <database-username> #Enter the target database username password: <user-password> #Enter the target database password max-connections: 30 max-retries: 10 retry-wait-duration-ms: 1000
For database specific examples, please refer to one of our Target database setup pages.
VI. Run Replicant in snapshot mode #
Replicant is now ready to run in snapshot mode. The snapshot will only transfer existing data from the source database to the target database. If you would like to transfer real-time changes in addition to the snapshot, skip step VI and proceed to steps VII and VIII to run Replicant in full mode.
-
Execute the following command from
REPLICANT_HOME
to run Replicant in snapshot mode:./bin/replicant snapshot conf/conn/<source-database-name>.yaml \ conf/conn/<destination-database-name>.yaml \ --extractor conf/src/<source-database-name>.yaml \ --applier conf/dst/<destination-database-name>.yaml \ --filter filter/<source-database-name>_filter.yaml
The proceeding steps are only required if you intend to run Replicant in real-time mode.
VII. Set up heartbeat table #
-
Create a heartbeat table in the catalog/schema you are going to replicate with the following DDL:
CREATE TABLE <catalog>.<schema>.replicate_io_cdc_heartbeat( \ timestamp <data_type_equivalent_to_long>)
-
Grant
INSERT
,UPDATE
, andDELETE
privileges to the user configured for Replicant. -
From
REPLICANT_HOME
, navigate to the heartbeat table’s configuration.vi conf/src/<source-database-name>.yaml
-
Under the Realtime Section, make the necessary changes as follows
heartbeat: enable: true catalog: <catalog_name> #if the source database supports catalog, change the catalogue name accordingly schema: <schema_name> #if the source database supports schema, change the schema name accordingly interval-ms: 10000
VIII. Run Replicant in full mode #
-
From
REPLICANT_HOME
, enter the following to run Replicant in full mode:./bin/replicant full conf/conn/<source-database-name>.yaml \ conf/conn/<destination-database-name>.yaml \ --extractor conf/src/<source-database-name>.yaml \ --applier conf/dst/<destination-database-name>.yaml \ --filter filter/<source-database-name>_filter.yaml \
Upgrade Arcion Replicant #
We recommend that you always use the latest version of Replicant. That way, you can enjoy the latest features and various quality improvmements.
You can grab the latest version of Replicant from our Arcion Self-hosted page.
To get the most out of your upgrades, follow the tips below:
- Keep the configuration files isolated from the
replicant-cli
directory tree. During version upgrades, they might get overwritten. - After upgrade, copy your
replicant.lic
license file to the newREPLICANT_HOME
. - Copy the contents of your older
$REPLICANT_HOME/lib
directory to the new versionlib
directory$REPLICANT_HOME/lib
.
Set up a specific database pipeline #
Different source and target databases may have slightly more specific and different setup instructions than the general guidelines provided in this Quickstart Guide. Follow the six steps below for a pipeline specific setup for Replicant:
- Source Database Setup
- Target Database Setup
- Running Replicant
- Performance Enhancing and Troubleshoot