Destination S3 #

The extracted replicant-cli will be referred to as the $REPLICANT_HOME directory in the proceeding steps.

I. Setup Connection Configuration #

From $REPLICANT_HOME navigate to the sample S3 connection configuration file
```
vi conf/conn/s3.yaml
```

If you store your connection credentials in AWS Secrets Manager, you can tell Replicant to retrieve them. For more information, see Retrieve credentials from AWS Secrets Manager.

Otherwise, you can put your credentials like usernames and passwords in plain form like the sample below:

type: S3

access-key: "REPLICANT" #Replace REPLICANT with the access key of AWS user created from IAM management
secret-key: "Replicant#123" #Replace Replicant#123 with the secret Key of the AWS User (note: make sure the specified user has  AmazonS3FullAccess)

bucket: "bucket_name" #Replace bucket_name with the name of your S3 bucket
root: "root_dir" #Replace root_dir with the name of the directory inside the s3 bucket where the CSV files will be stored

stage:
  type: SHARED_FS
  root-dir: /home/user/stage #Enter the path of the directory where CSV files will be staged before uploading to S3

#file-format: CSV #Possible file formats are CSV or JSON
max-connections: 50 #Maximum number of connections Replicant can open in the target s3

II. Set up Applier Configuration #

From $REPLICANT_HOME navigate to the sample S3 applier configuration file
```
vi conf/dst/s3.yaml        
```

If necessary, make the necessary changes as follows:

snapshot:
  #threads: 16
  #max-file-size: 33_554_432 #32MB
  #delimiter: `,` #CSV files created will have provided delimiter
  #quote: `”`
  #escape:   `\`
  #include-header: false #enable or disable toggle column names as header in CSV file

#realtime:
  #threads: 16

III. S3 File Format #

When data is being loaded into S3, the data is first converted to either a CSV or JSON file. To better understand the data format for the CSV and JSON converted files, see S3 File Format.

Important: We highly recommended that you read the S3 File Format page when using S3 as the Target system.