Oracle GoldenGate – Parallel Replicat

Oracle GoldenGate – Parallel Replicat

Initially, when Oracle GoldenGate was launched, there was only one EXTRACT and REPLICAT processes. Later in OGG 11.2.x, Oracle introduced Integrated Extract. So, the normal or already existing Extract was renamed to Classic Extract.Similarly, the Replicat process was also renamed to Classic Replicat. Oracle GoldenGate became a powerful tool for replication and migration. As there were many version upgrades happened in OGG, there was also many processes got introduced which you can see in the below diagram.,

From the above table, you can see a new type of replicat process has been introduced from OGG 12.3, which is call “PARALLEL REPLICAT”. Integrated Replicat itself is very fast in applying the data to the target as it has parallelism concepts in it. But this Parallel Replicat process is even more faster than the Integrated Replicat process.

• Parallel Replicat has a highly scalable apply engine which achieves a apply rate up to 1 million+ operations per second.
• It is 5X faster than Integrated Replicat
• Applies a single large transaction in parallel
• Parallelizes apply by carefully handling the dependent transactions
• Dependency calculation and parallelism outside the database
• Ability to Parallelize a Single Large Transaction
• Processes large transactions faster as they are parallelized
• Dependencies are still considered while parallelizing large transaction
• Ability controlled by a SPLIT_TRANS_RECS parameter which specifies the transaction split size (in records). Default is 100,000.

There are two types of Parallel Replicat processes available.

• Integrated Parallel Replicat

• Non-Integrated Parallel Replicat

Let us see about the architecture and working concepts of this newly introduced Parallel Replicat process,

Like Integrated Replicat, Parallel Replicat process have background processes and each one of them plays it’s role in order to apply the transactions or data to the target server. Below is the architecture diagram of the Parallel Replicat process.,

So, what these background processes does? Check below,

MAPPERS

The Mappers read the trail file and map records, forward the mapped records to the Master.

APPLIERS

Applies the records to the target database.

MASTER

The Master process consists of two separate threads, Collater and Scheduler.

COLLATER

The Collater is responsible for managing and communicating with the Mappers, along with receiving the mapped transactions and reordering them into a single in-order stream.

SCHEDULER

The Scheduler is responsible for managing and communicating with the Appliers, along with reading transactions from the Collater, batching them, and scheduling them to Appliers.

As per above diagram, we do have some special parameters for the Parallel Replicat according to which it behaves. Let’s see what these parameters do,

MAP_PARALLELISM

Configures number of mappers. This controls the number of threads used to read the trail file. The default value is 2.

APPLY_PARALLELISM

Configures number of appliers. This controls the number of connections in the target database used to apply the changes. The default value is 4.

MIN_APPLY_PARALLELISM | MAX_APPLY_PARALLELISM

The Apply parallelism is auto-tuned. You can set a minimum and maximum value to define the ranges in which the Replicat automatically adjusts its parallelism. There are no defaults. Do not use with APPLY_PARALLELISM at same time.

SPLIT_TRANS_REC

Specifies that large transactions should be broken into pieces of specified size and applied in parallel. Dependencies between pieces are still honored. Disabled by default.

COMMIT_SERIALIZATION

Enables commit FULL serialization mode, which forces transactions to be committed in trail order.

The difference between a Integrated Parallel Replicat and Non-Integrated Parallel Replicat is

• • APPLIER process runs outside of the database in Non-Integrated Parallel Replicat where in Integrated mode of Parallel Replicat, the APPLIER process will run inside the database.
  • Integrated Replicat process can be configured only when your Oracle RDBMS version is 12.2 and above.

Now, let’s configure a Non-Integrated Parallel Replicat process.

I have installed Oracle GoldenGate 12.3.0.1.2 version,

Add the parameters for the Parallel Replicat process.

Add the Parallel Replicat process using the below command.

You can see the Parallel Replicat process has been added now.

How do we know if the replicat added is a parallel or integrated or classic? You can see a keyword PARALLEL when seeing the info of the replicat process.

Let’s start the Parallel Replicat process now.

Now the replicat is running.

Below is the report file of the Parallel Replicat process PREP which has been added now.

GGSCI (ogg123-2.localdomain as ggadmin@ORCL) 12> view report PREP


***********************************************************************
                 Oracle GoldenGate Delivery for Oracle
    Version 12.3.0.1.2 OGGCORE_12.3.0.1.0_PLATFORMS_171208.0005_FBO
   Linux, x64, 64bit (optimized), Oracle 12c on Dec  9 2017 01:06:26

Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved.


                    Starting at 2018-11-18 19:29:33
***********************************************************************

Operating System Version:
Linux
Version #1 SMP Wed Feb 2 18:40:23 EST 2011, Release 2.6.32-100.28.5.el6.x86_64
Node: ogg123-2.localdomain
Machine: x86_64
                         soft limit   hard limit
Address Space Size   :    unlimited    unlimited
Heap Size            :    unlimited    unlimited
File Size            :    unlimited    unlimited
CPU Time             :    unlimited    unlimited

Process id: 9554

Description:

2018-11-18 19:29:33  WARNING OGG-02904  Replication of PARTIAL XML containing NCHAR/NVARCHAR/NCLOB data may cause divergence.

***********************************************************************
**            Running with the following parameters                  **
***********************************************************************

2018-11-18 19:29:33  INFO    OGG-03059  Operating system character set identified as UTF-8.

2018-11-18 19:29:33  INFO    OGG-02695  ANSI SQL parameter syntax is used for parameter parsing.

2018-11-18 19:29:34  INFO    OGG-01360  REPLICAT is running in Parallel mode.
replicat PREP
userid ggadmin, password ***
MAP_PARALLELISM 2
MIN_APPLY_PARALLELISM 2
MAX_APPLY_PARALLELISM 8
SPLIT_TRANS_RECS 100
MAP source.t1, TARGET target.t1;

2018-11-18 19:29:34  INFO    OGG-06067  Spawned Applier with pid 9,564.

2018-11-18 19:29:34  INFO    OGG-06067  Spawned Applier with pid 9,565.

2018-11-18 19:29:34  INFO    OGG-06067  Spawned Applier with pid 9,566.

2018-11-18 19:29:34  INFO    OGG-06067  Spawned Applier with pid 9,567.

2018-11-18 19:29:34  INFO    OGG-06067  Spawned Applier with pid 9,569.

2018-11-18 19:29:34  INFO    OGG-06067  Spawned Mapper with pid 9,568.

2018-11-18 19:29:34  INFO    OGG-06067  Spawned Mapper with pid 9,570.

2018-11-18 19:29:36  INFO    OGG-06451  Triggers will be suppressed by default.

2018-11-18 19:29:36  INFO    OGG-01815  Virtual Memory Facilities for: COM
    anon alloc: mmap(MAP_ANON)  anon free: munmap
    file alloc: mmap(MAP_SHARED)  file free: munmap
    target directories:
    /u01/app/ogg/dirtmp.

2018-11-18 19:29:37  INFO    OGG-06604  Database ORCL CPU info: CPU Count 1, CPU Core Count 0, CPU Socket Count 0.

***********************************************************************
**                     Run Time Messages                             **
***********************************************************************


2018-11-18 19:30:09  INFO    OGG-06067  Spawned Applier with pid 9,624.

As we know, the replicat process has it’s own checkpoint table which is mainly used for recovery purpose. This Parallel Replicat also has it’s own checkpoint table. But this does not take the default or the checkpoint table which is mentioned in the ./GLOBALS parameter

The checkpoint table in here is ggadmin.ggchkpt

This checkpoint table should have an entry in it for the parallel replicat process PREP. But, we will not see it in here.

This is because, the Parallel Replicat process creates it’s own checkpoint table. It uses the same schema name which is mentioned in the ./GLOBALS parameter, but the table name will be created with an extension as below,

Querying this checkpoint table will give the information of the parallel replicat process.

We have multiple applier processes, but only 3 were used and hence we can see the information for the 3 processes. So, this is something different when comes to Parallel Replicat process. Always do not check for the default checkpoint table which will be created.

So, we have created Non-Intgrated Replicat process. Let’s try to create a Integrated Parallel Replicat process now. Add the parameters for the Integrated Parallel Replicat process.

Add the replicat IPREP now. The keyword used in here is “PARALLEL INTEGRATED”.

Let’s start the Integrated Parallel Replicat process IPREP.

Oops! The process is stopped. Why? Let us see the report file to check for the error messages.

As I mentioned earlier in this post, Integrated Parallel Replicat process is only supported when the Database version 12.2 and above. The database which I have used in here is 12.1.x and hence this got failed.

Finally, below is the comparison report between various Replicat processes available in Oracle GoldenGate.

Hope the post gave you much information about the new replicat process “Parallel Replicat” which was introduced in OGG 12.3. Your comments are always welcome.

Cheers 🙂