Oracle GoldenGate Veridata 23c Basic Performance Tuning

Tuning Oracle GoldenGate Veridata is essential for several reasons, particularly when working with large datasets or complex replication environments. GoldenGate Veridata is a tool used for data comparison, which is crucial for verifying the integrity of data during replication processes between source and target systems.

Here are the key reasons why tuning is important:

Large Datasets: When comparing large volumes of data, the performance of GoldenGate Veridata can significantly impact the speed and efficiency of the data verification process. Proper tuning ensures that it can handle large datasets without timeouts or excessive resource consumption.

System Resources: Tuning helps to balance CPU, memory, and disk usage, preventing the system from being overloaded during the verification process. Without optimization, the system can become slow or even unresponsive, especially if it’s running in a production environment.

Memory Consumption: By tuning memory parameters, you can optimize how much memory Veridata uses. This reduces the likelihood of running out of memory during data comparisons, which can lead to failures or slow performance.

Processing Speed: Tuning settings related to the comparison algorithms, batch sizes, and comparison threads can drastically improve the speed of data comparison. This is important, especially in high-volume replication environments where comparing large datasets can otherwise take too long.

Parallel Processing: Tuning enables the optimization of parallel processing, allowing multiple threads to work simultaneously on different chunks of data, which can significantly speed up the comparison process.

This article explains the basic tuning tips of Oracle GoldenGate Veridata 23c.

Memory Configurations

It is important to tune the memory parameters of Veridata Server, Agents and for Sorting (if required) to achieve better performance.

Veridata Server

The memory parameter for Veridata Server will be in the file “oggvdt_cainput.properties” file which will be under “/config/”.

Navigate to the config directory which will be under Veridata23c home.

cd /veridata23_home/config

You will see the file “oggvdt_cainput.properties” and set / change the values of below parameters.

/veridata23c_home/config/oggvdt_cainput.properties file,
jvm.memory.xms=32g
jvm.memory.xmx=96g

For example, here, Veridata 23c home is /scratch/vdt23c

Once the values are set, please restart the server to ensure new values are reflected.

Veridata Agent

Veridata Agent is responsible for fetching the data from the sources and targets. Tune the memory of the Agent for a better performance. By default, it is set to 1GB. You need to adjust the memory as per the requirement.

The memory parameter for Veridata Agent will be present in the “agent.sh” file which will be under each Agent_Home, i.e., where the agent is deployed and configured.

For example,

To set maximum heap size to 64GB,

edit “agent.sh”, using the vi and modify the below parameter based on the requirement.

USER_MEM_ARGS="-Xmx1024M -Xms1024M"

USER_MEM_ARGS="-Xmx1024M -Xms1024M" to USER_MEM_ARGS="-Xmx64g -Xms1024M"

Sort Memory

The Sort Memory is tuned only when you see slowness in the sorting phase. This can be tuned by setting the below parameter in the file “veridata.cfg”.

server.max_sort_memory

This file will be under the “/Veridata23c_Home/config/veridata/veridata.cfg”

By default, it takes around 2GB. You can increase the value based on the requirement.

Sort Disk Size

This is mainly used for improving the performance of the Sorting phase. A temporary disk storage will be used for this. By default, Veridata takes the /tmp directory. However, it can be changed.

Providing a good amount of storage for this directory is always recommended. Also, it is better to provide different directory for Source and Target as it will not impact the I/O of the disk.

Veridata Tuning Parameters

Below are few essential tuning parameters explained

Max Concurrent Comparison Threads

Parallel Hints

Batch Size

Max Concurrent Comparison Threads

Setting this parameter helps to improve the performance of the Initial Compare phase. The data will be compared in parallel.

The default value is 4.

The calculation of setting this parameter is done as below,

Max Concurrent Comparison Threads = Total CPU - 2

For example, if the total CPU is 64, then it will be as follows,

Max Concurrent Comparison Threads = 64 – 2, which is 62. It can be between the range 4 and 62. In this case, it is not necessary to set this directly to 62. You can start with an average number like 30 or 32 and then increase it as per the performance.

Parallel Hints

The parallel hints can be set for both “Initial Compare” and “Confirm-Out-Of-“Sync” phases. It will add load to the database, so, request you to check DB load and then provide the numbers to the Parallel Hints.

Initial Compare

Source Oracle optimizer hint – optimizer for Source database, e.g. PARALLEL(x, 8)
Target Oracle optimizer hint – optimizer for Target database, e.g. PARALLEL(x, 8)

Confirm-Out-Of-Sync

The same applies for Confirm-Out-Of-Sync.

Source Oracle optimizer hint – optimizer for Source database, e.g. PARALLEL(x, 8)
Target Oracle optimizer hint – optimizer for Target database, e.g. PARALLEL(x, 8)

Batch Size

The Batch Size can be set for both Initial Compare and Confirm-Out-Of-Sync phases. Based on the batch size provided, the data will be fetched in batches and then it will be compared.

Initial Compare

For, Initial Compare, the batch size parameter can be found in the

Connections -> Connection Configuration -> Properties

This is to provide flexibility to the users to provide different batch sizes for each Sources and Targets.

Confirm-Out-Of-Sync

For Confirm-Out-Of-Sync, you can set the batch size under the profiles itself.

Web UI Tuning Tips

Ensure to only access Veridata Web Ui in a single web browser tab. Opening Veridata in multiple web browser tabs may cause synchronization issue and is not recommended.

Summary

Tuning GoldenGate Veridata is crucial for ensuring optimal performance, resource utilization, speed, and reliability during data comparison processes. Proper tuning allows the tool to handle large data volumes efficiently while minimizing system overloads and downtimes, improving scalability, and enabling faster issue resolution. Without it, replication environments might experience performance bottlenecks, leading to delays and potential data inconsistencies.

Cheers 🙂