Home

Architecture Install Configure Test

 

Configuration of Unidirectional Oracle GoldenGate Replication

  

 

 

Table of Contents

 

Document Control      

Introduction 

Overview of GoldenGate Installation 

1.      Installation of GoldenGate

    1.1 Download GoldenGate Binaries

    1.2 Create Custom Subdirectories

    1.3 Create GoldenGate Subidrectories from GGSCI

2.  Create GoldenGate Admin User

3. Enable Supplemental Logging

4. Configuration of Change Data Capture Processes

    4.1.1  Add Checkpoint Table

    4.1.2 Create and Start Extract ECLMDBA1

    4.1.3 Create and Start DataPump PCLMDBA1

    4.1.4 Add Extract EDRPROD1

    4.1.5 Create and Start DataPump PDRPROD1

5. Create Consistent Copy of Source Database for Target

6. Preparing Target Database for Replication 

7. Starting Replication on Target

    7.1 Create and Start Replicat RDRPROD1

    7.2 Create and Start Replicat RCLMDBA1

8. Testing Replication from Source to Target

9. Creating LINUX OS GoldenGate Monitoring Scripts

 

 

 

Document Control

 

Change Record

                                                    

Date

Author

Version

Change Reference

 

 

 

 

 

 

November 21, 2011

Vladimir Grigorian

1.0

First Draft

 

 

Reviewers

Name

Position

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Distribution

Copy No.

Name

Location

 

 

 

 

 

1          

 

 

 

2          

 

 

 

3

 

 

 

 

This publication may not be reproduced, stored in a retrieval system, or transmitted in whole or in part, in any form, or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permission of Vladimir Grigorian.

 

 

 

Introduction

 

Purpose

The purpose of this document is document installation, configuration, testing and monitoring setup of Oracle GoldenGate

In this document details are provided on various parameters, their location and in some cases recommended settings. 

 

Audience

This document is intended for:

·         DBAs and Architects

 

 

 

Assumptions

Following assumptions have been verified prior to installing and configuring Oracle GoldenGate.

1.       All specified Hardware Requirements for customer’s platform have been met.

2.       All specified Operating System Requirements for source and destination have been satisfied.

3.       All Oracle recommended settings have been set on database level.

 

 

 

 

 

 

 

 

Overview of GoldenGate Installation

 

 

The following outlines the intended setup for this configuration

 

Table 1. Server Information

 

 

Source

Destination

Hostname

source

target

OS

Oracle Enterprise Linux 6.1

Oracle Enterprise Linux 6.1

GoldenGate Home

/opt/oracle/gg/

/opt//oracle/gg

Oracle Instance

SOURCE

TARGET

Oracle Version

11.2.0.2

11.2.0.2

GoldenGate version

11.1.1.1

11.1.1.1

 

Table 2. GoldenGate Processes Information

 

 

Source

Destination

Extract

EDRPROD1,

ECLMDBA1

 

DataPump

PDRPROD1,

PCLMDBA1

 

Replicat

 

RDRPROD1,

RCLMDBA1

Trails

Dirdat/d1

Dirdat/c1

Dirdat/c2

Dirdat/d2

GoldenGate Admin

GGATE

GGATE

User schemas

DRPROD, CLMDBA

DRPROD, CLMDBA

 

Fig 1. Sequence of Steps

 

 

 

 

 

The following outlines replication processes data flow in end state configuration:

Fig 2. Data Flow in SOURCE – TARGET GoldenGate Configuration

 

 

 

 

 

 

 

 

 

 

1.     Installation of GoldenGate

 

 

1.1 Download Goldengate Binaries

 

Download GoldenGate binaries from Oracle edelivery :

https://edelivery.oracle.com/EPD/Download/get_form?egroup_aru_number=13730870.

Make sure that you are using the correct OS version of binaries.

 

Unzip V26185-01.zip into this directory /opt/oracle/gg/.

Untar the file tar -xvf fbo_ggs_Linux_x64_ora11g_64bit.tar

  

1.2 Create Custom Sub-Directories

 

Create gg directory:  /opt/oracle/gg/ as oracle user. Create discard (for discarded transactions), log (for email alerts) and scripts (for diagnostic scripts) directories under this directory.

 

1.3 Create Golden Gate Subdirectories from GGSCI

 

Create subdirectories from Oracle GoldenGate  :

 

[oracle@source gg]$ ./ggsci

 

Oracle GoldenGate Command Interpreter for Oracle

Version 11.1.1.1 OGGCORE_11.1.1_PLATFORMS_110421.2040

Linux, x64, 64bit (optimized), Oracle 11g on Apr 21 2011 22:42:14

 

Copyright (C) 1995, 2011, Oracle and/or its affiliates. All rights reserved.
 
 
 
GGSCI (source) 1> create subdirs
 
Creating subdirectories under current directory /opt/app1/oracle/gg
 
Parameter files                /opt/oracle/gg/dirprm: created
Report files                   /opt/oracle/gg/dirrpt: created
Checkpoint files               /opt/oracle/gg/dirchk: created
Process status files           /opt/oracle/gg/dirpcs: created
SQL script files               /opt/oracle/gg/dirsql: created
Database definitions files     /opt/oracle/gg/dirdef: created
Extract data files             /opt/oracle/gg/dirdat: created
Temporary files                /opt/oracle/gg/dirtmp: created
Veridata files                 /opt/oracle/gg/dirver: created
Veridata Lock files            /opt/oracle/gg/dirver/lock: created
Veridata Out-Of-Sync files     /opt/oracle/gg/dirver/oos: created
Veridata Out-Of-Sync XML files /opt/oracle/gg/dirver/oosxml: created
Veridata Parameter files       /opt/oracle/gg/dirver/params: created
Veridata Report files          /opt/oracle/gg/dirver/report: created
Veridata Status files          /opt/oracle/gg/dirver/status: created
Veridata Trace files           /opt/oracle/gg/dirver/trace: created
Stdout files                   /opt/oracle/gg/dirout: created

 

 

 

2.     Configure GoldenGate Admin User

 

 

Oracle GoldenGate Admin “GGATE” – could be named differently - is not a standard, automatically configured user. The user is manually created or created as part of a default, unchanged sample implementation script, for instance to support the heartbeat mechanism, which is nothing more and nothing less than just an implementation of a best practice. A heartbeat is generated by updating a source table, updating the intercepted result by the data pump (optional, but configured here) and by the Replicat, using the TOKEN mechanism and the standard GGHEADER tokens. So for this to happen, the update to the heartbeat table at source has to be intercepted by Extract and stored in a trail, so data pump and Replicat have access to that information, can enrich it and can insert results in the target heartbeat table owned by GGATE.

The following should be done on both source and destination for bidirectional replication. The configuration of DDL can be done on source and destination.

 

2.1 Create GGATE Tablespace

 

 

SYS / SOURCE> create tablespace ggate datafile '+DATA_01/TARGET/datafile/ts_test_ggate01.dbf' size 500m autoextend on;

 

Tablespace created.

Elapsed: 00:00:01.16

 

Optionally, you can put this tablespace on autoextend with maxsize.

 

 

 

 

 

2.2 Create GGATE User

 

 

SYS / SOURCE> create user ggate identified by ****  default tablespace ggate temporary tablespace temp;
 
User created.
 
Elapsed: 00:00:00.12
SYS / SOURCE> grant resource, connect  to ggate;
 
Grant succeeded.
 
Elapsed: 00:00:00.03
SYS / SOURCE> grant select any table to ggate;
 
Grant succeeded.
 
Elapsed: 00:00:00.00
SYS / SOURCE> grant  create table  to ggate;
 
Grant succeeded.
 
Elapsed: 00:00:00.01
SYS / SOURCE> grant flashback any table  to ggate;
 
Grant succeeded.
 
Elapsed: 00:00:00.00
SYS / SOURCE> grant execute on dbms_flashback  to ggate;
 
Grant succeeded.
 
Elapsed: 00:00:00.17
SYS / SOURCE> grant execute on utl_file to ggate;
 
Grant succeeded.
 
Elapsed: 00:00:00.09
SYS / SOURCE> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;
 
Database altered.
 
Elapsed: 00:00:00.28
 
 
2.3 Grant GGATE User Replication Privileges
 
 
SYS / SOURCE> @marker_setup
 
Marker setup script
 
You will be prompted for the name of a schema for the GoldenGate database objects.
NOTE: The schema must be created prior to running this script.
NOTE: Stop all DDL replication before starting this installation.
 
Enter GoldenGate schema name:GGATE
 
Elapsed: 00:00:00.06
 
Marker setup table script complete, running verification script...
Please enter the name of a schema for the GoldenGate database objects:
Setting schema name to GGATE
 
MARKER TABLE
-------------------------------
OK
Elapsed: 00:00:00.05
 
MARKER SEQUENCE
-------------------------------
OK
Elapsed: 00:00:00.00
 
Script complete.
SYS / SOURCE> @ddl_setup
 
GoldenGate DDL Replication setup script
 
Verifying that current user has privileges to install DDL Replication...
Elapsed: 00:00:00.00
 
You will be prompted for the name of a schema for the GoldenGate database objects.
NOTE: For an Oracle 10g source, the system recycle bin must be disabled. For Oracle 11g and later, it can be enabled.
NOTE: The schema must be created prior to running this script.
NOTE: Stop all DDL replication before starting this installation.
 
Enter GoldenGate schema name:GGATE
 
You will be prompted for the mode of installation.
To install or reinstall DDL replication, enter INITIALSETUP
To upgrade DDL replication, enter NORMAL
Enter mode of installation:INITIALSETUP
 
Working, please wait ...
Spooling to file ddl_setup_spool.txt
Checking for sessions that are holding locks on Oracle Golden Gate metadata tables ...
Check complete.
Elapsed: 00:00:00.00
Elapsed: 00:00:00.08
Elapsed: 00:00:00.00
Elapsed: 00:00:00.11
Elapsed: 00:00:00.00
Elapsed: 00:00:00.02
Elapsed: 00:00:00.01
Elapsed: 00:00:00.02
Elapsed: 00:00:00.00
Using GGATE as a GoldenGate schema name, INITIALSETUP as a mode of installation.
Working, please wait ...
DDL replication setup script complete, running verification script...
Please enter the name of a schema for the GoldenGate database objects:
Setting schema name to GGATE
DDLORA_GETTABLESPACESIZE STATUS:
Line/pos   Error
---------- ------------------------------------------------------------------
No errors  No errors
Elapsed: 00:00:00.01
CLEAR_TRACE STATUS:
Line/pos   Error
---------- ------------------------------------------------------------------
No errors  No errors
Elapsed: 00:00:00.01
CREATE_TRACE STATUS:
Line/pos   Error
---------- ------------------------------------------------------------------
No errors  No errors
Elapsed: 00:00:00.01
TRACE_PUT_LINE STATUS:
Line/pos   Error
---------- ------------------------------------------------------------------
No errors  No errors
Elapsed: 00:00:00.01
INITIAL_SETUP STATUS:
Line/pos   Error
---------- ------------------------------------------------------------------
No errors  No errors
Elapsed: 00:00:00.01
DDLVERSIONSPECIFIC PACKAGE STATUS:
Line/pos   Error
---------- ------------------------------------------------------------------
No errors  No errors
Elapsed: 00:00:00.01
DDLREPLICATION PACKAGE STATUS:
Line/pos   Error
---------- ------------------------------------------------------------------
No errors  No errors
Elapsed: 00:00:00.01
DDLREPLICATION PACKAGE BODY STATUS:
Line/pos   Error
---------- ------------------------------------------------------------------
No errors  No errors
Elapsed: 00:00:00.01
DDL HISTORY TABLE
-----------------------------------
OK
Elapsed: 00:00:00.02
DDL HISTORY TABLE(1)
-----------------------------------
OK
Elapsed: 00:00:00.01
DDL DUMP TABLES
-----------------------------------
OK
Elapsed: 00:00:00.02
DDL DUMP COLUMNS
-----------------------------------
OK
Elapsed: 00:00:00.01
DDL DUMP LOG GROUPS
-----------------------------------
OK
Elapsed: 00:00:00.02
DDL DUMP PARTITIONS
-----------------------------------
OK
Elapsed: 00:00:00.01
DDL DUMP PRIMARY KEYS
-----------------------------------
OK
Elapsed: 00:00:00.02
DDL SEQUENCE
-----------------------------------
OK
Elapsed: 00:00:00.00
GGS_TEMP_COLS
-----------------------------------
OK
Elapsed: 00:00:00.02
GGS_TEMP_UK
-----------------------------------
OK
Elapsed: 00:00:00.01
DDL TRIGGER CODE STATUS:
Line/pos   Error
---------- ------------------------------------------------------------------
No errors  No errors
Elapsed: 00:00:00.01
DDL TRIGGER INSTALL STATUS
-----------------------------------
OK
Elapsed: 00:00:00.07
DDL TRIGGER RUNNING STATUS
-----------------------------------
ENABLED
Elapsed: 00:00:00.06
STAYMETADATA IN TRIGGER
-----------------------------------
OFF
Elapsed: 00:00:00.00
DDL TRIGGER SQL TRACING
-----------------------------------
0
Elapsed: 00:00:00.01
DDL TRIGGER TRACE LEVEL
-----------------------------------
0
Elapsed: 00:00:00.00
LOCATION OF DDL TRACE FILE
-------------------------------------------------------------------------------------------------------------------------
/opt/app/oracle/diag/rdbms/SOURCE/SOURCE/trace/ggs_ddl_trace.log
Elapsed: 00:00:00.00
Analyzing installation status...
Elapsed: 00:00:00.33
STATUS OF DDL REPLICATION
-------------------------------------------------------------------------------------------------------------------------
SUCCESSFUL installation of DDL Replication software components
Elapsed: 00:00:00.00
Script complete.
 
SYS / SOURCE> @role_setup
 
GGS Role setup script
 
This script will drop and recreate the role GGS_GGSUSER_ROLE
To use a different role name, quit this script and then edit the params.sql script to change the gg_role parameter to the preferred name. (Do not run the script.)
 
You will be prompted for the name of a schema for the GoldenGate database objects.
NOTE: The schema must be created prior to running this script.
NOTE: Stop all DDL replication before starting this installation.
 
Enter GoldenGate schema name:GGATE
Wrote file role_setup_set.txt
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.01
Role setup script complete
Grant this role to each user assigned to the Extract, GGSCI, and Manager processes, by using the following SQL command:
GRANT GGS_GGSUSER_ROLE TO <loggedUser>
where <loggedUser> is the user assigned to the GoldenGate processes.
 
SYS / SOURCE> grant ggs_ggsuser_role to ggate;
Grant succeeded.
Elapsed: 00:00:00.03
SYS / SOURCE> @ddl_enable
Trigger altered.
Elapsed: 00:00:00.00
SYS / SOURCE> @ddl_pin GGATE
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.06
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.00
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.00
SYS / SOURCE>

 

    3. Enable Supplemental Logging

 

After refreshes from production logging may be disabled, which will cause replication to stall (everything is running, but nothing gets replicated).

 

SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;

 

--Verify that it is enabled.

 

SQL> SELECT SUPPLEMENTAL_LOG_DATA_MIN FROM V$DATABASE;
 
SUPPLEMENTAL_LOG_DATA_MIN
-------------------------<
YES

 

To add trandata through GoldenGate:

Run the following in source as Oracle user to replicate from:

SOURCE / SOURCE> select 'ADD TRANDATA SOURCE.'||tname from tab;

Results running in GGSCI for all tables in schema:

GGSCI> ADD TRANDATA SOURCE.CUSTOMERS
GGSCI> ADD TRANDATA SOURCE.ORDERS
...
GGSCI> Logging of supplemental redo log data is enabled for table SOURCE.CUSTOMERS
GGSCI> Logging of supplemental redo log data is enabled for table SOURCE.ORDERS

 

This has to be done for all tables in sche

 

 

 

 

4. Configuration of GoldenGate Change Data Capture Processes

 

The following describes GoldenGate extract, replicat and datapump configuration.

In addition, it describes other GG processes such as manager, globals, etc,

For better understanding of these processes please refer to GoldenGate documentation and table 2 above.

Here are parameter files for all processes:

Table 2. Parameter Files

SOURCE TARGET
GGSCI (source) 2> edit params ./GLOBALS
--Globals parameter file on SOURCE and TARGET--
GGSCHEMA GGATE
CHECKPOINTTABLE GGATE.CHKPTAB
GGSCI (target) 2> edit params ./GLOBALS
--Globals parameter file on SOURCE and TARGET--
GGSCHEMA GGATE
CHECKPOINTTABLE GGATE.CHKPTAB
 
GGSCI (source) 3> edit params mgr
-------------------------------------------------------------------
-- GoldenGate Manager                                                                       --
-- Last updated on 20111015 by Vladimir Grigorian    --
-------------------------------------------------------------------
PORT 7809
USERID ggate, PASSWORD ***
AUTOSTART EXTRACT pdrprod1
AUTOSTART EXTRACT eclmdba1
AUTOSTART EXTRACT pclmdba1
AUTOSTART EXTRACT edrprod1
AUTORESTART EXTRACT eclmdba1, RETRIES 3, WAITMINUTES 1, RESETMINUTES 60
AUTORESTART EXTRACT pclmdba1, RETRIES 3, WAITMINUTES 1, RESETMINUTES 60
AUTORESTART EXTRACT pdrprod1, RETRIES 3, WAITMINUTES 1, RESETMINUTES 60
AUTORESTART EXTRACT edrprod1, RETRIES 3, WAITMINUTES 1, RESETMINUTES 60
--Specifies to log the lag time as a warning in the event log
LAGCRITICALMINUTES 5
--Specifies how often to report lag info to the event log
LAGREPORTMINUTES 60
LAGINFOMINUTES 0
--Manages trail files to conserve space--
PURGEOLDEXTRACTS /opt/oracle/gg/dirdat/*, USECHECKPOINTS, MINKEEPDAYS
 
GGSCI (target) 3> edit params mgr
-------------------------------------------------------------------
-- GoldenGate Manager                                                                       --
-- Last updated on 20111017 by Vladimir Grigorian    --
-------------------------------------------------------------------
PORT 7809
USERID ggate, PASSWORD ***
AUTOSTART REPLICAT rdrprod1
AUTOSTART REPLICAT rclmdba1
AUTORESTART EXTRACT rdrprod1, RETRIES 3, WAITMINUTES 1, RESETMINUTES 60
AUTORESTART EXTRACT rclmdba1, RETRIES 3, WAITMINUTES 1, RESETMINUTES 60
--Specifies to log the lag time as a warning in the event log
LAGCRITICALMINUTES 5
--Specifies how often to report lag info to the event log
LAGREPORTMINUTES 60
LAGINFOMINUTES 0
--Manages trail files to conserve space--
PURGEOLDEXTRACTS /opt/oracle/gg/dirdat/*, USECHECKPOINTS, MINKEEPDAYS
 
GGSCI (source) 5> edit params ECLMDBA1
--------------------------------------------------------------------
-- Local extract eclmdba1 for CLMDBA schema                       --
-- Updated on 2011 Nov 18 by Vladimir Grigorian       --
--------------------------------------------------------------------
--extract group--
extract ECLMDBA1
--export sid
setenv (ORACLE_SID=source)
--connection to database--
userid ggate, password ***
--reporting--
ReportCount Every 30 Minutes, Rate
Report at 01:00
ReportRollover at 01:15
--discard--
DiscardFile dirrpt/DRPROD1.dis, Append
DiscardRollover at 02:00 ON SUNDAY
--Redo log options--
TRANLOGOPTIONS COMPLETEARCHIVEDLOGTIMEOUT 600 ASMUSER SYS@+ASM, ASMPASSWORD asmpassword
--bound recovery--
BR BROFF
--path and name for local destination trail--
exttrail /opt/app1/oracle/gg/dirdat/c1
--DDL support--
ddl include mapped objname clmdba.*
--DML support--
table clmdba.*;
GGSCI (target) 4> edit params RCLMDBA1
 
-------------------------------------------------------------------
-- Replicat for CLMDBA Schema                                    --
-- Last updated on Nov 18 2011 by Vladimir Grigorian --
-------------------------------------------------------------------
--Replicat group --
replicat RCLMDBA1
--export sid
setenv (ORACLE_SID=TARGET)
--source and target definitions--
AssumeTargetDefs
--target database login --
userid ggate, password ***
--PERFORMANCE PARMETERS--
--Double SQL transaction in 1 operation--
GROUPTRANSOPS 2000
--group similar transactions--
--BATCHSQL--
--error handling--
DDLERROR DEFAULT IGNORE RETRYOP
--file for dicarded transaction --
discardfile /opt/oracle/gg/discard/RCLMDBA1_discard.txt, purge, megabytes 50
--Reporting--
ReportCount Every 30 Minutes, Rate
Report at 01:00
ReportRollover at 01:15
DiscardRollover at 02:00 ON SUNDAY
--ddl support--
--Supress constraints, SUPRESSTRIGGERS is not available in this version--
DBOPTIONS DEFERREFCONST
DDL INCLUDE ALL
DDLOPTIONS REPORT
DDLERROR 942 DISCARD INCLUDE OPTYPE DROP
REPERROR (1403, DISCARD)
--HANDLECOLLISIONS--
--Specify table mapping ---
map clmdba.*, target clmdba.* ;
 
GGSCI (source) 6> edit params EDRPROD1
--------------------------------------------------------------------
-- Local extract for DRPROD schema                               --
-- Updated on 2011 Nov 30 by Vladimir Grigorian       --
--------------------------------------------------------------------
--extract group--
extract EDRPROD1
--export sid
setenv (ORACLE_SID=source)
--connection to database--
userid ggate, password ***
--reporting--
ReportCount Every 30 Minutes, Rate
Report at 01:00
ReportRollover at 01:15
--discard--
DiscardFile dirrpt/EDRPROD1.dis, Append
DiscardRollover at 02:00 ON SUNDAY
--Redo log options--
TRANLOGOPTIONS COMPLETEARCHIVEDLOGTIMEOUT 600 ASMUSER SYS@+ASM, ASMPASSWORD asmpassword
--bound recovery--
BR BROFF
--path and name for local destination trail--
exttrail /opt/app1/oracle/gg/dirdat/d1
--DDL support--
ddl include mapped objname drprod.*
--DML support--
table drprod.*;
 
GGSCI (target) 4> edit params RDRPROD1
 
-------------------------------------------------------------------
-- Replicat for CLMDBA Schema                                    --
-- Last updated on Nov 18 2011 by Vladimir Grigorian --
-------------------------------------------------------------------
--Replicat group --
replicat RDRPROD1
--export sid
setenv (ORACLE_SID=TARGET)
--source and target definitions--
AssumeTargetDefs
--target database login --
userid ggate, password ***
--PERFORMANCE PARMETERS--
--Double SQL transaction in 1 operation--
GROUPTRANSOPS 2000
--group similar transactions--
--BATCHSQL--
--error handling--
DDLERROR DEFAULT IGNORE RETRYOP
--file for dicarded transaction --
discardfile /opt/oracle/gg/discard/RCLMDBA1_discard.txt, purge, megabytes 50
--Reporting--
ReportCount Every 30 Minutes, Rate
Report at 01:00
ReportRollover at 01:15
DiscardRollover at 02:00 ON SUNDAY
--ddl support--
--Supress constraints, SUPRESSTRIGGERS is not available in this version--
DBOPTIONS DEFERREFCONST
DDL INCLUDE ALL
DDLOPTIONS REPORT
DDLERROR 942 DISCARD INCLUDE OPTYPE DROP
REPERROR (1403, DISCARD)
--HANDLECOLLISIONS--
--Specify table mapping ---
map drprod.*, target drprod.* ;
 
GGSCI (source) 7> edit params PCLMDBA1
--------------------------------------------------------------------
-- DataPump pclmdba1 for CLMDBA schema                               --
-- Updated on 2011 Nov 18 by Vladimir Grigorian       --
--------------------------------------------------------------------
--dataPump group--
extract pclmdba1
PassThru
--hostname and port for trail--
rmthost target, mgrport 7809
--path and name for remote destination trail--
rmttrail /opt/app/oracle/gg/dirdat/c2
Table CLMDBA.* ;
 
 
GGSCI (source) 8> edit params PDRPROD1
--------------------------------------------------------------------
-- DataPump pclmdba1 for DRPROD schema                            --
-- Updated on 2011 Nov 30 by Vladimir Grigorian       --
--------------------------------------------------------------------
--dataPump group--
extract PDRPROD1
PassThru
--hostname and port for trail--
rmthost target, mgrport 7809
--path and name for remote destination trail--
rmttrail /opt/app/oracle/gg/dirdat/d2
table DRPROD.* ;
 
 

 

4.1.1 Add Checkpoint Table

 

 

GGSCI (source) 3> ADD CHECKPOINTTABLE GGATE.CHKPTAB

 

Successfully created checkpoint table GGATE.CHKPTAB.

 

 

 

4.1.2 Create and start extract ECLMDBA1

 

GGSCI (source) 12> add extract ECLMDBA1, tranlog, begin now

 

2011-11-08 10:57:42  INFO    OGG-01749  Successfully registered EXTRACT ECLMDBA1 to start managing log retention at SCN 5161842.

EXTRACT added.

 

 

GGSCI (source) 13> ADD EXTTRAIL /opt/oracle/gg/dirdat/c1, EXTRACT ECLMDBA1, megabytes 100
EXTTRAIL added.
 
 
GGSCI (source) 14> info exttrail *
 
       Extract Trail: /opt/oracle/gg/dirdat/c1
             Extract: ECLMDBA1
               Seqno: 0
                 RBA: 0
           File Size: 100M
 
 
GGSCI (source) 15> edit params ECLMDBA1
 
---------------------------------------------------------------------
-- Local extract eclmdba1 for CLMDBA schema                       --
-- Updated on 2011 Nov 8 by Vladimir Grigorian        --
---------------------------------------------------------------------
--extract group--
extract ECLMDBA1
--export sid
setenv (ORACLE_SID=SOURCE)
--connection to database--
userid ggate, password ***
--reporting--
ReportCount Every 30 Minutes, Rate
Report at 01:00
ReportRollover at 01:15
--discard--
DiscardFile dirrpt/ECLMDBA1_discard.dis, Append
DiscardRollover at 02:00 ON SUNDAY
--Redo log options--
TRANLOGOPTIONS COMPLETEARCHIVEDLOGTIMEOUT 600 ASMUSER SYS@+ASM, ASMPASSWORD asmpassword
--bound recovery--
BR BROFF
--path and name for local destination trrail--
exttrail /opt/oracle/gg/dirdat/c1
--DDL support--
ddl include mapped objname clmdba.*
--DML support--
table clmdba.*;
 
 
GGSCI (source) 17> start extract ECLMDBA1
 
Sending START request to MANAGER ...
EXTRACT ECLMDBA1 starting
 
 
 
GGSCI (source) 18> info all
 
Program     Status      Group       Lag           Time Since Chkpt
 
MANAGER     RUNNING
EXTRACT     RUNNING     ECLMDBA1    00:00:00      00:03:55
 
 
GGSCI (source) 19>
 
 

 

4.1.3  Create and Start DataPump PCLMDBA1

 

 

GGSCI (source) 19> edit params pclmdba1
---------------------------------------------------------------------
-- DataPump pclmdba1 for CLMDBA schema                           --
-- Updated on 2011 Nov 8 by Vladimir Grigorian       --
---------------------------------------------------------------------
--dataPump group--
extract pclmdba1
PassThru
--hostname and port for trail--
rmthost target, mgrport 7809
--path and name for remote destination ttrail--
rmttrail /opt/oracle/gg/dirdat/c2
table CLMDBA.* ;
 
 
GGSCI (source) 25> ADD EXTRACT pclmdba1, EXTTRAILSOURCE /opt/oracle/gg/dirdat/c1, begin now
EXTRACT added.
 
 
GGSCI (source) 26> ADD RMTTRAIL /opt/oracle/gg/dirdat/c2, EXTRACT pclmdba1, MEGABYTES 100
RMTTRAIL added.
 
 
GGSCI (source) 27> start extract pclmdba1
 
Sending START request to MANAGER ...
EXTRACT PCLMDBA1 starting
 
 
GGSCI (source) 28> info all
 
Program     Status      Group       Lag           Time Since Chkpt
 
MANAGER     RUNNING
EXTRACT     RUNNING     ECLMDBA1    00:00:00      00:00:03
EXTRACT     RUNNING     PCLMDBA1    00:00:00      00:02:11
 
 
GGSCI (source) 29>
 
 
 
4.1.4. Add Extract EDRPROD1
 
 
GGSCI (source) 29> add extract EDRPROD1, tranlog, begin now
 
2011-11-08 11:10:27  INFO    OGG-01749  Successfully registered EXTRACT EDRPROD1 to start managing log retention at SCN 5167304.
EXTRACT added.
 
 
GGSCI (source) 30> ADD EXTTRAIL /opt/oracle/gg/dirdat/d1, EXTRACT EDRPROD1, megabytes 100
EXTTRAIL added.
 
 
GGSCI (source) 31> edit params EDRPROD1
---------------------------------------------------------------------
-- Local extract for DRPROD schema                               --
-- Updated on 2011 Nov 8 by Vladimir Grigorian       --
---------------------------------------------------------------------
--extract group--
extract EDRPROD1
--export sid
setenv (ORACLE_SID=SOURCE)
--connection to database--
userid ggate, password ***
--reporting--
ReportCount Every 30 Minutes, Rate
Report at 01:00
ReportRollover at 01:15
--discard--
DiscardFile dirrpt/EDRPROD1_discard.dis, Append
DiscardRollover at 02:00 ON SUNDAY
--Redo log options--
TRANLOGOPTIONS COMPLETEARCHIVEDLOGTIMEOUT 600 ASMUSER SYS@+ASM, ASMPASSWORD asmpassword
--bound recovery--
BR BROFF
--path and name for local destination trrail--
exttrail /opt/oracle/gg/dirdat/d1
--DDL support--
ddl include mapped objname drprod.*
--DML support--
table drprod.*;
 
GGSCI (source) 32> start extract EDRPROD1
 
Sending START request to MANAGER ...
EXTRACT EDRPROD1 starting
 
 
 
GGSCI (source) 33> info all
 
Program     Status      Group       Lag           Time Since Chkpt
 
MANAGER     RUNNING
EXTRACT     RUNNING     ECLMDBA1    00:00:00      00:00:08
EXTRACT     RUNNING     EDRPROD1    00:00:00      00:04:22
EXTRACT     RUNNING     PCLMDBA1    00:00:00      00:00:00
 
 
GGSCI (source) 34>
 

 

4.1.5 Create and Start DataPump PDRPROD1

 

 

GGSCI (source) 36> edit params PDRPROD1
---------------------------------------------------------------------
-- DataPump pclmdba1 for DRPROD schema                            --
-- Updated on 2011 Nov 8 by Vladimir Grigorian       --
---------------------------------------------------------------------
--dataPump group--
extract PDRPROD1
PassThru
--hostname and port for trail--
rmthost target, mgrport 7809
--path and name for remote destination ttrail--
rmttrail /opt/oracle/gg/dirdat/d2
table DRPROD.* ;
 
GGSCI (source) 37> ADD EXTRACT pdrprod1, EXTTRAILSOURCE /opt/oracle/gg/dirdat/d1, begin now
EXTRACT added.
 
 
GGSCI (source) 38> ADD RMTTRAIL /opt/oracle/gg/dirdat/d2, EXTRACT pdrprod1, MEGABYTES 100
RMTTRAIL added.
 
 
GGSCI (source) 39> info rmttrail *
 
       Extract Trail: /opt/oracle/gg/dirdat/c1
             Extract: ECLMDBA1
               Seqno: 6
                 RBA: 1089
           File Size: 100M
 
       Extract Trail: /opt/oracle/gg/dirdat/d1
             Extract: EDRPROD1
               Seqno: 0
                 RBA: 0
           File Size: 100M
 
       Extract Trail: /opt/oracle/gg/dirdat/c2
             Extract: PCLMDBA1
               Seqno: 0
                 RBA: 0
           File Size: 100M
 
       Extract Trail: /opt/oracle/gg/dirdat/d2
             Extract: PDRPROD1
               Seqno: 0
                 RBA: 0
           File Size: 100M
 
 
 
 
GGSCI (source) 40> start pdrprod1
 
Sending START request to MANAGER ...
EXTRACT PDRPROD1 starting
 
 
GGSCI (source) 41> info all
 
Program     Status      Group       Lag           Time Since Chkpt
 
MANAGER     RUNNING
EXTRACT     RUNNING     ECLMDBA1    00:00:00      00:00:07
EXTRACT     RUNNING     EDRPROD1    00:00:00      00:00:07
EXTRACT     RUNNING     PCLMDBA1    00:00:00      00:00:05
EXTRACT     RUNNING     PDRPROD1    00:00:00      00:01:29

 

 

 

In order to not miss any data, before starting any instantiation method, you must make sure that all open transactions that existed when the real-time extract was started are completed.

The best source for this information is the V$TRANSACTION performance view (GV$TRANSACTION in a RAC database). After starting your extract, determine what transactions exist in this view. When a transaction is complete, it will no longer exist in this view. You may have to query this view many times. If a particular transaction is running for longer than expected, you will need to investigate who is running that transaction and what that transaction doing. Ultimately, you may discover that you will need to kill the session that owns the long running transaction in order to begin the instantiation of your target system.

 

There are several ways to copy (instantiate) GoldenGate source database to target, including GoldenGate’s own, RMAN, Oracle DataPump, etc.

 

We will use either Datapump (export/import) or RMAN. Either way will work.

 

 

5.1 Instantiation of Source Database to Target with Oracle DataPump

 

SOURCE:/opt/app1/oracle/gg> sqlplus /nolog
 
SQL*Plus: Release 11.2.0.2.0 Production on Tue Nov 8 11:23:39 2011
 
Copyright (c) 1982, 2010, Oracle.  All rights reserved.
 
 / > connect / as sysdba
 
Connected.
 
SYS / SOURCE> select to_char(current_scn,999999999999999999) from v$database ;
 
TO_CHAR(CURRENT_SCN
-------------------
            5172101
 
1 row selected.
 
Elapsed: 00:00:00.03
SYS / SOURCE>
 
SOURCE:/opt/app1/oracle/gg> expdp directory=EXPDP_DIR full=y parallel=1 dumpfile=SOURCE_GoldenGate_%u.dmp flashback_scn= 5172101
 
Export: Release 11.2.0.2.0 - Production on Tue Nov 8 11:25:05 2011
 
Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.
 
Username: system
Password:
 

The parameter flashback_scn is used for consistent copy of database and is passed from query above.

Please note that OGG uses "CSN" and not "SCN", which are functionally interchangeable but syntactically distinct.

 

 

5.2 Instantiation of Source database to Target with RMAN

 

An alternative to Export/Import is RMAN duplicate command. Please see Metalink note 340848.1

After restoring on target, check alert log to make note of SCN for replicat start:

 

 

 TARGET:/opt/app/oracle/diag/rdbms/TARGET/TARGET/trace> vi alert_TARGET.log
Switch of datafile 85 complete to datafile copy
  checkpoint is 5397864
Switch of datafile 86 complete to datafile copy
  checkpoint is 5397864
alter database add supplemental log data(ALL) columns
SUPLOG: Previous supplemental logging attributes at scn = 0
SUPLOG:  minimal = OFF, primary key = OFF
SUPLOG:  unique = OFF, foreign key = OFF, all column = OFF
SUPLOG:  procedural replication = OFF
SUPLOG: New supplemental logging attributes at scn = 0
SUPLOG:  minimal = ON, primary key = OFF
SUPLOG:  unique = OFF, foreign key = OFF, all column = ON
SUPLOG:  procedural replication = OFF
Completed: alter database add supplemental log data(ALL) columns
alter database open resetlogs
RESETLOGS after incomplete recovery UNTIL CHANGE 5397864

 

So, SCN to be used as OGG CSN is 5397864

 

 

 

 

 

 

 

 

4.1 Run Delete Cascade Constraints and Triggers on Destination

 

You need to disable any database triggers or cascade-delete referential integrity constraints on your target tables. The reason is to prevent duplicate changes, because GoldenGate is already replicating the results of the trigger and cascade-delete operations from the source database. If you didn’t disable the constraints and triggers, GoldenGate would replicate those changes; then the triggers and cascadedelete constraints would also fire, causing duplicate changes on the target database.

 

Run the following two scripts on target only.

 

SQL> @/opt/oracle/gg/scripts/ disable_cascade_delete_constraints.sql;
SQL> @/opt/oracle/gg/scripts/triggers.sql;

 

They will prompt you for schema and create two sets of spooled scripts for disable and enable constraints and triggers.

Then run the created spooled scripts:

 

SQL> @/opt/oracle/gg/scripts/DRPROD.disable_cascade_delete_constraints.sql;
SQL> @/opt/oracle/gg/scripts/DRPROD.disable_triggers.sql;
 
TARGET:/opt/oracle/gg/scripts> ls -ltr
total 216
-rw-r--r-- 1 oracle dba  7530 Oct 15 16:07 DRPROD.enable_cascade_delete_constraints.sql
-rw-r--r-- 1 oracle dba 74296 Oct 15 16:08 DRPROD.enable_triggers.sql
-rw-r--r-- 1 oracle dba   285 Oct 15 16:12 enable_triggers.sql
-rw-r--r-- 1 oracle dba   364 Oct 15 16:12 enable_cascade_delete_constraints.sql
-rw-r--r-- 1 oracle dba   289 Oct 15 16:12 disable_triggers.sql
-rw-r--r-- 1 oracle dba   369 Oct 15 16:12 disable_cascade_delete_constraints.sql
-rwxrwxr-x 1 oracle dba  1043 Oct 19 13:58 check_gg_process.ksh
-rwxr-xr-x 1 oracle dba  3504 Oct 29 14:43 ggs_lag.ksh.bkp
-rwxr-xr-x 1 oracle dba   757 Oct 29 14:58 ggs.ksh
-rwxr-xr-x 1 oracle dba  3335 Oct 29 15:41 ggs_lag_test.ksh.bkp
-rwxrwxr-x 1 oracle dba  3440 Oct 29 15:45 ggs_lag_test.ksh
-rwxr-xr-x 1 oracle dba  3452 Oct 29 15:52 ggs_lag.ksh
-rw-r--r-- 1 oracle dba     0 Nov  9 15:29 CLMDBA.disable_cascade_delete_constraints.sql
-rw-r--r-- 1 oracle dba     0 Nov  9 15:29 CLMDBA.disable_triggers.sql
-rw-r--r-- 1 oracle dba  7782 Nov  9 15:31 DRPROD.disable_cascade_delete_constraints.sql
-rw-r--r-- 1 oracle dba 74297 Nov  9 15:31 DRPROD.disable_triggers.sql

 

 

 

 

 

 

 

 

 

 

 

 

Configure manager and ./GLOBALS with parameters from table above.

 

 

7.1 Create and Start Replicat RDRPROD1

 

Add replicats using aftercsn 5397864 from alert log information from restore above:

 

GGSCI (target) 2> info all
Program     Status      Group       Lag           Time Since Chkpt
MANAGER     RUNNING
 
 
GGSCI (target) 4> add replicat RDRPROD1, checkpointtable ggate.CHKPTAB, exttrail /opt/oracle/gg/dirdat/d2
REPLICAT added.
 
 
GGSCI (target) 4> edit params RDRPROD1
--------------------------------------------------------------------
-- Replicat for DRPROD Schema                                    --
-- Last updated by Vladimir Grigorian                --
--------------------------------------------------------------------
--CHECKPARAMS
--Replicat group --
replicat RDRPROD1
--export sid
setenv (ORACLE_SID=TARGET)
--source and target definitions--
AssumeTargetDefs
--target database login --
userid ggate, password ***
--PERFORMANCE PARMETERS--
--Double SQL transaction in 1 operation--
GROUPTRANSOPS 2000
--group similar transactions--
--BATCHSQL--
--error handling--
DDLERROR DEFAULT IGNORE RETRYOP
--file for dicarded transaction --
discardfile /opt/oracle/gg/discard/RDRPROD1_discard.txt, purge, megabytes 500
--Reporting--
ReportCount Every 30 Minutes, Rate
Report at 01:00
ReportRollover at 01:15
DiscardRollover at 02:00 ON SUNDAY
--ddl support--
--HANDLECOLLISIONS--
--Supress triggers and constraints--
DBOPTIONS DEFERREFCONST
DDL INCLUDE ALL
DDLOPTIONS REPORT
DDLERROR 942 DISCARD INCLUDE OPTYPE DROP
REPERROR (1403, DISCARD)
--Enable tracing--
--TRACE2--
--Exclude for table does not exist error--
--MAPEXCLUDE drprod.test*
--Specify table mapping ---
map drprod.*, target drprod.* ;
 
 
GGSCI (target) 5> start replicat RDRPROD1, aftercsn 5397864
 
Sending START request to MANAGER ...
REPLICAT RDRPROD1 starting
 
If replicat needs to be started as of current time:
 
GGSCI (target) 9>   alter replicat rdrprod1 begin now
REPLICAT altered.
GGSCI (target) 49> start RDRPROD1
 
Sending START request to MANAGER ...
REPLICAT RDRPROD1 starting
 
GGSCI (target) 10> info all
 
Program     Status      Group       Lag           Time Since Chkpt
 
MANAGER     RUNNING
REPLICAT    STOPPED     RDRPROD1    00:00:00      00:00:03
 
 
 
 

 7.2. Create and Start Replicat RCLMDBA1

 

 
GGSCI (target) 10> add replicat RCLMDBA1, checkpointtable ggate.CHKPTAB, exttrail /opt/oracle/gg/dirdat/c2
REPLICAT added.
 
GGSCI (target) 4> edit params RCLMDBA1
 
--------------------------------------------------------------------
-- Replicat for CLMDBA Schema                                    --
-- Last updated on Nov 18 2011 by Vladimir Grigorian --
--------------------------------------------------------------------
--Replicat group --
replicat RCLMDBA1
--export sid
setenv (ORACLE_SID=TARGET)
--source and target definitions--
AssumeTargetDefs
--target database login --
userid ggate, password ***
--PERFORMANCE PARMETERS--
--Double SQL transaction in 1 operation--
GROUPTRANSOPS 2000
--group similar transactions--
--BATCHSQL--
--error handling--
DDLERROR DEFAULT IGNORE RETRYOP
--file for dicarded transaction --
discardfile /opt/oracle/gg/discard/RCLMDBA1_discard.txt, purge, megabytes 500
--Reporting--
ReportCount Every 30 Minutes, Rate
Report at 01:00
ReportRollover at 01:15
DiscardRollover at 02:00 ON SUNDAY
--ddl support--
--Supress constraints, SUPRESSTRIGGERS is not available in this version--
DBOPTIONS DEFERREFCONST
DDL INCLUDE ALL
DDLOPTIONS REPORT
DDLERROR 942 DISCARD INCLUDE OPTYPE DROP
REPERROR (1403, DISCARD)
--HANDLECOLLISIONS--
--Specify table mapping ---
map clmdba.*, target clmdba.* ;
 
 

If data is static you can start replicat as of current time.

 
GGSCI (target) 11> alter replicat RCLMDBA1 begin now
REPLICAT altered.
 
 
GGSCI (target) 12> start replicat RCLMDBA1
 
Sending START request to MANAGER ...
REPLICAT RCLMDBA1 starting
 
 
GGSCI (target) 13>
 
GGSCI (target) 13> info all
 
Program     Status      Group       Lag           Time Since Chkpt
 
MANAGER     RUNNING
REPLICAT    RUNNING     RCLMDBA1    00:00:00      00:00:02
REPLICAT    RUNNING     RDRPROD1    00:00:00      00:00:09
 
 
 
 
 

To test replication, we will create a table (CUSTOMERS) and insert 100 rows in it. You can adjust number of rows if you want to estimate lag performance by adjusting CONNECT BY LEVEL <= 100; to a larger  value.

 
CLMDBA / SOURCE> CREATE TABLE
CUSTOMERS
(CUSTOMER_ID NUMBER PRIMARY KEY
,CUSTOMER_NAME VARCHAR2(30) NOT NULL
,CUSTOMER_REGISTRATION DATE
,CURRDATE DATE);
 
Table created.
 
Elapsed: 00:00:00.14
CLMDBA / SOURCE> CREATE SEQUENCE
CUSTOMER_ID_SEQ
INCREMENT BY 2
START WITH 2
NOMAXVALUE
NOMINVALUE
NOCYCLE
CACHE 500;  
 
Sequence created.
 
Elapsed: 00:00:00.03
CLMDBA / SOURCE> INSERT INTO CUSTOMERS SELECT CUSTOMER_ID_SEQ.NEXTVAL,
DBMS_RANDOM.STRING('P',DBMS_RANDOM.VALUE(1,30)),
(SYSDATE-1825)+DBMS_RANDOM.VALUE(1,1820),
CURRENT_TIMESTAMP FROM DUAL CONNECT BY LEVEL <= 100; 
 
100 rows created.
 
Elapsed: 00:00:00.03
CLMDBA / SOURCE> commit;

 

 

On destination:

 

GGSCI (target) 17> stats RCLMDBA1
 
Sending STATS request to REPLICAT RCLMDBA1 ...
 
Start of Statistics at 2011-11-10 17:15:19.
 
DDL replication statistics:
 
*** Total statistics since replicat started     ***
        Operations                                   2.00
        Mapped operations                            2.00
        Unmapped operations                          0.00
        Other operations                             0.00
        Excluded operations                          0.00
        Errors                                       0.00
        Retried errors                               0.00
        Discarded errors                             0.00
        Ignored errors                               0.00
 
Replicating from CLMDBA.CUSTOMERS to CLMDBA.CUSTOMERS:
 
*** Total statistics since 2011-11-10 17:14:19 ***
        Total inserts                              100.00
        Total updates                                0.00
        Total deletes                                0.00
        Total discards                               0.00
        Total operations                           100.00
 
*** Daily statistics since 2011-11-10 17:14:19 ***
        Total inserts                              100.00
        Total updates                                0.00
        Total deletes                                0.00
        Total discards                               0.00
        Total operations                           100.00
 
*** Hourly statistics since 2011-11-10 17:14:19 ***
        Total inserts                              100.00
        Total updates                                0.00
        Total deletes                                0.00
        Total discards                               0.00
        Total operations                           100.00
 
*** Latest statistics since 2011-11-10 17:14:19 ***
        Total inserts                              100.00
        Total updates                                0.00
        Total deletes                                0.00
        Total discards                               0.00
        Total operations                           100.00
 
End of Statistics.
 
 
GGSCI (target) 18> info all
 
Program     Status      Group       Lag           Time Since Chkpt
 
MANAGER     RUNNING
REPLICAT    RUNNING     RCLMDBA1    00:00:00      00:00:04
REPLICAT    RUNNING     RDRPROD1    00:00:00      00:00:05
 
 
GGSCI (target) 19>

 

 

On source:

 

CLMDBA / SOURCE> drop sequence CUSTOMER_ID_SEQ;
 
Sequence dropped.
 
Elapsed: 00:00:00.03
CLMDBA / SOURCE> drop table customers;

 

 

 

 

 

GoldenGate needs some monitoring enabled. The following are settings that serve the goal of:

1)      Restarting failed GoldenGate processes

2)      Emailing alerts to DBAs if they are down

The first feature –configuration of AUTOSTART and AUTORESTART parameter in manager - has already been configured in prior steps, but not explained. These mean only if manager is running it will try to restart processes automatically. Also, it purges old trail files.

 

GGSCI (source) 3> edit params mgr
-------------------------------------------------------------------
-- GoldenGate Manager                                                                       --
-- Last updated on 20111015 by Vladimir Grigorian    --
-------------------------------------------------------------------
PORT 7809
USERID ggate, PASSWORD ***
AUTOSTART EXTRACT pdrprod1
AUTOSTART EXTRACT eclmdba1
AUTOSTART EXTRACT pclmdba1
AUTOSTART EXTRACT edrprod1
AUTORESTART EXTRACT eclmdba1, RETRIES 3, WAITMINUTES 1, RESETMINUTES 60
AUTORESTART EXTRACT pclmdba1, RETRIES 3, WAITMINUTES 1, RESETMINUTES 60
AUTORESTART EXTRACT pdrprod1, RETRIES 3, WAITMINUTES 1, RESETMINUTES 60
AUTORESTART EXTRACT edrprod1, RETRIES 3, WAITMINUTES 1, RESETMINUTES 60
--Specifies to log the lag time as a warning in the event log
LAGCRITICALMINUTES 5
--Specifies how often to report lag info to the event log
LAGREPORTMINUTES 60
LAGINFOMINUTES 0
--Manages trail files to conserve space--
PURGEOLDEXTRACTS /opt/oracle/gg/dirdat/*, USECHECKPOINTS, MINKEEPDAYS

 

 

The second feature – email alerts – is self explanatory.

 

SOURCE

 

SOURCE:/opt/oracle/gg/scripts> ls -ltr
total 40
-rwxrwxr-x 1 oracle oinstall  601 Oct 29 09:52 ggs.ksh
-rwxrwxr-x 1 oracle oinstall 3496 Oct 29 15:53 ggs_lag.ksh
-rwxrwxr-x 1 oracle oinstall 1760 Oct 30 19:23 check_gg_process.ksh

 

1.      Lag Script

 

Lag script ggs_lag emails DBAs if lag time exceeds 30 minutes (can be adjusted).

 

 

SOURCE:/opt/oracle/gg/scripts> more ggs_lag.ksh
#!/bin/ksh
############################################
# Name: ggs_lag.ksh #
# UPDATED: on 2011-10-19 by VG
# PURPOSE: TO MONITOR LAG OF GOLDEN GATE #
# NOTE: THIS SCRIPT CALLS ggs.ksh #
# THIS SCRIPT NOTIFY IF LAG IS MORE THEN 30 MIN #
# ONLY FOR FOR EXT AND PMP PROCESS GROUP #
###########################################
export GGATE=/opt/app1/oracle/gg
alias gate='clear;cd $GGATE;./ggsci'
export PATH=/opt/app1/oracle/gg:/u01/app/oracle/product/11.2.0/db_1/bin:/usr/sbin:/usr/lib/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local
/sbin:/usr/sbin:/sbin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/usr/lib:/opt/app1/oracle/gg:
LOGDIR=/opt/oracle/gg/log
EMAILFile=$LOGDIR/ggs_email.log
BOX=$(uname -a | awk '{print $2}')
 
##########################################################################
# RUNNING SCRIPT TO GET GOLDEN GATE INFORMATION #
##########################################################################

 

 
 
/opt/oracle/gg/scripts/ggs.ksh > $LOGDIR/ggs_1.log
 
#to check when script was running
 
echo "script ggsksh completed from ggs_lag at `date`" >> /tmp/ggs_check.log
 
##################################################################################
## FORMATING INFORMATION: change cut -d":" -f 1,4 TO cut -d":" -f 1,2 ##
## to getinformation about lag instead of checkpoint ##
## this command grep only EXT_ and PMP_ if you need more pattern ##
## if you need more pattern to be greped please add to '(EXT_|PMP_|pattern)' ##
##################################################################################
 
cat $LOGDIR/ggs_1.log|egrep -i '(EXT_|PMP_)'|cut -d":" -f 1,2,4| tr ":" " "|tr -s '[:space:]'|cut -d" " -f1,2,3,4,5,6 > $LOGDIR/ggs_2.log
 
# uncomment below command if you want to get lag and checkpoint both information #
 
cat $LOGDIR/ggs_1.log|egrep -i '(EXT_|PMP_|DART)'|cut -d":" -f 1,2,4| tr ":" " "|tr -s '[:space:]'|cut -d" " -f1,22,3,4,5,6 > $LOGDIR/ggs_2.lo
g
 
# uncomment below command if you want to get lag information about running process #
 
cat $LOGDIR/ggs_1.log|grep RUNNING|cut -d":" -f 1,2,4| tr ":" " "|tr -s '[:space:]'|cut -d" " -f1,2,3,4,5,6 > $LOGDIR/ggs_2.log
 
##########################################################################
## CHECKING FOR LAG MORE THEN 30 MIN FOR ABENDED PROCESS ##
##########################################################################
 
##########################################################################
## CHECKING FOR LAG MORE THEN 30 MIN FOR ABENDED PROCESS ##
##########################################################################
 
awk '{if ( $4 > 00 || $5 >=30 ) {print $1 " " $3 " HAS LAG of " $4" hour " $5 " min -- at -- " d "\n"} else {{print "NO LAG FOR " $3 " " d >>
"/tmp/ggs_lag_fine.log" }}' d="$(date)" $LOGDIR/ggs_2.log > $LOGDIR/ggs_email.log
 
# uncomment below command if you want to get lag and checkpoint both information #
 
awk '{if ($4 >=30 || $5>=30 ) {print $1 " " $3 " has lag of "$4" min with checkpoint of "$5" min -- at -- " d "\n"} else {print "NO LAG FOR "
 $3 " "d > "/tmp/ggs_lag_fine.log" }}' d="$(date)" $LOGDIR/ggs_2.log > $LOGDIR/ggs_email.log
 
##########################################################
## SENDING EMAIL IF ERRORS ARE IN LOGFILE ###
##########################################################
 
if [ -s $EMAILFile ]; then
#echo "ERRORS FOUND"
mailx -s "GoldenGate Lag Exceeding 30 mins Found on: $BOX" grigorianvlad@yahoo.com < $EMAILFile
#echo "ERRORS NOT FOUND"
fi
 
################# SCRIPT END ######################

 

Here is email alert example (no lag here, the script was changed for the purpose of testing at lag => 0):

 

 

 

 

2.      Check_gg_process.ksh

This script checks if GoldenGate processes are running on a server and emails DBAs if they are not.

#*************************************************************
# Test to see if Oracle GoldenGate extract is running
# Updated on 2011-10-19 by VG  
#*************************************************************
#!/bin/ksh
EMAIL_LIST="grigorianvlad@yahoo.com"
host=`hostname`
gghome=/opt/app1/oracle/gg
check=`ps -ef|grep drprod1|grep -v "grep drprod1"|wc -l`;
check_num=`expr $check`
if [ $check_num -le 0 ]
then
echo "GoldenGate Extract DRPROD1 is down on $host." > $gghome/DRPROD1.out
tail -200 $gghome/dirrpt/DRPROD1.rpt >> $gghome/DRPROD1.out
        mailx -s "GoldenGate Extract DRPROD1 is down on $host." $EMAIL_LIST < $gghome/DRPROD1.out
fi
 
check=`ps -ef|grep eclmdba1|grep -v "grep eclmdba1"|wc -l`;
check_num=`expr $check`
if [ $check_num -le 0 ]
then
echo "GoldenGate Extract ECLMDBA1 is down on $host." > $gghome/ECLMDBA1.out
tail -200 $gghome/dirrpt/ECLMDBA1.rpt >> $gghome/ECLMDBA1.out
        mailx -s "GoldenGate Extract ECLMDBA1 is down on $host." $EMAIL_LIST < $gghome/ECLMDBA1.out
fi
 
check=`ps -ef|grep pclmdba1|grep -v "grep pclmdba1"|wc -l`;
check_num=`expr $check`
if [ $check_num -le 0 ]
then
echo "GoldenGate DataPump PCLMDBA1 is down on $host." > $gghome/PCLMDBA1.out
tail -200 $gghome/dirrpt/PCLMDBA1.rpt >> $gghome/PCLMDBA1.out
        mailx -s "GoldenGate DataPump PCLMDBA1 is down on $host." $EMAIL_LIST < $gghome/PCLMDBA1.out
fi
 
DESTINATION:
 
#*************************************************************
# Test to see if Oracle GoldenGate replicat is running
# Updated on 2011-10-19 by VG
#*************************************************************
#!/bin/ksh
EMAIL_LIST="grigorianvlad@yahoo.com"
host=`hostname`
gghome=/opt/app/oracle/gg
check=`ps -ef|grep rdrprod1|grep -v "grep rdrprod1"|wc -l`;
check_num=`expr $check`
if [ $check_num -le 0 ]
then
echo "GoldenGate Replicat RDRPROD1 is down on $host." > $gghome/RDRPROD1.out
tail -200 $gghome/dirrpt/RDRPROD1.rpt >> $gghome/RDRPROD1.out
        mailx -s "GoldenGate Replicat RDRPROD1 is down on $host." $EMAIL_LIST < $gghome/RDRPROD1.out
fi
check=`ps -ef|grep rclmdba1|grep -v "grep rclmdba1"|wc -l`;
check_num=`expr $check`
if [ $check_num -le 0 ]
then
echo "GoldenGate Replicat RCLMDBA1 is down on $host." > $gghome/RCLMDBA1.out
tail -200 $gghome/dirrpt/RCLMDBA1.rpt >> $gghome/RCLMDBA1.out
        mailx -s "GoldenGate Replicat RCLMDBA1 is down on $host." $EMAIL_LIST < $gghome/RCLMDBA1.out
fi

 

 

 

Add both scripts to crontab with 60 and 30 mins interval:

 

#Script for monitoring GoldenGate replicat
*/60 * * * * /opt/oracle/gg/scripts/check_gg_process.ksh 2> /tmp/check_gg_process.log
*/30 * * * * /opt/oracle/gg/scripts/ggs_lag.ksh 2> /tmp/check_gg_lag.log

 

 Back to Top


 

Copyright © 2019 Vladimir Grigorian. All Rights Reserved