Thursday, 2 July 2015

Changing Sysdate / Systimestamp


Changing sysdate / systimestamp.
----------------------------------------

1. Server currently running with SGT timezone.


Conneting to the db using client, client OS timezone is SGT.


Scenario 1 )

[oracle@standalone2 ~]$ sqlplus manzoor/manzoor@source

SQL*Plus: Release 11.2.0.3.0 Production on Fri Jul 3 10:56:47 2015

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select systimestamp from dual;

SYSTIMESTAMP
---------------------------------------------------------------------------
03-JUL-15 10.56.54.251769 AM +08:00


-- Its retuned the SGT value.


Chaning the timezone in client side.


[oracle@standalone2 ~]$ export TZ=Asia/Calcutta
[oracle@standalone2 ~]$ date
Fri Jul  3 08:27:45 IST 2015
[oracle@standalone2 ~]$ sqlplus manzoor/manzoor@source

SQL*Plus: Release 11.2.0.3.0 Production on Fri Jul 3 08:27:49 2015

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select systimestamp from dual;

SYSTIMESTAMP
---------------------------------------------------------------------------
03-JUL-15 10.57.54.919229 AM +08:00


-- Its retunred SGT value, which means changing the tz on the client side will not make any
effect. The systimestamp which retunred from the database server.


Sysdate & Systimestamp function will make a call to the OS to get the time for the local connection,
and for the remote connection it will retrun the timzone in which the database started.


--> Shutdown the database.
--> set timezone to IST
--> start the database

[oracle@rhel11gr2rac1 admin]$ export TZ=Asia/Calcutta
[oracle@rhel11gr2rac1 admin]$ sqlplus / as sysdba
SQL> startup
ORACLE instance started.

Total System Global Area  521936896 bytes
Fixed Size          2229944 bytes
Variable Size     159385928 bytes
Database Buffers          356515840 bytes
Redo Buffers        3805184 bytes
Database mounted.
Database opened.
SQL> select systimestamp from dual;

SYSTIMESTAMP
---------------------------------------------------------------------------
03-JUL-15 08.30.28.585681 AM +05:30


-- Now from the client make a remote connection the server and select the timestamp.


[oracle@rhel11gr2rac1 ~]$ sqlplus manzoor/manzoor@source

SQL*Plus: Release 11.2.0.3.0 Production on Fri Jul 3 11:11:24 2015

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select systimestamp from dual;

SYSTIMESTAMP
---------------------------------------------------------------------------
03-JUL-15 08.31.03.026900 AM +05:30


-- Now the client returns the timestamp as per the setting in which db is restarted.

- You the below query to check in which TZ the database is started.


sql> SET SPACE 1 LINESIZE 80 PAGES 1000
sql> select to_char(ORIGINATING_TIMESTAMP,'YYYY/MM/DD HH24:MI:SS TZH:TZM') from V$DIAG_ALERT_EXT WHERE trim(COMPONENT_ID)='rdbms' and inst_id=1
     and MESSAGE_TEXT like ('PMON started with%') order by originating_timestamp;

TO_CHAR(ORIGINATING_TIMEST
--------------------------
2015/06/24 10:59:45 +08:00
2015/06/24 11:13:54 +08:00
2015/06/24 11:18:18 +08:00
2015/06/24 11:21:32 +08:00
2015/06/24 11:37:08 +08:00
2015/07/03 10:23:00 +08:00
2015/07/03 08:30:11 +05:30

7 rows selected.

-- We could see that the pmon process started with IST timezone. Hence the remote connections for which db is registered dynamically with the listener will return the time based on this timezone.


Some points
-----------

a) PMON only reads the OS environment variables when the database is started.
b) If TZ is changed after the database is started starts PMON will not pick up the changed value.
c) When dynamic database service registration is enabled to a listener, PMON both registers the service name and
   also the environment variables for the database associated with that service name.
d) This means that connections via the dynamically registered service handler will continue to use the old TZ.
e) If the listener is restarted and a also has a static service registration, connection via the listener will use the TZ set in the listeners environment until PMON first re-registers the service handler.
   When the listener has two service registrations, one static and one dynamic, then the dynamic one will be used.

Listener Registration
---------------------

a) The way registration typically works, The dispatcher registration is going to look for a default listener on the default port (1521)
   on the hardware IP address. If things are not in that standard format, local_listener should be used to point the database to where the listener is.



Eg.

Scenario

1) DB is running with SG timezone.  (Below output from local connection)


SQL> select systimestamp from dual;

SYSTIMESTAMP
-------------------------------------
30-OCT-15 08.39.16.829936 AM +08:00




2) Listener is running with SG timezone. (Below output from remote connection)



[oracle@rhel11gr2rac1 admin]$ sqlplus sys/admin@purchase as sysdba

SQL> select systimestamp from dual;

SYSTIMESTAMP
---------------------------------------------------------------------------
30-OCT-15 08.39.56.457772 AM +08:00



3) Now restarting the listener with differnet Timezone eg (IST), which means db will continue to run in
   SG timezone but only the listener will be running in IST timezone.


[oracle@rhel11gr2rac1 admin]$ export TZ=Asia/Kolkata
[oracle@rhel11gr2rac1 admin]$ lsnrctl stop listener_seccdb
[oracle@rhel11gr2rac1 admin]$ lsnrctl start listener_seccdb

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1600)))
STATUS of the LISTENER
------------------------
Alias                     listener_seccdb
Version                   TNSLSNR for Linux: Version 12.1.0.2.0 - Production
Start Date                30-OCT-2015 06:12:07
Uptime                    0 days 0 hr. 0 min. 0 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /oracle/product/12.1.0.2/db_1/network/admin/listener.ora
Listener Log File         /u01/app/oraInventory/diag/tnslsnr/rhel11gr2rac1/listener_seccdb/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1600)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=rhel11gr2rac1.manzoor.com)(PORT=1600)))
Services Summary...
Service "purchase" has 1 instance(s).
  Instance "purchase", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully


4) DB is statically registered in the listener, note that listener is running in port 1600 which is not the default (1521) port,
      hence dynamic register to the listener will not happen. It has to be done by setting up local_listener parameter to
      register db to the listener, here in our case we have registered the db statically in the listener.

        Recall the point
      "If the listener is restarted and a also has a static service registration, connection via the listener will use the TZ set in the
      listeners environment until PMON first re-registers the service handler.
      When the listener has two service registrations, one static and one dynamic, then the dynamic one will be used"


LISTENER_SECCDB =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1600))
      (ADDRESS = (PROTOCOL = TCP)(HOST = rhel11gr2rac1.manzoor.com)(PORT = 1600))
    )
  )
SID_LIST_LISTENER_SECCDB =
(SID_LIST =
  (SID_DESC =
        (ORACLE_HOME=/oracle/product/12.1.0.2/db_1)
        (SID_NAME = purchase)
  )
)

      So as per above remote connection now will have the timezone as IST.


[oracle@rhel11gr2rac1 admin]$ sqlplus sys/admin@purchase as sysdba

SQL> select systimestamp from dual;

SYSTIMESTAMP
---------------------------------------------------------------------------
30-OCT-15 06.19.03.769312 AM +05:30


(As you can see remote connection is showing as +5:30 Which is IST)




A) Point to note.

      a) if the listener is running with default port then again a dynamic registraion will happen and pmon register the db
            db to the listener with db timzone, hence once dynamic registration is done any further remote connection will
            use the db timezone.

      b) systimestamp will retrun as IST during remote connection and if any insert happen to any table it will be as IST.

            Eg.

[oracle@rhel11gr2rac1 trace]$ sqlplus manzoor/ahamed@purchase


SQL> select systimestamp from dual;

SYSTIMESTAMP
------------------------------------
30-OCT-15 06.24.29.160008 AM +05:30


-Remote connection is establieshed and it is showing IST timezone since listener is runnig with IST timezone.


SQL> desc emp1;
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 EMPNO                                              NUMBER
 JOIN_DATE                                          DATE


SQL> insert into emp1 values (1,sysdate);

1 row created.

SQL> commit;

Commit complete.


SQL> insert into emp1 values (1,sysdate);

1 row created.

SQL> commit;

Commit complete.



SQL> alter session set nls_date_format = 'DD-MM-YY HH24:MI:SS';

Session altered.

SQL> select * from emp1 order by 2;

     EMPNO JOIN_DATE
---------- -----------------
         1 30-10-15 06:25:52
         1 30-10-15 06:27:23


--Now lets make local connection and insert an row.

[oracle@rhel11gr2rac1 ~]$ sqlplus manzoor/ahamed

SQL> insert into emp1 values (2,sysdate);

1 row created.

SQL> commit;

Commit complete.

SQL> select * from emp1 order by 2;

     EMPNO JOIN_DATE
---------- -----------------
         1 30-10-15 06:25:52
         1 30-10-15 06:27:23
         2 30-10-15 08:58:52

-- Now the data inserted in SG timezone.

-- Now lets run backup runing remote connection.

[oracle@rhel11gr2rac1 admin]$ rman target sys/admin@purchase

Recovery Manager: Release 12.1.0.2.0 - Production on Fri Oct 30 06:31:38 2015

Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved.

connected to target database: PURCHASE (DBID=3978345698)


--As you can see the connection established at 6:31 which is IST time.

Triggered the backup.

RMAN> backup incremental level 1 database;

Starting backup at 30-OCT-15
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1


--


In another session connect to the db and ran the below query to check the



alter session set nls_date_format = 'DD-MM-YY HH24:MI:SS';

SQL> select start_time, end_time, input_bytes, output_bytes/1024/1024/1024 "Output GB", INPUT_TYPE, status, ELAPSED_SECONDS/60/60 "Hours"
from V$rman_backup_job_details order by 1;



START_TIME        END_TIME          INPUT_BYTES  Output GB INPUT_TYPE    STATUS                       Hours
----------------- ----------------- ----------- ---------- ------------- ----------------------- ----------
13-09-15 07:17:02 13-09-15 07:17:11     3670016 .003456116 DATAFILE INCR COMPLETED                    .0025
13-09-15 08:24:14 13-09-15 08:24:22     3670016 .003456116 DATAFILE INCR COMPLETED               .002222222
13-09-15 08:25:13 13-09-15 08:25:20     3670016 .003456116 DATAFILE INCR COMPLETED               .001944444
13-09-15 08:27:07 13-09-15 08:27:14     3670016 .003456116 DATAFILE INCR FAILED                  .001944444
13-09-15 08:29:22 13-09-15 08:29:29     3670016 .003456116 DATAFILE INCR FAILED                  .001944444
13-09-15 08:33:09 13-09-15 08:33:16     3670016 .003456116 DATAFILE INCR COMPLETED               .001944444
06-10-15 10:35:06 06-10-15 10:42:05  3339788800 2.75684404 DB INCR       FAILED                  .116388889
30-10-15 06:00:49 30-10-15 08:31:56  2254438400  .57598114 DB INCR       COMPLETED               2.51861111
30-10-15 06:33:08 30-10-15 09:03:17   382509056  .00970459 DB INCR       RUNNING                     2.5025
30-10-15 08:29:47 30-10-15 08:30:13           0          0 DB INCR       FAILED                  .007222222


-- As you can see the above backup (status RUNNING ) started at 6:33 and the endtime is taken as SG time, so the number of
hours its running shows as 2.5 hours even though it is just started 1 minute back.

Lets query once the backup is completed.


Now backup is completed.

channel ORA_DISK_3: backup set complete, elapsed time: 00:00:27
Finished backup at 30-OCT-15



START_TIME        END_TIME          INPUT_BYTES  Output GB INPUT_TYPE    STATUS                       Hours
----------------- ----------------- ----------- ---------- ------------- ----------------------- ----------
11-09-15 09:19:44 11-09-15 09:26:47  4993318912 1.68067169 DB FULL       FAILED                       .1175
13-09-15 07:17:02 13-09-15 07:17:11     3670016 .003456116 DATAFILE INCR COMPLETED                    .0025
13-09-15 08:24:14 13-09-15 08:24:22     3670016 .003456116 DATAFILE INCR COMPLETED               .002222222
13-09-15 08:25:13 13-09-15 08:25:20     3670016 .003456116 DATAFILE INCR COMPLETED               .001944444
13-09-15 08:27:07 13-09-15 08:27:14     3670016 .003456116 DATAFILE INCR FAILED                  .001944444
13-09-15 08:29:22 13-09-15 08:29:29     3670016 .003456116 DATAFILE INCR FAILED                  .001944444
13-09-15 08:33:09 13-09-15 08:33:16     3670016 .003456116 DATAFILE INCR COMPLETED               .001944444
06-10-15 10:35:06 06-10-15 10:42:05  3339788800 2.75684404 DB INCR       FAILED                  .116388889
30-10-15 06:00:49 30-10-15 08:31:56  2254438400  .57598114 DB INCR       COMPLETED               2.51861111
30-10-15 06:33:08 30-10-15 09:03:37  2254438400 .015541077 DB INCR       COMPLETED               2.50805556
30-10-15 08:29:47 30-10-15 08:30:13           0          0 DB INCR       FAILED                  .007222222


Now the backup is completed and shows it took 2.5 hours to complete as per report (so dont be surprised).


To check the environment in which a process is started you can use below. (in Linux)

Linux

a)  Get the process id.
[oracle@rhel11gr2rac1 trace]$ ps -ef | grep tns
oracle    7045     1  0 06:12 ?        00:00:00 /oracle/product/12.1.0.2/db_1/bin/tnslsnr listener_seccdb –inherit

b)  View the environment details of that process. (environment details of each process will be stored in
/proc/[process id]/environ binary file.

[oracle@rhel11gr2rac1 trace]$ strings /proc/7045/environ | grep TZ
TZ=Asia/Kolkata
Solaris
a)  Get the process id

-bash-3.2$ ps -ef | grep pmon
  oracle 11310     1   0   Oct 29 ?           0:02 ora_pmon_source

b)  View the environment details of that process.
-bash-3.2$ pargs -e 11310 | grep TZ

envp[18]: TZ=Asia/Kuala_Lumpur



Oracle Note ids to refer

How to Change the "Database Time" ( SYSDATE and SYSTIMESTAMP) To An Other Time / Timezone (Doc ID 1988586.1)
Timestamps & time zones - Frequently Asked Questions (Doc ID 340512.1)



We can also set listener to show up more than 1 timezone. Refer

How To setup TNS listener to Show More Than one Timezone (Doc ID 399448.1)
























Wednesday, 2 October 2013

Oracle Golden Gate High Availability using Oracle Clusterware

OGG High Availability using Oracle clusterware
=======================================
1) Oracle Golden Gate cluster high availability Pre-requsite.

a. Oracle Golden gate runs on one server at any time.
b. In the event of the failure on one node, the oracle GG can be started on the another node.
c. In order to resume processing on the another node, we need to maintain/store the
recover relagted files (checkpoint file & trailfiles) on shared location.
d. Oracle ACFS is the recommended cluster file system for Oracle Golden Gate binaries and
trail files in Real Application Cluster configurations for ease of management and high availability.


Note: ACFS can be used for Oracle Golden Gate trail files with no restrictions. Oracle GoldenGate installation can be done on ACFS and you can also store the recovery-related files in a cluster configuration in ACFS to make them accessible to all nodes. However if your Oracle Grid Infrastructure version is older than 11.2.0.3 then ACFS mounted on multiple servers concurrently does not currently support file locking, thus you would need to mount ACFS on only one server.

If ACFS is mounted on one server at a time then file locking is supported in pre 11.2.0.3 Grid Infrastructure releases. This file locking issue has been resolved in Oracle Grid Infrastructure release 12c and the fix has been back ported up to version 11.2.0.3.


2) Oracle clusterware

a) Oracle clusterware provides the capability to manage the third-party applications.
b) There are commands to register an application and instruct Oracle Clusterware how to manage the application in a clustered environment.
c) This capability will be used to register the Oracle GoldenGate manager process as an application managed through Oracle Clusterware.
d) Oracle Clusterware can be installed standalone without an Oracle RAC database and still manage a cluster of  servers and various applications running on these servers. As such Oracle Clusterware can also be installed on more than just the database servers to form a single cluster.

3) Oracle Golden Gate Installations.

a) You may choose to perform a local installation on every server, or a single installation on a shared file system. You will need shared storage for the recovery-related files. On a Unix/Linux platform you can use a symbolic link to a central location for the shared directories.

4) Virtual IP.

a) Oracle Clusterware uses the concept of a Virtual IP address (VIP) to manage high availability for applications that require incoming network traffic (including the Oracle RAC database).
b) A VIP is an IP address on the public subnet that can be used to access a server. If the server hosting the VIP were to go down, then Oracle Clusterware will migrate the VIP to a surviving server to minimize interruptions for the application  accessing the server (through the VIP).
c) This concept enables faster failovers compared to time-out based failovers on a server's actual IP address in case of a server failure.
d) For Oracle GoldenGate, you should use a VIP to access the manager process to isolate access to the manager process from the physical server that is running Oracle GoldenGate. Remote pumps must use the VIP to contact the Oracle GoldenGate manager. The VIP must be an available IP address on the public subnet and cannot be determined through DHCP.
Ask a system administrator for an available fixed IP address for Oracle GoldenGate managed through Oracle Clusterware.

5. We need to instruct Oracle clusterware  how to start, stop, check process.

   i) Start
a) Oracle GoldenGate manager is the process that starts all other Oracle GoldenGate processes. The only process that Oracle Clusterware should start is the manager process. Use the AUTOSTART parameter in the manager parameter file to start extract and replicat processes. You can use wild cards (AUTOSTART ER *) to start all extract and replicat processes.
b) Also note that once manager is started through Oracle Clusterware, it is Oracle Clusterware that manages its availability. If you would stop manager through the command interface ggsci, then Oracle Clusterware will attempt to restart it.  Use the Oracle Clusterware commands to stop Oracle GoldenGate and prevent Oracle Clusterware from attempting to restart it.

   ii) check
a) The validation whether Oracle GoldenGate is running is equivalent to making sure the Oracle GoldenGate manager runs.

   iii) Stop
a) Stop must stop all Oracle GoldenGate processes, including manager. Stop may be called during a planned downtime (e.g. a server is taken out of a cluster for maintenance reasons) and/or if you manually instruct Oracle Clusterware to relocate Oracle GoldenGate to a different server (e.g. to change the load on a server). If a server crashes then all processes will go down with it, in which case they can be started on another server.


Setup
=====


1 .   As of now below setup is running.

a. Source is two node rac where the GG is configured using ACFS.
        b. Configured one extract and one pump process on the source.
c. Target is standalone db.
d. One Replicat process is configured in the target.



2. Now we need to register the Golden gate in Oracle cluster ware. We need to use oracle clusterware commands to create, register and set privileges on the VIP and the Oracle Golden gate application.Once registered, use the Oracle Clusterware commands to start, relocate and stop Oracle GoldenGate.


3. Add an application VIP.

a) The first step is to create an application VIP. The VIP will be used to access Oracle GoldenGate.
    Oracle Clusterware will assign the VIP to a physical server, and migrate the VIP if that server were to go down or   if you instruct Clusterware to do so.


b. Update the below vip in the /etc/hosts file on both the nodes , ( the vip should be on the same subnet of the public ip).

########## VIP FOR GOLDENGATE ################################

192.168.0.22    goldengate-vip.manzoor.com      goldengate-vip


c. Create a application VIP using below command as root user.


[root@rhel11gr2rac1 bin]# cd /grid/11.2/bin

[root@rhel11gr2rac1 bin]# ./appvipcfg -help
Production Copyright 2007, 2008, Oracle.All rights reserved
Unknown option: help

  Usage: appvipcfg create -network= -ip= -vipname=
                          -user=[-group=] [-failback=0 | 1]
                   delete -vipname=


To identifiy the network number execute the below command.

[root@rhel11gr2rac1 bin]# ./crsctl stat res -p | grep  -ie.network -ie subnet | grep -ie name -ie subnet
NAME=ora.net1.network
USR_ORA_SUBNET=192.168.0.0

here ora.net1 in NAME denotes the network number which is 1, and the USR_ORA_SUBNET denotes the subnet under which
the vip will be created.

Execute the below command to create the application vip.

./appvipcfg create -network=1 -ip=192.168.0.22 -vipname=goldengate-vip -user=root
Production Copyright 2007, 2008, Oracle.All rights reserved
2013-10-01 23:18:11: Creating Resource Type
2013-10-01 23:18:11: Executing /grid/11.2/bin/crsctl add type app.appvip_net1.type -basetype ora.cluster_vip_net1.type -file /grid/11.2/crs/template/appvip.type
2013-10-01 23:18:11: Executing cmd: /grid/11.2/bin/crsctl add type app.appvip_net1.type -basetype ora.cluster_vip_net1.type -file /grid/11.2/crs/template/appvip.type
2013-10-01 23:18:13: Create the Resource
2013-10-01 23:18:13: Executing /grid/11.2/bin/crsctl add resource goldengate-vip -type app.appvip_net1.type -attr "USR_ORA_VIP=192.168.0.22,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL='owner:root:rwx,pgrp:root:r-x,other::r--,user:root:r-x',HOSTING_MEMBERS=rhel11gr2rac1.manzoor.com,APPSVIP_FAILBACK="
2013-10-01 23:18:13: Executing cmd: /grid/11.2/bin/crsctl add resource goldengate-vip -type app.appvip_net1.type -attr "USR_ORA_VIP=192.168.0.22,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL='owner:root:rwx,pgrp:root:r-x,other::r--,user:root:r-x',HOSTING_MEMBERS=rhel11gr2rac1.manzoor.com,APPSVIP_FAILBACK="


d) Now allow the oracle clusterware owner (eg. oracle or grid) to run the script to start the VIP.

execute the below as root.

./crsctl setperm resource goldengate-vip -u user:oracle:r-x


e) As oracle user start the vip.


[oracle@rhel11gr2rac1 bin]$ ./crsctl start resource goldengate-vip
CRS-2672: Attempting to start 'goldengate-vip' on 'rhel11gr2rac1'
CRS-2676: Start of 'goldengate-vip' on 'rhel11gr2rac1' succeeded

f) Check the status of the vip.

[oracle@rhel11gr2rac1 bin]$ ./crsctl stat res goldengate-vip
NAME=goldengate-vip
TYPE=app.appvip_net1.type
TARGET=ONLINE
STATE=ONLINE on rhel11gr2rac1


e) Now we can able to ping the vip from the other nodes. Test it in node 2.

[root@rhel11gr2rac2 ~]# ping 192.168.0.22
PING 192.168.0.22 (192.168.0.22) 56(84) bytes of data.
64 bytes from 192.168.0.22: icmp_seq=1 ttl=64 time=2.29 ms
64 bytes from 192.168.0.22: icmp_seq=2 ttl=64 time=0.389 ms
64 bytes from 192.168.0.22: icmp_seq=3 ttl=64 time=0.372 ms

--- 192.168.0.22 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.372/1.018/2.295/0.903 ms



4) Now develop and agent script.

a) Oracle Clusterware runs resource-specific commands through an entity called an agent.
The agent script must be able to accept 5 parameter values: start, stop, check, clean and abort (optional).

b) Now we will create an script to  and will also place the script in the shared location, here we have placed the script under the gg home which will be accessed on both the nodes. (This is the sample script provided by oracle we can also have a customized script as per our requirement).

Script name = gg_monitor_start.sh


#!/bin/sh
#goldengate_action.scr
. ~oracle/.bash_profile
[ -z "$1" ]&& echo "ERROR!! Usage $0 "&& exit 99
GGS_HOME=/golden_gate
#specify delay after start before checking for successful start
start_delay_secs=5
#Include the Oracle GoldenGate home in the library path to start GGSCI
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${GGS_HOME}
#set the oracle home to the database to ensure Oracle GoldenGate will get
#the right environment settings to be able to connect to the database
export ORACLE_HOME=/u01/app/oracle/product/11.2/db
export CRS_HOME=/grid/11.2
#Set NLS_LANG otherwise it will default to US7ASCII

export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
logfile=/tmp/crs_gg_start.log

###########################
function log
###########################
{
DATETIME=`date +%d/%m/%y-%H:%M:%S`
echo $DATETIME "goldengate_action.scr>>" $1
echo $DATETIME "goldengate_action.scr>>" $1 >> $logfile
}
#check_process validates that a manager process is running at the PID
#that Oracle GoldenGate specifies.
check_process () {
dt=`date +%d/%m/%y-%H:%M:%S`
if ( [ -f "${GGS_HOME}/dirpcs/MGR.pcm" ] )
then
pid=`cut -f8 "${GGS_HOME}/dirpcs/MGR.pcm"`
if [ ${pid} = `ps -e |grep ${pid} |grep mgr |awk '{ print $1 }'` ]
then
#manager process is running on the PID . exit success
echo $dt "manager process is running on the PID . exit success">> /tmp/check.out
exit 0
else
#manager process is not running on the PID
echo $dt "manager process is not running on the PID" >> /tmp/check.out
exit 1
fi
else
#manager is not running because there is no PID file
echo $ dt"manager is not running because there is no PID file" >> /tmp/check.out
exit 1
fi
}
#call_ggsci is a generic routine that executes a ggsci command
call_ggsci () {
log "entering call_ggsci"
ggsci_command=$1
#log "about to execute $ggsci_command"
log "id= $USER"
cd ${GGS_HOME}
ggsci_output=`${GGS_HOME}/ggsci << EOF
${ggsci_command}
exit
EOF`
log "got output of : $ggsci_output"
}

case $1 in
'start')
#Updated by Sourav B (02/10/2011)
# During failover if the “mgr.pcm” file is not deleted at the node crash
# then Oracle clusterware won’t start the manager on the new node assuming the
# manager process is still running on the failed node. To get around this issue
# we will delete the “mgr.prm” file before starting up the manager on the new
# node. We will also delete the other process files with pc* extension and to
# avoid any file locking issue we will first backup the checkpoint files and then
# delete them from the dirchk directory.After that we will restore the checkpoint
# files from backup to the original location (dirchk directory).
log "removing *.pc* files from dirpcs directory..."
cd $GGS_HOME/dirpcs
rm -f *.pc*
log "creating tmp directory to backup checkpoint file...."
cd $GGS_HOME/dirchk
mkdir tmp
log "backing up checkpoint files..."
cp *.cp* $GGS_HOME/dirchk/tmp
log "Deleting checkpoint files under dirchk......"
rm -f *.cp*
log "Restore checkpoint files from backup to dirchk directory...."
cp $GGS_HOME/dirchk/tmp/*.cp* $GGS_HOME/dirchk
log "Deleting tmp directory...."
rm -rf tmp
log "starting manager"
call_ggsci 'start manager'


#there is a small delay between issuing the start manager command
#and the process being spawned on the OS . wait before checking
log "sleeping for start_delay_secs"
sleep ${start_delay_secs}
#check whether manager is running and exit accordingly
check_process
;;
'stop')
#attempt a clean stop for all non-manager processes
call_ggsci 'stop er *'
#ensure everything is stopped
call_ggsci 'stop er *!'
#stop manager without (y/n) confirmation
call_ggsci 'stop manager!'
#exit success
exit 0
;;
'check')
check_process
exit 0
;;
'clean')
#attempt a clean stop for all non-manager processes
call_ggsci 'stop er *'
#ensure everything is stopped
call_ggsci 'stop er *!'
#in case there are lingering processes
call_ggsci 'kill er *'
#stop manager without (y/n) confirmation
call_ggsci 'stop manager!'
#exit success
exit 0
;;
'abort')
#ensure everything is stopped
call_ggsci 'stop er *!'
#in case there are lingering processes
call_ggsci 'kill er *'
#stop manager without (y/n) confirmation
call_ggsci 'stop manager!'
#exit success
exit 0
;;
esac

c) Now we need to add a clusterware resource for the ggate application. As oracle user execute the
below command.


[oracle@rhel11gr2rac1 bin]$ ./crsctl add resource ggateapp -type cluster_resource -attr "ACTION_SCRIPT=/golden_gate/gg_monitor_start.sh,CHECK_INTERVAL=30,START_DEPENDENCIES='hard(goldengate-vip) pullup(goldengate-vip)', STOP_DEPENDENCIES='hard(goldengate-vip)'"


where ggateapp - is the application name we have given for golden gate resource.

START_DEPENDENCIES: there is a hard start dependency on goldengate-vip. This indicates that the VIP and the ggateapp application should  always start together.

STOP_DEPENDENCIES: there is a hard stop dependency on goldengate-vip. This indicates that the VIP and the ggateapp application should always stop together.

d) Now set the ownership of the oracle golden gate application if it is different from the oracle clusterware owner eg(ggowner) If oracle goldengate owner is same then ignore the below.

As root execute the below command.

./crsctl setperm resource ggateapp -o ggowner


e) Now start the resource using oracle user.

[oracle@rhel11gr2rac1 bin]$ ./crsctl start res ggateapp
CRS-2672: Attempting to start 'ggateapp' on 'rhel11gr2rac1'
CRS-2676: Start of 'ggateapp' on 'rhel11gr2rac1' succeeded

[oracle@rhel11gr2rac1 bin]$ ./crsctl status res ggateapp
NAME=ggateapp
TYPE=cluster_resource
TARGET=ONLINE
STATE=ONLINE on rhel11gr2rac1


[oracle@rhel11gr2rac1 bin]$ ./crsctl stop res ggateapp
CRS-2673: Attempting to stop 'ggateapp' on 'rhel11gr2rac1'
CRS-2677: Stop of 'ggateapp' on 'rhel11gr2rac1' succeeded


-- Now lets check the status in ggsci.


[oracle@rhel11gr2rac1 golden_gate]$ ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Apr 23 2012 08:32:14

Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved.



GGSCI (rhel11gr2rac1.manzoor.com) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     STOPPED
EXTRACT     STOPPED     PTBLS       00:00:00      00:00:29
EXTRACT     STOPPED     XTBLS       00:00:02      00:00:27



--- Its showing stopped..


[oracle@rhel11gr2rac1 bin]$ ./crsctl start res ggateapp
CRS-2672: Attempting to start 'ggateapp' on 'rhel11gr2rac1'
CRS-2676: Start of 'ggateapp' on 'rhel11gr2rac1' succeeded

[oracle@rhel11gr2rac1 golden_gate]$ ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Apr 23 2012 08:32:14

Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved.



GGSCI (rhel11gr2rac1.manzoor.com) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     PTBLS       00:00:00      00:00:10
EXTRACT     RUNNING     XTBLS       00:00:01      00:00:09


--- Now lets relocate the ggapp to the another node (-- scheduled downtime)


[oracle@rhel11gr2rac1 bin]$ ./crsctl relocate resource ggateapp -f
CRS-2673: Attempting to stop 'ggateapp' on 'rhel11gr2rac1'
CRS-2677: Stop of 'ggateapp' on 'rhel11gr2rac1' succeeded
CRS-2673: Attempting to stop 'goldengate-vip' on 'rhel11gr2rac1'
CRS-2677: Stop of 'goldengate-vip' on 'rhel11gr2rac1' succeeded
CRS-2672: Attempting to start 'goldengate-vip' on 'rhel11gr2rac2'
CRS-2676: Start of 'goldengate-vip' on 'rhel11gr2rac2' succeeded
CRS-2672: Attempting to start 'ggateapp' on 'rhel11gr2rac2'
CRS-2676: Start of 'ggateapp' on 'rhel11gr2rac2' succeeded

-- Lets check the gg process on node 2.

[oracle@rhel11gr2rac2 golden_gate]$ ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Apr 23 2012 08:32:14

Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved.



GGSCI (rhel11gr2rac2.manzoor.com) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     PTBLS       00:00:00      00:00:04
EXTRACT     RUNNING     XTBLS       00:00:05      00:00:06



[oracle@rhel11gr2rac1 bin]$ ./crsctl status resource ggateapp
NAME=ggateapp
TYPE=cluster_resource
TARGET=ONLINE
STATE=ONLINE on rhel11gr2rac2


-- Now lets check how the failover is working.

Lets crash the node 2...(power of from vmware home)

Below are the staus after the node 2 is down....

Cluster Resources
--------------------------------------------------------------------------------
ggateapp
      1        ONLINE  OFFLINE
goldengate-vip
      1        ONLINE  OFFLINE                               STARTING



Cluster Resources
--------------------------------------------------------------------------------
ggateapp
      1        ONLINE  ONLINE       rhel11gr2rac1
goldengate-vip
      1        ONLINE  ONLINE       rhel11gr2rac1



-- Could see the ggateapp resource and goldengate-vip has been failed over from
node 2 to node 1...


Below is the output from ggsci.


GGSCI (rhel11gr2rac1.manzoor.com) 3> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     ABENDED     PTBLS       00:00:00      00:04:10
EXTRACT     RUNNING     XTBLS       00:00:00      00:00:09



-- The manager and the extract process are started but the pump extract has been abended with the below
error.

2013-10-02 22:20:56  ERROR   OGG-01031  There is a problem in network communication, a remote file problem, encryption keys for target and source do not matc
h (if using ENCRYPT) or an unknown error. (Reply received is Unable to open file "./dirdat/XT000016" (error 11, Resource temporarily unavailable)).

2013-10-02 22:20:56  ERROR   OGG-01668  PROCESS ABENDING.



Source --


GGSCI (rhel11gr2rac1.manzoor.com) 17> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     ABENDED     PTBLS       00:00:00      00:08:18
EXTRACT     RUNNING     XTBLS       00:00:00      00:00:00


GGSCI (rhel11gr2rac1.manzoor.com) 18> info PTBLS

EXTRACT    PTBLS     Last Started 2013-10-02 22:24   Status ABENDED
Checkpoint Lag       00:00:00 (updated 00:08:24 ago)
Log Read Checkpoint  File ./dirdat/XT000019
                     2013-10-02 22:16:24.415084  RBA 1111



Target --


GGSCI (standalone2.manzoor.com) 2> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
REPLICAT    RUNNING     RTBLS       00:00:00      00:00:05


GGSCI (standalone2.manzoor.com) 3> info rtbls

REPLICAT   RTBLS     Last Started 2013-10-02 20:38   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:07 ago)
Log Read Checkpoint  File ./dirdat/XT000016
                     2013-10-02 22:16:24.422522  RBA 1991


GGSCI (standalone2.manzoor.com) 4> send rtbls status

Sending STATUS request to REPLICAT RTBLS ...
  Current status: At EOF
  Sequence #: 16
  RBA: 1991
  0 records in current transaction


--- The replicat process shows it is completed all the data and currently it is at end of file.



-- Now we will do an et (extact trail) rollover on the source and


Source--

GGSCI (rhel11gr2rac1.manzoor.com) 21> alter extract ptbls etrollover

2013-10-02 22:34:04  INFO    OGG-01520  Rollover performed.  For each affected output trail of Version 10 or higher format, after
starting the source extract, issue ALTER EXTSEQNO for that trail's reader (either pump EXTRACT or REPLICAT) to move the reader's scan to the new trail file;  it will not happen automatically.
EXTRACT altered.


GGSCI (rhel11gr2rac1.manzoor.com) 23> start extract ptbls

Sending START request to MANAGER ...
EXTRACT PTBLS starting


GGSCI (rhel11gr2rac1.manzoor.com) 24> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     PTBLS       00:00:00      00:00:58
EXTRACT     RUNNING     XTBLS       00:00:00      00:00:08

GGSCI (rhel11gr2rac1.manzoor.com) 31> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     PTBLS       00:00:00      00:00:05
EXTRACT     RUNNING     XTBLS       00:00:00      00:00:00

GGSCI (rhel11gr2rac1.manzoor.com) 32> info ptbls

EXTRACT    PTBLS     Last Started 2013-10-02 22:35   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:02 ago)
Log Read Checkpoint  File ./dirdat/XT000020
                     2013-10-02 22:20:46.216461  RBA 1111



--- Target

in target we need to move the replicat to start again from the next sequnce since we have
did an ET rollover on the source.

GGSCI (standalone2.manzoor.com) 3> stop replicat rtbls

Sending STOP request to REPLICAT RTBLS ...
Request processed.


GGSCI (standalone2.manzoor.com) 4> alter replicat rtbls extseqno 17 extrba 0
REPLICAT altered.



GGSCI (standalone2.manzoor.com) 5> start replicat rtbls


-- Now lets update some data on source.

SQL> select count(*) from emp;

  COUNT(*)
----------
      4000

SQL> begin
  2     for i in 4001..5000 loop
  3             insert into emp values (i, dbms_random.string('U',30),30);
  4     END LOOP;
  5     commit;
  6  end;
  7  /

PL/SQL procedure successfully completed.

SQL> select count(*) from emp;

  COUNT(*)
----------
      5000


-- Lets check whether it replicated to target.




SQL> select count(*) from emp;

  COUNT(*)
----------
      5000




---- Reference -- Oracle White Paper—Oracle GoldenGate high availability with Oracle Clusterware

Note -  1527310.1, 

Sunday, 14 July 2013

GG using ACFS

Oracle Golden Gate setup in 11gr2 Using ACFS.
=============================================


NOTE:1371067.1 - How To Install/Reinstall Or Deinstall ACFS Modules/Installation Manually?
How To Create A New ACFS Volume & Filesystem And Set The ACFS Filesystem Ownership To A Non-Grid/Oracle OS User? [ID 1322383.1] To BottomTo Bottom
How To Convert A General Purpose ACFS Filesystem To CRS Managed On Release 11.2.0.X? [ID 1517113.1]




1) Create and mount the ACFS file system on both the nodes.


Identify the raw disk which has to be used for creating the ASM disk, we are using oracleasm libraries for
managing the asm devices.

[root@rhel11gr2rac1 ~]# /etc/init.d/oracleasm querydisk /dev/s* | grep "is not marked"
Device "/dev/sda" is not marked as an ASM disk
Device "/dev/sda1" is not marked as an ASM disk
Device "/dev/sda2" is not marked as an ASM disk
Device "/dev/sda3" is not marked as an ASM disk
Device "/dev/sdb" is not marked as an ASM disk
Device "/dev/sdb1" is not marked as an ASM disk
Device "/dev/sdc" is not marked as an ASM disk
Device "/dev/sdd" is not marked as an ASM disk
Device "/dev/sde" is not marked as an ASM disk
Device "/dev/sdf" is not marked as an ASM disk
Device "/dev/sdg" is not marked as an ASM disk
Device "/dev/sdh" is not marked as an ASM disk
Device "/dev/sdi" is not marked as an ASM disk
Device "/dev/sdj" is not marked as an ASM disk
Device "/dev/sdk" is not marked as an ASM disk
Device "/dev/sdk1" is not marked as an ASM disk
Device "/dev/sdl" is not marked as an ASM disk




From the above we need to use the disk /dev/sdk1 for the ACFS file system. Create the asm disk in
node 1.


[root@rhel11gr2rac1 ~]# /etc/init.d/oracleasm createdisk ACFSDISK /dev/sdk1
Marking disk "ACFSDISK" as an ASM disk:                    [  OK  ]

Scan the disk in the node 2, and check whether the created disk is listing there.

[root@rhel11gr2rac2 ~]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks:               [  OK  ]

[root@rhel11gr2rac2 ~]# /etc/init.d/oracleasm listdisks | grep "ACF*"
ACFSDISK


Create the Diskgroup using this disk.


SQL> select path, header_status , inst_id from GV$asm_disk where header_status = 'PROVISIONED';

PATH                                     HEADER_STATU    INST_ID
---------------------------------------- ------------ ----------
ORCL:ACFSDISK                            PROVISIONED           2
ORCL:ACFSDISK                            PROVISIONED           1


SQL> CREATE DISKGROUP ACFDG EXTERNAL REDUNDANCY
     DISK 'ORCL:ACFSDISK'
     ATTRIBUTE 'compatible.asm' = '11.2', 'compatible.advm' = '11.2';

Diskgroup created.


Create the ACFS Volume.

SQL> select name, total_mb, usable_file_mb from V$asm_diskgroup WHERE NAME = 'ACFDG'

NAME                             TOTAL_MB USABLE_FILE_MB
------------------------------ ---------- --------------
ACFDG                                9759           9707

SQL> ALTER DISKGROUP ACFDG ADD VOLUME ACFSVOLUME size 9650M;

Diskgroup altered.

SQL> select name, total_mb, usable_file_mb from V$asm_diskgroup WHERE name = 'ACFDG';

NAME                             TOTAL_MB USABLE_FILE_MB
------------------------------ ---------- --------------
ACFDG                                9759             40

Using asmcmd verify the volume that we have created.


$asmcmd

ASMCMD> volinfo -a
Diskgroup Name: ACFDG

         Volume Name: ACFSVOLUME
         Volume Device: /dev/asm/acfsvolume-184
         State: ENABLED
         Size (MB): 9664
         Resize Unit (MB): 32
         Redundancy: UNPROT
         Stripe Columns: 4
         Stripe Width (K): 128
         Usage:
         Mountpath:

Now create a mount pain to mount this acfs volume on both the nodes.

# mkdir /golden_gate



Create the ACFS filesystem on the acfsvolume.


# [root@rhel11gr2rac1 ~]# mkfs -t acfs -b 4k /dev/asm/acfsvolume-184
mkfs.acfs: version                   = 11.2.0.3.0
mkfs.acfs: on-disk version           = 39.0
mkfs.acfs: volume                    = /dev/asm/acfsvolume-184
mkfs.acfs: volume size               = 10133438464
mkfs.acfs: Format complete.

Register the acfs filesystem on the new acfs volume.

[root@rhel11gr2rac1 ~]# acfsutil registry -f -a /dev/asm/acfsvolume-184 /golden_gate
acfsutil registry: mount point /golden_gate successfully added to Oracle Registry


Now mont the file system.

[root@rhel11gr2rac1 ~]# mount.acfs -o all


or

mount -t acfs /dev/asm/acfsvolume-184 /golden_gate


[root@rhel11gr2rac1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              21G   18G  2.0G  90% /
/dev/sda1             289M   17M  258M   6% /boot
tmpfs                 816M  563M  253M  69% /dev/shm
/dev/sdb1              20G  173M   19G   1% /emcontrol
/dev/asm/acfsvolume-184
                      9.5G   56M  9.4G   1% /golden_gate


[root@rhel11gr2rac1 ~]# ls -ld /golden_gate/
drwxrwx--- 4 root dba 4096 Jul 13 17:14 /golden_gate/

[root@rhel11gr2rac1 ~]# chown oracle:oinstall /golden_gate/

[root@rhel11gr2rac1 ~]# ls -ld /golden_gate/
drwxrwx--- 4 oracle oinstall 4096 Jul 13 17:14 /golden_gate/




Now mount the diskgroup on the node2.

ASMCMD> mount acfdg
ORA-15032: not all alterations performed
ORA-15202: cannot create additional ASM internal change segment
ORA-15041: diskgroup "ACFDG" space exhausted (DBD ERROR: OCIStmtExecute)

Unable to mount the diskgroup on the node 2, shows space exhausted, as per oracle support
we need to remove some files in the diskgroup, but here we have used to mount acfs so we
will try the reduce the size to 9g in node 1.



[root@rhel11gr2rac1 ~]# acfsutil size -500m /golden_gate/
acfsutil size: ACFS-03162: Warning: The file system was resized, but an error occurred while resizing the ADVM volume.
File system size now 9184 MB, volume size 9664 MB.


Not able to resize the volume online, so drop and recreate the volumegroup.

SQL> alter diskgroup acfdg drop volume acfsvolume
     /

Diskgroup altered.


SQL> ALTER DISKGROUP ACFDG ADD VOLUME ACFSVOLUME size 9100m;

Diskgroup altered.

SQL> select name, total_mb, usable_file_mb from V$asm_diskgroup WHERE NAME = 'ACFDG';

NAME                             TOTAL_MB USABLE_FILE_MB
------------------------------ ---------- --------------
ACFDG                                9759            581


[oracle@rhel11gr2rac1 ~]$ asmcmd
ASMCMD> volinfo -a
Diskgroup Name: ACFDG

         Volume Name: ACFSVOLUME
         Volume Device: /dev/asm/acfsvolume-184
         State: ENABLED
         Size (MB): 9120
         Resize Unit (MB): 32
         Redundancy: UNPROT
         Stripe Columns: 4
         Stripe Width (K): 128
         Usage:
         Mountpath:

ASMCMD> exit

[oracle@rhel11gr2rac1 ~]$ su - root
Password:

[root@rhel11gr2rac1 ~]# mkfs -t acfs -b 4k /dev/asm/acfsvolume-184
mkfs.acfs: version                   = 11.2.0.3.0
mkfs.acfs: on-disk version           = 39.0
mkfs.acfs: volume                    = /dev/asm/acfsvolume-184
mkfs.acfs: volume size               = 9563013120
mkfs.acfs: Format complete.

[root@rhel11gr2rac1 ~]# acfsutil registry -f -a /dev/asm/acfsvolume-184 /golden_gate
acfsutil registry: ACFS-03161: Device : /dev/asm/acfsvolume-184 : Mount Point : /golden_gate already exists in the Oracle Registry

[root@rhel11gr2rac1 ~]# mount.acfs -o all

[root@rhel11gr2rac1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              21G   18G  2.4G  88% /
/dev/sda1             289M   17M  258M   6% /boot
tmpfs                 816M  563M  253M  69% /dev/shm
/dev/sdb1              20G  173M   19G   1% /emcontrol
/dev/asm/acfsvolume-184
                      9.0G   83M  8.9G   1% /golden_gate


check in node 2.



ASMCMD> volinfo -a
Diskgroup Name: ACFDG

         Volume Name: ACFSVOLUME
         Volume Device: /dev/asm/acfsvolume-184
         State: ENABLED
         Size (MB): 9120
         Resize Unit (MB): 32
         Redundancy: UNPROT
         Stripe Columns: 4
         Stripe Width (K): 128
         Usage: ACFS
         Mountpath: /golden_gate

ASMCMD> exit
[oracle@rhel11gr2rac2 ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              21G   15G  5.3G  74% /
/dev/sda1             289M   17M  258M   6% /boot
tmpfs                 816M  563M  253M  69% /dev/shm
/dev/asm/acfsvolume-184
                      9.0G   83M  8.9G   1% /golden_gate


Now it is mounted.

Register this mountpoint in CRS.

Get the information of acfs volumes.


[root@rhel11gr2rac1 ~]# acfsutil registry
Mount Object:
  Device: /dev/asm/acfsvolume-184
  Mount Point: /golden_gate
  Disk Group: ACFDG
  Volume: ACFSVOLUME
  Options: none
  Nodes: all

[root@rhel11gr2rac1 bin]# ./srvctl add filesystem -d /dev/asm/acfsvolume-184 -g ACFDG -v ACFSVOLUME -m /golden_gate -u oracle
PRCA-1023 : Unable to add ACFS file system resource ora.acfdg.acfsvolume.acfs for disk group acfdg and volume acfsvolume
PRCA-1046 : File system for volume device /dev/asm/acfsvolume-184 exists in the ACFS registry


To convert an ACFS mount registry based file system, first delete it from the registry.  You don't need to unmount or stop it.
You must be root or asmadmin to perform this operation.

So first delete it from the registry.

This operation will not remove or otherwise disturb the files that might already be stored on that filesystem:

[root@rhel11gr2rac1 bin]# acfsutil registry -d /dev/asm/acfsvolume-184
acfsutil registry: successfully removed ACFS volume /dev/asm/acfsvolume-184 from Oracle Registry


[root@rhel11gr2rac1 bin]# ./srvctl add filesystem -d /dev/asm/acfsvolume-184 -g ACFDG -v ACFSVOLUME -m /golden_gate -u oracle

Now start the filesystem.

[oracle@rhel11gr2rac1 ~]$ srvctl start filesystem -d /dev/asm/acfsvolume-184 -n rhel11gr2rac1
[oracle@rhel11gr2rac1 ~]$ srvctl start filesystem -d /dev/asm/acfsvolume-184 -n rhel11gr2rac2

[oracle@rhel11gr2rac1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.acfdg.acfsvolume.acfs
               ONLINE  ONLINE       rhel11gr2rac1            mounted on /golden_
                                                             gate
               ONLINE  ONLINE       rhel11gr2rac2            mounted on /golden_
                                                             gate



Install the ogg on /golden_gate mount point.


Unzip and untar the software.

[oracle@rhel11gr2rac1 golden_gate]$ ./ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Apr 23 2012 08:32:14

Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved.



GGSCI (rhel11gr2rac1.manzoor.com) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     STOPPED


GGSCI (rhel11gr2rac1.manzoor.com) 2> create subdirs

Creating subdirectories under current directory /golden_gate

Parameter files                /golden_gate/dirprm: already exists
Report files                   /golden_gate/dirrpt: created
Checkpoint files               /golden_gate/dirchk: created
Process status files           /golden_gate/dirpcs: created
SQL script files               /golden_gate/dirsql: created
Database definitions files     /golden_gate/dirdef: created
Extract data files             /golden_gate/dirdat: created
Temporary files                /golden_gate/dirtmp: created
Stdout files                   /golden_gate/dirout: created

Connect to db

SQL> alter database add supplemental log data (all) columns;

Database altered.


SQL> create tablespace golden_gate datafile '+DATA/layatest/datafile/golden_gate01.dbf' size 500m;

Tablespace created.

SQL> create user ggate identified by Gate_123 default tablespace golden_gate;

User created.

Grant the below privs:-

exec dbms_goldengate_auth.grant_admin_privilege('ggate');
GRANT CONNECT,RESOURCE TO ggate;
GRANT SELECT ANY DICTIONARY, SELECT ANY TABLE TO ggate;
GRANT FLASHBACK ANY TABLE TO ggate;
GRANT CREATE TABLE TO ggate;
grant dba to ggate;
grant insert any table, update any table, delete any table to ggate;
GRANT EXECUTE ON DBMS_FLASHBACK TO ggate;
GRANT EXECUTE ON UTL_FILE TO ggate;

Execute the below sql scripts.

SQL> @/golden_gate/marker_setup.sql
SQL> @/golden_gate/ddl_setup.sql
SQL> @/golden_gate/ddl_enable.sql
SQL> @/golden_gate/role_setup.sql
SQL> @/golden_gate/ddl_pin.sql


Connect to ggsci.

> edit params ./GLOBALS

GGSCHEMA GGATE
CHECKPOINTTABLE GGATE.CHKPTBL



=== MGR process parameters============================

--------------------------------------
----------MGR PROCESS PARAMETERS -----
--------------------------------------
USERID GGATE@NCMSSIT12, PASSWORD Gate_123
PORT 7810
DYNAMICPORTLIST 7900-7950
LAGCRITICALMINUTES 5

=======================================================


> start mgr

GGSCI (rhel11gr2rac1.manzoor.com) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING


GGSCI (rhel11gr2rac1.manzoor.com) 2>

GGSCI (rhel11gr2rac1.manzoor.com) 2> dblogin userid ggate password Gate_123
Successfully logged into database.

GGSCI (rhel11gr2rac1.manzoor.com) 3> add checkpointtable ggate.chkptbl

Successfully created checkpoint table ggate.chkptbl.




Test user
=========


SQL> create tablespace manzoor datafile '+DATA/layatest/datafile/manzoor01.dbf' size 300m;

Tablespace created.

SQL> create user manzoor identified by manzoor default tablespace manzoor;

User created.

SQL> grant connect, resource to manzoor;

Grant succeeded.

SQL> grant dba to manzoor;

Grant succeeded.

SQL> conn manzoor/manzoor
Connected.

SQL> create table dept (dept_id number, department_name varchar2(30));

Table created.

SQL> alter table dept add constraint pk_dept_id primary key (dept_id);

Table altered.

SQL> create table emp (emp_id number primary key, emp_name varchar2(30), dept_id number);

Table created.

SQL> alter table emp add constraint fk_dept_id foreign key (dept_id) references dept (dept_id);

Table altered.



GGSCI (rhel11gr2rac1.manzoor.com) 3> dblogin userid ggate, password Gate_123
Successfully logged into database.

GGSCI (rhel11gr2rac1.manzoor.com) 6> add trandata manzoor.dept

Logging of supplemental redo data enabled for table MANZOOR.DEPT.

GGSCI (rhel11gr2rac1.manzoor.com) 7> add trandata manzoor.emp

Logging of supplemental redo data enabled for table MANZOOR.EMP.


GGSCI (rhel11gr2rac1.manzoor.com) 8> edit params xtbls

EXTRACT XTBLS
setenv (NLS_LANG="AMERICAN_AMERICA.AL32UTF8")
userid ggate@layatest, password Gate_123
tranlogoptions dblogreader
discardfile ./dirdat/xtbls.dsc, append
exttrail ./dirdat/XT
ddl include mapped
table manzoor.dept;
table manzoor.emp;


GGSCI (rhel11gr2rac1.manzoor.com) 10> ADD EXTRACT XTBLS, TRANLOG, BEGIN NOW, THREADS 2
EXTRACT added.

GGSCI (rhel11gr2rac1.manzoor.com) 16> add exttrail ./dirdat/XT, extract xtbls, megabytes 200
EXTTRAIL added.

GGSCI (rhel11gr2rac1.manzoor.com) 25> REGISTER EXTRACT XTBLS LOGRETENTION

2013-07-13 19:35:05  INFO    OGG-01749  Successfully registered EXTRACT XTBLS to start managing log retention at SCN 1868707.

GGSCI (rhel11gr2rac1.manzoor.com) 2> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     XTBLS       00:00:02      00:00:07


If you check the status in node 2 then it will look like as below.

GGSCI (rhel11gr2rac2.manzoor.com) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     STOPPED
EXTRACT     ABENDED     XTBLS       00:00:06      00:00:07




sql

SQL> begin
  2     for i in 1..10 loop
  3             insert into manzoor.dept values (dbms_random.value(1,10),dbms_random.string('U',30));
  4     end loop;
  5  commit;
  6  end;
  7  /

PL/SQL procedure successfully completed.

Lets check whether it is captured in the extract.


[oracle@rhel11gr2rac2 dirdat]$ pwd
/golden_gate/dirdat
[oracle@rhel11gr2rac2 dirdat]$ ls -lrt
total 8
-rw-rw-rw- 1 oracle oinstall  256 Jul 13 20:04 xtbls.dsc
-rw-rw-rw- 1 oracle oinstall 2842 Jul 13 20:09 XT000000

------- Yet to complete---------------------------------------------------


http://www.oracle11ggotchas.com/articles/TheImportanceofBoundedRecovery.htm