Just playing around with Data Guard & Oracle Restart in the current release 11gR2 – it’s really cool! Look at Joel Goodmans introduction into Oracle Restart, if you never heared of it before. At first, I installed Oracle Database 11gR2 Grid Infrastructure for Standalone Server to get Oracle Restart, then Oracle Database 11gR2 Software only. I did not use ASM in this setup – everything is running inside a VM on my notebook with only one single hard drive really, so ASM is of not much help here 
I continued to create a Primary Database & a Physical Standby Database pretty much in the same way (on the command line) as described on the Whitepaper 11g Data Guard on the command line on my Downloads page – of course, the directories must be remapped to the 11gR2 ORACLE_HOME etc., but almost identically else. I also created the Data Guard Broker Configuration in the same way as in the Whitepaper described. Now Oracle Restart kicks in:
First registering the listener:
$ srvctl add listener -o /u01/app/oracle/product/11.2.0/dbhome_1
$ srvctl add database -d prima -o /u01/app/oracle/product/11.2.0/dbhome_1 -r PRIMARY
Above added the Primary to the Oracle Local Registry (OLR)
$ srvctl add database -d physt -o /u01/app/oracle/product/11.2.0/dbhome_1 -r PHYSICAL_STANDBY -s MOUNT
Above added the Physical Standby to the OLR. -s is the startmode which defaults to OPEN. My Standby shall get mounted automatically. I would choose OPEN if I am inclined to use the 11g New Feature Real-Time Query – supposed I had purchased Active Data Guard.
I will now add an Application Service to my Configuration – with the same purpose demonstrated already in this posting, but without having to use DBMS_SERVICE. Instead, Oracle Restart will take care for the start of the services in my Data Guard environment according to the assigned Database Role. The creation of a Database Trigger for that purpose is no longer necessary. Instead, the usage of Oracle Restart is recommended:
$ srvctl add service -d prima -s prod -l PRIMARY -e SELECT -m BASIC
-l is the Database Role in which this service prod is supposed to get started, -e is the Failover Type and -m the Failover Method. Next I configure Failover Retries (-z) and Failover Delays (-w):
$ srvctl modify service -d prima
-s prod
-z 180
-w 1
The same has to be done for the Standby Database also – the -l PRIMARY parameter takes care that the service will not be offered unless this database becomes Primary:
$ srvctl add service -d physt -s prod -l PRIMARY -e SELECT -m BASIC $ srvctl modify service -d physt -s prod -z 180 -w 1
The nect command is going to start the service actually. We need to do this only for the first time manually - again without needing DBMS_SERVICE for it:
$ srvctl start service -d prima -s prod
Now let’s investigate the properties of our new service:
$ srvctl config service -d prima -s prod
Service name: prod Service is enabled Cardinality: SINGLETON Disconnect: false Service role: PRIMARY Management policy: AUTOMATIC DTP transaction: false AQ HA notifications: false Failover type: SELECT Failover method: BASIC TAF failover retries: 180 TAF failover delay: 1 Connection Load Balancing Goal: LONG Runtime Load Balancing Goal: NONE TAF policy specification: NONE
Thes service is now up and running:
$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 06-SEP-2010 08:15:36
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=uhesse)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date 06-SEP-2010 07:40:29
Uptime 0 days 0 hr. 35 min. 8 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/11.2.0/dbhome_1/
network/admin/listener.ora
Listener Log File /u01/app/oracle/product/11.2.0/dbhome_1/log/diag/
tnslsnr/uhesse/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=uhesse)(PORT=1521)))
Services Summary...
Service "logst_DGMGRL" has 1 instance(s).
Instance "logst", status UNKNOWN, has 1 handler(s) for this service...
Service "physt" has 1 instance(s).
Instance "physt", status READY, has 1 handler(s) for this service...
Service "physt_DGB" has 1 instance(s).
Instance "physt", status READY, has 1 handler(s) for this service...
Service "physt_DGMGRL" has 1 instance(s).
Instance "physt", status UNKNOWN, has 1 handler(s) for this service...
Service "prima" has 1 instance(s).
Instance "prima", status READY, has 1 handler(s) for this service...
Service "prima_DGB" has 1 instance(s).
Instance "prima", status READY, has 1 handler(s) for this service...
Service "prima_DGMGRL" has 1 instance(s).
Instance "prima", status UNKNOWN, has 1 handler(s) for this service...
Service "prod" has 1 instance(s).
Instance "prima", status READY, has 1 handler(s) for this service...
The command completed successfully
I have prepared a connect descriptor prod in the same way as in this posting shown already. Using it to connect as system user. Now I kill the SMON process of my Primary Instance (Do not do that at home). Then a select on v$instance in my user system session waits for the reestablishing of the connection (maximum 180 seconds with my settings above) – after less than 5 seconds, the ohasd (Oracle High Availabilty Services Daemon) recognizes this and restarts the Primary Instance. After less than 20 seconds, I see my select on v$instance come through. This was a Transparent Application Failover without another Instance or Database in the game Really cool, isn’t it? That is even reflected in v$session:
SYSTEM@prod > select username,FAILED_OVER,
FAILOVER_METHOD,FAILOVER_TYPE from v$session;
USERNAME FAI FAILOVER_M FAILOVER_TYPE
------------------------------ --- ---------- -------------
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
PUBLIC NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
NO NONE NONE
SYSTEM YES BASIC SELECT
NO NONE NONE
31 rows selected.
Now let’s do a switchover and see how Oracle Restart will take care to start the prod service on the new Primary and how the connection again is reestablished there:
$ dgmgrl sys/oracle@prima
DGMGRL for Linux: Version 11.2.0.1.0 - Production
Copyright (c) 2000, 2009, Oracle. All rights reserved.
Welcome to DGMGRL, type "help" for information.
Connected.
DGMGRL> show configuration
Configuration - myconf
Protection Mode: MaxAvailability
Databases:
prima - Primary database
physt - Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS
DGMGRL> switchover to physt
Performing switchover NOW, please wait...
New primary database "physt" is opening...
Operation requires shutdown of instance "prima" on database "prima"
Shutting down instance "prima"...
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
Operation requires startup of instance "prima" on database "prima"
Starting instance "prima"...
ORACLE instance started.
Database mounted.
Switchover succeeded, new primary is "physt"
The switchover was done as usual. Now what about the service and the session?
$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 06-SEP-2010 08:45:25
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date 06-SEP-2010 07:40:29
Uptime 0 days 1 hr. 4 min. 57 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/11.2.0/dbhome_1/
network/admin/listener.ora
Listener Log File /u01/app/oracle/product/11.2.0/dbhome_1/log/diag/
tnslsnr/uhesse/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=uhesse)(PORT=1521)))
Services Summary...
Service "logst_DGMGRL" has 1 instance(s).
Instance "logst", status UNKNOWN, has 1 handler(s) for this service...
Service "physt" has 1 instance(s).
Instance "physt", status READY, has 1 handler(s) for this service...
Service "physt_DGB" has 1 instance(s).
Instance "physt", status READY, has 1 handler(s) for this service...
Service "physt_DGMGRL" has 1 instance(s).
Instance "physt", status UNKNOWN, has 1 handler(s) for this service...
Service "prima" has 1 instance(s).
Instance "prima", status READY, has 1 handler(s) for this service...
Service "prima_DGB" has 1 instance(s).
Instance "prima", status READY, has 1 handler(s) for this service...
Service "prima_DGMGRL" has 1 instance(s).
Instance "prima", status UNKNOWN, has 1 handler(s) for this service...
Service "prod" has 1 instance(s).
Instance "physt", status READY, has 1 handler(s) for this service...
The command completed successfully
The service was started on the new Primary by Oracle Restart automatically. Now the session:
SYSTEM@prod > select instance_name from v$instance;
INSTANCE_NAME
----------------
physt
SYSTEM@prod > select username,FAILED_OVER,FAILOVER_METHOD,FAILOVER_TYPE
from v$session where username='SYSTEM';
USERNAME FAI FAILOVER_M FAILOVER_TYPE
------------------------------ --- ---------- -------------
SYSTEM YES BASIC SELECT
The session was again “indestructable” and survived the switchover by failing over to the new Primary. When I reboot my server, ohasd together with DMON now take care that Listener, Primary, Standby and Service all get started in the correct order.
I must admit that I am really enthusiastic about these new features in 11gR2 and how smooth it all fits together especially on the Data Guard part. Kudos to our Developement Teams!
#1 by emre baransel on September 7, 2010 - 08:56
Restart is a useful feature !! Glad to be compatible with dataguard
Thanks Uwe!!
#2 by Uwe Hesse on September 7, 2010 - 13:39
You’re welcome
Thank you for your nice feedback!
#3 by Stéphane Fromholtz on September 16, 2010 - 15:32
Having configured a Data Guard config using Physical standby and your previous article with dbms_service and the famous trigger, I’m happy to see that there is now a clean way to do this in 11g R2.
It really seems to be the best option for a Data Guard configuration with 11g. Thanks for this article, that will probably be helpful for the next Data Guard setup.
#4 by Uwe Hesse on September 20, 2010 - 09:49
Stéphane,
glad that you found the article helpful
I am not sure whether the new method with Oracle Restart is “more clean” than the old method with the Database Event Trigger before, though. It is just feasible to use Oracle Restart (if you have it setup) for that purpose also.
In other words: I would not take the effort to implement Oracle Restart only to get my services in a Data Guard Environment properly managed. That is more an additional benefit, in my view.
The big advantage of Oracle Restart is that it will start each component in the appropriate order and especially, it also will RESTART it (if possible), should it fail. That was the reason to give it that name, I guess
#5 by Stéphane Fromholtz on September 24, 2010 - 08:32
I did read another article about the memory usage of Grid Infrastructure (http://www.pythian.com/news/9179/oracle-11gr2-grid-infrastructure-memory-footprint/). It talks about 500Mb (see comments below for a fix) to run Grid Infra on a node. This is quite large for what it does to my opinion.
As you said, I’ll perhaps keep the trigger and the service for an 11g Data Guard setup.
#6 by TGASCARD on October 13, 2010 - 16:48
Hi Uwe,
You need to be careful with Data Guard and Oracle Restart. I found Bug 9645789: SWITCHOVER BY DATAGUARD BROKER FAILS BY ORA-16535.
I think it is better to use a trigger than srvctl add service.
Thierry
#7 by Kotesh on October 15, 2010 - 15:37
Uwe,
what is the command to add logical standby to olr. Is it srvctl add database -db_name -r logical_standby -s mount
Regards,
Kotesh
#8 by monto on October 19, 2010 - 18:52
Uwe,
I’m a regular visitor of your blog and i have question can i use oracle 11gr2 RAC(primary) and 11gr2(oracle restart) as standby which i assume i should be able to ,and configure an application service how should be by tns entry for this ,is it possible to put multiple address list when using scan name?What is that i need to do to make this service failover to standby database when using jdbc thin driver.
Thanks
#9 by Uwe Hesse on October 25, 2010 - 14:08
Thierry,
thank you for mentioning this bug that I did not notice yet. The Metalink Note only talks about a scenario with a RAC-Primary. Not sure whether this bug is also hitting in Single Instance setups. Will keep an eye on it
#10 by Uwe Hesse on October 25, 2010 - 14:24
Kotesh,
yes
(look at http://download.oracle.com/docs/cd/E11882_01/server.112/e17120/restart004.htm#i1008403 ) , but you wouldn’t give -s mount for a logical standby because it is always open
#11 by Uwe Hesse on October 25, 2010 - 14:37
Monto,
Yes, you can use the SCAN entries in a tnsnames (or in a thin JDBC coding) from RAC Primary (first SCAN) and RAC Standby (second SCAN), very similar as you would do for two Single Instance DBs. We don’t call that Oracle Restart then but Clusterware resp. Grid Infrastructure. Oracle Restart is the Marketing Name for “Clusterware on a Single Instance Architecture” 
thanks for visiting my Blog regularly
#12 by tanos on May 13, 2011 - 15:36
Hello
This arcticle is very helpfull.
I got a problem with the service creation
srvctl add service -d MG -s test-appli -l PRIMARY -e SELECT -m BASIC
I receive this error :
PRKO-2001 : Invalid command line syntax
So i check my resource
crsctl status resource ora.mg.db -p
and i find a server_pool associate is that normal.
The detail of my resource could you compare with yours?
NAME=ora.mg.db
TYPE=ora.database.type
ACL=owner:oracle:rwx,pgrp:oinstall:rwx,other::r–
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=
ACTIVE_PLACEMENT=1
AGENT_FILENAME=%CRS_HOME%/bin/oraagent%CRS_EXE_SUFFIX%
AUTO_START=restore
CARDINALITY=%CRS_SERVER_POOL_SIZE%
CHECK_INTERVAL=1
CHECK_TIMEOUT=600
CLUSTER_DATABASE=true
DB_UNIQUE_NAME=MG
DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=database) PROPERTY(DB_UNIQUE_NAME= CONCAT(PARSE(%NAME%, ., 2), %USR_ORA_DOMAIN%, .)) ELEMENT(INSTANCE_NAME= %GEN_USR_ORA_INST_NAME%)
DEGREE=1
DESCRIPTION=Oracle Database resource
ENABLED=1
FAILOVER_DELAY=0
FAILURE_INTERVAL=60
FAILURE_THRESHOLD=1
GEN_AUDIT_FILE_DEST=/u01/app/oracle/admin/MG/adump
GEN_USR_ORA_INST_NAME=
GEN_USR_ORA_INST_NAME@SERVERNAME(aix_rac1)=MG
HOSTING_MEMBERS=
INSTANCE_FAILOVER=0
LOAD=1
LOGGING_LEVEL=1
MANAGEMENT_POLICY=AUTOMATIC
NLS_LANG=
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
ORACLE_HOME=/u01/app/oracle/11.2.0
PLACEMENT=restricted
PROFILE_CHANGE_TEMPLATE=
RESTART_ATTEMPTS=2
ROLE=PRIMARY
SCRIPT_TIMEOUT=60
SERVER_POOLS=ora.MG
SPFILE=
START_DEPENDENCIES=weak(type:ora.listener.type,global:type:ora.scan_listener.type,uniform:ora.ons,uniform:ora.eons) hard(ora.DATA.dg,ora.FRA.dg)
START_TIMEOUT=600
STATE_CHANGE_TEMPLATE=
STOP_DEPENDENCIES=hard(intermediate:ora.asm,shutdown:ora.DATA.dg,shutdown:ora.FRA.dg)
STOP_TIMEOUT=600
UPTIME_THRESHOLD=1h
USR_ORA_DB_NAME=
USR_ORA_DOMAIN=
USR_ORA_ENV=
USR_ORA_FLAGS=
USR_ORA_INST_NAME=
USR_ORA_OPEN_MODE=open
USR_ORA_OPI=false
USR_ORA_STOP_MODE=immediate
VERSION=11.2.0.1.0
#13 by Uwe Hesse on May 14, 2011 - 20:58
Tanos,
please take into account that the example in the posting is with Oracle Restart on a Single Instance system. That might have a different behavior as your environment which seems to be Grid Infrastructure on a RAC cluster. I can right now not reproduce your environment, because I have no MAA (RAC Primary + RAC Standby) setup available.You may consider to contact Oracle Support, though. They are so good with those troubleshooting issues that they can even make a living from it
#14 by Sigrid Keydana on October 20, 2011 - 15:15
Hi Uwe,
may I still add a comment / ask a question here? When I do a switchover in dgmrl in a DataGuard+Restart environment, the new primary (which, when still a standby, I added with -s mount -r physical_standby) is not opened, but mounted only…
Is this normal behavior, meaning you have to modify the Restart config before doing a switchover, – or shouldn’t Restart have done this modification automatically, exchanging the mount vs. open configs of the databases automatically?
In that case, I wonder what might be wrong in my setup…
Many thanks,
Sigrid
#15 by Uwe Hesse on October 20, 2011 - 16:52
Sigrid,
indeed, after the sitchover the new Primary is supposed to reach the status OPEN without an additonal command from your side. If you look at the posting closer, you see that it is done that way: The last select of the user system would be impossible (because the connection was not as sysdba) in status MOUNT.
#16 by Sigrid Keydana on October 27, 2011 - 10:06
Hi Uwe,
many thanks for the answer! This is strange however, because on our system (which is 11.2.0.2.3 for the restart as well as the database home) it does not work like that – we have to reconfigure before the switchover in order to get the new primary opened… I guess we have to inspect the restart logs and try to find out why :-;
Ciao
Sigrid
#17 by wendy on January 1, 2012 - 17:05
Hi, Uwe:
This is Wendy. I find this post of you regarding Oracle Restart and Dataguard. I already configured my Dataguard and Primary DB with dbms_service and database triggers per your other post. Is it worth to reconfigure it with oracle restart?
#18 by Uwe Hesse on January 1, 2012 - 17:11
Hi Wendy,
in my opinion, it is more a matter of taste whether you use Oracle Restart or the Trigger approach, in order to achieve Client Connectivity. With the Grid Infrastructure already installed, I would use it when creating a Data Guard Configuration.
I would not remove my working trigger configuration to replace it with the Oracle Restart solution just to get the same result, though.
#19 by wendy on January 1, 2012 - 17:40
So is it possible I can configure ORacle Restart while keeping the dbms_services and triggers?
#20 by Uwe Hesse on January 1, 2012 - 23:04
I would not recommend to configure the service with Oracle Restart (as described in the posting you saw) AND also the trigger. It should be okay to configure the Databases with Oracle Restart additionally, though.
So:
srvctl add database
but not
srvctl add service
when you want to keep the trigger & dbms_service configuration.
#21 by Wendy on January 2, 2012 - 04:58
ok, Thanks. Have a happy new year.