Oracle RAC Administration - Part 9: Hands on administration | Database Journal
Oracle
SHARE
Facebook X Pinterest WhatsApp

Oracle RAC Administration – Part 9: Hands on administration

Written By
Tarry Singh
Tarry Singh
Nov 16, 2006
2 minute read

Brief intro


As a continuing part of the hands-on articles, we will take a deeper look at such things as errors in the installation (which are actually not errors), ESX host tuning for time synchronization (without which the whole RHEL RAC installation means nothing) and some SRVCTL commands.


Errors come and errors go


Let’s take a look at a couple of screen shots and some of the typical Cluster Ready Services (CRS) errors and tricks to bring your RAC services and applications online.


This is a typical placement error that I get on every installation. I think it may have to do with the time issue; we will come to that. If you are using an ESX server to test/develop your RAC, then the information to test and fix your time synchronization issues will certainly come very handy.




It really doesn’t mean a thing. I do run into this every time I do my virtual machine restart. 

[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stat -t
Name           Type           Target    State     Host
————————————————————
ora….SM1.asm application    ONLINE    UNKNOWN   vm1rh4
ora….H4.lsnr application    ONLINE    UNKNOWN   vm1rh4
ora.vm1rh4.gsd application    ONLINE    UNKNOWN   vm1rh4
ora.vm1rh4.ons application    ONLINE    UNKNOWN   vm1rh4
ora.vm1rh4.vip application    ONLINE    ONLINE    vm1rh4
ora….SM2.asm application    ONLINE    UNKNOWN   vm2rh4
ora….H4.lsnr application    ONLINE    UNKNOWN   vm2rh4
ora.vm2rh4.gsd application    ONLINE    UNKNOWN   vm2rh4
ora.vm2rh4.ons application    ONLINE    UNKNOWN   vm2rh4
ora.vm2rh4.vip application    ONLINE    ONLINE    vm2rh4
ora….SM3.asm application    ONLINE    OFFLINE
ora….H4.lsnr application    ONLINE    OFFLINE
ora.vm3rh4.gsd application    ONLINE    UNKNOWN   vm3rh4
ora.vm3rh4.ons application    ONLINE    UNKNOWN   vm3rh4
ora.vm3rh4.vip application    ONLINE    UNKNOWN   vm1rh4
ora….SM4.asm application    ONLINE    OFFLINE
ora….H4.lsnr application    ONLINE    OFFLINE
ora.vm4rh4.gsd application    ONLINE    ONLINE    vm4rh4
ora.vm4rh4.ons application    ONLINE    ONLINE    vm4rh4
ora.vm4rh4.vip application    ONLINE    UNKNOWN   vm4rh4

This is really a VM issue. I am using a 1.2 G vMEM, 2vCPUs, and decent SCSI Virtual Disks (VMFS-2 file system on a VMDK format). Restarting the services does not go as expected. Let’s see what happens if you try to do a crs_stat –stop and then a crs_stop –start.

[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stop -all
Attempting to stop `ora.vm4rh4.gsd` on member `vm4rh4`
Attempting to stop `ora.vm4rh4.ons` on member `vm4rh4`
Stop of `ora.vm4rh4.gsd` on member `vm4rh4` succeeded.
Stop of `ora.vm4rh4.ons` on member `vm4rh4` succeeded.
Attempting to stop `ora.vm2rh4.LISTENER_VM2RH4.lsnr` on member `vm2rh4`
Attempting to stop `ora.vm1rh4.LISTENER_VM1RH4.lsnr` on member `vm1rh4`
Stop of `ora.vm2rh4.LISTENER_VM2RH4.lsnr` on member `vm2rh4` succeeded.
Attempting to stop `ora.vm2rh4.ASM2.asm` on member `vm2rh4`
Stop of `ora.vm1rh4.LISTENER_VM1RH4.lsnr` on member `vm1rh4` succeeded.
Attempting to stop `ora.vm1rh4.ASM1.asm` on member `vm1rh4`
Stop of `ora.vm2rh4.ASM2.asm` on member `vm2rh4` succeeded.
Attempting to stop `ora.vm2rh4.vip` on member `vm2rh4`
Stop of `ora.vm2rh4.vip` on member `vm2rh4` succeeded.
Stop of `ora.vm1rh4.ASM1.asm` on member `vm1rh4` succeeded.
Attempting to stop `ora.vm1rh4.vip` on member `vm1rh4`
Stop of `ora.vm1rh4.vip` on member `vm1rh4` succeeded.

As you can see, it just does not restart all of the services when we do the crs_stat –t.

[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stat -t
Name           Type           Target    State     Host
————————————————————
ora….SM1.asm application    OFFLINE   OFFLINE
ora….H4.lsnr application    OFFLINE   OFFLINE
ora.vm1rh4.gsd application    ONLINE    UNKNOWN   vm1rh4
ora.vm1rh4.ons application    ONLINE    UNKNOWN   vm1rh4
ora.vm1rh4.vip application    OFFLINE   OFFLINE
ora….SM2.asm application    OFFLINE   OFFLINE
ora….H4.lsnr application    OFFLINE   OFFLINE
ora.vm2rh4.gsd application    ONLINE    UNKNOWN   vm2rh4
ora.vm2rh4.ons application    ONLINE    UNKNOWN   vm2rh4
ora.vm2rh4.vip application    OFFLINE   OFFLINE
ora….SM3.asm application    ONLINE    OFFLINE
ora….H4.lsnr application    ONLINE    OFFLINE
ora.vm3rh4.gsd application    ONLINE    UNKNOWN   vm3rh4
ora.vm3rh4.ons application    ONLINE    UNKNOWN   vm3rh4
ora.vm3rh4.vip application    ONLINE    UNKNOWN   vm1rh4
ora….SM4.asm application    ONLINE    OFFLINE
ora….H4.lsnr application    ONLINE    OFFLINE
ora.vm4rh4.gsd application    OFFLINE   OFFLINE
ora.vm4rh4.ons application    OFFLINE   OFFLINE
ora.vm4rh4.vip application    ONLINE    UNKNOWN   vm4rh4

And this is bizarre , of course, I will test it on a ESX 3.0 with more capacity and see if it vanishes, but now the task is to start all the services one by one, which is not easy when you have “ora….H4.lsnr” names. So, when you do the following, you get full names of the services.

[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stat

In my case, it gave these and now you have the full names.

NAME=ora.brianic.brianic1.inst
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.brianic.brianic2.inst
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.brianic.brianic3.inst
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.brianic.brianic4.inst
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.brianic.db
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.brianic.fokeserv.brianic1.srv
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm1rh4
NAME=ora.brianic.fokeserv.brianic2.srv
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm2rh4
NAME=ora.brianic.fokeserv.brianic3.srv
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm3rh4
NAME=ora.brianic.fokeserv.brianic4.srv
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm4rh4
NAME=ora.brianic.fokeserv.cs
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm4rh4
NAME=ora.vm1rh4.ASM1.asm
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm1rh4.LISTENER_VM1RH4.lsnr
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm1rh4.gsd
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm1rh4
NAME=ora.vm1rh4.ons
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm1rh4
NAME=ora.vm1rh4.vip
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm2rh4.ASM2.asm
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm2rh4.LISTENER_VM2RH4.lsnr
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm2rh4.gsd
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm2rh4
NAME=ora.vm2rh4.ons
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm2rh4
NAME=ora.vm2rh4.vip
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm3rh4.ASM3.asm
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm3rh4.LISTENER_VM3RH4.lsnr
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm3rh4.gsd
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm3rh4
NAME=ora.vm3rh4.ons
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm3rh4
NAME=ora.vm3rh4.vip
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm4rh4.ASM4.asm
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm4rh4.LISTENER_VM4RH4.lsnr
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.vm4rh4.gsd
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm4rh4
NAME=ora.vm4rh4.ons
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vm4rh4
NAME=ora.vm4rh4.vip
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
Tarry Singh

I have been active in several industries since 1991. While working in the maritime industry I have worked for several Fortune 500 firms such as NYK, A.P. Møller-Mærsk Group. I made a career switch, emigrated, learned a new language and moved into the IT industry starting 2000. Since then I have been a Sr. DBA, (Technical) Project Manager, Sr. Consultant, Infrastructure Specialist (Clustering, Load Balancing, Networks, Databases) and (currently) Virtualization/Cloud Computing Expert and Global Sourcing in the IT industry. My deep understanding of multi-cultural issues (having worked across the globe) and international exposure has not only helped me successfully relaunch my career in a new industry but also helped me stay successful in what I do. I believe in "worknets" and "collective or swarm intelligence". As a trainer (technical as well as non-technical) I have trained staff both on national and international level. I am very devoted, perspicacious and hard working.

Database Journal Logo

DatabaseJournal.com publishes relevant, up-to-date and pragmatic articles on the use of database hardware and management tools and serves as a forum for professional knowledge about proprietary, open source and cloud-based databases--foundational technology for all IT systems. We publish insightful articles about new products, best practices and trends; readers help each other out on various database questions and problems. Database management systems (DBMS) and database security processes are also key areas of focus at DatabaseJournal.com.

facebook
linkedin
x
rss

Company

Contact us Advertise with us

Categories

Database News Features Scripts Mysql MS SQL Oracle MS Access DB2 PostgreSQL SAP PHP

Property of TechnologyAdvice. © 2026 TechnologyAdvice. All Rights Reserved

Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.

Terms of Service Privacy Policy California - Do Not Sell My Information