So I go ahead and stop them all first...
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stop ora.vm1rh4.gsd
Attempting to stop `ora.vm1rh4.gsd` on member `vm1rh4`
Stop of `ora.vm1rh4.gsd` on member `vm1rh4` succeeded.
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stop ora.vm2rh4.gsd
Attempting to stop `ora.vm2rh4.gsd` on member `vm2rh4`
Stop of `ora.vm2rh4.gsd` on member `vm2rh4` succeeded.
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stop ora.vm3rh4.gsd
Attempting to stop `ora.vm3rh4.gsd` on member `vm3rh4`
Stop of `ora.vm3rh4.gsd` on member `vm3rh4` succeeded.
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stop ora.vm3rh4.ons
Attempting to stop `ora.vm3rh4.ons` on member `vm3rh4`
Stop of `ora.vm3rh4.ons` on member `vm3rh4` succeeded.
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stop ora.vm2rh4.ons
Attempting to stop `ora.vm2rh4.ons` on member `vm2rh4`
Stop of `ora.vm2rh4.ons` on member `vm2rh4` succeeded.
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stop ora.vm1rh4.ons
Attempting to stop `ora.vm1rh4.ons` on member `vm1rh4`
Stop of `ora.vm1rh4.ons` on member `vm1rh4` succeeded.
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stop ora.vm3rh4.vip
Attempting to stop `ora.vm3rh4.vip` on member `vm1rh4`
Stop of `ora.vm3rh4.vip` on member `vm1rh4` succeeded.
CRS-1016: Resources depending on 'ora.vm3rh4.vip' are running
Check status
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application OFFLINE OFFLINE
ora....H4.lsnr application OFFLINE OFFLINE
ora.vm1rh4.gsd application OFFLINE OFFLINE
ora.vm1rh4.ons application OFFLINE OFFLINE
ora.vm1rh4.vip application OFFLINE OFFLINE
ora....SM2.asm application OFFLINE OFFLINE
ora....H4.lsnr application OFFLINE OFFLINE
ora.vm2rh4.gsd application OFFLINE OFFLINE
ora.vm2rh4.ons application OFFLINE OFFLINE
ora.vm2rh4.vip application OFFLINE OFFLINE
ora....SM3.asm application OFFLINE OFFLINE
ora....H4.lsnr application OFFLINE OFFLINE
ora.vm3rh4.gsd application OFFLINE OFFLINE
ora.vm3rh4.ons application OFFLINE OFFLINE
ora.vm3rh4.vip application OFFLINE OFFLINE
ora....SM4.asm application OFFLINE OFFLINE
ora....H4.lsnr application OFFLINE OFFLINE
ora.vm4rh4.gsd application OFFLINE OFFLINE
ora.vm4rh4.ons application OFFLINE OFFLINE
ora.vm4rh4.vip application OFFLINE OFFLINE
Then (fortunately) I just need the –all command to start all services.
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_start -all
Attempting to start `ora.vm1rh4.vip` on member `vm1rh4`
Attempting to start `ora.vm2rh4.vip` on member `vm2rh4`
Attempting to start `ora.vm3rh4.vip` on member `vm3rh4`
Attempting to start `ora.vm4rh4.vip` on member `vm4rh4`
Start of `ora.vm2rh4.vip` on member `vm2rh4` succeeded.
Attempting to start `ora.vm2rh4.ASM2.asm` on member `vm2rh4`
Start of `ora.vm4rh4.vip` on member `vm4rh4` succeeded.
Attempting to start `ora.vm4rh4.ASM4.asm` on member `vm4rh4`
Start of `ora.vm3rh4.vip` on member `vm3rh4` succeeded.
Start of `ora.vm1rh4.vip` on member `vm1rh4` succeeded.
Attempting to start `ora.vm1rh4.ASM1.asm` on member `vm1rh4`
Attempting to start `ora.vm3rh4.ASM3.asm` on member `vm3rh4`
Start of `ora.vm2rh4.ASM2.asm` on member `vm2rh4` succeeded.
Attempting to start `ora.vm2rh4.LISTENER_VM2RH4.lsnr` on member `vm2rh4`
Start of `ora.vm2rh4.LISTENER_VM2RH4.lsnr` on member `vm2rh4` succeeded.
Start of `ora.vm1rh4.ASM1.asm` on member `vm1rh4` succeeded.
Attempting to start `ora.vm1rh4.LISTENER_VM1RH4.lsnr` on member `vm1rh4`
Start of `ora.vm1rh4.LISTENER_VM1RH4.lsnr` on member `vm1rh4` succeeded.
Start of `ora.vm3rh4.ASM3.asm` on member `vm3rh4` succeeded.
Attempting to start `ora.vm3rh4.LISTENER_VM3RH4.lsnr` on member `vm3rh4`
Start of `ora.vm4rh4.ASM4.asm` on member `vm4rh4` succeeded.
Start of `ora.vm3rh4.LISTENER_VM3RH4.lsnr` on member `vm3rh4` succeeded.
Attempting to start `ora.vm4rh4.LISTENER_VM4RH4.lsnr` on member `vm4rh4`
Start of `ora.vm4rh4.LISTENER_VM4RH4.lsnr` on member `vm4rh4` succeeded.
CRS-1002: Resource 'ora.vm1rh4.ons' is already running on member 'vm1rh4'
CRS-1002: Resource 'ora.vm2rh4.ons' is already running on member 'vm2rh4'
Attempting to start `ora.vm1rh4.gsd` on member `vm1rh4`
CRS-1002: Resource 'ora.vm3rh4.ons' is already running on member 'vm3rh4'
CRS-1002: Resource 'ora.vm4rh4.ons' is already running on member 'vm4rh4'
These errors don't mean anything; the installation ran faster than the
console command--the services were already started, causing the error messages.
As you see now...
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE vm1rh4
ora....H4.lsnr application ONLINE ONLINE vm1rh4
ora.vm1rh4.gsd application ONLINE ONLINE vm1rh4
ora.vm1rh4.ons application ONLINE ONLINE vm1rh4
ora.vm1rh4.vip application ONLINE ONLINE vm1rh4
ora....SM2.asm application ONLINE ONLINE vm2rh4
ora....H4.lsnr application ONLINE ONLINE vm2rh4
ora.vm2rh4.gsd application ONLINE ONLINE vm2rh4
ora.vm2rh4.ons application ONLINE ONLINE vm2rh4
ora.vm2rh4.vip application ONLINE ONLINE vm2rh4
ora....SM3.asm application ONLINE ONLINE vm3rh4
ora....H4.lsnr application ONLINE ONLINE vm3rh4
ora.vm3rh4.gsd application ONLINE ONLINE vm3rh4
ora.vm3rh4.ons application ONLINE ONLINE vm3rh4
ora.vm3rh4.vip application ONLINE ONLINE vm3rh4
ora....SM4.asm application ONLINE ONLINE vm4rh4
ora....H4.lsnr application ONLINE ONLINE vm4rh4
ora.vm4rh4.gsd application ONLINE ONLINE vm4rh4
ora.vm4rh4.ons application ONLINE ONLINE vm4rh4
ora.vm4rh4.vip application ONLINE ONLINE vm4rh4
[oracle@vm1rh4 ~]$
After having successfully completed the 4-node installation:
We print out all of our CRS services:
[oracle@vm1rh4 ~]$ $ORA_CRS_HOME/bin/crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....c1.inst application ONLINE ONLINE vm1rh4
ora....c2.inst application ONLINE ONLINE vm2rh4
ora....c3.inst application ONLINE ONLINE vm3rh4
ora....c4.inst application ONLINE ONLINE vm4rh4
ora.brianic.db application ONLINE ONLINE vm1rh4
ora....ic1.srv application ONLINE ONLINE vm1rh4
ora....ic2.srv application ONLINE ONLINE vm2rh4
ora....ic3.srv application ONLINE ONLINE vm3rh4
ora....ic4.srv application ONLINE ONLINE vm4rh4
ora....serv.cs application ONLINE ONLINE vm4rh4
ora....SM1.asm application ONLINE ONLINE vm1rh4
ora....H4.lsnr application ONLINE ONLINE vm1rh4
ora.vm1rh4.gsd application ONLINE ONLINE vm1rh4
ora.vm1rh4.ons application ONLINE ONLINE vm1rh4
ora.vm1rh4.vip application ONLINE ONLINE vm1rh4
ora....SM2.asm application ONLINE ONLINE vm2rh4
ora....H4.lsnr application ONLINE ONLINE vm2rh4
ora.vm2rh4.gsd application ONLINE ONLINE vm2rh4
ora.vm2rh4.ons application ONLINE ONLINE vm2rh4
ora.vm2rh4.vip application ONLINE ONLINE vm2rh4
ora....SM3.asm application ONLINE ONLINE vm3rh4
ora....H4.lsnr application ONLINE ONLINE vm3rh4
ora.vm3rh4.gsd application ONLINE ONLINE vm3rh4
ora.vm3rh4.ons application ONLINE ONLINE vm3rh4
ora.vm3rh4.vip application ONLINE ONLINE vm3rh4
ora....SM4.asm application ONLINE ONLINE vm4rh4
ora....H4.lsnr application ONLINE ONLINE vm4rh4
ora.vm4rh4.gsd application ONLINE ONLINE vm4rh4
ora.vm4rh4.ons application ONLINE ONLINE vm4rh4
ora.vm4rh4.vip application ONLINE ONLINE vm4rh4
[oracle@vm1rh4 ~]$
As you can see, this is a very real challenge in the VMware environment. It
is not yet ripe to be deployed in production--not because of the problems we
encountered here, but also issues with OSs like RHEL on time synchronization! Let's
take a look at what I did to resolve these issues.
Fixing the Time Synchronization issue on VMware ESX Server host for
RHEL/Centos 4.2
The first major steps are these:
Editing the following files for ESX 2.x Servers
/etc/ntp.conf
/etc/ntp/step-tickers
[root@esxhost]# esxcfg-firewall --enableService ntpClient
For ESX Server 3.0 only, run the following command. This opens
the appropriate ports and enables the NTP daemon to talk with the external
server.
[root@esxhost]# esxcfg-firewall
--enableService ntpClient
Restarting your ntp daemon, service
ntpd restart.
Disabling the VMware tools in guests
- Installing the ntp daemon as a service
chkconfig --level 345 ntpd on
Set your local hardware clock to NTP server by
doing : hwclock –systohc
Editing ntp.conf
In your ESX files, after making backups of the ntp.conf
files, they should look like this:
restrict default kod nomodify notrap
server 0.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org
driftfile /etc/ntp/drift
Editing step-tickers
Here the listed servers should be your known NTP servers. Then your step-tickers file looks like
this:
0.pool.ntp.org
1.pool.ntp.org
2.pool.ntp.org
pool.ntp.org
And finally checking by running ntpq –p to get detailed
realtime check on the NTP activities. And you are done.
Conclusion:
In the next
article, we will continue administering our ASM, making a new service and
trying to disable and enable a particular instance in order to perform, say, an
OS patch work or any regular maintenance.
»
See All Articles by Columnist Tarry Singh
» 
Comment and Contribute
