by Vincent Chan
Step 2: Install Oracle Clusterware Software
Clusterware requires two files the Oracle Cluster Registry (OCR) and the
voting disk on shared raw devices or on Cluster File System (CFS). The Shared
Configuration file and quorum file in Oracle 9i RAC has been renamed to Oracle
Cluster Registry and voting disk respectively. These files must be accessible
to all nodes in the cluster.
avoid single point of failure, the OCR and voting disk can now be multiplexed
by the database. You can create up to two OCR and up to three voting disks.
2a. Oracle Clusterware pre-installation checks
Cluster Verification Utility (CVU) reduces the complexity
and time it takes to install RAC. The tool scans all the required components in
the cluster environment to ensure all criteria are met for a successful
installation. Additional information on CVU can be found at http://download-east.oracle.com/docs/cd/B19306_01/rac.102/b14197/appsupport.htm#BEHIJAJC
Install cvuqdisk RPM prior to running CVU:
rpm -iv /stage/clusterware/rpm/cvuqdisk-1.0.1-1.rpm
To check for shared storage accessibility, run:
/stage/clusterware/cluvfy/runcluvfy.sh comp ssa -n salmon1,salmon2
Upgrade OCFS to the minimum required version if you receive
the following warning:
OCFS shared storage discovery skipped because OCFS version 1.0.14 or later is required.
The procedure to upgrade OCFS is located at http://oss.oracle.com/projects/ocfs/dist/documentation/How_To_Upgrade.txt
To perform Oracle Clusterware pre-installation checks, run:
/stage/clusterware/cluvfy/runcluvfy.sh stage pre crsinst -n salmon1,salmon2 verbose
CVU reports the error below if non-routable IP addresses
such as 192.168.*.*, 172.*.*.* or 10.*.*.* are used for the public interface
Could not find a suitable set of interfaces for VIPs.
Node connectivity check failed.
The error can be safely ignored. As a workaround, invoke the
Virtual IP Configuration Assistant (VIPCA) manually during the installation on
the second node and a suitable network interface (eth0) will be detected.
2b. Install Oracle Clusterware software
There are two approaches to installing the Oracle
1. Upgrade the existing Shared Configuration file to Oracle
10g OCR format
2. Create new OCR files.
With option 1, the OUI will automatically upgrade the Oracle
9i Shared Configuration file to the 10g OCR format and the OCR file locator,
/etc/oracle/ocr.loc points to the location of the upgraded Shared Configuration
file (/ocfs/prod1/srvm). After the upgrade, the file,
/var/opt/oracle/srvConfig.loc is updated to /dev/null.
[oracle@salmon1]$ more /var/opt/oracle/srvConfig.loc
During the upgrade, the OUI does not provide the option of
specifying multiple OCR and voting disk file locations. You can however, use
ocrconfig and crsctl to manually multiplex the files.
With option 2, you simply perform a fresh Oracle Clusterware
installation. You have to shut down all Oracle 9i RAC processes, which include
the database instances, listeners, Global Services Daemons and Oracle Cluster
Manager and rename the Shared Configuration pointer file, srvConfig.loc to
prevent the OUI from detecting the existing Oracle 9i RAC environment.
During the installation, you will be prompted with the
option of multiplexing the OCR and voting disk.
My preference is to perform a fresh install. In this demonstration,
we will use option 2 to install the Oracle 10g Clusterware software.
Mount the Oracle Clusterware CD or download the software
from OTN. The OUI should be launched on only the first node. During
installation, the installer automatically copies the software to the second
1. Welcome: Click on "Next"
2. Specify Home Details:
a. Name: OraCr10g_home
b. Path: /u01/app/oracle/product/crs
3. Product-Specific Prerequisite Checks: Verify that all
checks are successful. Click on "Next."
4. Specify Cluster Configuration:
Cluster Name: crs
a. Public Node Name: salmon1.dbsconsult.com
b. Private Node Name: salmon1-priv.dbsconsult.com
c. Virtual Host Name: salmon1-vip.dbsconsult.com
d. Public Node Name: salmon2.dbsconsult.com
e. Private Node Name: salmon2-priv.dbsconsult.com
f. Virtual Host Name: salmon2-vip.dbsconsult.com
5. Specify Network Interface Usage:
a. Interface Name: eth0 Subnet: 192.168.0.0 Interface Type: Public
b. Interface Name: eth3 Subnet: 10.10.10.0 Interface Type: Private
6. Specify Oracle Cluster Registry (OCR) Location:
a. Select "OCR Configuration"
b. Specify OCR Location: /ocfs/prod1/ocr1
c. Specify OCR Mirror Location: /ocfs/prod1/ocr2
7. Specify Voting Disk Location:
a. Select "Voting Disk Configuration"
b. Voting Disk Location: /ocfs/prod1/vdisk1
c. Additional Voting Disk 1 Location: /ocfs/prod1/vdisk2
d. Additional Voting Disk 2 Location: /ocfs/prod1/vdisk3
8. Summary: Click on "Install"
9. Execute Configuration scripts: Execute /u01/app/oracle/product/crs/root.sh from another
window as the root user on each node one at a time.
The root.sh script creates the OCR keys, formats the voting disks and starts the CSSD,
CRSD and EVMD processes.
10. Execute /u01/app/oracle/product/crs/root.sh as the root user on the second node.
The root.sh script invokes the VIPCA automatically and fails with an error "The given
interface(s), "eth0" is not public. Public interfaces should be used to configure virtual IPs."
IP address, 192.168.x.x is considered non-routable by CVU and thus it fails to find a suitable
public interface. A workaround is to run VIPCA manually.
11. As the root user, manually invokes VIPCA on the second node:
12. Welcome: Click on "Next"
13. VIP Configuration Assistant, Step 1 of 2: Network
a. Select "eth0"
14. VIP Configuration Assistant, Step 2 of 2: Virtual IPs for
a. Node name: salmon1
b. IP Alias Name: salmon1-vip
c. IP address: 192.168.0.186
d. Subnet Mask: 255.255.255.0
e. Node name: salmon2
f. IP Alias Name: salmon2-vip
g. IP address: 192.168.0.187
h. Subnet Mask: 255.255.255.0
15. Summary: Click on "Finish"
16. Configuration Assistant Progress Dialog: Verify that all
checks are successful. Click on "OK."
17. Configuration Results: Click on "Exit."
18. Return to the Execute Configuration scripts screen on
the first node and click on "OK."
19. Configuration Assistants: Verify that all checks are
successful. The OUI does a Clusterware
post-installation check at the end. If the CVU fails, correct the problem and re-run the
/u01/app/oracle/product/crs/bin/cluvfy stage -post crsinst n salmon1,salmon2
20. End of Installation: Click "Exit."
[oracle@salmon1]$ more /etc/oracle/ocr.loc
[oracle@salmon1]$ srvctl status nodeapps -n salmon1
VIP is running on node: salmon1
GSD is running on node: salmon1
PRKO-2016 : Error in checking condition of listener on node: salmon1
ONS daemon is running on node: salmon1
[oracle@salmon1]$ /u01/app/oracle/product/crs/bin/olsnodes -n
[oracle@salmon1]$ ps -ef | egrep "cssd|crsd|evmd"