Upgrade Oracle 9i RAC to Oracle 10g RAC - Page 2

January 31, 2006

Step 2: Install Oracle Clusterware Software

Oracle Clusterware requires two files – the Oracle Cluster Registry (OCR) and the voting disk on shared raw devices or on Cluster File System (CFS). The Shared Configuration file and quorum file in Oracle 9i RAC has been renamed to Oracle Cluster Registry and voting disk respectively. These files must be accessible to all nodes in the cluster.

To avoid single point of failure, the OCR and voting disk can now be multiplexed by the database. You can create up to two OCR and up to three voting disks.

2a. Oracle Clusterware pre-installation checks

Cluster Verification Utility (CVU) reduces the complexity and time it takes to install RAC. The tool scans all the required components in the cluster environment to ensure all criteria are met for a successful installation. Additional information on CVU can be found at http://download-east.oracle.com/docs/cd/B19306_01/rac.102/b14197/appsupport.htm#BEHIJAJC

Install cvuqdisk RPM prior to running CVU:

rpm -iv /stage/clusterware/rpm/cvuqdisk-1.0.1-1.rpm

To check for shared storage accessibility, run:

/stage/clusterware/cluvfy/runcluvfy.sh comp ssa -n salmon1,salmon2

Upgrade OCFS to the minimum required version if you receive the following warning:

WARNING:
OCFS shared storage discovery skipped because OCFS version 1.0.14 or later is required.

The procedure to upgrade OCFS is located at http://oss.oracle.com/projects/ocfs/dist/documentation/How_To_Upgrade.txt

To perform Oracle Clusterware pre-installation checks, run:

/stage/clusterware/cluvfy/runcluvfy.sh stage –pre crsinst -n salmon1,salmon2 –verbose

CVU reports the error below if non-routable IP addresses such as 192.168.*.*, 172.*.*.* or 10.*.*.* are used for the public interface (eth0).

ERROR:
Could not find a suitable set of interfaces for VIPs.
Node connectivity check failed.

The error can be safely ignored. As a workaround, invoke the Virtual IP Configuration Assistant (VIPCA) manually during the installation on the second node and a suitable network interface (eth0) will be detected.

2b. Install Oracle Clusterware software

There are two approaches to installing the Oracle Clusterware:

With option 1, the OUI will automatically upgrade the Oracle 9i Shared Configuration file to the 10g OCR format and the OCR file locator, /etc/oracle/ocr.loc points to the location of the upgraded Shared Configuration file (/ocfs/prod1/srvm). After the upgrade, the file, /var/opt/oracle/srvConfig.loc is updated to /dev/null.

[oracle@salmon1]$ more /var/opt/oracle/srvConfig.loc
srvconfig_loc=/dev/null

During the upgrade, the OUI does not provide the option of specifying multiple OCR and voting disk file locations. You can however, use ocrconfig and crsctl to manually multiplex the files.

With option 2, you simply perform a fresh Oracle Clusterware installation. You have to shut down all Oracle 9i RAC processes, which include the database instances, listeners, Global Services Daemons and Oracle Cluster Manager and rename the Shared Configuration pointer file, srvConfig.loc to prevent the OUI from detecting the existing Oracle 9i RAC environment.

During the installation, you will be prompted with the option of multiplexing the OCR and voting disk.

My preference is to perform a fresh install. In this demonstration, we will use option 2 to install the Oracle 10g Clusterware software.

Mount the Oracle Clusterware CD or download the software from OTN. The OUI should be launched on only the first node. During installation, the installer automatically copies the software to the second node.

[oracle@salmon1]$ /stage/clusterware/runInstaller

1. Welcome: Click on "Next"
2. Specify Home Details:
   a. Name: OraCr10g_home
   b. Path: /u01/app/oracle/product/crs
3. Product-Specific Prerequisite Checks: Verify that all checks are successful. Click on "Next."
4. Specify Cluster Configuration:
   Cluster Name: crs
   a. Public Node Name: salmon1.dbsconsult.com
   b. Private Node Name: salmon1-priv.dbsconsult.com
   c. Virtual Host Name: salmon1-vip.dbsconsult.com
   d. Public Node Name: salmon2.dbsconsult.com
   e. Private Node Name: salmon2-priv.dbsconsult.com
   f. Virtual Host Name: salmon2-vip.dbsconsult.com 5. Specify Network Interface Usage:
   a. Interface Name: eth0 Subnet: 192.168.0.0 Interface Type: Public
   b. Interface Name: eth3 Subnet: 10.10.10.0 Interface Type: Private
6. Specify Oracle Cluster Registry (OCR) Location:
   a. Select "OCR Configuration"
   b. Specify OCR Location: /ocfs/prod1/ocr1
   c. Specify OCR Mirror Location: /ocfs/prod1/ocr2
7. Specify Voting Disk Location:
   a. Select "Voting Disk Configuration"
   b. Voting Disk Location: /ocfs/prod1/vdisk1
   c. Additional Voting Disk 1 Location: /ocfs/prod1/vdisk2
   d. Additional Voting Disk 2 Location: /ocfs/prod1/vdisk3
8. Summary: Click on "Install"
9. Execute Configuration scripts: Execute /u01/app/oracle/product/crs/root.sh from another window as the root user on each node one at a time. The root.sh script creates the OCR keys, formats the voting disks and starts the CSSD, CRSD and EVMD processes.
10. Execute /u01/app/oracle/product/crs/root.sh as the root user on the second node. The root.sh script invokes the VIPCA automatically and fails with an error – "The given interface(s), "eth0" is not public. Public interfaces should be used to configure virtual IPs." IP address, 192.168.x.x is considered non-routable by CVU and thus it fails to find a suitable public interface. A workaround is to run VIPCA manually.
11. As the root user, manually invokes VIPCA on the second node:
   /u01/app/oracle/product/crs/bin/vipca
12. Welcome: Click on "Next"
13. VIP Configuration Assistant, Step 1 of 2: Network Interfaces:
   a. Select "eth0"
14. VIP Configuration Assistant, Step 2 of 2: Virtual IPs for cluster nodes:
   a. Node name: salmon1
   b. IP Alias Name: salmon1-vip
   c. IP address: 192.168.0.186
   d. Subnet Mask: 255.255.255.0
   e. Node name: salmon2
   f. IP Alias Name: salmon2-vip
   g. IP address: 192.168.0.187
   h. Subnet Mask: 255.255.255.0
15. Summary: Click on "Finish"
16. Configuration Assistant Progress Dialog: Verify that all checks are successful. Click on "OK."
17. Configuration Results: Click on "Exit."
18. Return to the Execute Configuration scripts screen on the first node and click on "OK."
19. Configuration Assistants: Verify that all checks are successful. The OUI does a Clusterware
post-installation check at the end. If the CVU fails, correct the problem and re-run the
following command:
/u01/app/oracle/product/crs/bin/cluvfy stage -post crsinst –n salmon1,salmon2
20. End of Installation: Click "Exit."

[oracle@salmon1]$ more /etc/oracle/ocr.loc
ocrconfig_loc=/ocfs/prod1/ocr1
ocrmirrorconfig_loc=/ocfs/prod1/ocr2
local_only=FALSE
[oracle@salmon1]$ srvctl status nodeapps -n salmon1
VIP is running on node: salmon1
GSD is running on node: salmon1
PRKO-2016 : Error in checking condition of listener on node: salmon1
ONS daemon is running on node: salmon1
[oracle@salmon1]$ /u01/app/oracle/product/crs/bin/olsnodes -n
salmon1 1
salmon1 2
[oracle@salmon1]$ ps -ef | egrep "cssd|crsd|evmd"