Upgrade Oracle 9i RAC to Oracle 10g RAC

by Vincent Chan

This article provides the procedures for converting Oracle 9i (9.2.0.4) RAC to Oracle 10g (10.2.0.1) RAC on Red Hat Enterprise Linux 3 (RHEL3).

Oracle Database 10g and Oracle Real Application Cluster (RAC) 10g itself, boast many new and exciting features that simplify database manageability and provide higher availability and scalability. Compared to its predecessors, Oracle 10g RAC is much easier to install, use and manage, thereby reducing the total cost of ownership.

Overview

The RAC cluster comprises two Intel x86 servers running on RHEL3 (Kernel 2.4.21-27). Each node has access to a shared storage and connectivity to the public and private network.

This article is structured into the following steps:

Unless otherwise specified, you should execute all steps on both nodes.

Here’s an overview of our Oracle 9i RAC environment:

And an overview of the Oracle 10g RAC environment:

Step 1: Preliminary Installation

1a. Verify software package versions

Install the required packages. Additional information can be obtained from http://download-east.oracle.com/docs/cd/B19306_01/install.102/b14203/prelinux.htm#sthref380

• binutils-2.14
  
• compat-db-4.0.14-5
  
• compat-gcc-7.3-2.96.128
  
• compat-gcc-c++-7.3-2.96.128
  
• compat-libstdc++-7.3-2.96.128
  
• compat-libstdc++-devel-7.3-2.96.128
  
• gcc-3.2
  
• glibc-2.3.2-95.27
  
• make-3.79
  
• openmotif-2.2.3
  
• setarch-1.3-1

1b. Verify kernel parameters

Verify the following kernel parameters. Additional information can be obtained from http://download-east.oracle.com/docs/cd/B19306_01/install.102/b14203/prelinux.htm#sthref418

[root@salmon1]# sysctl -a | grep shm

kernel.shmmni = 4096

kernel.shmall = 2097152

kernel.shmmax = 2147483648

[root@salmon1]# sysctl -a | grep sem

kernel.sem = 250        32000   100     128

[root@salmon1]# sysctl -a | grep -i ip_local

net.ipv4.ip_local_port_range = 1024     65000

[root@salmon1]# sysctl -a | grep -i file-max

fs.file-max = 65536

[root@salmon1]# sysctl -a | egrep “rmem_default|rmem_max|wmem_default| wmem_max”

net.core.rmem_default = 262144

net.core.wmem_default = 262144

net.core.rmem_max = 262144

net.core.wmem_max = 262144

1c. Verify users and groups

Verify that the oracle user primary group is oinstall and the secondary group is dba.

[root@salmon1]# id oracle

uid=500(oracle) gid=501(oinstall) groups=501(oinstall),500(dba)

Verify that the user nobody exists on the node.

[root@salmon1]# id nobody

uid=99(nobody) gid=99(nobody) groups=99(nobody)

1d. Edit the oracle user environment file

Verify that the oracle user primary group is oinstall and the secondary group is dba.

[oracle@salmon1]$ more .bash_profile

# .bash_profile

# Get the aliases and functions

if [ -f ~/.bashrc ]; then

        . ~/.bashrc

fi

export PATH=$PATH:$HOME/bin

export ORACLE_SID=prod1a

export ORACLE_BASE=/u01/app/oracle

export ORACLE_HOME=$ORACLE_BASE/product/crs

export ORA_CRS_HOME=$ORACLE_BASE/product/crs

export LD_LIBRARY_PATH=$ORACLE_HOME/lib

export PATH=$ORACLE_HOME/bin:$PATH

umask 022

1e. Configure the oracle user shell limits

[root@salmon1]# more /etc/security/limits.conf

*                soft    nproc            2047

*                hard    nproc            16384

*                soft    nofile           1024

*                hard    nofile           65536

[root@salmon1]# grep pam_limits /etc/pam.d/login

 session    required     /lib/security/pam_limits.so

1f. Configure public and private network

Using the information below, make the necessary changes to network interface devices eth0 (public) and eth3 (private).

[root@salmon1]# redhat-config-network

1g. Edit the /etc/hosts file

Add the entries for Virtual IP (VIP) addresses in /etc/hosts. VIP is required in Oracle 10g RAC to increase availability by eliminating the need to wait on network timeout which may take as long as 10 minutes. Refer to http://metalink.oracle.com/metalink/plsql/ml2_documents.showDocument?p_database_id=NOT&p_id=264847.1 for additional information on VIP configuration.

127.0.0.1   localhost.localdomain  localhost

10.10.10.84  salmon1-priv.dbsconsult.com        salmon1-priv

10.10.10.85  salmon2-priv.dbsconsult.com        salmon2-priv

192.168.0.184  salmon1.dbsconsult.com           salmon1

192.168.0.185  salmon2.dbsconsult.com           salmon2

192.168.0.186  salmon1-vip.dbsconsult.com     salmon1-vip

192.168.0.187  salmon2-vip.dbsconsult.com     salmon2-vip

Verify the hostname and the configured network interface devices.

[root@salmon1]# hostname

salmon1.dbsconsult.com

[root@salmon1]# /sbin/ifconfig

1h. Establish user equivalence with SSH

During the Oracle Clusterware and RAC installation, the Oracle Universal Installer (OUI) has to be able to copy the software as oracle to all RAC nodes without being prompted for a password. This can be accomplished using ssh instead of rsh.

To establish user equivalence, generate the user’s public and private keys as the oracle user on both nodes.

[oracle@salmon1]$ ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_dsa):

Created directory ‘/home/oracle/.ssh’.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/oracle/.ssh/id_dsa.

Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.

The key fingerprint is:

5d:8c:42:97:eb:42:ae:52:52:e9:59:20:2a:d3:6f:59 oracle@salmon1.dbsconsult.com

The public key on each node is copied to both nodes. Execute the following on each node.

[oracle@salmon1]$ ssh salmon1 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

The authenticity of host ‘salmon1 (192.168.0.184)’ can’t be established.

RSA key fingerprint is 00:d9:70:08:bc:fd:b5:e4:e3:df:a3:c7:d8:46:1e:a5.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added ‘salmon1,192.168.0.184’ (RSA) to the list of known hosts.

oracle@salmon1’s password:

[oracle@salmon1]$ ssh salmon2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

The authenticity of host ‘salmon2 (192.168.0.185)’ can’t be established.

RSA key fingerprint is 00:d9:70:08:bc:fd:b5:e4:e3:df:a3:c7:d8:46:1e:a5.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added ‘salmon2,192.168.0.185’ (RSA) to the list of known hosts.

oracle@salmon2’s password:

Test the connection on every node. Verify that you are not prompted for password when you run the following the second time.

• ssh salmon1 date
  
• ssh salmon2 date
  
• ssh salmon1-priv date
  
• ssh salmon2-priv date
  
• ssh salmon1.dbsconsult.com date
  
• ssh salmon2.dbsconsult.com date
  
• ssh salmon1-priv.dbsconsult.com date
  
• ssh salmon2-priv.dbsconsult.com date

1i. Configure hangcheck timer kernel module

The hangcheck timer kernel module monitors the system’s health and restarts a failing RAC node. It uses two parameters, hangcheck_tick (defines the system checks frequency) and hangcheck_margin (defines the maximum hang delay before a RAC node is reset), to determine if a node is failing.

The hangcheck timer should have already been configured for Oracle 9i (9.2.0.4) RAC. Verify that the following line is in /etc/rc.d/rc.local.

[root@salmon1]# grep insmod /etc/rc.d/rc.local

insmod hangcheck-timer hangcheck_tick=30 hangcheck_margin=180

1j. Create pfile

Save a copy of the pfile for the database upgrade.

SQL> create pfile=’/tmp/pfile_singleinst’ from spfile;