Installation gotchas for Oracle Linux , Oracle RAC 10gR2 and VMware ESX 3.0.1

April 5, 2007

Brief intro

As I said on my blog, I have been trying to setup Oracle Linux 10gR2 RAC on Oracle Enterprise Linux 4.4 with VMware ESX 3.0.1 as our chosen platform for Virtualization. This article mentions several issues that we will try to cover all in one go. If that's not possible, we'll finish it up in the next article.

What I used?

  • HP DL box , 4way SMP
  • ESX 3.0.1 (VMTN license which is soon to expire as it does not exist anymore)
  • Oracle Linux 4.4
  • Oracle 10gR2

Is Oracle Enterprise Linux installation all smooth sailing?

Not really. After carefully selecting all of the possible options, (you do know that several binaries like ASM and OCFS come along with the Oracle Linux), while going through the installation I still came across several anomalies on some rpms.

Cluvfy.sh to the rescue

Upon running the cluvfy, after the clusterware installation and before the database software installation, we had some errors.

[oracle@vm01 cluvfy]$ ./runcluvfy.sh stage -pre dbinst -n vm01,vm02 -r 10gR2 -verbose


Check: Package existence for "compat-gcc-7.3-2.96.128"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  vm02                            missing                         failed
  vm01                            missing                         failed
Result: Package existence check failed for "compat-gcc-7.3-2.96.128".
Check: Package existence for "compat-gcc-c++-7.3-2.96.128"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  vm02                            missing                         failed
  vm01                            missing                         failed
Result: Package existence check failed for "compat-gcc-c++-7.3-2.96.128".
Check: Package existence for "compat-libstdc++-7.3-2.96.128"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  vm02                            missing                         failed
  vm01                            missing                         failed
Result: Package existence check failed for "compat-libstdc++-7.3-2.96.128".
Check: Package existence for "compat-libstdc++-devel-7.3-2.96.128"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  vm02                            missing                         failed
  vm01                            missing                         failed
Result: Package existence check failed for "compat-libstdc++-devel-7.3-2.96.128".
Check: Package existence for "openmotif-2.2.3"
  Node Name                       Status                          Comment
  ------------------------------  ------------------------------  ----------------
  vm02                            missing                         failed
  vm01                            missing                         failed
Result: Package existence check failed for "openmotif-2.2.3".

Check: Kernel parameter for "wmem_max"
  Node Name     Configured                Required                  Comment
  ------------  ------------------------  ------------------------  ----------
  vm02          131071                    262144                    failed -- I made a typo in my sysctl.conf!!
  vm01          262144                    262144                    passed
Result: Kernel parameter check failed for "wmem_max".
Pre-check for database installation was unsuccessful on all the nodes.
[oracle@vm01 cluvfy]$

Remedy?

Normally you are not expected to miss out on those rpm's, (when I say this, I am comparing to an RHEL4 or Centos4 installation). I chose all of the legacy development binaries and still I had to go to http://rpm.pbone.net/ and download the following binaries.

  • compat-libstdc++-7.3-2.96.128.i386.rpm
  • openmotif-2.2.3-10.RHEL4.5.i386.rpm
  • compat-gcc-7.3-2.96.128.i386.rpm
  • compat-libstdc++-devel-7.3-2.96.128.i386.rpm
  • compat-gcc-c++-7.3-2.96.128.i386.rpm
  • libaio-0.3.96-7.i386.rpm.

Why libaio-0.3.96-7.i386.rpm? Simply because the cluvfy.sh in the GUI failed me on this rpm. Don't forget to do the rpm -e to other conflicting rpms before installing the compat*.rpms'.

Running the cluvfy again was finally successful.

[oracle@vm01 sysadm]$ cd /u01/app/oracle/orasoft/clusterware/cluvfy
[oracle@vm01 cluvfy]$ ./runcluvfy.sh stage -pre dbinst -n vm01,vm02 -r 10gR2 -verbose
Performing pre-checks for database installation
Checking node reachability...
Check: Node reachability from node "vm01"
  Destination Node                      Reachable?
  ------------------------------------  ------------------------
  vm01                                  yes
  vm02                                  yes
Result: Node reachability check passed from node "vm01".

Checking user equivalence...
Check: User equivalence for user "oracle"
  Node Name                             Comment
  ------------------------------------  ------------------------
  vm02                                  passed
  vm01                                  passed
Result: User equivalence check passed for user "oracle".
Checking administrative privileges...
Check: Existence of user "oracle"
  Node Name     User Exists               Comment
  ------------  ------------------------  ------------------------
  vm02          yes                       passed
  vm01          yes                       passed
Result: User existence check passed for "oracle".
Check: Existence of group "dba"
  Node Name     Status                    Group ID
  ------------  ------------------------  ------------------------
  vm02          exists                    115
  vm01          exists                    115
Result: Group existence check passed for "dba".
Check: Membership of user "oracle" in group "dba" [as Primary]
  Node Name         User Exists   Group Exists  User in Group  Primary       Comment
  ----------------  ------------  ------------  ------------  ------------  ------------
  vm02              yes           yes           yes           yes           passed
  vm01              yes           yes           yes           yes           passed
Result: Membership check for user "oracle" in group "dba" [as Primary] passed.
.
.
.
.
Checking node application existence...

Checking existence of VIP node application
  Node Name     Required                  Status                    Comment
  ------------  ------------------------  ------------------------  ----------
  vm02          yes                       exists                    passed
  vm01          yes                       exists                    passed
Result: Check passed.
Checking existence of ONS node application
  Node Name     Required                  Status                    Comment
  ------------  ------------------------  ------------------------  ----------
  vm02          no                        exists                    passed
  vm01          no                        exists                    passed
Result: Check passed.
Checking existence of GSD node application
  Node Name     Required                  Status                    Comment
  ------------  ------------------------  ------------------------  ----------
  vm02          no                        exists                    passed
  vm01          no                        exists                    passed
Result: Check passed.

Pre-check for database installation was successful.
[oracle@vm01 cluvfy]$

Swap space dilemma: Extending a vmdk disk

I also had a problem with a swap space issue. I had 2Gb RAM per node. I had allocated 2Gb RAM to a separate VMDK disk. However, the OUI still complained that I had less swap space than required. I used the following steps to extend the VMDK disk size.

1.    Do make a back up of your previous disk and data!

2.    Typing vmkfstools will give you the whole syntax of the utility

3.    vmkfstools -X 4GB swap.vmdk (This way you can extend the file size but this does not show immediately on your OS , use partitionmagic, diskpart for Windows OS and try this cool utility called gparted ( http://gparted.sourceforge.net/download.php) for linux OS.

4.    Copy the gparted.iso file, boot your VM (after cleanly shutting it down) and use the easy to follow feature to expand the partition in the GUI.

5.    Reboot your machine and voila! You have your new extended partition.

Another way to do this is (in the case of our swap file):

1.    Delete the swap.vmdk file and remove it from the ESX server permanently!

2.    Add a new disk on the same SCSI node, which means if it was vm01_1.vmdk for your original swap disk, then the newly created disk may luckily acquire the same name , if not change it to same name.

3.    Doing fdisk -ls will show you that this newly created disk is not partitioned, so partition it.

4.    Edit your etc/fstab file to match the partition, lets say the partition is sdb1:

mkswap /dev/sdb1

5.    To enable the swap file immediately do swapon /dev/hdb2

6.    To enable it at boot time, edit the previous entry in your etc/fstab like this: /dev/sdb2 swap swap defaults 0 0

7.    Boot your vm and continue with the database installation.

Conclusion

Installation and optimal performance of Oracle RAC on VMware's ESX server is easy to say but complex to accomplish. However, if you are a manager and are in dire need of providing your DBAs and Developers an environment, which is a lot cheaper than the money you spent in production, then you'd want to go for it. The great thing of having your RAC on ESX is that you can back it all up before initiating a change in the system. Someday (not too distant) we will have RACs and all kinds of mission critical clusters running on a hypervisor--but you don't have to wait for that day, you can get started right away!

» See All Articles by Columnist Tarry Singh








The Network for Technology Professionals

Search:

About Internet.com

Legal Notices, Licensing, Permissions, Privacy Policy.
Advertise | Newsletters | E-mail Offers