Brief intro
As I said on my blog, I have
been trying to setup Oracle Linux 10gR2 RAC on Oracle Enterprise Linux 4.4 with
VMware ESX 3.0.1 as our chosen platform for Virtualization. This article
mentions several issues that we will try to cover all in one go. If that’s not
possible, we’ll finish it up in the next article.
What I used?
- HP DL box , 4way SMP
-
ESX 3.0.1 (VMTN license which is soon to expire as it does not
exist anymore) - Oracle Linux 4.4
- Oracle 10gR2
Is Oracle Enterprise Linux installation all smooth sailing?
Not really. After carefully selecting all
of the possible options, (you do know that several binaries like ASM and OCFS
come along with the Oracle Linux), while going through the installation I still
came across several anomalies on some rpms.
Cluvfy.sh to the rescue
Upon running the cluvfy, after the clusterware installation and before the
database software installation, we had some errors.
[oracle@vm01 cluvfy]$ ./runcluvfy.sh stage -pre dbinst -n vm01,vm02 -r 10gR2 -verboseCheck: Package existence for “compat-gcc-7.3-2.96.128”
Node Name Status Comment
—————————— —————————— —————-
vm02 missing failed
vm01 missing failed
Result: Package existence check failed for “compat-gcc-7.3-2.96.128”.
Check: Package existence for “compat-gcc-c++-7.3-2.96.128”
Node Name Status Comment
—————————— —————————— —————-
vm02 missing failed
vm01 missing failed
Result: Package existence check failed for “compat-gcc-c++-7.3-2.96.128”.
Check: Package existence for “compat-libstdc++-7.3-2.96.128”
Node Name Status Comment
—————————— —————————— —————-
vm02 missing failed
vm01 missing failed
Result: Package existence check failed for “compat-libstdc++-7.3-2.96.128”.
Check: Package existence for “compat-libstdc++-devel-7.3-2.96.128”
Node Name Status Comment
—————————— —————————— —————-
vm02 missing failed
vm01 missing failed
Result: Package existence check failed for “compat-libstdc++-devel-7.3-2.96.128”.
Check: Package existence for “openmotif-2.2.3”
Node Name Status Comment
—————————— —————————— —————-
vm02 missing failed
vm01 missing failed
Result: Package existence check failed for “openmotif-2.2.3”.Check: Kernel parameter for “wmem_max”
Node Name Configured Required Comment
———— ———————— ———————— ———-
vm02 131071 262144 failed — I made a typo in my sysctl.conf!!
vm01 262144 262144 passed
Result: Kernel parameter check failed for “wmem_max”.
Pre-check for database installation was unsuccessful on all the nodes.
[oracle@vm01 cluvfy]$
Remedy?
Normally you are not
expected to miss out on those rpm’s, (when I say this, I am comparing to an
RHEL4 or Centos4 installation). I chose all of the legacy development binaries
and still I had to go to http://rpm.pbone.net/ and download the following
binaries.
- compat-libstdc++-7.3-2.96.128.i386.rpm
- openmotif-2.2.3-10.RHEL4.5.i386.rpm
- compat-gcc-7.3-2.96.128.i386.rpm
- compat-libstdc++-devel-7.3-2.96.128.i386.rpm
- compat-gcc-c++-7.3-2.96.128.i386.rpm
- libaio-0.3.96-7.i386.rpm.
Why libaio-0.3.96-7.i386.rpm?
Simply because the cluvfy.sh in the GUI failed me on this rpm. Don’t forget
to do the rpm -e to other conflicting rpms before installing the compat*.rpms’.
Running the cluvfy again was finally successful.
[oracle@vm01 sysadm]$ cd /u01/app/oracle/orasoft/clusterware/cluvfy
[oracle@vm01 cluvfy]$ ./runcluvfy.sh stage -pre dbinst -n vm01,vm02 -r 10gR2 -verbose
Performing pre-checks for database installation
Checking node reachability…
Check: Node reachability from node “vm01”
Destination Node Reachable?
———————————— ————————
vm01 yes
vm02 yes
Result: Node reachability check passed from node “vm01”.Checking user equivalence…
Check: User equivalence for user “oracle”
Node Name Comment
———————————— ————————
vm02 passed
vm01 passed
Result: User equivalence check passed for user “oracle”.
Checking administrative privileges…
Check: Existence of user “oracle”
Node Name User Exists Comment
———— ———————— ————————
vm02 yes passed
vm01 yes passed
Result: User existence check passed for “oracle”.
Check: Existence of group “dba”
Node Name Status Group ID
———— ———————— ————————
vm02 exists 115
vm01 exists 115
Result: Group existence check passed for “dba”.
Check: Membership of user “oracle” in group “dba” [as Primary]
Node Name User Exists Group Exists User in Group Primary Comment
—————- ———— ———— ———— ———— ————
vm02 yes yes yes yes passed
vm01 yes yes yes yes passed
Result: Membership check for user “oracle” in group “dba” [as Primary] passed.
.
.
.
.
Checking node application existence…Checking existence of VIP node application
Node Name Required Status Comment
———— ———————— ———————— ———-
vm02 yes exists passed
vm01 yes exists passed
Result: Check passed.
Checking existence of ONS node application
Node Name Required Status Comment
———— ———————— ———————— ———-
vm02 no exists passed
vm01 no exists passed
Result: Check passed.
Checking existence of GSD node application
Node Name Required Status Comment
———— ———————— ———————— ———-
vm02 no exists passed
vm01 no exists passed
Result: Check passed.Pre-check for database installation was successful.
[oracle@vm01 cluvfy]$
Swap space dilemma: Extending a vmdk disk
I also had a problem with a swap space issue. I had 2Gb RAM per node. I had
allocated 2Gb RAM to a separate VMDK disk. However, the OUI still complained that
I had less swap space than required. I used the following steps to extend the
VMDK disk size.
1. Do make a back
up of your previous disk and data!
2. Typing vmkfstools
will give you the whole syntax of the utility
3. vmkfstools -X
4GB swap.vmdk (This way you can extend the file size but this does not show
immediately on your OS , use partitionmagic, diskpart for Windows OS and try
this cool utility called gparted ( http://gparted.sourceforge.net/download.php)
for linux OS.
4. Copy the gparted.iso
file, boot your VM (after cleanly shutting it down) and use the easy to follow
feature to expand the partition in the GUI.
5.
Reboot your machine and voila! You have your new extended partition.
Another way to do this is
(in the case of our swap file):
1. Delete the swap.vmdk
file and remove it from the ESX server permanently!
2. Add a new disk
on the same SCSI node, which means if it was vm01_1.vmdk for your original swap
disk, then the newly created disk may luckily acquire the same name , if not
change it to same name.
3. Doing fdisk -ls
will show you that this newly created disk is not partitioned, so partition it.
4. Edit your etc/fstab
file to match the partition, lets say the partition is sdb1:
mkswap /dev/sdb1
5. To enable the
swap file immediately do swapon
/dev/hdb2
6. To enable it at
boot time, edit the previous entry in your etc/fstab like this: /dev/sdb2 swap swap
defaults 0 0
7.
Boot your vm and continue with the database installation.
Conclusion
Installation and optimal
performance of Oracle RAC on VMware’s ESX server is easy to say but complex to
accomplish. However, if you are a manager and are in dire need of providing
your DBAs and Developers an environment, which is a lot cheaper than the money
you spent in production, then you’d want to go for it. The great thing of
having your RAC on ESX is that you can back it all up before initiating a
change in the system. Someday (not too distant) we will have RACs and all kinds
of mission critical clusters running on a hypervisor–but you don’t have to
wait for that day, you can get started right away!
