Oracle RAC Administration – Part 3: Administering the Clusterware Components

Brief intro

Database administration can be a very daunting task. Will that
change with all the cool technologies like Virtualization or market
developments like Offshoring? NO WAY! You will still need a DBA or a typical
“IT Versatilist,” if you will, who will do the necessary tasks. Sure, things
might get easier as the Oracle Enterprise Manager will be more robust.
Administration tools like Spotlight on RAC will get even more advanced. However,
the fact remains that you must still be able to troubleshoot your RAC. The need for these
skills will not go away. Ok so in the future, (as I see it happening) maybe
your RAC will float in a “Secure Virtual Vault” somewhere on the web, but someone
will need to watch it. On the other hand, it could be the Extended
RAC (RAC nodes geographically separated) thus providing real High Availability.
Administration is administration–you will need it no matter what.

So here, we will take good look at the administration
aspects of “Oracle Clusterware Components”

What is Clusterware composed of?

The Oracle Clusterware is comprised primarily of two
components: the voting disk and the OCR (Oracle Cluster Registry). The voting
disk is nothing but a file that contains and manages information of all the
node memberships and the OCR is a file that manages the cluster and RAC
configuration. Let’s take a quick look at administering the voting disks and
the OCR.

Administering voting disks

Backup and Recovery:

First, let’s look at backing up the voting disks by running
the following command:

dd if=voting_disk_name of=backup_file_name

This operation needs to be performed on all voting disks. Here,
clearly you see that the if (input file) is the source file (replace the voting_disk_name
with your voting disk) and the of (output file) is the destination backup file
containing all information of the voting disk contents. You can do a lot of
stuff with dd, such as splitting a file (using bs = block size, I just
did it yesterday, to restore my Linux hacked iPAQ with WinCE), converting case
sensitivity. Type dd –help for more information. Running the command with the
names of the files reversed will help you recover your voting disk file(s).

dd if=backup_file_name of=voting_disk_name

You can use the ocopy command in Windows environments or
use the crsctl commands to copy and administer the files. Also, note that if
you have multiple voting disks, which are not abnormal to have, you can use the
crsctl command to add and delete the voting disks. For instance:

crsctl delete css votedisk path

Here you delete the disk and the path, which is the complete
path of the location of the file, and below you add your new or backup files by
doing the following:

crsctl add css votedisk path

This way you can either statically or
dynamically add or remove your voting disks in your RAC.

You must, however, note that if your cluster is down, then
you can use the -force option

crsctl add css votedisk path -force

to modify the voting disk configuration. This way you don’t
end up interfering with other Clusterware daemons. Using it in your active
configuration may corrupt your configuration.

Administering Oracle Cluster Registry

OCR contains information pertaining to instance-to-node
mapping, node list and resource profiles for customized applications in your Clusterware.
Let’s take a look at some of the following administrative tasks:

Adding, Replacing, Managing & Removing OCR

You can’t have more than two OCRs.You can add an OCR either
after an upgrade or after installing the RAC installation. If you already
mirror the OCR, then you do not need to add an OCR location; Oracle does that
automatically. If your OCR is on the network, do create a target file before
performing any tasks! In addition, you must be logged in as the root user
to run the ocrconfig tool.

So if you created a single OCR, then add by doing the
following. :

ocrconfig -replace ocr destination_file or disk

Here, do the following to add a mirror file.

ocrconfig -replace ocrmirror destination_file or disk

Replacing OCR works the same way. Do however check that the
file to be replaced is online, the Clusterware is running on that particular
node and that if it is on a CFS or on the network.

To replace OCR do the following:

ocrconfig -replace ocr destination_file or disk

and to replace the OCR mirror:

ocrconfig -replace ocrmirror destination_file or disk

Repairing the OCR comes in handy if the nodes where you may
have been working are shutdown. Typing ocrconfig
-repair brings those nodes back on line. More specifically, you
can do this (to repair your OCR mirror):

ocrconfig –repair ocrmirror device_name 

This command repairs the OCR configuration locally. Also,
note that you cannot do this on running Clusterware daemons.

To remove an OCR, you need to have at least one OCR online.
You may want to do this to reduce overhead or for other storage reasons, such
as stopping a mirror to move it to SAN, RAID etc. Carry out the following steps
:

• Check if at least one OCR is online

• Remove the OCR or OCR mirror

ocrconfig -replace ocr OR ocrconfig -replace ocrmirror

Using the –backuploc option allows you to save/move the OCR
file to a safe location. Type ocrconfig –option to see all the
commands.

Conclusion:

The Oracle
Clusterware has an HA (High Availability) framework that provides a robust infrastructure
to manage any application. The Oracle Clusterware daemons make sure that all
applications startup during system startup. Failed applications are started
automatically to maintain the HA aspect of the RAC cluster. It is possible to
configure all of the administrative aspects (like monitoring frequencies,
startup, shutdown) of the RAC cluster.

This article examined the two most important components of Oracle
Clusterware. In our next article, we will cover the backup and recovery
administration of OCR using OCR backup files.

»


See All Articles by Columnist Tarry Singh