Synopsis. Oracle 10g’s new Automatic Storage
Management (ASM) features allow an Oracle DBA to take advantage of a robust,
flexible, scalable file storage system that any Oracle database instance can
access. This article – the first in this series — provides a high-level
overview of the ASM architecture, and should be helpful to any DBA
contemplating the adoption of ASM for storing Oracle database files as part of
a high-volume storage solution.
With the possible exception of having to rebuild a database
server from a "cold metal" starting point, I have to admit that the
scenario that holds the most dread for me as an Oracle DBA is to run out of
disk space for a production database’s datafiles or tempfiles. Though, in most
cases this situation can be rectified easily – as long as there is sufficient
disk space available, of course! – it always seems that these challenges seem
to occur either at the busiest time of my client’s business day, or between
02:00 and 03:00 on an early Sunday morning when I am scheduled to be out of
town enjoying a long weekend. And I have also encountered my share of
unexpected (and unlikely) disk media failures. For example, I once watched five
disks in a 42-disk mirrored storage array fail within a three-week period after
three years of relatively solid performance. The pattern never repeated itself,
but it did teach me the value of a good mirroring strategy!
While a storage area network (SAN) and other logical volume
manager (LVM) storage solutions like EMC are certainly options for guaranteed
storage reliability, the costs of these solutions can be prohibitive for
smaller IT organizations. The good news is that Oracle 10g supplies a new set
of disk media storage features called Automatic Storage Management (ASM)
that are aimed directly at insuring a storage solution for smaller enterprise
databases. This series of articles will delve into ASM’s rich features and
provide examples of how to implement some simple ASM disk configurations for
evaluation purposes so that your DBA team can make some decisions about whether
ASM is worth pursuing for your organization. I will concentrate this article on
a general review of ASM features as well as a brief explanation of the ASM
architecture.
Automatic Storage Management Features
Out-Of-The-Box Functionality. First off, ASM was designed
specifically for the storage of Oracle database files. This means that it
is ready "out of the box" for use against any Oracle database that it
services. Though ASM appears to be the intended replacement for Oracle Cluster
File System (OCFS) for the Real Applications Cluster (RAC) environment, that
doesn’t mean that your database needs to be deployed under RAC to take
advantage of ASM’s myriad features; ASM can be deployed on a single
processor (SMP) server just as easily as it can be deployed on a
multiple-node RAC cluster. Moreover, it is simple to configure using the
graphic interface provided by the Database Configuration Assistant (DBCA), an
existing Oracle tool that just about every Oracle DBA has used. Oracle 10g also
provides the standard Enterprise Manager (EM) interface for managing an ASM
instance once it has been created.
Flexible Storage Configuration. Since ASM is in
essence its own file management system, this means that I can add
more physical storage, change storage configurations, or remove
extraneous storage without having to shut down my Oracle database to change
out physical hardware. This is a big advantage over traditional hardware
configurations, and is certainly featured in storage-area networks (SANs).
Using ASM also means that I can turn over the majority of my
space management tasks to the ASM instance and let it monitor the growth of my
database’s tablespaces. In addition, ASM maintains pre-existing Oracle
database functionality, so regular Oracle databases can continue to operate as
usual. New database files can be created as ASM files managed by the ASM
instance, existing database files can be migrated to ASM as time and
opportunity permit, and other database files can continue to exist as
"traditional" database files without ASM file management.
Fault Tolerance. I also expect any file management
system to insure that once I write database information to disk, it will never
be lost – unless, of course, I tell the database to delete that data. One way
that DBAs usually guarantee against the inevitable failure of disk hardware is
through mirroring drives via redundant arrays of inexpensive disks (aka
RAID) and striping (writing portions of disk files to different disks so
that if one disk is lost the information is recoverable from another disk in
the set). This guards against the inevitable failure of disk hardware, since
even with extremely high estimated mean time between failures (MTBF), a disk
drive will almost certainly fail eventually.
Most operating systems provide controls to set up and
monitor disk storage to take advantage of these redundancy features. Like any
good file management system, ASM provides fault tolerance by allowing me to
write duplicate or triplicate copies of any database files’ contents. And if
my current server and disk storage system already support fault tolerance, so
much the better! ASM can also take advantage of existing vendor-supplied
fault tolerance mechanisms for increased guarantees to safe data storage.
Since ASM handles the mirroring of data, there is no need to
purchase a third-party Logical Volume Manager (LVM). Mirroring is applied on a file-by-file
basis, rather than on a per-volume basis, so the same disk group may
contain a combination of files protected by mirroring (or perhaps not even
mirrored at all, if mirroring is not required for some files). ASM also
supports Oracle Real Application Clusters (RAC), so there is no need for
a separate Cluster LVM or a Cluster File System.
I/O Load Balancing. ASM is responsible for providing
equal distribution of I/O loads across all available disk storage resources,
so there is a measurable performance improvement. For example, ASM will split a
datafile into its component extents and then spread those extents evenly across
all defined disks that ASM is managing; those extents are then tracked via an
indexing technique. ASM also automatically manages data via selection of
desired reliability and performance characteristics instead of manually
manipulating data file storage characteristics, and therefore it tends to
reduce (or even eliminate) manual tuning and retuning of I/O.
A big advantage to this approach is that when ASM storage
capacity changes, ASM doesn’t need to re-stripe all data – it just moves
enough data in proportion to the amount of added (or reduced!) storage,
thus redistributing the datafile’s extents evenly and keeping a balanced data
load across all disks. Moreover, since ASM can accomplish the rebalancing act
while the database is active, there is virtually no impact on database
availability. I can also instruct ASM to increase the speed of a
space rebalancing operation if I know that sufficient system resources are
available, or I can tell ASM to reduce the speed of the rebalancing
operation to limit the impact on the ASM I/O subsystem.
Productivity Increases. Here is one final, intangible
benefit of ASM: an increase in my productivity as a DBA. Since I am
freed up from physical and logical space management worries, I can now
concentrate my limited and valuable time on more critical database tuning
issues, like poorly-structured SQL statements and inefficient logical storage structures
that tend to be the real culprits behind database performance issues.
The ASM Instance
The ASM instance is at the heart of Automatic Storage
Management. Rather than "reengineer the wheel," Oracle decided to
adopt the basic architecture of a normal Oracle database instance. An ASM
instance never actually opens a database; instead, it is responsible for
storing and processing the metadata that is required to make available the
files stored within the ASM storage system to non-ASM Oracle databases
(henceforth referred to as database instances).
In addition to the normal database background processes like
CKPT, DBWR, LGWR, SMON, and PMON, an ASM instance uses at least two additional
background processes to manage data storage operations. The Rebalancer
process, RBAL,
coordinates the rebalance activity for ASM disk groups, and the Actual ReBalance
processes, ARBn,
handle the actual rebalance of data extent movements. There are usually several
ARB background processes (ARB0, ARB1, and so forth). I will talk more about
disk rebalancing operations in the final article in this series.
Starting Up and Shutting Down an ASM Instance. The
ASM instance can be can be started in NOMOUNT mode, and ASM disk groups are
mounted via the MOUNT command. Similarly, disk groups can be taken offline with
the SHUTDOWN command. When an ASM instance is started and appropriate ASM disk
groups are mounted, all ASM files contained on those disk groups are made accessible
to any Oracle database. Any database instance can use the ASM disk groups as
targets for creating new ASM-managed files.
Managing an ASM Instance. Managing access to the ASM
instance is relatively straightforward. The DBA can issue the ALTER SYSTEM ENABLE RESTRICTED SESSION; command to
prevent database instances from accessing ASM disk groups while maintenance is
being performed against the ASM instance or any ASM disk groups. Alternatively,
issuing the ALTER SYSTEM DISABLE RESTRICTED
SESSION; command
re-enables access to the ASM instance.
The only way to connect to an ASM instance is via OS
authentication (i.e. SYSDBA or SYSOPER) because an ASM instance does not
have a data dictionary per se. Connecting to an ASM via a remote connection
implies, of course, that a password file must be used. Normally, the
SYSDBA privilege is granted through the use of an operating system group (on UNIX
or Linux, typically the dba group). Since by default members of that
group have SYSDBA privilege on all instances on the node, including the ASM
instance, any user that connects to the ASM instance with the SYSDBA privilege
has administrative access to all ASM disk groups in the system.
The SYSOPER privilege is also supported in ASM instances and
limits the set of allowable SQL commands to the minimum required for basic
operation of an already-configured system. A user with SYSOPER permissions can
start up or shut down an ASM instance, take disk groups online or offline,
issue a manual rebalance request, and verify a disk group. However, more
advanced command sets like those for creating new ASM disk groups, resizing ASM
disks, adding new ASM disks to existing ASM disk groups, and dropping ASM disk
groups are reserved to users with the SYSDBA privilege.
The ASM Storage Hierarchy
ASM provides a flexible storage hierarchy for managing
access to its disk files, starting with the smallest unit of storage, the Allocation
Unit (AU). All ASM disk space is partitioned in AUs of 1 MB each, and
blocks will never be split across any allocation unit.
ASM Files. Next in
the hierarchy are the actual ASM files themselves. With the exception of
trace files and operating system files, ASM can store every
type of Oracle database file, including control files, server parameter files (SPFILEs),
datafiles, tempfiles, online redo logs, archived redo logs, flashback logs, DataPump
dump sets. In addition, ASM can store all types of files managed by Recovery
Manager (RMAN), including backup sets, archived redo log image copies, datafile
image copies, and autobackups.
ASM Disk Groups. The highest level of storage unit in
an ASM instance is the ASM Disk Group. An ASM disk group can support multiple
files that belong to multiple Oracle database instances. Each ASM disk
group consists of one or more ASM Disks, and an ASM Disk belongs to one
and only one ASM Disk Group. All ASM files in the ASM Disk Group will be spread
across all the ASM disks in the ASM disk group; however, a single ASM File
will never span more than one ASM disk group.
ASM disk groups are accessible by either ASM instances or
database instances, and database instances can access the contents of ASM files
directly, communicating with an ASM instance only to get information about the
layout of these files. An ASM instance supports the addition of new ASM
disks, deletion of existing ASM disks and disk groups, and modifications
to existing ASM disk groups (e.g. adding new ASM disks).
ASM File Naming Conventions and File Name Aliasing.
ASM uses a four-layer file name to identify an ASM file. The ASM file name
consists of the ASM disk group name, the database instance name,
and the file type; the final file name component consists of three
tokens, a file type token (e.g. CF for control file) and two integer
values. The combination of these file name components insures that an ASM
file name is always unique and can be easily tied back to its owning ASM disk
group and database instance. Since the ASM file naming convention can be
cumbersome, ASM also supports file name aliases for easier referencing
of the ASM file instead of using the fully-qualified ASM file name.
Implementing ASM Storage In Oracle Databases
Every database instance that uses ASM for file storage will
also need two new processes. The Rebalancer background process (RBAL)
handles global opens of all ASM disks in the ASM Disk Groups, while the ASM
Bridge process (ASMB)
connects as a foreground process into the ASM instance when the regular
database instance starts. ASMB facilitates communication between the ASM
instance and the regular database, including handling physical file changes
like data file creation and deletion.
ASMB exchanges messages between both servers for statistics
update and instance health validation. These two processes are automatically
started by the database instance when a new Oracle file type – for example, a tablespace’s
datafile — is created on an ASM disk group. When an ASM instance mounts a disk
group, it registers the disk group and connect string with Group Services. The
database instance knows the name of the disk group, and can therefore use it to
locate connect information for the correct ASM instance.
ASM Instance Failure Scenarios. There are some
implications for any database instance that is using ASM files for storage:
When an ASM instance is shut down, any database instance that is using
that ASM instance to store ASM files will lose contact with those ASM-managed
files. Oracle 10g overcomes this potential failure scenario by allowing multiple
ASM instances to share ASM disk groups, so if one ASM instance should fail,
another ASM instance can continue to manage the changes to data stored on those
disk groups.
Next Steps
The next article in this series will focus on a simple
implementation of Automated Storage Management. It provides examples of how to
create an example ASM instance in both the Red Hat Enterprise Linux (RHEL) and
Windows NT environments as well as a demonstration of how to implement basic
ASM storage for an existing Oracle database instance.
References and Additional Reading
While there is no substitute for direct experience, reading
the manual is not a bad idea, either. I have drawn upon the following Oracle
10g documentation for the deeper technical details of this article:
B10739-01 Oracle Database
Administrator’s Guide, Chapter 12
B10743-01 Oracle Database
Concepts, Chapter 14
B10755-01 Oracle Database
Reference
B10759-01 Oracle Database
SQL Reference
