Oracle DBA in an Oracle Applications World - Part 2
May 28, 2008
In a prior article about E-Business Suite, I provided an overview of the cloning process, which is essentially three steps: gather, copy and implement. Virtually all of the documentation regarding cloning assumes you already have replicants in place. Replicants you ask? Like the ones in Blade Runner? The ones who are visually indistinguishable from [the production environment] are used for dangerous and degrading work in [your test, dev and QA environments]? Well, almost the same. What I mean here is that your test/dev/QA machines are virtually indistinguishable from those used in production. But, how do you get there?
What follows next is a simple but effective means of replicating your production environment for the first time on a new machine. First, lets take care of some semantics. With respect to the non-production servers, regardless of their purpose (test, development or QA), lets refer to them generically as the clone server(s) or clone(s).
Assuming the source and clone servers are physically similar, the basic requirement to get started is to install/match the operating system version or kernel, create the same mount points/partitions on the clones, and create the same (key) user accounts of oracle and applmgr (plus any other people accounts). Networking/IP address/ports are other setup areas that need to be addressed, which really isnt any different than adding a new server into your environment.
But, you ask, dont I have to install Oracle on the clone database server? You can if you want to, but it is easier to just copy the Oracle RDBMS and Apps software installation footprints from the sources to the clones. This isnt just ORALCE_BASE and below; youll need to get pointer files such as oratab and oraInst.loc. With a direct copy, theres no need to recompile the binaries.
Relative to the clones, the paths for the software installations are more than likely to be local. Special files (datafiles, redo logs, controlfiles, archived redo logs) may not be. You have to find a home or new set of paths for these files. If we assume new locations (because the source files are on a network file system or SAN) are required, what is an easy way to make the name/path changes? One way is to use file name convert parameters within the init.ora file. Another way is to (and you have to do this anyway) use the newly created controlfile, specifically, the SQL script used to create a new controlfile.
On the source database, backup the controlfile to a usable version/file that can be edited. Youll be using the new SID name in that file, and that NEWSID value is what ties the log and datafiles to the controlfile which in turn is what binds the controlfile(s) to the init.ora file.
The clones can have different host names. Reconfiguring the cloned installations to recognize the new hostnames takes place during the implementation. The database name can (and more than likely) be different as well. Dont forget to edit the init.ora file for other changes such as archive log and dump locations.
To provide some perspective on cloning the database tier, look at it this way: what if your production database server needs to be replaced, or you were just cloning the database (forget about this being for Oracle Apps, for the moment) onto a different server for development or QA users? Its the same thing here. Recover the database via using a new init.ora file and re-create the controlfiles. If the copy of the source was taken while the database was open, and assuming archivelog mode, perform recovery on the clone.
Applying the configuration scripts
Running the RapidClone Perl script starts an interview process. Default/current values are shown and you can accept or reject the suggested value. If you reject, you will be prompted to enter new/updated information. One thing to keep in mind is the difference between a SID or server named config file versus a SID or server named file system path. By copying files over from the sources, you will inherit path names reflecting the source. What matters is that configuration files are named reflecting the new SID or server/host name.
If you make a mistake, or if the script encounters an error, simply re-run the script. Sample output (it looks the same on the apps tier and database tier) from the script is shown below, including an error. Key words to look for (replace with your values) are OLD and NEWSID, the_password, domain name, and host/server name.
:oldappsvr:/u003/app/applmgr/OLDSID/comntop/clone/bin>perl adcfgclone.pl appsTier Enter the APPS password [APPS]: the_password First Creating a new context file for the cloned system. The program is going to ask you for information about the new system: Provide the values required for creation of the new APPL_TOP Context file. Do you want to use a virtual hostname for the target node (y/n) [n] ?: Target system database SID [OLDSID]:NEWSID Target system domain name [xyz.com]: Target system database server node [oldsvr]:newsvr Target system database domain name [xyz.com]: Does the target system have more than one application tier server node (y/n) [y] ?:y Is the target system APPL_TOP divided into multiple mount points (y/n) [n] ?: Target system APPL_TOP mount point [/u003/app/applmgr/OLDSID/appltop]: Target system COMMON_TOP directory [/u003/app/applmgr/OLDSID/comntop]: Target system 8.0.6 ORACLE_HOME directory [/u003/app/applmgr/OLDSID/8.0.6]: Target system iAS ORACLE_HOME directory [/u003/app/applmgr/OLDSID/iAS]: Do you want to preserve the Display set to oldappssvr:0.0 (y/n) [y] ?:n Target system Display [newappsvr:0.0]: Location of the JDK on the target system [/usr/j2se/j2sdk1.4.2_13]:/usr/jdk/jdk1.5.0_15 Target system JRE_TOP [/usr/j2se/j2sdk1.4.2_13]:/usr/jdk/jdk1.5.0_15 Do you want to preserve the port values from the source system on the target system (y/n) [y] ?: Web Listener port is 8002 Complete port information available at /u003/app/applmgr/OLDSID/appltop/admin/out/NEWSID_newappsvr/portpool.lst UTL_FILE_DIR on database tier consists of the following directories. 1. /usr/tmp 2. /usr/tmp 3. /u002/app/oracle/product/OLDSID/9.2.0/appsutil/outbound/NEWSID_newsvr 4. /usr/tmp Choose a value which will be set as APPLPTMP value on the target node : Creating the new APPL_TOP Context file from : /u003/app/applmgr/OLDSID/appltop/ad/11.5.0/admin/template/adxmlctx.tmp The new APPL_TOP context file has been created : /u003/app/applmgr/OLDSID/appltop/admin/NEWSID_oldappsvr.xml Log file located at /u003/app/applmgr/OLDSID/comntop/clone/bin/CloneContext_04180252.log Running Rapid Clone with command: perl adclone.pl java=../jre/1.3.1 mode=apply stage=/u003/app/applmgr/OLDSID/comntop/clone/bin/.. component=appsTier method=CUSTOM appctxtg=/u003/app/applmgr/OLDSID/appltop/admin/NEWSID_oldappsvr.xml showProgress contextValidated=true The 806 installer uses the oratab file. The file oratab needs to exist at: /var/opt/oracle/oratab Rapid Clone will use the oraInst.loc file. The file oraInst.loc needs to exist at: /var/opt/oracle/oraInst.loc Please run the script /tmp/orainstRoot.sh as root Restart adclone.pl after performing required corrections. ERROR: Failed to execute adclone.pl Please check logfile.
So, which tier was being implemented via the adclone.pl script? Several ways to tell, but one of the easiest is based on the questions regarding these four locations:
Is the target system APPL_TOP divided into multiple mount points (y/n) [n] ?: Target system APPL_TOP mount point [/u003/app/applmgr/OLDSID/appltop]: Target system COMMON_TOP directory [/u003/app/applmgr/OLDSID/comntop]: Target system 8.0.6 ORACLE_HOME directory [/u003/app/applmgr/OLDSID/8.0.6]: Target system iAS ORACLE_HOME directory [/u003/app/applmgr/OLDSID/iAS]:
Had it been the database tier, the questions would have been like:
Number of DATA_TOP's on the target system : Target system DATA_TOP 1 [/u004/app/oradata/OLDSID]: Target system DATA_TOP 2 [/u005/app/oradata/OLDSID]: Target system DATA_TOP 3 [/u008/app/oradata/OLDSID]: Target system DATA_TOP 4 [/u009/app/oradata/OLDSID]:
Additionally, the database tier questions in the interview leave no doubt as to which tier youre dealing with. I left the error in place to show the effect of what happens when ancillary files (those not normally under ORACLE_BASE) are missing.
What if some of the servers (in the Apps context, not the machine context) are split across the Apps server (machine or node) and the database server? Everything on the Apps node is run as applmgr, and everything run on the database node for the database component is run as oracle. You will need to have applmgr as an account (and owner) of the applications tier components installed on this node. In other words, you run the adclone script twice, once for the oracle user/owner, and once for the applmgr user/owner.
One way to see if the apps tier is split across physical nodes is to examine the state of processes at the end of the log file generated by adstrtal.sh script.
The report format is: <Service> <Script> <Status> Oracle Apache Server NEWSID_newappsvr adapcctl.sh Started OracleTNSListener80APPS_NEWSID_newappsvr adalnctl.sh Started OracleFormsServer-Forms60NEWSID_newappsvr adfrmctl.sh Started Oracle Metrics Client NEWSID_newappsvr adfmcctl.sh Started Oracle Metrics Server NEWSID_newappsvr adfmsctl.sh Started Oracle Fulfillment Server NEWSID_newappsvr jtffmctl.sh Started Oracle Discoverer services NEWSID_newappsvr addisctl.sh Started Oracle Restricted Apache Server NEWSID_newappsvr adaprstctl.sh Disabled Oracle Apache Server NEWSID_newappsvr for PL/SQL adapcctl.sh Disabled Oracle TCF SocketServer NEWSID_newappsvr adtcfctl.sh Disabled OracleConcMgrNEWSID_newappsvr adcmctl.sh Disabled OracleReportServer-Rep60_NEWSID adrepctl.sh Disabled Oracle ICSM NEWSID_newappsvr ieoicsm.sh Disabled Oracle iProcurement Bulk Loader NEWSID_newappsvr icxblkctl.sh Disabled ServiceControl is exiting with status
Where is the Concurrent Manager running? Given that it is disabled (see OracleConcMgrNEWSID_newappsvr), probably not on the server named newappsvr. It would be reasonable to assume that it is running on the database server. The corresponding log file there shows:
The report format is: <Service> <Script> <Status> OracleTNSListener80APPS_NEWSID_newsvr adalnctl.sh Started OracleConcMgrNEWSID_newsvr adcmctl.sh Started OracleReportServer-Rep60_OCVREP adrepctl.sh Started Oracle Apache Server NEWSID_newsvr adapcctl.sh Disabled Oracle Restricted Apache Server NEWSID_newsvr adaprstctl.sh Disabled Oracle Apache Server NEWSID_newsvr for PL/SQL adapcctl.sh Disabled Oracle TCF SocketServer NEWSID_newsvr adtcfctl.sh Disabled OracleFormsServer-Forms60NEWSID_newsvr adfrmctl.sh Disabled Oracle Metrics Client NEWSID_newsvr adfmcctl.sh Disabled Oracle Metrics Server NEWSID_newsvr adfmsctl.sh Disabled Oracle ICSM NEWSID_newsvr ieoicsm.sh Disabled Oracle Fulfillment Server NEWSID_newsvr jtffmctl.sh Disabled Oracle iProcurement Bulk Loader NEWSID_newsvr icxblkctl.sh Disabled Oracle Discoverer services NEWSID_newsvr addisctl.sh Disabled ServiceControl is exiting with status 0
The output also reflects the scripts used to start/stop each component individually.
One other key piece of information concerns ports, and by extension, URLs. The URL used to get to the Apps logon page will be the same as what is in production, except for one item, and that is the host or server name. If you kept the same port assignments, just update the base URL to reflect the new server name. Port assignments are recorded in the portpool.lst file under $APPL_TOP/admin/out/NEWSID_newappsvr. Port 8002 was used in production, and 8002 will be used on the clone.
[applmgr newappsvr] $ more portpool.lst Web Listener Port : 8002 Database Port : 1523 RPC Port : 1628 Reports Port : 7002 OPROC Manager Port : 8102 Web PLSQL Port : 8202 Servlet Port : 8800 Forms Listener Port : 9002 Metrics Server Data Port : 9102 Metrics Server Req. Port : 9202 JTF Fulfillment Server Port : 9302 Map Viewer Servlet Port : 9802 OEM Web Utility Port : 10002 VisiBroker OrbServer Agent Port : 10102 MSCA Server Port : 10202 MSCA Dispatcher Port : 10302 OACORE Servlet Port Range : 16020-16029 Discoverer Servlet Port Range : 17020-17029 Forms Servlet Port Range : 18020-18039 TCF port : -1 XMLSVCS Servlet Port Range : 19020-19029 Java Object Cache Port : 12345
Cloning an Oracle Apps environment using RapidClone is relatively easy once the replicants have been created/established. Assuming no changes to the apps tier (e.g., new forms), a subsequent clone may only consist of cloning the database. The tricky part is getting the very first clone in place, and with that, you have the overhead of performing steps with more complexity. But, as I hope I have done in this article, once the overall process has been explained, you can see that making the first clone isnt that hard to do.