RAC Attack - Oracle Cluster Database at Home/RAC Attack 12c/Clusterware Callouts
Prev: Create Database
Explore RAC (e)
The goal of this lab is to demonstrate Oracle Fast Application Notification (FAN) Callouts. In versions prior to 11g, these were also known as Oracle Clusterware Callouts.
This feature is a relatively little-known capability for Oracle Clusterware to fire a script (or a whole directory full of them) to perform whatever tasks you may want performed when a cluster-wide event happens.
For more information, consult the documentation here: http://docs.oracle.com/cd/E16655_01/rac.121/e17887/hafeats.htm#BABHAJAH
For this exercise, we’ll configure some FAN callout scripts on each node and then trigger various cluster events to see how each one triggers the callout script.
- Start with a normal, running cluster with both nodes up and running.
- From a shell prompt (logged in as oracle) on each server, navigate to /u01/app/12.1.0/grid/racg/usrco. Create file there called callout1.sh using vi (or your favorite editor). The contents of the file should be this: #!/bin/ksh umask 022 FAN_LOGFILE=/tmp/`hostname`_uptime.log echo $* "reported="`date` >> $FAN_LOGFILE &
- Make sure that the permissions on the file are set to 755 using the following command: [oracle@<node_name> ~]$ chmod 755 /u01/app/12.1.0/grid/racg/usrco/callout1.sh
- Monitor the logfiles for clusterware on each node. On each node, start a new window and run the following command: [oracle@<node_name> ~]$ tail -f /u01/app/12.1.0/grid/log/`hostname -s`/crsd/crsd.log
- Next, we need to trigger an event that will cause the callout to fire. One such event is node shutdown. Shutdown the clusterware on node collabn2. [root@collabn2 ~]# crsctl stop crs CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'collabn2' CRS-2673: Attempting to stop 'ora.crsd' on 'collabn2' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'colleabn2' CRS-2673: Attempting to stop 'ora.rac.db' on 'collabn2' CRS-2673: Attempting to stop 'ora.DATA.SHARED.advm' on 'collabn2' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'collabn2' ... CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'collabn2' has completed CRS-4133: Oracle High Availability Services has been stopped.
- Following this command, watch the logfiles you began monitoring in step 2 above. Because we set long timeouts on our test cluster, you might have to wait for a few minutes before you see anything.
Note the use of backticks around the hostname and date commands.
- You should eventually observe entries noting that the node has failed and shortly following that, you should observe an entry placed in the /tmp/<hostname>_uptime.log file indicating that the node is down.
- Note which members run the clusterware callout script. (A surviving member could run commands to notify clients and/or application servers that one of the cluster nodes has died.)
One popular use for clusterware callouts is to notify administrators (possibly via email) that a cluster event has occurred. You may use the arguments to the script (you’ll see the arguments in the logfile we’ve created) to conditionally perform notification as well. For example, you may not want to notify anyone unless a node crashes unexpectedly. By testing some of these arguments, you may be able to send notifications only when desired.