RAC Attack - Oracle Cluster Database 12c at Home

Introduction

Architecture

To better understand the RAC Installation, this picture illustrates the architecture that is implemented when following the book.

IP Addresses

In order to install a fully functional RAC, the following IP addresses are required:

2 public IPs, one for each node, for the primary OS network interface
2 public IPs, one for each node, for the Virtual IP
3 public IPs, one for each SCAN listener
2 private IPs, one for each node, for the cluster private interconnect

In the book, the public addresses belong to the network 192.168.78.0/24, and the private addresses belong to the network 172.16.100.0/24.

Technical choices

The book aims to provide instructions as simple as possible to get a basic RAC installation on your laptop. There are many, many advanced topologies and topics that are not covered here. If you are curious about technical possibilities, just ask a volunteer, he/she will be glad to explain you something more.

Hardware Requirements

This handbook will walk you through the process of creating a two-node Oracle RAC cluster on your own laptop or desktop computer.

Hardware Minimum Requirements

Most modern laptop and desktop computers should be powerful enough to run a two-node virtual RAC cluster. In a nutshell, these are the recommended minimums:

Modern CPU (most of laptops produced after 2011 should be ok)
8Gb memory
40Gb of free disk space
- 9Gb Software Staging
- 29Gb - 2VMs + 2 ASM disks
Windows 64bit (XP, Vista or 7) (Linux & Mac have been tested as well and differences to the Windows instructions are included in the book)

Software Components

Before starting you need to know what software will be installed. If attending an event, would be a good idea to download the software in advance to your laptop in order to avoid the download during the labs. The copyrighted software is not distributable so the volunteers will not be able to give you all the required software components. But organizers may have set up a proxy server to speed up downloads or provide at least the free software.

Windows 7 64 bit

This book covers Windows 7 64 bit as host even if all operating systems that can run VirtualBox 64-bit can be used. 64 bits are mandatory since Oracle 12c for Linux 32-bit is not available.

VirtualBox

This book uses VirtualBox as many Oracle specialists consider it as a mature and free virtualization solution, fully compatible with Oracle Software: https://www.virtualbox.org/. The VirtualBox version 4.3.26 and older versions from 4.2.12 up to 4.2.18 have been tested successfully with OEL6 and Oracle 12c. The latest successfully tested VirtualBox version is 5.0.8

A few virtualbox versions (4.3.14 to 22 if I remember well [1]) had serious problems on Windows. EL 6.4 to 6.7 have been tried without problems. [2]

Putty

One of the preferred SSH clients for Windows. You can download it here:

http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe

Vnc Viewer

The servers will be installed without X server, so you'll need VNC to get the graphics:

http://www.realvnc.com/download/get/1295/

Oracle Software

Oracle Enterprise Linux 6

Can be downloaded from http://edelivery.oracle.com/linux/.

Login to: http://edelivery.oracle.com/linux with your Oracle Account
On the Media Pack Search page, select:

Oracle Linux

x86 64 bit

GO

Click on: Oracle Linux Release 6 Update 4 Media Pack for x86_64 (64 bit)

Click on the download button at the first line:

Once the download of the file V37084-01.iso is finished, rename it to Oracle_Linux_6_64.iso to distinguish it easily from other Oracle files.

Oracle Grid Infrastructure and Database 12c (12.1.0.1)

Can be downloaded from http://edelivery.oracle.com/.

Login to: http://edelivery.oracle.com/ with your Oracle Account
On the Media Pack Search page, select:

Oracle Database

Linux x86-64

GO

Click on: Oracle Database 12c Release 1 (12.1.0.1.0) Media Pack for Linux x86-64

Click on the download button for the following four packs:

Oracle Database 12c Release 1 (12.1.0.1.0) for Linux x86-64 (Part 1 of 2)
Oracle Database 12c Release 1 (12.1.0.1.0) for Linux x86-64 (Part 2 of 2)
Oracle Database 12c Release 1 Grid Infrastructure (12.1.0.1.0) for Linux x86-64 (Part 1 of 2)
Oracle Database 12c Release 1 Grid Infrastructure (12.1.0.1.0) for Linux x86-64 (Part 2 of 2)

The following files are downloaded:

V38501-01_1of2.zip
V38501-01_2of2.zip
V38500-01_1of2.zip
V38500-01_2of2.zip

After uncompressing them, there will be two folders: "grid" and "database" containing the full installation.

Prepare Host

VirtualBox Setup

Run Oracle VM VirtualBox Setup.
On the welcome page, click Next:

Leave the default component selection, click Next:

Click Next:

Unless you have pending downloads or batches over the network, click Yes:

Click Install:

The installation is completed. Click Finish to open VirtualBox:

VirtualBox Network Configuration

Run VirtualBox Manager.
From the main screen, choose File > Preferences > Network.

Double click on VirtualBox Host-Only Ethernet Adapter. VirtualBox will suggest a default address of 192.168.56.1. Please update the settings as shown and click Ok twice.

IPv4 Address:

IPv4 Network Mask:

Install Linux

Create VirtualBox VM

On your computer, create a new folder racattack12c on a location of your choice that will contain all the virtual disks.

At the end of the installation the files will look like this:

In VirtualBox, from the main screen, click the New icon in the upper left hand corner.

Type in collabn1 for the Name of the VM. Choose Linux for the Type and Oracle (64 bit) for the Version and click Next.

Type 3072 in the Size field and click Next. If you only have 8GB total ram, decrease this to 2510, otherwise your laptop may slow down

Leave Create a virtual hard drive now selected and click Create.

Leave VDI (VirtualBox Disk Image) selected and click Next.

Leave Dynamically allocated selected and click Next.

Type in the full path or choose through the browse icon the Location for the disk file to be created.

It's better to use the common folder racattack12c previously created for all the virtual disks.

The file should be named collabn1.vdi.

Type in 30.00 GB in the Size field and click Create.

VirtualBox VM Settings

From the main screen, select the virtual machine collabn1 and click the Settings icon in the upper left hand corner.

Open the USB sub-menu. Uncheck the Enable USB Controller check box.

Open the Storage sub-menu. Delete the Empty disk under the IDE Controller.

Click on Remove to confirm.

Click on the Add CD/DVD Device.

Click on the Choose disk.

Locate and open the Oracle_Linux_6_64.iso file.

Open the Network sub-menu. Under the Adapter 1 tab, change the Attached to: dropdown to Host-only Adapter.

Choose the Adapter 2 tab. Check the box for Enable Network Adapter. Change the Attached to: dropdown to Internal Network and type the name rac-priv in the Name field.

Choose the Adapter 3 tab. Check the box for Enable Network Adapter. Change the Attached to: dropdown to NAT.

Select the Shared Folders tab. Click Add Shared Folder.

Enter the path where you have downloaded the Oracle installation media and enter a name for your folder.

Enter the Folder Name ""12cR1"".

Check Auto-mount and click OK.

Click on Ok to save the modifications.
Review the summary of the new virtual machine.

OS Installation

Click on Start to boot the virtual machine. If asked to select a start-up disk, choose Oracle_Linux_6_64.iso and click Start again.

The Oracle Linux install screen will automatically come up. Hit enter to begin the install.

Skip the CD testing by pressing the right arrow key and press enter.

From this point on you can use the mouse to navigate. To unlock the mouse and keyboard from the VM, hit the right Ctrl key. Click Next.

Leave English (English) selected and click Next.

Leave U.S. English selected and click Next (unless you have another preferred keyboard layout).

Leave Basic Storage Devices selected and click Next.

Click Yes, discard any data for the warning box stating The storage device below may contain data.

In the hostname field, type collabn1.racattack.
- Click Configure Network in the bottom left corner.
- Highlight System eth0 and click the Edit... button.
- Click the Connect automatically checkbox.
- Click the IPv4 Settings tab.
- Click the Method: dropdown and select Manual.
- Click Add and type 192.168.78.51 for the address.
- Leave Netmask at 24.
- In the DNS servers: box, type 192.168.78.51, 192.168.78.52.
- In the Search domains: box, type racattack.
- Click the Apply button.

Highlight System eth1 and click the Edit... button.
- Click the Connect automatically checkbox.
- Click the IPv4 Settings tab.
- Click the Method: dropdown and select Manual.
- Click Add and type 172.16.100.51.
- Change Netmask to 24.
- Click the Apply button.

Highlight System eth2 and click the Edit... button.
- Click the Connect automatically checkbox.
- Click the IPv4 Settings tab.
- Click the Method: dropdown and select Automatic (DHCP) addresses only.
- Click the Apply button.
- Click Close to close the network configuration menu and click Next.

Leave the timezone settings as they are and click Next.

Enter the Root Password as racattack and click Next.

Click Use Anyway when warned about the weak password.

Click Use All Space for the installation type and click Next.

Click Write changes to disk when warned about Writing storage configuration to disk.

Leave Basic Server as the default installation type. Click Next.

The operating system will take a while to install.

When the installation is complete, click Reboot.

The reboot no longer shows messages, just a progress bar.
- You can see the progress by hitting F8 while the server is booting.
- The system should boot to a command line login prompt.

Linux Post Installation

To connect to the new created virtual machine, create a connection in Putty for the new VM called collabn1 with the IP Address of 192.168.78.51.

Optionally, create another connection for collabn2 with the IP Address of 192.168.78.52 that will be created later as a clone of collabn1.

Open the collabn1 connection by clicking Open.

Click Yes on the Security Alert dialog box.

Enter root at the login as: prompt and racattack at the password: prompt.

Turn off and disable the firewall IPTables.

[root@collabn1 ~]# service iptables stop
iptables: Flushing firewall rules:                         [  OK  ]
iptables: Setting chains to policy ACCEPT: filter          [  OK  ]
iptables: Unloading modules:                               [  OK  ]

[root@collabn1 ~]# chkconfig iptables off

[root@collabn1 ~]# chkconfig --list iptables
iptables        0:off   1:off   2:off   3:off   4:off   5:off   6:off

Disable SELinux. Open the config file and change the SELINUX variable from enforcing to disabled.

[root@collabn1 ~]# vim /etc/selinux/config # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted

Verify that all the network interfaces are up.

[root@collabn1 ~]# ip l 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 08:00:27:4f:8a:0b brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 08:00:27:99:7e:95 brd ff:ff:ff:ff:ff:ff 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 08:00:27:e2:63:7f brd ff:ff:ff:ff:ff:ff

Add the DVD as repository.

Go to the VBOX menu => Storage => Add the DVD device to the server Device -> CD/DVD Devices -> Oracle_Linux_6_4.iso

Run the following commands:

mkdir /media/cdrom
mount /dev/cdrom /media/cdrom/
cd /etc/yum.repos.d
mkdir /tmp/old.yum.repos.d
mv * /tmp/old.yum.repos.d

cat <<EOF > ol64.repo
[OL64]
name=Oracle Linux 6.4 x86_64
baseurl=file:///media//cdrom
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY
gpgcheck=1
enabled=1
EOF

Install the database preinstall package oracle-rdbms-server-11gR2-preinstall.

This package installs all the package requirements for Oracle 11g (but will work also for 12c), configures kernel parameters, creates oracle user and groups and sets the user limits.

[root@collabn1 ~]# yum install -y oracle-rdbms-server-11gR2-preinstall
....
Transaction Summary
========================================================================
Install      21 Package(s)
Upgrade       2 Package(s)

Total download size: 43 M

....

Installed:
  oracle-rdbms-server-11gR2-preinstall.x86_64 0:1.0-8.el6

....

Complete!

Install additional RPMs that will be used to install and administer the servers.

[root@collabn1 ~]# rpm -ivh /media/cdrom/Packages/kernel-uek-devel-$(uname -r).rpm

[root@collabn1 ~]# yum install -y tigervnc-server.x86_64 xclock man parted.x86_64 unzip.x86_64 xterm lsof bind  xorg-x11-twm oracleasm-support
Install      12 Package(s)
.... 
Total download size: 22 M
....
Complete!

Uninstall NTP.

[root@collabn1 ~]# yum remove -y ntp .... Removed: ntp.x86_64 0:4.2.4p8-3.el6 .... Complete! [root@collabn1 ~]#

Unmount the DVD before continuing with the next steps.

[root@collabn1 ~]# umount /media/cdrom

Install VirtualBox Addons

Make the VirtualBox guest additions available to the OS by clicking Devices->Install Guest Additions.

Mount the virtual CD-Rom created by VirtualBox.

[root@collabn1 ~]# mount /dev/cdrom /media/cdrom
 mount: block device /dev/sr0 is write-protected, mounting read-only

Install the VirtualBox Guest Additions. The error related to the step Installing the Window System drivers is ok to ignore, it will not appear on newest VBox versions.

sh /media/cdrom/VBoxLinuxAdditions.run

Dismount the cdrom.

umount /media/cdrom

Configure Bind DNS

If BIND DNS not installed the run #yum install -y bind (after step 9 in Linux Post Installation before unmount /media/cdrom)

Enable BIND DNS to start at boot time.

chkconfig named on

Change named directory permissions.

touch /var/named/racattack

chgrp named /var/named/racattack

chmod 664 /var/named/racattack

chmod g+w /var/named

Backup the BIND configuration file.

cp /etc/named.conf /etc/named.conf.org

Change /etc/named.conf permissions.

chmod 664 /etc/named.conf

Run the following command or edit the /etc/named.conf file to change the named configuration manually.

sed -i -e 's/listen-on .*/listen-on port 53 { 192.168.78.51; };/' \
-e 's/allow-query .*/allow-query     { 192.168.78.0\/24; localhost; };\n        allow-transfer  { 192.168.78.0\/24; };/' \
-e '$azone "racattack" {\n  type master;\n  file "racattack";\n};\n\nzone "in-addr.arpa" {\n  type master;\n  file "in-addr.arpa";\n};' \
/etc/named.conf

In bold the lines that have been modified from the default.

options {
       listen-on port 53 { 192.168.78.51; };
       listen-on-v6 port 53 { ::1; };
       directory       "/var/named";
       dump-file       "/var/named/data/cache_dump.db";
       statistics-file "/var/named/data/named_stats.txt";
       memstatistics-file "/var/named/data/named_mem_stats.txt";
       allow-query     { 192.168.78.0/24; localhost; };
       allow-transfer  { 192.168.78.0/24; };
       recursion yes;

       dnssec-enable yes;
       dnssec-validation yes;
       dnssec-lookaside auto;

       /* Path to ISC DLV key */
       bindkeys-file "/etc/named.iscdlv.key";

       managed-keys-directory "/var/named/dynamic";
};

logging {
       channel default_debug {
               file "data/named.run";
               severity dynamic;
       };
};

zone "." IN {
       type hint;
       file "named.ca";
};

include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

zone "racattack" {
 type master;
 file "racattack";
};

zone "in-addr.arpa" {
 type master;
 file "in-addr.arpa";
};

Create the zone file for the racattack domain on collabn1 by running the following command:

(Copy & Paste the whole box) echo '$TTL 3H @ IN SOA collabn1 hostmaster ( 101 ; serial 1D ; refresh 1H ; retry 1W ; expire 3H ) ; minimum NS collabn1 NS collabn2 localhost A 127.0.0.1 collabn1 A 192.168.78.51 collabn1-vip A 192.168.78.61 collabn1-priv A 172.16.100.51 collabn2 A 192.168.78.52 collabn2-vip A 192.168.78.62 collabn2-priv A 172.16.100.52 collabn-cluster-scan A 192.168.78.251 collabn-cluster-scan A 192.168.78.252 collabn-cluster-scan A 192.168.78.253' \ > /var/named/racattack

Create the reverse zone file on collabn1.

(Copy & Paste the whole box) echo '$TTL 3H @ IN SOA collabn1.racattack. hostmaster.racattack. ( 101 ; serial 1D ; refresh 1H ; retry 1W ; expire 3H ) ; minimum NS collabn1.racattack. NS collabn2.racattack. 51.78.168.192 PTR collabn1.racattack. 61.78.168.192 PTR collabn1-vip.racattack. 51.100.16.172 PTR collabn1-priv.racattack. 52.78.168.192 PTR collabn2.racattack. 62.78.168.192 PTR collabn2-vip.racattack. 52.100.16.172 PTR collabn2-priv.racattack. 251.78.168.192 PTR collabn-cluster-scan.racattack. 252.78.168.192 PTR collabn-cluster-scan.racattack. 253.78.168.192 PTR collabn-cluster-scan.racattack.' \ > /var/named/in-addr.arpa

Generate the rndc.key file.

[root@collabn1 ~]# rndc-confgen -a -r /dev/urandom wrote key file "/etc/rndc.key" [root@collabn1 ~]# chgrp named /etc/rndc.key [root@collabn1 ~]# chmod g+r /etc/rndc.key [root@collabn1 ~]# ls -lrta /etc/rndc.key -rw-r----- 1 root named 77 Nov 10 09:19 /etc/rndc.key

Restart the named service.

[root@collabn1 ~]# service named restart Stopping named: [ OK ] Starting named: [ OK ]

Check that the parameter PEERDNS is set to no in /etc/sysconfig/network-scripts/ifcfg-eth2 to prevent the resolv.conf from being overwritten by the dhcp client:

DEVICE=eth2 TYPE=Ethernet UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=dhcp HWADDR=xx:xx:xx:xx:xx DEFROUTE=yes PEERDNS=no PEERROUTES=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME="System eth2" USERCTL=no note: I (Yury) found that the following two should be set to NO to => DEFROUTE=no, PEERROUTES=no

If it was set to yes previously, restart the network and verify that the file /etc/resolv.conf contains now the correct nameservers:

[root@collabn1 ~]# service network restart Shutting down interface eth0: [ OK ] Shutting down interface eth1: [ OK ] Shutting down interface eth2: [ OK ] Shutting down loopback interface: [ OK ] Bringing up loopback interface: [ OK ] Bringing up interface eth0: [ OK ] Bringing up interface eth1: [ OK ] Bringing up interface eth2: Determining IP information for eth2... done. [ OK ]

/etc/resolv.conf should contain:

[root@collabn1 ~]# cat /etc/resolv.conf ; generated by /sbin/dhclient-script nameserver 192.168.78.51 nameserver 192.168.78.52 search racattack

Check that the master DNS on collabn1 is working.

[root@collabn1 ~]# nslookup collabn-cluster-scan.racattack Server: 192.168.78.51 Address: 192.168.78.51#53 Name: collabn-cluster-scan.racattack Address: 192.168.78.251 Name: collabn-cluster-scan.racattack Address: 192.168.78.252 Name: collabn-cluster-scan.racattack Address: 192.168.78.253

Prepare Linux for Oracle

Modify the pam.d login file to use limits.

sed -i -e '/session    required     pam_selinux.so open/i\
session    required     \/lib64\/security\/pam_limits.so\
session    required     pam_limits.so' /etc/pam.d/login

Change the password for the oracle user to racattack.

passwd oracle

Create and change the ownership of the directories for Clusterware and the Database installations.

mkdir -p /u01/app

chown oracle:oinstall /u01/app

Modify the oracle user so that it belongs to the vboxsf group.

usermod -G oinstall,dba,vboxsf oracle

id oracle

if BIND DNS not installed the run #yum install -y bind (after step 9 in Linux Post Installation before unmount /media/cdrom)

Create Cluster

Create VirtualBox Shared Storage

Shutdown the server:

shutdown -h now

In the VirtualBox Manager, select the machine collabn1, click Settings -> Storage.

Select Controller SATA and click on the Add Hard Disk button:

Click on Create New Disk:

Leave VDI (VirtualBox Disk Image) selected and click Next.

For the shared storage, select Fixed size and click Next.

Change the location of the file to the folder racattack12 previously created.

Enter asm1.vdi as disk name.

Enter 5.00Gb as disk size.

Click Create.

Now the disk is physically created with a size of 5Gb, so it can take more time to complete.

Repeat the steps to create three more disks:

asm2.vdi (5.00GB)
asm3.vdi (5.00GB)
asm4.vdi (5.00GB)

Finally, four virtual disks should have been created for asm.

Click OK.

Make all the asm devices sharable: from VirtualBox Manager, click File -> Virtual Media Manager.

For each asm devices: right-click -> Modify:

Select Shareable and click OK. Do it for the four asm devices.

Start the machine collabn1.

You should see now four more disks named /dev/sd*: [root@collabn1 ~]# ls -l /dev/sd* brw-rw----. 1 root disk 8, 0 Jul 28 10:16 /dev/sda brw-rw----. 1 root disk 8, 1 Jul 28 10:16 /dev/sda1 brw-rw----. 1 root disk 8, 2 Jul 28 10:16 /dev/sda2 brw-rw----. 1 root disk 8, 16 Jul 28 10:16 /dev/sdb brw-rw----. 1 root disk 8, 32 Jul 28 10:16 /dev/sdc brw-rw----. 1 root disk 8, 48 Jul 28 10:16 /dev/sdd brw-rw----. 1 root disk 8, 64 Jul 28 10:16 /dev/sde

Configure Storage Persistent Naming

Once the new disks are visible by the server, add a primary partition on each of them with fdisk.

fdisk /dev/sdb

n

p

1

<enter>

w

Repeat the same step for sdc, sdd and sde.

When finished, all disks have at least a partition:

ls -l /dev/sd?1

Configure the option -g for the scsi_id command to expect an UUID from the shared devices.

echo "options=-g" > /etc/scsi_id.config

Test to see if scsi_id - returns data

[root@collabn1 ~]#  /sbin/scsi_id -g -u -d /dev/sdb

Prepare the file /etc/udev/rules.d/99-oracle-asmdevices.rules by running the following script:

#!/bin/bash
#file ~/createOracleAsmUdevRules.sh

i=1
 # ol6 / rhel6 / centos 6 
cmd="/sbin/scsi_id -g -u -d"
for disk in sdb sdc sdd sde ; do 
         cat <<EOF >> /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="$cmd /dev/\$parent", \
 RESULT=="`$cmd /dev/$disk`", NAME="asm-disk$i", OWNER="oracle", GROUP="dba", MODE="0660"
EOF
         i=$(($i+1)) 
done

Once completed, the file should look like the following (except for the scsi UUID).

Note: The lines beginning with "KERNEL" should not be splitted.

cat /etc/udev/rules.d/99-oracle-asmdevices.rules

Reload the udev rules and restart udev:

/sbin/partprobe /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

/sbin/udevadm test /block/sdb/sdb1

/sbin/udevadm test /block/sdc/sdc1

/sbin/udevadm test /block/sdd/sdd1

/sbin/udevadm test /block/sde/sde1

/sbin/udevadm control --reload-rules

/sbin/start_udev

/etc/init.d/oracleasm scandisks

/etc/init.d/oracleasm listdisks

The new disks are ready for use:

ls -l /dev/asm*

Clone VirtualBox VM

Shutdown the VM.

shutdown -h now

Clone the disk collabn1.vdi: from VirtualBox Manager, click File -> Virtual Media Manager.

Right-click on the disk collabn1.vdi and click Copy...

Leave the file collabn1.vdi, click Next.

Leave VDI as file type. Click Next.

Leave Dynamically allocated. Click Next.

Type in the full path or choose through the browse icon the Location for the disk file to be created.

Again, it is better to use the common folder racattack12c previously created for all the virtual disks.

The file should be named collabn2.vdi.

Click Copy.

Click Close to exit from the Virtual Media Manager.

Create Second VirtualBox VM

In VirtualBox, click New icon in the upper left corner to create the new VM.
Type in collabn2 for the Name of the VM. Choose Linux for the Type and Oracle (64 bit) for the Version and click Next.

Type 3072 in the Size field and click Next. If you only have 8GB total ram, decrease this to 2510, otherwise your laptop may slow down

Select Use an existing virtual hard drive file and use the folder icon to locate the file collabn2.vdi previously created.

Click Create to complete the creation of the second VM.

There are now two virtual machines; collabn2 being a clone of collabn1.

From the main screen, select the virtual machine collabn2 and click the Settings icon in the upper left hand corner.

Open the USB sub-menu. Uncheck the Enable USB Controller check box.

Open the Network sub-menu. Under the Adapter 1 tab, change the Attached to: dropdown to Host-only Adapter.

Choose the Adapter 2 tab. Check the box for Enable Network Adapter. Change the Attached to: dropdown to Internal Network and type the name rac-priv in the Name field.

Choose the Adapter 3 tab. Check the box for Enable Network Adapter. Change the Attached to: dropdown to NAT.

Select the Shared Folders tab. Click Add Shared Folder.

Enter the path where you've downloaded the Oracle installation media and enter a name for your folder.

Check Auto-mount and click OK.

Open the Storage sub-menu.

Click on Controller SATA and then on Add Disk:

Click on Choose existing disk:

Select the disk asm1.vdi and click Open.

Repeat the operation to add all remaining asm disks.

Click on Ok to save the modifications.
Review the summary of the new virtual machine.

Start the collabn2 VM.

Configure Second Linux VM

The network won't be available yet because we reinitialized the network adapters so the following changes will need to be done in the VM. Log in as root/racattack. Type the command system-config-network. Device configuration is already highlighted. Press Enter.

eth0 (eth0) - Ethernet is already highlighted. Press Enter.

Go down to the Static IP line. Change the IP to 192.168.78.52. Go down to Ok and press Enter.

Select eth1 (eth1) - Ethernet and press Enter. Go down to the Static IP line. Change the IP to 172.16.100.52. Go down to Ok and press Enter.

Tab down to Save and press Enter.

Arrow down to DNS configuration and press Enter.

In the Hostname field, change to collabn2.racattack. Tab down to Ok and press Enter.

Tab down to Save&Quit and press Enter.

Check that the parameter PEERDNS is set to no in /etc/sysconfig/network-scripts/ifcfg-eth2 to prevent the resolv.conf from being overwritten by the dhcp client:

PEERDNS=no

Remove the udev network rules file. It will be regenerated on the next reboot with the new MAC addresses.

rm -f /etc/udev/rules.d/70-persistent-net.rules

Remove the HWADDR and UUID lines in the network adapter configuration files.

sed -i -e '/HWADDR/d' -e '/UUID/d' /etc/sysconfig/network-scripts/ifcfg-eth[0-2]

Reboot the collabn2 VM by typing reboot.

Start the collabn1 VM as well and start Putty sessions for both.

Verify that all the network interfaces are up.

ip l

Verify that all the ASM devices are visible.

ls -l /dev/asm*

Complete DNS Setup

Open a session on collabn2.
Stop the DNS service.

service named stop

Remove the actual DNS files.

rm -f /var/named/racattack /var/named/in-addr.arpa

Modify the file /etc/named.conf by using the following command:

http://en.wikibooks.org/wiki/RAC_Attack_-_Oracle_Cluster_Database_at_Home/RAC_Attack_12c/Complete_DNS_Setup

sed -i -e 's/listen-on .*/listen-on port 53 { 192.168.78.52; };/' \
-e 's/type master;/type slave;\n masters  {192.168.78.51; };/' \
/etc/named.conf

so that at the end it looks like this one:

options {
       listen-on port 53 { 192.168.78.52; };
       listen-on-v6 port 53 { ::1; };
       directory       "/var/named";
       dump-file       "/var/named/data/cache_dump.db";
       statistics-file "/var/named/data/named_stats.txt";
       memstatistics-file "/var/named/data/named_mem_stats.txt";
       allow-query     { 192.168.78.0/24; localhost; };
       allow-transfer  { 192.168.78.0/24; };
       recursion yes;

       dnssec-enable yes;
       dnssec-validation yes;
       dnssec-lookaside auto;

       /* Path to ISC DLV key */
       bindkeys-file "/etc/named.iscdlv.key";

       managed-keys-directory "/var/named/dynamic";
};

logging {
       channel default_debug {
               file "data/named.run";
               severity dynamic;
       };
};

zone "." IN {
       type hint;
       file "named.ca";
};

include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

zone "racattack" {
 type slave;
 masters  { 192.168.78.51; };
 file "racattack";
};

zone "in-addr.arpa" {
 type slave;
 masters  { 192.168.78.51; };
 file "in-addr.arpa";
};

Start the named service.

service named start

Check that both the master on collabn1 and slave on collabn2 DNS servers are working.

Run all the following statements to verify the correct configuration of both DNS servers:

http://en.wikibooks.org/wiki/RAC_Attack_-_Oracle_Cluster_Database_at_Home/RAC_Attack_12c/Linux_Post_Installation_No_Internet

[root@collabn2 ~]# dig @collabn1 collabn1.racattack
[root@collabn2 ~]# dig @collabn1 collabn2.racattack
[root@collabn2 ~]# dig @collabn1 collabn1-vip.racattack
[root@collabn2 ~]# dig @collabn1 collabn2-vip.racattack
[root@collabn2 ~]# dig @collabn1 collabn1-priv.racattack
[root@collabn2 ~]# dig @collabn1 collabn2-priv.racattack
[root@collabn2 ~]# dig @collabn1 collabn-cluster-scan.racattack

[root@collabn2 ~]# dig @collabn2 collabn1.racattack
[root@collabn2 ~]# dig @collabn2 collabn2.racattack
[root@collabn2 ~]# dig @collabn2 collabn1-vip.racattack
[root@collabn2 ~]# dig @collabn2 collabn2-vip.racattack
[root@collabn2 ~]# dig @collabn2 collabn1-priv.racattack
[root@collabn2 ~]# dig @collabn2 collabn2-priv.racattack
[root@collabn2 ~]# dig @collabn2 collabn-cluster-scan.racattack

VNC Server Setup

Configure VNC Server with the oracle account (passwords won't be displayed):

su - oracle

vncserver :1

racattack

Open a vncviewer on your laptop and fill the ip address of collabn1 followed by :1.

Enter the password racattack when prompted:

The graphical interface is ready for the Grid Infrastructure Installation.

Install Grid Infrastructure

Prepare for GI Install

Verify that the VirtualBox Shared folder containing the installation media is mounted correctly and that Oracle has access to it:

su - oracle

df -k

12cR1 976728060 863308936 113419124 89% /media/sf_12cR1

ls -l /media/sf_12cR1

You can decide to unzip it from your laptop (using Windows native functionalities, WinZip, 7-Zip or other tools or directly from the server.

cd /media/sf_12cR1

unzip linuxamd64_12c_database_1of2.zip

unzip linuxamd64_12c_database_2of2.zip

unzip linuxamd64_12c_grid_1of2.zip

unzip linuxamd64_12c_grid_2of2.zip

Once the package decompression is completed, you'll end up with two directories, grid and database containing the installation files.

ls -l

drwxrwx---. 1 root vboxsf 0 Jul 3 06:38 database

drwxrwx---. 1 root vboxsf 0 Jun 10 08:15 grid

Install the cvuqdisk package as root:

rpm -Uvh /media/sf_12cR1/grid/rpm/cvuqdisk-1.0.9-1.rpm

Do the same on node collabn2:

rpm -Uvh /media/sf_12cR1/grid/rpm/cvuqdisk-1.0.9-1.rpm

Install Grid Infrastructure

From the VNC remote session, run the installation of Grid Infrastructure as oracle:

/media/sf_12cR1/grid/runInstaller

The Installation starts displaying a Splash Screen:

Leave Skip software updates and click Next:

Leave Install and Configure Oracle Grid Infrastructure for a Cluster and click Next:

Leave Configure a Standard Cluster and click Next:

Leave Typical Installation and click Next:

In the SCAN Name box, enter collabn-cluster-scan.racattack

In the central box, only the first node is displayed. Click Add... and enter the network names of the second node:

Public Hostname: collabn2.racattack

Virtual Hostname: collabn2-vip.racattack

Click SSH connectivity...

Enter racattack as the password of oracle user and click Setup.

The setup of SSH equivalency is initiated.

Click OK when it's completed.

Click on Identify Network Interfaces...

Check that the correct addresses are set:

eth0 is marked as Public

eth1 is marked as Private

eth2 is marked as Do Not Use

Click OK to close the Identify Network Interfaces window. Finally, click Next

Some prerequisite validation is done at this step. It can require some minutes to complete.

Since the SCAN Name chosen is too long (more than 15 chars.) Oracle proposes a different name as the Cluster Name. Click Yes.

Leave the default Oracle Base and Software Location.

In the Cluster Registry Storage Type combo box, select Oracle Automatic Storage Management Enter racattack as the SYSASM password. Select dba as OSASM group. Click Next.

racattack is a weak password and Oracle raise an alert. Click Yes to continue:

Leave DATA as Disk Group Name.

Select External redundancy (it is safe to do this in a RAC Attack event!). The Candidate Disks list is empty. Click on Change Discovery Path... Enter /dev/asm* in the Disk Discovery Path text field. Click Ok.

Check two disks that will be immediately allocated to the new disk group.

Click Next

Leave the default Inventory Directory. Click Next.

Check Automatically run configuration script and then Use "root" user credentials.

Enter racattack as the password of root and click Next.

It takes some time to complete the prerequisite checks.

If everything is correct, you will notice two checks that have failed:

Physical Memory
Device Checks for ASM

Check "Ignore All" and click Next.

Click Yes to confirm that you want to skip some prerequisites.

The summary page appears. Optionally, click Save Response File and choose a location to have the response file saved for this installation session.

Review the summary and click Install to start the installation.

The installation starts. It may take more than an hour, depending on your laptop hardware.

After a while, the installation asks a confirmation before running some scripts as root. Click Yes to continue.

The installation continues, again, it may take longtime to complete. Once it's finished, click on Close to exit.

Check Cluster Status After GI Install

Once your Grid Infrastructure installation is finished, you can get the status of the cluster components:

. oraenv

+ASM1

crsctl stat res -t

Optional step: in order to increase the resistance of your nodes to the huge latency of a Virtualbox environment, you can increase the timeout of CRS before it causes a fencing (restart) of the node.

You'll need to stop the second node while applying the configuration to the first node.

ssh collabn2

su -

. oraenv

+ASM2

crsctl stop crs

exit

su -

. oraenv

+ASM1

crsctl get css misscount

crsctl set css misscount 90

crsctl get css disktimeout

crsctl set css disktimeout 600

ssh collabn2

. oraenv

+ASM2

crsctl start crs

The start command returns the prompt in few seconds. However it can take minutes before the whole stack is started entirely.

ASM Configuration

From a VNC session, run the ASM Configuration Assistant:

. oraenv

+ASM1

asmca

The asmca opens on the Disk Groups tab, showing the DATA diskgroup created by the installer. Click on Create.

Enter FRA as the disk group name.

Choose external (none) redundancy.

Check one disk to create a 5Gb diskgroup.

Click OK.

The diskgroup is being created.

The diskgroup is created.

Click on the Volumes tab, then on Create.

Enter SHARED as the volume name.

Choose the diskgroup DATA.

The size will be irrelevant in our labs, so enter 1.

Click OK.

The volume is created.

Select the ASM Cluster File Systems tab.

Click Create.

Select Cluster File System.

Mountpoint: /shared

Check Automount.

User Name: oracle

Group Name: dba

Volume: SHARED

Click OK.

To complete the filesystem creation, run this script as root.

/u01/app/oracle/cfgtoollogs/asmca/scripts/acfs_script.sh

Finally, the filesystem is mounted on both nodes.

df -k

/dev/asm/shared-336 1048576 80176 968400 8% /shared

Install Database Software

In a VNC session, run the installer as oracle user.

/media/sf_12cR1/database/runInstaller

The splash screen appears.

Uncheck I wish to receive security updates via My Oracle Support and click Next.

Click Yes to confirm that you do not want to be informed about security updates.

Select Skip software updates and click Next.

Select Install database software only and click Next.

Select Oracle Real Application Clusters database installation and click Next.

Verify that all nodes are selected and click Next.

Click Next to accept the default language selection.

Select Enterprise Edition and click Next.

If you have previously created the ACFS 1Gb filesystem, you will need to specify another location now.

Oracle Base: /u01/app/oracle

Software location: /u01/app/oracle/product/12.1.0/dbhome_1

Click Next.

Leave dba for all groups except for the OSOPER group (leave it empty) and click Next.

The installer checks all prerequisites.

Review the installation summary, optionally save a response file, and click Install.

The installation can take half an hour on common laptop configurations.

right before completing, the installer ask to run a script on both nodes as root. Do not click OK.

Execute the script on both nodes as root.
Click OK on the previous window, then click Close to exit the Installer.

Create Database

Run the Database Configuration Assistant:

/u01/app/oracle/product/12.1.0/dbhome_1/bin/dbca

The splash screen appears.

Select Create Database and click Next.

Select Create a database with default configuration.

Global Database Name: RAC.racattack

Storage Type: Automatic Storage Management (ASM)

Database Files Location: +DATA

Fast Recovery Area: +FRA

Database Character Set: AL32UTF8 - Unicode UTF-8 Universal character set

Administrative password: racattack

Check Create As Container Database

Pluggable Database Name: PDB (*)

Click Next.

A warning is shown because the password is weak. Click Yes to continue.
The prerequisite checks may fail due to space requirements. It's safe to ignore by checking Ignore all.

Click Next.

Review the summary page and click Finish.

The database creation is finished. Click Exit, then Close.

RAC Attack - Oracle Cluster Database at Home/RAC Attack 12c/Print Book

Introduction

Architecture

IP Addresses

Technical choices

Hardware Requirements

Hardware Minimum Requirements

Software Components

Windows 7 64 bit

VirtualBox

Putty

Vnc Viewer

Oracle Software

Oracle Enterprise Linux 6

Oracle Grid Infrastructure and Database 12c (12.1.0.1)

Prepare Host

VirtualBox Setup

VirtualBox Network Configuration

Install Linux

Create VirtualBox VM

VirtualBox VM Settings

OS Installation

Linux Post Installation

Install VirtualBox Addons

Configure Bind DNS

Prepare Linux for Oracle

Create Cluster

Create VirtualBox Shared Storage

Configure Storage Persistent Naming

Clone VirtualBox VM

Create Second VirtualBox VM

Configure Second Linux VM

Complete DNS Setup

VNC Server Setup

Install Grid Infrastructure

Prepare for GI Install

Install Grid Infrastructure

Check Cluster Status After GI Install

ASM Configuration

Install Database Software

Create Database