Add SCAN VIPs to Oracle RAC with zero “Listener” Downtime

Add SCAN VIPs to Oracle RAC with zero “Listener” Downtime

Intro

Infrastructure teams are usually keener to maintain network stability rather than altering their existing configuration every other day. This means that once the application and database hosts are provisioned and running, the DBA won’t witness a topology change unless a migration of the servers/applications is in the pipes. But for highly available environments like Oracle RAC, the DBA’s help can still be required when the network changes impact the related Clusterware resources.

In my case, my client had a RAC One Node database in an ODA with only one scan IP address (don’t ask me why) which made the whole environment a bit too shaky. Hence the DNS configuration had to add another IP for SCAN entry

What would be the downtime in order to update RAC with a newly added scan VIP in the DNS server? 

The short answer is “0”, but I needed to refresh my memory, so I checked a few resources and articles online but all of them were mentioning a mandatory stop/start for both scan and scan listener during the change.

My guts were telling otherwise as I remember that some srvctl modify commands could apply changes online. In this particular case, as long as you are not changing the IP address/port of an existing SCAN[listener] there is no need to stop anything.

To see how it works, I did a test in a two-node RAC Cluster lab but it works the same on a RAC-One node.

 

Check the 3rd VIP added in the DNS server

$ nslookup  ecl-oda-scan  Server: 10.10.2.2
 Address: 10.10.2.2#53 
Name: ecl-oda-scan.evilcorp.com
Address: 10.10.30.49, Address: 10.10.30.48,Address: 10.10.30.50 <== added IP

Check the existing Scan Configuration in the Cluster

$ srvctl config scan
SCAN name: ecl-oda-scan, Network: 1
Subnet IPv4: 10.10.30.0/255.255.255.0/bond0, static
SCAN 1 IPv4 VIP: 10.10.30.49
SCAN VIP is enabled.
SCAN 2 IPv4 VIP: 10.10.30.48
SCAN VIP is enabled

$ srvctl status scan_listener
 SCAN listener LISTENER_SCAN1 is running on node ecl-oda-0
 SCAN listener LISTENER_SCAN2 is running on node ecl-oda-1

Check the existing Scan Listener Configuration in the Cluster

$ srvctl config  scan_listener
SCAN Listeners for network 1:
Registration invited nodes:
Registration invited subnets:
Endpoints: TCP:1521
SCAN Listener LISTENER_SCAN1 exists and enabled
SCAN Listener LISTENER_SCAN2 exists and enabled

$ srvctl status scan_listener
 SCAN listener LISTENER_SCAN1 is running on node ecl-oda-0
 SCAN listener LISTENER_SCAN2 is running on node ecl-oda-1

 Note:  Now we can start updating the change at the Clusterware Level.

 

1. Run srvctl modify scan command to create additional SCAN VIP resources

There’s no need to stop/restart the existing scan resources since we just added a new scan VIP in the DNS zones  

$ srvctl modify scan -scanname ecl-oda-scan

$ srvctl config scan
 SCAN name: ecl-oda-scan, Network: 1
 Subnet IPv4: 10.10.30.0/255.255.255.0/bond0, static
 SCAN 1 IPv4 VIP: 10.10.30.49
 SCAN VIP is enabled.
 SCAN 2 IPv4 VIP: 10.10.30.48
 SCAN VIP is enabled
SCAN 3 IPv4 VIP: 10.10.30.50
SCAN VIP is enabled.

$ srvctl status  scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node ecl-oda-0
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node ecl-oda-1
SCAN VIP scan3 is enabled
SCAN VIP scan3 is not running

 

2. Create Oracle Clusterware resource for the additional SCAN listener to go with the additional SCAN VIP  

Here again, no need to stop any of the existing scan listeners first

$ srvctl modify scan_listener -update

$ srvctl status  scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node ecl-oda-0
SCAN Listener LISTENER_SCAN2 is enabled
SCAN listener LISTENER_SCAN2 is running on node ecl-oda-1
SCAN Listener LISTENER_SCAN3 is enabled
SCAN listener LISTENER_SCAN3 is not running

 

3. Start the third scan resource added to the Clusterware 

The trick is to only start the added scan by using the scan number as an argument

Syntax (19c)  srvctl start scan [-scannumber ordinal_number] [-node node_name]

$ srvctl start scan -scannumber 3
SCAN VIP scan3 is running on node ecl-oda-0

$ srvctl modify scan_listener -update

 

4. Start the third scan Listener Resource added to the Clusterware 

$ srvctl start scan_listener -i 3
SCAN VIP scan3 is running on node ecl-oda-0

 

5. Confirm that the changes are now effective

$ srvctl status  scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node ecl-oda-0
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node ecl-oda-1
SCAN VIP scan3 is enabled SCAN VIP scan3 is running on node ecl-oda-0

$ srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node ecl-oda-0
SCAN Listener LISTENER_SCAN2 is enabled
SCAN listener LISTENER_SCAN2 is running on node ecl-oda-1
SCAN Listener LISTENER_SCAN3 is enabled SCAN listener LISTENER_SCAN3 is running on node ecl-oda-0

 

Conclusion

This example confirms that not all SCAN modifications require a shutdown of your scan/listener resources. If it’s a new scan VIP that is added then the downtime is equal to zero as the new resources can be started independently (using -scannumber and -I arguments). 

Just make sure you are using the right syntax according to the version of the Grid software you are running :

– 18c

srvctl modify scan –n  [scan_name] 
srvctl modify scan_listener -u

– 19c

srvctl modify scan –scanname [scan_name]
srvctl modify scan_listener -update

More from this Author