While installing 126.96.36.199 Grid Infrastructure , CVU reported IZ97457, IZ89165 and network parameters failed. I did check it manually it was all reported OK by somehow OUI was not getting round it… So started looking for bugs and Here is the BUG.
11gR2 OUI On AIX Pre-Requisite Check Gives Error “Patch IZ97457, IZ89165 Are Missing” [ID 1439940.1]
OUI checks specific fixes using instfix command instead looking at TL / SP level (oslevel -s). Recent TL4 has already included both patches. So both can be safely ignored.
On Network Parameters. if you check using no -a , you will find all parameters are set correctly (Make sure all after setting ipqmaxlen nodes are rebooted). However OUI still reporting network parameters expected value is XXXXX and current value is null , which again looks like BUG. Look at the below below bug which is mentioned as impacting 188.8.131.52 but fixed in 184.108.40.206 , but this is not correct.. its still impacting 220.127.116.11
Bug 10129309 – AIX: CVU does not check network tunables on AIX 18.104.22.168 [ID 10129309.8]
Check which Agents are in which TimeZones
SELECT target_name, timezone_region FROM mgmt_targets
WHERE target_type = ‘&Instance_Type’
AND target_name = ‘&Target_Name’;
SQL to detect problematic agents with Skewed Timings
SELECT agent, timezone_region, difference “seconds”,
FROM (SELECT t.target_name agent, t.timezone_region,
FROM mgmt_emd_ping p, mgmt_targets t
WHERE p.target_guid = t.target_guid
AND p.status = 1)
WHERE difference NOT BETWEEN -91 AND 91
ORDER BY difference
Set the TimeZone of an Agent
I had several problems with deploying latest OMS PSU.
1. After deploying the patch I was not able to start OMS so I was advised to copy emCORE.jar file from any other environment where this patch was not applied.
2. After copying emCORE.jar I had some wierd problem. OMS was up but OEM home page was displaying the no. of targets only , Alerts and all other info were completely missing.
Finally We could figure out somehow when i download the patch one of the class file getting corrupted. So I downloaded one more time and copied to AIX box from local laptop… this time it work. So IDEA is to have a mount point with all the necessary software loaded .. its kind of Software repository which can be remotely mounted to other servers. So no need to copy software everytime you build a new environment.
One more thing, I have noticed OPATCH was dead slow so download latest OPatch 22.214.171.124.4 .. ths version is specially tuned to reduce CPU cycles and memory requirement during patch deployment.
As I have instructions handy so you may find them handy too :)
OMS PATCHING (11727299)
opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./11727299
emctl stop oms -all
$ORACLE_HOME/bin/rcuJDBCEngine sys/<sys_pwd>@dbhost:1521:dbname JDBC_SCRIPT post_install_script.sql $PWD $ORACLE_HOME
$ORACLE_HOME/bin/rcuJDBCEngine sys/<sys_pwd>@dbhost:1521:dbname JDBC_SCRIPT post_install_patch.sql $PWD $ORACLE_HOME
AGENT PATCHING (9345906)
opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./9345906
emctl stop agent
emctl start agent
I always insist to enable flash back database in Development / Test database. Reason as simple incase Genius developers make a mistake .. a human bevior only then intead of taking too much overhead on DBAs its easier to flashback quickly as rolling back using RMAN.
However recently I had seen few nasty block corruptions on Index as well on one a table too.. I found reason too.. those indexes and tables were in nologgin mode. So idea is whenever you create database always include force logging clause.
There is another seprate debate what about nologging operations. For that i would have another post probably.
We have been seeing VIP tied up with Public IPs. When I installed 126.96.36.199 first time i was amazed to see ips tied up with Private IPs as well. Oracle started this feature with 188.8.131.52 only. During Installation I have not provided them , just supplied two private IPs for redundant purpose.
I quite like the idea using IPs for high availability purpose. Apparently there is specific need for this feature i.e. MULTICAST. Oracle GI installation may fail if multicast is not enabled. Oracle has released a BUF Fix as solution to fix this problem.
These IPs are HAIP (High availability Virtual IPs) . Oracle RAC uses them for redunant purpose as well as for load balancing. If a private interconnect interface fails then Cluster transparently moves the corresponding HAIP address to remaining working interface.
Problem :: root.sh will fail on second node during GI Installation.
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node hexatech1 , and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
Failed to start Oracle Clusterware stack
Solution :: Login to support.oracle.com , and follow below notes.
Bug 9974223 – Allow multicast communications on 184.108.40.206 [ID 9974223.8]
Oracle Grid Infrastructure 220.127.116.11 Installation or Upgrade may fail due to Multicasting Requirement [ID 1212703.1]
First post. I have been thinking to do write ups about Oracle products. I have done few in past on google.blogspot.. As most of the Oracle geeks hitting wordpress so i thought i should follow the leaders or you say treand setters :)
Anyway here you go.