1.       Rebooting the server can be done in two ways. One way is to Turn Off the power from the ILOM interface. This approach did not work in our case. The other way is to log in the cell server and invoke the reboot command.

2.       During this activity it is prudent to have an active console window, so the overall progress can be monitored. There is a button in the ILOM interface screen which can be used to open such a console window. However, in this occasion, a window was opened via SSH to the server and manually initiating the console. Please note that the console may time out after a certain period of time. It is therefore prudent to start it up just before is needed.

 

INCREASE THE POWER LIMMIT IN ASM TO SPEED UP THE REBALANCING.

$ sqlplus / as sysdba

SQL> show parameter POWER

asm_power_limit                      integer     2

SQL> alter system set asm_power_limit = 9 scope = both;

SQL> show parameter POWER

asm_power_limit                      integer     9

 

MONITOR THE PROCESS FROM THE ILOM

# ssh

 

Oracle(R) Integrated Lights Out Manager

-> start /SP/console

 

# ssh docexacel13-ilom

Oracle(R) Integrated Lights Out Manager

-> reset /SYS

Are you sure you want to reset /SYS (y/n)? y

Performing hard reset on /SYS

 

After a short while, output will start streaming in the console.
Monitor its progress until the login prompt appears.
Then, proceed with the checks, as described below.

 

sqlplus / as sysdba

SQL> select * from gv$asm_operation;

 INST_ID GROUP_NUMBER OPERA STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE

         1            1 REBAL WAIT          9

         1            3 REBAL WAIT          9

         3            1 REBAL WAIT          9

         3            3 REBAL RUN           9          9      91743     553951       8149          56

         4            1 REBAL WAIT          9

         4            3 REBAL WAIT          9

         2            1 REBAL RUN           9          9     116410     630450       9268          55

         2            3 REBAL WAIT          9

8 rows selected.

 

Keep monitoring until the rebalancing is complete, i.e. all groups are in WAIT state.

#dcli -g /home/oracle/cell_group -l root cellcli -e list alerthistory |grep 7445

 

# cellcli

Cell Efficiency Ratio: 9,022

CellCLI> list cell detail

         name:                  

         bbuTempThreshold:       60

         bbuChargeThreshold:     800

         bmcType:                IPMI

         cellVersion:            OSS_11.2.3.2.0_LINUX.X64_121226

         cpuCount:               24

         diagHistoryDays:        7

         fanCount:               8/8

         fanStatus:              normal

         flashCacheMode:         WriteThrough

         id:                     1238FM500A

         interconnectCount:      3

         interconnect1:          bondib0

         iormBoost:              0.0

         ipaddress1:             192.168.1.1/22

         kernelVersion:          2.6.32-400.1.1.el5uek

         locatorLEDStatus:       off

         makeModel:              Oracle Corporation SUN FIRE X4270 M3 SAS

         metricHistoryDays:      7

         notificationMethod:     snmp

         notificationPolicy:     critical,warning,clear

         offloadEfficiency:      9,021.8

         powerCount:             2/2

         powerStatus:            normal

         releaseVersion:         11.2.3.2.0

         releaseTrackingBug:     14212264,16042459

         snmpSubscriber:         host=mgmt6.uk.centricaplc.com,port=1830,community=public

                                 host=mgmt8.uk.centricaplc.com,port=1830,community=public

         status:                 online

         temperatureReading:     26.0

         temperatureStatus:      normal

         upTime:                 0 days, 0:02

         cellsrvStatus:          running

         msStatus:               running

         rsStatus:               running

 

CellCLI> list physicaldisk

         19:0            K60M7N          normal

         19:1            KZNBBL          normal

         19:2            K5ZLEN          normal

         19:3            K60P5N          normal

         19:4            K3ZRAN          normal

         19:5            K2AH4N          normal

         19:6            K3YWGN          normal

         19:7            K6015N          normal

         19:8            K5ZTGN          normal

         19:9            K608XN          normal

         19:10           K60NWN          normal

         19:11           K9U7TN          normal

         FLASH_1_0       5L0046ZR        normal

         FLASH_1_1       5L0046ZN        normal

         FLASH_1_2       5L0046W8        normal

         FLASH_1_3       5L0046SY        normal

         FLASH_2_0       5L0046LD        normal

         FLASH_2_1       5L0046NY        normal

         FLASH_2_2       5L0046RD        normal

         FLASH_2_3       5L0046PL        normal

         FLASH_4_0       5L0046SQ        normal

         FLASH_4_1       5L0047A4        normal

         FLASH_4_2       5L00470Q        normal

         FLASH_4_3       5L00472Q        normal

         FLASH_5_0       5L0046LF        normal

         FLASH_5_1       5L0046V8        normal

         FLASH_5_2       5L004773        normal

         FLASH_5_3       5L0046TP        normal

 

CellCLI> list griddisk

         DATA_CD_00_3mgmt      active

         DATA_CD_01_3mgmt      active

         DATA_CD_02_3mgmt      active

         DATA_CD_03_3mgmt      active

         DATA_CD_04_3mgmt      active

         DATA_CD_05_3mgmt      active

         DATA_CD_06_3mgmt      active

         DATA_CD_07_3mgmt      active

         DATA_CD_08_3mgmt      active

         DATA_CD_09_3mgmt      active

         DATA_CD_10_3mgmt      active

         DATA_CD_11_docexacel13mgmt      active

         DBFS_DG_CD_02_3mgmt   active

         DBFS_DG_CD_03_3mgmt   active

         DBFS_DG_CD_04_3mgmt   active

         DBFS_DG_CD_05_3mgmt   active

         DBFS_DG_CD_06_3mgmt   active

         DBFS_DG_CD_07_3mgmt   active

         DBFS_DG_CD_08_3mgmt   active

         DBFS_DG_CD_09_3mgmt   active

         DBFS_DG_CD_10_3mgmt   active

         DBFS_DG_CD_11_3mgmt   active

         RECO_CD_00_3mgmt      active

         RECO_CD_01_3mgmt      active

         RECO_CD_02_3mgmt      active

         RECO_CD_03_3mgmt      active

         RECO_CD_04_3mgmt      active

         RECO_CD_06_3mgmt      active

         RECO_CD_07_3mgmt      active

         RECO_CD_08_3mgmt      active

         RECO_CD_09_3mgmt      active

         RECO_CD_10_3mgmt      active

         RECO_CD_11_13mgmt      active

 

When the status for all groups is observed to be “WAIT”, we can return the power degree parameter to its original value.

sqlplus / as sysdba

SQL> alter system set asm_power_limit = 2 scope = both;

System altered.

SQL> show parameter POWER

asm_power_limit                      integer     2