These have been followed for PP1.

NOTE – For the list griddisk attributes name,asmmodestatus,asmdeactivationoutcome last step for each cell ensure that status is ONLINE
(not SYNCING) before moving on to the next storage server.

The following query in sqlplus / as sysasm is useful for checking that you have 12 disks for each cell
for diskgroups 1 and 3 (DATA and RECO) and 10 for diskgroup 2 (DBFS)
select group_number, substr(path,1,15), count(1) from v$asm_disk group by group_number, substr(path,1,15);


1. Check griddisk status by verifying the griddisk attribute “asmdeactivationoutcome” = “Yes” for all griddisks on this cell.  Do not proceed if a griddisk is returned using the following command.  The following command should return no rows.
dcli -g /home/oracle/cell_group -l root cellcli -e “list griddisk where asmdeactivationoutcome != ‘Yes’ attributes name,asmmodestatus,asmdeactivationoutcome”

2. Determine amount of bytes to be flushed by determining how much is dirty is in the flash cache.  This will provide the number of bytes of data that needs to be de-staged to disk per cell, which will give an indication of how long the flush will take. 
dcli -g /home/oracle/cell_group -l root cellcli -e “list metriccurrent attributes name,metricvalue where name like \’FC_BY_DIRTY.*\’ “

3. Flush the flashcache across all cells

To perform this step, it is recommended to have two separate sessions: one to execute the command below and the other to monitor its progression in the next step.

Issue the following command to begin the flush in one of the two sessions:
dcli -g /home/oracle/cell_group -l root cellcli -e “alter flashcache all flush”

If any errors occur, they will be displayed in this session, otherwise, this session will show a successful flush across all cells.

4. Check the flush status across all cells

a. Execute the following command every few minutes in the second session to monitor the progress.  As dirty blocks are de-staged to disk, this count will reduce to zero (0).  This will take some time and you can determine a time estimate as you execute the following command over time: 
dcli -g /home/oracle/cell_group -l root cellcli -e “list metriccurrent attributes name,metricvalue where name like \’FC_BY_DIRTY.*\’ “

b. The following command should return  “working” for each flash disk on each cell while the cache is being flushed and “completed” when it is finished. Execute the following command in the second session:
dcli -g /home/oracle/cell_group -l root cellcli -e “LIST CELLDISK ATTRIBUTES name, flushstatus, flusherror” | grep FD


The following steps are to be executed individually on each cell, one at a time.  All steps that must be performed directly on a cell use the cellcli utility.

 Log onto the first cell that will have the write back flash cache disabled.

5. Drop the flashcache for this cell after the flush completes
cellcli -e drop flashcache

6. Inactivate all griddisks on the cell
cellcli -e alter griddisk all inactive

7. Shut down the cellsrv service
cellcli -e alter cell shutdown services cellsrv

8. Reset the cell flash cache state to writethrough
cellcli -e “alter cell flashCacheMode=writethrough”

9. Restart the cellsrv service
cellcli -e alter cell startup services cellsrv

10. Reactivate the griddisks on the cell
cellcli -e alter griddisk all active

11. Recreate the flash cache
cellcli -e create flashcache all

12. Check the status of this cell flash cache state
cellcli -e list cell detail | grep flashCacheMode

13. Check the griddisks of the cell

Before moving on to the next cell, check the attribute “asmModestatus” of all of the griddisks and make sure they are all “ONLINE” and the attribute “asmdeactivationoutcome” is set to “Yes”.  It may be necessary to execute the following command several times until the “asmModestatus” shows “ONLINE”. 
cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome


checking on all nodes
dcli -g /home/oracle/cell_group -l root cellcli -e “list cell detail” | grep flashCacheMode