ASM resilvering – or – how to recover your crashed cluster – the Environment

We use the same environment we installed here which consists of two virtual machines running Oracle Enterprise Linux 5 Update 3 64-bit with Oracle 11g Release 2 (Grid Infrastructure and Oracle binaries) installed. All data is placed in ASM with normal redundancy. Storage is exported from two Netapp Filers via iSCSI.

Disk Storage Layout

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

 Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14         535     4192965   82  Linux swap / Solaris
/dev/sda3             536        3916    27157882+  83  Linux

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

 Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1       10240    10485744   83  Linux

Disk /dev/sdc: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

 Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       10240    10485744   83  Linux

Disk /dev/sdd: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

 Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1       10240    10485744   83  Linux

Disk /dev/sde: 21.4 GB, 21474836480 bytes
64 heads, 32 sectors/track, 20480 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

 Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1       20480    20971504   83  Linux

Disk /dev/sdf: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

 Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1       10240    10485744   83  Linux

Disk /dev/sdg: 21.4 GB, 21474836480 bytes
64 heads, 32 sectors/track, 20480 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

 Device Boot      Start         End      Blocks   Id  System
/dev/sdg1               1       20480    20971504   83  Linux

ASM Disks and Device mapping

• DISK001A    /dev/sdd1
• DISK001B    /dev/sdb1
• DISK002A    /dev/sdf1
• DISK002B    /dev/sdc1
• DISK003A    /dev/sde1
• DISK003B    /dev/sdg1

ASM disk groups and failure groups mapping

Inside ASM we have two disk groups:

• Disk Group DATA (normal redundancy and two failure groups)
  • which stores
    • Voting Disk and
    • OCR
    • one copy of redo logs and control file of database “ORA11P”
  • and consists of the following disks:
    • Failure Group 1
      • DISK001A
      • DISK002A
    • Failure Group 2
      • DISK001B
      • DISK002B
• Disk Group DATA2 (normal redundancy and two failure groups)
  • which stores
    • all data files for database “ORA11P”
    • ADVM volume for ACFS file system mounted as shared oracle home
  • and consists of the following disks:
    • Failure Group 1
      • DISK003A
    • Failure Group 2
      • DISK003B