Increasing data checks performed by RMAN with "check logical" switch

By default RMAN checks for physical corruptions in data files when performing backups. For a logical check you can add “check logical” to your rman backup command. With this clause rman also checks the block integrity logically.

A few examples how to use it:

backup validate check logical database;

backup check logical database;

backup check logical incremental level 0 filesperset 4 database;

Checking for both logical and physical corruptions requires the init.ora parameter “db_block_checksum” set to “true” which is also the default. This parameter forces oracle to calculate a checksum for the block before writing it to disk. Doing logical checks with rman has a slight performance overhead when doing backups – typically 1 to 10 percent.

It is not recommended to turn this off for performance reasons bercause the performance overhead is minimal (< 2%) and you will be unable to detect corruptions!

RMAN in Data Guard configurations – Problems of restoring rman disk backups and standby site and vice versa

… or: the problem of backup association

From time to time i come across the same problems in data guard configurations using rman-to-disk: Backups taken on the standby database are not taken into account when trying to perform an rman recovery on the primary site and vice versa.

This is an expected behavior according to the oracle documentation:

The accessibility of a backup is different from its association. In a Data Guard environment, the recovery catalog considers disk backups as accessible only to the database with which it is associated, whereas tape backups created on one database are accessible to all databases. If a backup file is not associated with any database, then the row describing it in the recovery catalog view shows null for the SITE_KEY column. By default, RMAN associates files whose SITE_KEY is null with the database to which it is connected as TARGET.

RMAN commands such as BACKUP, RESTORE, and CROSSCHECK work on any accessible backup. For example, for a RECOVER COPY operation, RMAN considers only image copies that are associated with the database as eligible to be recovered. RMAN considers the incremental backups on disk and tape as eligible to recover the image copies. In a database recovery, RMAN considers only the disk backups associated with the database and all files on tape as eligible to be restored.

To illustrate the differences in backup accessibility, assume that databases prod and standby1 reside on different hosts. RMAN backs up datafile 1 on prod to /prmhost/disk1/df1.dbf on the production host and also to tape. RMAN backs up datafile 1 on standby1 to /sbyhost/disk2/df1.dbf on the standby host and also to tape. If RMAN is connected to database prod, then you cannot use RMAN commands to perform operations with the /sbyhost/disk2/df1.dbf backup located on the standby host. However, RMAN does consider the tape backup made on standby1 as eligible to be restored.

Note: This is a quote from the 11g R1 documentation. The same text apprears in 10g R2 documentation as well.

Managing the AWR (automatic workload repository)

Overview on the AWR

The automatic workload repository “AWR” was introduced with Oracle 10g. It collects performance data in intervals (default: 60 minutes) and keeps them by default for one week. The data is stored in the SYSAUX tablespace.

The AWR collects:

  • Wait events
  • time model statistics (view: V$SESS_TIME_MODEL and V$SYS_TIME_MODEL)
  • active session hitory information (view: V$ACTIVE_SESSION_HISTORY)
  • system statistics (view: V$SYSSTAT, V$SESSTAT)
  • resource intensive sql statements
  • information on object usage

For AWR to collect these data the database parameter STATISTICS_LEVEL has to be set to TYPICAL (= the defaul) or ALL. Setting it to BASIC disables data collection!

Changing the AWR settings

Changing the snapshot interval and retention time

For changing the snapshot interval and/or the retention time use the following syntax:

exec dbms_workload_repository.modify_snapshot_settings(
interval => 60
retention => 525600);

The example printed above changes the retention time to one year (60 minutes per hour * 24 hours a day * 365 days per year = 525600). It does not alter the snapshot interval (60 minutes is the default). If you want to you can alter this as well but keep in mind you need aditional storage to do so. Setting the interval value to zero completely disables data collection.

According to the oracle documentation depending on the size of the database the SYSAUX tablespace which contains the AWR objects can be as big as 5 gb in default configuration (60 minutes snapshot interval, 7 days retention time). So if you want to keep your data for one year you might end up with a SYSAUX of 260 GB (assuming a linear growth with the retention time).

Manually creating a snapshot

You can of course create a snapshot maually by executing:

exec dbms_workload_repository.create_snapshot;

Query available snapshots

To query the available snapshot you can use this query:

select snap_id, begin_interval_time, end_interval_time
from dba_hist_snapshot
order by snap_id;

Removing a snapshot

To remove a snapshot just do:

exec dbms_workload_repository.drop_snapshot_range
  (low_snap_id=>1, high_snap_id=>10);

Oracle 11 Release 2 Install Guide – Install Oracle RAC

System configuration

Hardware configuration

  • two virtual machines (VMWARE)
    • 1 vCPU
    • 4 GB RAM
    • 40 GB local Disk
  • Storage exported via ISCSI
    • 4 LUNs with 10 GB each
    • 2 LUNs with 30 GB each

Operating system configuration

  • Oracle Enterprise Linux 5.3 x86_64 (Kernel 2.6.18-128.el5)
  • Installed packages: default system + development packages

Cluster configuration

  • Cluster Name: „RAC“
  • Grid Binary installation on local disk
  • OCR, Voting and datafiles stored in ASM
  • Oracle HOME stored on ACFS file system (this menas we will have a SHARED home directory among all nodes!)
  • Oracle HOME will be installed unser user „ora11p“

Installation – Requirements and Preparations

Installation Steps outlined

  1. Installation of Oracle 11g Release 2 Grid Infrastructure → done here
  2. Installation of Oracle 11g Release 2 Database (rac installation)
    1. Review system requirements
    2. Install database software (aka „binaries“)
    3. Create database

general Requirements

All requirments from grid infrastructure installation apply here as well, for instance:

  • Kernel parameter
  • Limits (esp. configure limits for new user „ora11p“ which will hold the binary database installation)
  • Synchronous time
  • dns-resolvable SCAN name
  • working public and private interconnect
  • shared and accessible storage
  • at least ONE better two more disk groups created:
    • one for database files and binary installation
    • one for flashback recovery area
  • Note: SSH equivalence will be set up by installer

Preparations

Create ACFS file system for storing oracle binary installation

  • Create further ACFS mount directory
    mkdir -p /u01/app/oracle/product/11.2.0/ora11p
  • Create ADVM volume

step4_010

  • Create ACFS file system

step4_011

  • DO NOT register ACFS file system mountpoint for the oracle home directory! We will do this later (when creating the database) as clusterware resource
  • Mount ACFS file system on both nodes ( “mount /dev/asm/ora11p_home-132 /u01/app/oracle/product/11.2.0/ora11p”)

step4_014

 

Create User on both nodes:

useradd -u 501 -g dba -d /u01/app/oracle/product/11.2.0/ora11p ora11p
passwd ora11p
chown -R root:dba /u01/app/oracle/product
chmod -R 775 /u01/app/oracle/product
chown -R ora11p:dba /u01/app/oracle/product/11.2.0/ora11p

Create .bash_profile for user ora11p (note: changes on node A will be visible on node B cause were using ACFS; so changing profile file is needed only once)

export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/ora11p
export ORACLE_SID=ORA11P
if [ -t 0 ]; then
stty intr ^C
fi

One word on Cluster Verification utility (cluvfy)

  • should be used prior installation
  • At this point the user „ora11p“ does not have SSH equivalence configured
  • So there are two options:
    • start cluvfy after installer set up ssh equivanlence
    • set up SSH equivalence manually
  • I personally prefer to run cluvfy after installer set up ssh equivalence

However the database installer seems to run a more or less complete check automatically

If you want to start cluvfy here is the syntax:

cluvfy stage -pre dbinst -fixup -n nodeA,nodeB -r <release> -osdba <name of sysdba group on unix> -verbose
<release> is: 10gR1, 10gR2, 11gR1, 11gR2

Installation

Start installer als user „ora11p“ on any node

step4_015

For testing purposes i deselected update notifications… in productive environments this is highly recommended

step4_016

we will install database software only and create database later

step4_017

both nodes were discovered correctly….

step4_018

Because were installing as user „ora11p“ which is different from the user holding our infrastructure installation we need to create passwordless ssh connectitivty

!! you also need to select „user home is shared“ cause were using ACFS !!

step4_019

step4_019

SSH connectivity successfully established…

step4_020

Select language support

step4_021

Select Edition

step4_022

Select Options

step4_023

Select ORACLE_BASE and ORACLE_HOME; we set it in profile file so it is entered by the installer automatically

step4_024

step4_025

Installation Summary

step4_026

Install process

step4_027

step4_028

Installation nearly finished.. just start „root.sh“ as root on all nodes

step4_029

root.sh sample output

step4_030

Finished

step4_031

Where to go now?

  • We just installed all neccessary components for creating our first RAC database → this will be the next post
  • backup current configuration

Oracle Dataguard – automatic archive log file management on primary site

System Environment

Suppose you have an oracle data guard environment with one primary database and one physical standby database running.

You perform online backups with RMAN from the PRIMARY database. Your users complain about slowness during the night. You notice the slowness correlates with the time the online backup runs. Thanks to the physical standby database you can offload your backups to the standby database. This short guide shows how to do that.

First of all the ability to offload backups from primary to standby is a major plus in physical standby configurations. Note that this is not possible on logical standby configurations.

Furthermore the offload to the standby database requires to have a more or less separated storage (either separated storage boxes, separated disks, separated ports,….).General speaking: the more you separate primary from standby storage the better it is from performance point of view.

Implementation on primary site

Database requirements

In order to use the automatic deletion of archive logs on the primary site after they have been applied on the standby database you need to enable the FRA (flashback recovery area) on the primary database set setting the following two parameters:

db_recovery_file_dest='<path>’
db_recovery_file_dest_size=xyG

The value for db_recovery_file_dest must be a valid path and db_recovery_file_dest_size specified the total size of the FRA in GB (specified by the suffix “G”), MB (suffix “M”) and so on.

Note that the FRA will fill up to this value before starting to delete archive logs applied on standby. So if you configure the FRA to a size of 200 GB you will end up with 200 GB archive logs stored in the FRA.

You also need to unset the parameter archive_log_dest_n if it points to a local directory. Note that dataguard uses archive_log_dest_n parameter for specifying the remote archive log destination.

Once you set everything you can check it with “archive log list” from SQLPlus. The output should look like this:

SQL> archive log list
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     62686
Next log sequence to archive   62695
Current log sequence           62695

If you have configured the environment correctly the archive destination should show up as “USE_DB_RECOVERY_FILE_DEST”.

Backup on primary site

On primary site all you need to backup is the current control file. For use with EMC networker i am using the following script. If you backup with another backup software you just need to alter the SBT_TAPE configuration accordingly to the documentation or backup the control file to disk rather than to tape.

connect target sys/password@primary;
connect catalog rman_catalog/catalog_password@catalogd;
run {
 backup
 (current controlfile format 'ctrl_PRIMARY_s%s_p%p');
 sql 'alter database backup controlfile to trace';
 }
run {
copy current controlfile to '/u01/sicherung/controlfiles/ctl_PRIMARY.ctl';
}

The script shown above is just an example. You need to check and alter it to suit your needs. As you may assume the database in this example is named “PRIMARY”; the standby is named “STANDBY”.

You can see i am quite paranoid about control file backups: I backup them to tape, to trace in a readable format and directly to disk as a copy.

In restore scenarios this enables:

  • restore of the control file with rman
  • re-creatation of the control file from the dumped control file
  • copy the control file from  /u01/sicherung/controlfiles/ctl_PRIMARY.ctl to the destinations named in the database parameter “control_files”
    (Note: The file created with “copy controlfile” is not wrapped in an rman stream… therefore you can just “copy” it with cp for instance)

Implementation on standby site

Required INIT.ORA parameters

If you use dataguard in a Max Performance or Max Availability mode you need to set

_log_deletion_policy='ALL'

in your standby database init.ora. This is a know problem documented in Metalink Note 331924.1.

Required RMAN configuration

In order to enable automatic deletion on primary site you need to configure rman on standby site as follows:

CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON STANDBY

This tells oracle to make archive logs on primary size egible for deleteion after they have been applied on the standby site.

Backup scripts

Full backup

The following script shows how to perform an full backup with rman on the STANDBY database. Primary and standby databases share the same recovery catalog (because they are basically the same databases)!

My recommendation is to daily backup your database fully. With large databases a full backup might take longer than 24 hours. In this cases you can do daily incremental and weekly full backups.

connect target sys/password@standby;
connect catalog rman_catalog/catalog_password@catalogd;
run {
 CONFIGURE DEVICE TYPE SBT_TAPE PARALLELISM 4;
 backup check logical INCREMENTAL LEVEL 0 filesperset 8
 format 'FULL0_%d_%u'
 (database include current controlfile);
 backup not backed up 1 times filesperset 10
 (archivelog all delete input format  'al_STANDBY_%s_%p');
 backup
 (current controlfile format 'ctrl_STANDBY_s%s_p%p'); }
run {
copy current controlfile to '/u01/sicherung/controlfiles/ctl_STANDBY.ctl';
}

Archive log only backup

The following script shows how to backup the archive logs on STANDBY site. The script can be started any time.

I recommend to backup the archive logs every hour. Thanks to “backup not backed up 1 times” we only backup the archive logs once even if they are still on disk when we backup the archive logs once again.

connect target sys/password@standby;
connect catalog rman_catalog/catalog_password@catalogd;
run {
 CONFIGURE DEVICE TYPE SBT_TAPE PARALLELISM 4;
 backup not backed up 1 times as compressed backupset filesperset 10
 (archivelog all delete input format  'al_STANDBY_%s_%p');
 backup
 (current controlfile format 'ctrl_STANDBY_s%s_p%p');
 }
run {
copy current controlfile to '/u01/sicherung/controlfiles/ctl_STANDBY.ctl';
}

You also do not need to worry about RMAN deleting archive logs which are not yet applied on standby (in MAXIMUM PERFORMANCE mode). If rman tries to delete an archive log which is not yet applied on standby you see the following message:

archive log filename=/u02/oradata/STANDBY/arch/1_62487_646478938.dbf
  thread=1 sequence=62487
RMAN-08120: WARNING: archive log not deleted, not yet applied by standby

Test it

After you configured everything you need to TEST you configuration. This involves:

  • What happens to the primary database if the standby database is down and does not apply archive logs?
  • Does the archive logs expire once they are applied on the standby?
  • and so ….

Query the FRA

The following query queries the current status of the FRA and shows how much space is occupied in the FRA:

select * from v$flash_recovery_area_usage;

fra

In the screenshot above you can see the FRA is filled to 99.83 % with 58.29% reclaimable (this means: 58.29% of space occupied by archive logs in the FRA are eligible for deletetion because they are already applied on the standby database). This query was performed on a data guard enviroment running in MAXIMUM AVAILABLILITY mode.

Oracle 11 Release 2 Install Guide – ACFS and ADVM

ACFS and ADVM

System configuration

We will use the system configured and installed in part 2

General information

  • ADVM = ASM dynamic volume manager
  • ACFS = ASM cluster file system
  • Basic layout

step3_010

(Source: Oracle Database Storage Administrator’s Guide 11g Release 2)

ADVM – Advantages

  • Integrated in ASM; this can be an disadvantage as well :-)
  • Inhertits storage from ASM hence enables host-based mirroring (either 2- or 3-way-mirroring)
  • multiple volumes within a disk group can be created with an file system such as ext3, ext4, reiserfs, … on top of it and will support storage of any file type as the file system normally woud – EXCEPT files which can be place in ASM directly
  • ADVM volume dynamically resizeable

ADVM – Disadvantages

  • ADVM volumes may be resized online; but the used file system must support it as well (ext3 on OEL 5 does support online resizing but does not support online shrinking)
  • Storing files which can be stored in ASM directly in ADVM + file system is not supported
  • NFS on top of ADVM is also not supported
  • ASM configuration assistant (asmca) only supports creation of volumes / file system… delete a volume / file system requires command line

ACFS – Advantages

  • cluster file system on top of ASM and ADVM
  • as available as ASM is (inherits storage from ASM disk group and ADVM volume)
  • Supports storage of files which cannot be directly stored in ASM, i.e.
    • executables
    • trace files
    • log files
    • Supports even oracle database binary installations
  • On ACFS read-only Snapshots can be created
  • dynamically resizeableUseable accross plattforms
  • Thoughts
    • Do i need licenses for grid infrastructrue?
    • If not: What if grid infrastructure + ASM used to provide host-based mirroring and cluster file system for non-oracle applications, for instance web servers ACFS Mount registry: used for mouting ACFS and ADVM file system across reboots

ACFS – Disadvantages

  • for example storing database files in ACFS is not supported, according to the documentation
    „Oracle Support Services will not take calls and development will not fix bugs associated with storing unsupported file types in Oracle ACFS“
  • Only available with RedHat Server 5 or Oracle Enterprise Linux 5 !
  • Disk group compatible parameter COPATBILE.ASM and COMPATIBLE.ADVM must be set so 11.2
  • ASM configuration assistant (asmca) only supports creation of volumes / file system… delete a volume / file system requires command line

First steps with ADVM and ACFS

Create a disk group for use with ADVM and ACFS

Lets first create an additional disk group called „DATA2“ which consists for two iSCSI LUNs with 30 GB each

Preparation:

  • LUNs visible with „fdisk -l“
  • Partition created (one on each LUN)
  • disk labeled with „oracleasm createdisk <name> <devpath>“
  • Create disk group in ASM (remember to connect as „sys as sysASM“!)

step3_011

  • Notes on disk groups
    • AUSIZE of 4 MB recommended by Oracle documentation due to:
      • Increased I/O through the I/O subsystem if the I/O size is increased to the AU size.
      • Reduced SGA size to manage the extent maps in the database instance.
      • Faster datafile initialization if the I/O size is increased to the AU size.
      • Increased file size limits.
      • Reduced database open time.
    • Large AUSIZE requires as well
      • Increasing size of maximum IO request to at least 4 MB in operating system, drive, HBA, storage system
      • Larger stripe size in storage system (pre 11g R2: 1 MB stripe size, with 11g R2: 4 MB? → to be tested)

Read the documentation on COMPATIBLE parameters;most „cool“ features are only available with 11.2 COMPATIBLE parameter hence require 11g R2 database

Creating an ADVM and afterwards an ACFS

Create an ADVM

  • ACFS requires ADVM in which ACFS can be created
  • volcreate creates ADVM volume

step3_012

  • The command above shows minimal command creating an ADVM volume; redundancy is derived from disk group, our data group was created with „normal“ redundancy so the volume inherits this as well)
  • Creation with SQL also possible: „ALTER DISKGROUP data2 ADD VOLUME volume1 SIZE 10G;“

Create ACFS on top of ADVM

  • Requires ADVM in which ACFS can be created
  • volinfo shows detailed information
  • Especially device path is important for creating the file system

step3_013

  • create ACFS from operating system (only on one node)

step3_014

  • register acfs in registry to be mounted across reboots with „acfsutil“
  • ATTENTION: DO NOT register shared oracle home directories with acfsutil; this will be done later by the clusterware itself!
  • test of everything works by issueing „mount.acfs -o all“ on all nodes; the file system should be mounted and accessible

step3_015

Simple Performance tests

dd if=/dev/zero bs=1024k of=<path> count=1000 oflag=direct

→ direct I/O used; no caching, performed 10 times

  • write to ACFS
    • ACFS 2-way mirror: ~ 6 MB/s average
    • ACFS unprotected: ~ 12 MB/s averga
    • → expected… one disk, double I/O halfed throughput
  • direct write to iSCSI LUN: ~ 14.3 MB/s average

Attention: Tests were performed within a VMWare… so results are most likely not accurate… but we get an impression.. we will check this on real hardware later!

Calculate IOPS (or IO/s) for hard disks

From time to time i spend time looking for information how to calculate IOPS (or IO operations per second) for hard drives.

In the following table i outlined how to calculate the IOPS a disk can handle.  For this calculation you need at least TWO values. Most of the time you will be given with the RPMs (rotations per minute) and average seek time. You can also cope with rotation latency or IO time – you just need to play with the formulas.

 

 iops

 

Based on this approach you can easily calculate the data transfer rate (MBPS):

 

 iops2

As you can easily see larger IO requests yield to a higher throughput in MBPS. Due to this fact oracle recommends the SAME principle (SAME = stripe and mirror everything) with a stripe size of 1 MB.

Changing ADR Destination for 11g Listener

With 11g Release 1 Oracle introduced the Automatic Diagnostic Repository  – “ADR”. The ADR is designed as central repository for storing all oracle-related log files such as alert.log, listener.log and so on.

The log files are stored in XML format and in a subdirectory called “trace” in the pre-11g format.

By default the ADR-Home is located in $ORACLE_BASE. The path can be changed in the database by changing the parameter DIAGNOSTIC_DEST.

 

For the listener this change does not work. If you want to change the location the listener places its log files add the following parameter to the LISTENER.ORA file to change the path to the ADR:

ADR_BASE_listener = <path>

“listener” is the name of the listener to be changed. If your listener is named differently the parameter must be changed as well!

Building and using the kfed utility

When using ASM sometimes it it extremely helpful to get more information on the asm disk header. The “kfed” utility from oracle enables to dump the asm disk header and many more.

With this tool corruptions to the asm can be easily checked (and repaired).

 

 

Building kfed

This method works from 10g onwards to and including 11g R2:

 

-bash-3.2$ cd $ORACLE_HOME/rdbms/lib
-bash-3.2$ make -f ins_rdbms.mk ikfed
Linking KFED utility (kfed)
rm -f /u01/app/oracle/product/ora11r2p/rdbms/lib/kfed
gcc -o /u01/app/oracle/product/ora11r2p/rdbms/lib/kfed -m64 -L/u01/app/oracle/product/ora11r2p/rdbms/lib/
-L/u01/app/oracle/product/ora11r2p/lib/ -L/u01/app/oracle/product/ora11r2p/lib/stubs/ 
/u01/app/oracle/product/ora11r2p/lib/s0main.o /u01/app/oracle/product/ora11r2p/rdbms/lib/sskfeded.o
/u01/app/oracle/product/ora11r2p/rdbms/lib/skfedpt.o -ldbtools11 -lasmclnt11 -lcommon11 -lcell11 -lskgxp11
-lhasgen11 -lskgxn2 -lnnz11 -lzt11 -lxml11 -locr11 -locrb11 -locrutl11 -lhasgen11 -lskgxn2 -lnnz11 -lzt11
-lxml11 -lasmclnt11 -lcommon11 -lcell11 -lskgxp11 -lgeneric11  -lcommon11 -lgeneric11  -lclntsh 
`cat /u01/app/oracle/product/ora11r2p/lib/ldflags`    -lncrypt11 -lnsgr11 -lnzjs11 -ln11 -lnl11
-lnro11 `cat /u01/app/oracle/product/ora11r2p/lib/ldflags`    -lncrypt11 -lnsgr11 -lnzjs11 -ln11 -lnl11
-lnnz11 -lzt11 -lztkg11 -lztkg11 -lclient11 -lnnetd11  -lvsn11 -lcommon11 -lgeneric11 -lmm -lsnls11
-lnls11  -lcore11 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lxml11 -lcore11 -lunls11 -lsnls11 -lnls11
-lcore11 -lnls11 `cat /u01/app/oracle/product/ora11r2p/lib/ldflags`    -lncrypt11 -lnsgr11 -lnzjs11 -ln11
-lnl11 -lnro11 `cat /u01/app/oracle/product/ora11r2p/lib/ldflags`    -lncrypt11 -lnsgr11 -lnzjs11 -ln11 -lnl11
-lclient11 -lnnetd11  -lvsn11 -lcommon11 -lgeneric11   -lsnls11 -lnls11  -lcore11 -lsnls11 -lnls11 -lcore11
-lsnls11 -lnls11 -lxml11 -lcore11 -lunls11 -lsnls11 -lnls11 -lcore11 -lnls11 -lclient11 -lnnetd11  -lvsn11
-lcommon11 -lgeneric11 -lsnls11 -lnls11  -lcore11 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lxml11 -lcore11
-lunls11 -lsnls11 -lnls11 -lcore11 -lnls11   `cat /u01/app/oracle/product/ora11r2p/lib/sysliblist`
-Wl,-rpath,/u01/app/oracle/product/ora11r2p/lib -lm    `cat /u01/app/oracle/product/ora11r2p/lib/sysliblist`
-ldl -lm   -L/u01/app/oracle/product/ora11r2p/lib
test ! -f /u01/app/oracle/product/ora11r2p/bin/kfed ||\
           mv -f /u01/app/oracle/product/ora11r2p/bin/kfed /u01/app/oracle/product/ora11r2p/bin/kfedO
mv /u01/app/oracle/product/ora11r2p/rdbms/lib/kfed /u01/app/oracle/product/ora11r2p/bin/kfed
chmod 751 /u01/app/oracle/product/ora11r2p/bin/kfed

 

Using kfed to dump disk header

In the following example we use kfed to dump the asm disk header of an asm disk represented by device “/dev/sdg1”:

 

/u01/app/oracle/product/11.2.0/ora11p/bin/kfed read /dev/sdg1
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                            1 ; 0x002: KFBTYP_DISKHEAD
kfbh.datfmt:                          1 ; 0x003: 0x01
kfbh.block.blk:                       0 ; 0x004: T=0 NUMB=0x0
kfbh.block.obj:              2147483648 ; 0x008: TYPE=0x8 NUMB=0x0
kfbh.check:                  2733723458 ; 0x00c: 0xa2f14f42
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr: ORCLDISKDISK003A ; 0x000: length=16
kfdhdb.driver.reserved[0]:   1263749444 ; 0x008: 0x4b534944
kfdhdb.driver.reserved[1]:   1093873712 ; 0x00c: 0x41333030
kfdhdb.driver.reserved[2]:            0 ; 0x010: 0x00000000
kfdhdb.driver.reserved[3]:            0 ; 0x014: 0x00000000
kfdhdb.driver.reserved[4]:            0 ; 0x018: 0x00000000
kfdhdb.driver.reserved[5]:            0 ; 0x01c: 0x00000000
kfdhdb.compat:                186646528 ; 0x020: 0x0b200000
kfdhdb.dsknum:                        0 ; 0x024: 0x0000
kfdhdb.grptyp:                        2 ; 0x026: KFDGTP_NORMAL
kfdhdb.hdrsts:                        3 ; 0x027: KFDHDR_MEMBER
kfdhdb.dskname:                DISK003A ; 0x028: length=8
kfdhdb.grpname:                   DATA2 ; 0x048: length=5
kfdhdb.fgname:              CONTROLLER1 ; 0x068: length=11
kfdhdb.capname:                         ; 0x088: length=0
kfdhdb.crestmp.hi:             32925079 ; 0x0a8: HOUR=0x17 DAYS=0xc MNTH=0x9 YEAR=0x7d9
kfdhdb.crestmp.lo:           2888263680 ; 0x0ac: USEC=0x0 MSEC=0x1da SECS=0x2 MINS=0x2b
kfdhdb.mntstmp.hi:             32925206 ; 0x0b0: HOUR=0x16 DAYS=0x10 MNTH=0x9 YEAR=0x7d9
kfdhdb.mntstmp.lo:           1862809600 ; 0x0b4: USEC=0x0 MSEC=0x20e SECS=0x30 MINS=0x1b
kfdhdb.secsize:                     512 ; 0x0b8: 0x0200
kfdhdb.blksize:                    4096 ; 0x0ba: 0x1000
kfdhdb.ausize:                  4194304 ; 0x0bc: 0x00400000
kfdhdb.mfact:                    454272 ; 0x0c0: 0x0006ee80
kfdhdb.dsksize:                    5119 ; 0x0c4: 0x000013ff
kfdhdb.pmcnt:                         2 ; 0x0c8: 0x00000002
kfdhdb.fstlocn:                       1 ; 0x0cc: 0x00000001
kfdhdb.altlocn:                       2 ; 0x0d0: 0x00000002
kfdhdb.f1b1locn:                      2 ; 0x0d4: 0x00000002
kfdhdb.redomirrors[0]:                0 ; 0x0d8: 0x0000
kfdhdb.redomirrors[1]:                0 ; 0x0da: 0x0000
kfdhdb.redomirrors[2]:                0 ; 0x0dc: 0x0000
kfdhdb.redomirrors[3]:                0 ; 0x0de: 0x0000
kfdhdb.dbcompat:              186646528 ; 0x0e0: 0x0b200000
kfdhdb.grpstmp.hi:             32925079 ; 0x0e4: HOUR=0x17 DAYS=0xc MNTH=0x9 YEAR=0x7d9
kfdhdb.grpstmp.lo:           2887388160 ; 0x0e8: USEC=0x0 MSEC=0x283 SECS=0x1 MINS=0x2b
kfdhdb.vfstart:                       0 ; 0x0ec: 0x00000000
kfdhdb.vfend:                         0 ; 0x0f0: 0x00000000
kfdhdb.spfile:                        0 ; 0x0f4: 0x00000000
kfdhdb.spfflg:                        0 ; 0x0f8: 0x00000000
kfdhdb.ub4spare[0]:                   0 ; 0x0fc: 0x00000000
kfdhdb.ub4spare[1]:                   0 ; 0x100: 0x00000000
kfdhdb.ub4spare[2]:                   0 ; 0x104: 0x00000000
kfdhdb.ub4spare[3]:                   0 ; 0x108: 0x00000000
kfdhdb.ub4spare[4]:                   0 ; 0x10c: 0x00000000
kfdhdb.ub4spare[5]:                   0 ; 0x110: 0x00000000
kfdhdb.ub4spare[6]:                   0 ; 0x114: 0x00000000
kfdhdb.ub4spare[7]:                   0 ; 0x118: 0x00000000
kfdhdb.ub4spare[8]:                   0 ; 0x11c: 0x00000000
kfdhdb.ub4spare[9]:                   0 ; 0x120: 0x00000000
kfdhdb.ub4spare[10]:                  0 ; 0x124: 0x00000000
kfdhdb.ub4spare[11]:                  0 ; 0x128: 0x00000000
kfdhdb.ub4spare[12]:                  0 ; 0x12c: 0x00000000
kfdhdb.ub4spare[13]:                  0 ; 0x130: 0x00000000
kfdhdb.ub4spare[14]:                  0 ; 0x134: 0x00000000
kfdhdb.ub4spare[15]:                  0 ; 0x138: 0x00000000
kfdhdb.ub4spare[16]:                  0 ; 0x13c: 0x00000000
kfdhdb.ub4spare[17]:                  0 ; 0x140: 0x00000000
kfdhdb.ub4spare[18]:                  0 ; 0x144: 0x00000000
kfdhdb.ub4spare[19]:                  0 ; 0x148: 0x00000000
kfdhdb.ub4spare[20]:                  0 ; 0x14c: 0x00000000
kfdhdb.ub4spare[21]:                  0 ; 0x150: 0x00000000
kfdhdb.ub4spare[22]:                  0 ; 0x154: 0x00000000
kfdhdb.ub4spare[23]:                  0 ; 0x158: 0x00000000
kfdhdb.ub4spare[24]:                  0 ; 0x15c: 0x00000000
kfdhdb.ub4spare[25]:                  0 ; 0x160: 0x00000000
kfdhdb.ub4spare[26]:                  0 ; 0x164: 0x00000000
kfdhdb.ub4spare[27]:                  0 ; 0x168: 0x00000000
kfdhdb.ub4spare[28]:                  0 ; 0x16c: 0x00000000
kfdhdb.ub4spare[29]:                  0 ; 0x170: 0x00000000
kfdhdb.ub4spare[30]:                  0 ; 0x174: 0x00000000
kfdhdb.ub4spare[31]:                  0 ; 0x178: 0x00000000
kfdhdb.ub4spare[32]:                  0 ; 0x17c: 0x00000000
kfdhdb.ub4spare[33]:                  0 ; 0x180: 0x00000000
kfdhdb.ub4spare[34]:                  0 ; 0x184: 0x00000000
kfdhdb.ub4spare[35]:                  0 ; 0x188: 0x00000000
kfdhdb.ub4spare[36]:                  0 ; 0x18c: 0x00000000
kfdhdb.ub4spare[37]:                  0 ; 0x190: 0x00000000
kfdhdb.ub4spare[38]:                  0 ; 0x194: 0x00000000
kfdhdb.ub4spare[39]:                  0 ; 0x198: 0x00000000
kfdhdb.ub4spare[40]:                  0 ; 0x19c: 0x00000000
kfdhdb.ub4spare[41]:                  0 ; 0x1a0: 0x00000000
kfdhdb.ub4spare[42]:                  0 ; 0x1a4: 0x00000000
kfdhdb.ub4spare[43]:                  0 ; 0x1a8: 0x00000000
kfdhdb.ub4spare[44]:                  0 ; 0x1ac: 0x00000000
kfdhdb.ub4spare[45]:                  0 ; 0x1b0: 0x00000000
kfdhdb.ub4spare[46]:                  0 ; 0x1b4: 0x00000000
kfdhdb.ub4spare[47]:                  0 ; 0x1b8: 0x00000000
kfdhdb.ub4spare[48]:                  0 ; 0x1bc: 0x00000000
kfdhdb.ub4spare[49]:                  0 ; 0x1c0: 0x00000000
kfdhdb.ub4spare[50]:                  0 ; 0x1c4: 0x00000000
kfdhdb.ub4spare[51]:                  0 ; 0x1c8: 0x00000000
kfdhdb.ub4spare[52]:                  0 ; 0x1cc: 0x00000000
kfdhdb.ub4spare[53]:                  0 ; 0x1d0: 0x00000000
kfdhdb.acdb.aba.seq:                  0 ; 0x1d4: 0x00000000
kfdhdb.acdb.aba.blk:                  0 ; 0x1d8: 0x00000000
kfdhdb.acdb.ents:                     0 ; 0x1dc: 0x0000
kfdhdb.acdb.ub2spare:                 0 ; 0x1de: 0x0000

 

What does it all mean?

For most usages the following rows are most important:

 

Status of disk:

kfdhdb.hdrsts:                        3 ; 0x027: KFDHDR_MEMBER

Name of the disk:

kfdhdb.dskname:                DISK003A ; 0x028: length=8

 Name of disk group the disk belongs to:

kfdhdb.grpname:                   DATA2 ; 0x048: length=5

 Name of failure group the disk belongs to

kfdhdb.fgname:              CONTROLLER1 ; 0x068: length=11

Sector size of disk:

kfdhdb.secsize:                     512 ; 0x0b8: 0x0200

Blocksize of disk:

kfdhdb.blksize:                    4096 ; 0x0ba: 0x1000

Provision string for use with asm:

kfdhdb.driver.provstr: ORCLDISKDISK003A

–> which finally means: “ORCL:DISK003A”

AU size:

kfdhdb.ausize:                  4194304

 

In a followup i will dig all little bit deeper in ASM and provide samples of different headers in different combinations (external / normal / high redundancy, ….).

Using ASM with files (either locally or via NFS)

From time to time i need to configure additional and temporary storage for data migration or just testing purposes. Raw Devices are not recommended to use from 11g onwards.

So why not use the loopback device for that? With this scenario you can even place these disks on NFS and use it across nodes in a cluster:

 

1. Create the file

dd if=/dev/zero bs=1024k count=1000 of=/disk1

 

2. Setup loopback driver

losetup /dev/loop1 /disk1

 

3. Label device

oracleasm createdisk LOOP1 /dev/loop1

 

4. Use it with ASM