ext3 – beware of periodic file system checks

Users running Oracle on Red Hat or Oracle Enterprise Linux will most probaly use ext3 as default file system. It is robust, well maintained, a journaling file system and supports file system sizes up to 32 TB.

Database servers must be “high available” – so they are booted infrequently if ever.

Even if ext3 is a journaling file system there are periodic full file system checks which will check the entire file system just like ext2 did after a crash. On todays large file systems this can take hours. Thats especially bad if you just “quickly” rebooted the server after applying a patch and now facing a long running file system check.

Many users complained about this behavior and said ext3 is a journaling file system and should be running without periodic checks. On the other hand this is a security feature. Even with journaling corruptions may be introduced and remain undetected for a long time.

So lets take a look at the defaults chosen on Oracle Enterprise Linux 5 Update 3 when creating a ext3 file system:

[root@rac1 ~]# mkfs.ext3 /dev/asm/testvol1-132
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
131072 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
 32768, 98304, 163840, 229376

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 22 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

The last two lines are the ones of interest: “The file system will be checked every 22 mounts or 180 days, whichever comes first”. On database serverss 22 mounts seems a long time but 180 days are quite short. So administrators should change these values accordingly. Turning it off completely is a loss of data protection but still possible.

You can check the mount counts, maximum mount counts and check intervals with the following command:

tune2fs -l /dev/asm/testvol1-132
[..]
Mount count:              0
Maximum mount count:      22
[..]
Last checked:             Thu Oct  1 10:44:53 2009
Check interval:           15552000 (6 months)
Next check after:         Tue Mar 30 10:44:53 2010

You are free to change these values with tune2fs:

[root@rac1 ~]# tune2fs -c 100 /dev/asm/testvol1-132
tune2fs 1.39 (29-May-2006)
Setting maximal mount count to 100

[root@rac1 ~]# tune2fs -i 360 /dev/asm/testvol1-132
tune2fs 1.39 (29-May-2006)
Setting interval between checks to 31104000 seconds

…verifying:

tune2fs -l /dev/asm/testvol1-132
Filesystem created:       Thu Oct  1 10:44:53 2009
Last mount time:          n/a
Last write time:          Thu Oct  1 11:02:09 2009
Mount count:              0
Maximum mount count:      100
Last checked:             Thu Oct  1 10:44:53 2009
Check interval:           31104000 (12 months)
Next check after:         Sun Sep 26 10:44:53 2010

So based on this information:

To which values should max mount count and check interval be set?

The actual values depend completely on your requirements. But they should be set completely different for different volumes. This will prevent a full check on all your volumes at once.