Oracle 11g Release 2 – SSH equivalence gone after reboot

Today i came across a strange phenomenon:

I set up ssh equivalence during database installation but after rebooting the system some tool complained about misconfigured ssh equivalence. So i digged a little bit. I found out SSH is complaining about wrong directory permissions:

Sep 28 13:57:03 rac1 sshd[31620]: Authentication refused: bad ownership or modes for directory /u01/app/oracle/product/11.2.0/ora11p

The directory mentioned here is an ACFS shared oracle home and the home directory for the oracle user. After changing the mode for the directory to 755 ssh equivalence was restored. But after stopping and starting acfs resource the equivalence was gone again.

So i worked a little bit on that:

Lets first check directory permission with ACFS resource not started and not mounted:

[root@rac1 ~]# ll /u01/app/oracle/product/11.2.0/
total 4
drwxr-xr-x 2 ora11p dba 4096 Sep 13 00:02 ora11p
[root@rac1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              26G   13G   12G  52% /
/dev/sda1              99M   12M   83M  13% /boot
tmpfs                 1.9G  164M  1.7G   9% /dev/shm

Directory Mode is 775.

So now lets mount ACFS volume manually and check the permission again:

[root@rac1 ~]# mount /dev/asm/ora11p_home-132 /u01/app/oracle/product/11.2.0/ora11p/ -t acfs
[root@rac1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              26G   13G   12G  52% /
/dev/sda1              99M   12M   83M  13% /boot
tmpfs                 1.9G  164M  1.7G   9% /dev/shm
/dev/asm/ora11p_home-132
 10G  4.7G  5.4G  47% /u01/app/oracle/product/11.2.0/ora11p
[root@rac1 ~]# ll /u01/app/oracle/product/11.2.0/
total 16
drwxrwx--- 83 ora11p dba 12288 Sep 25 10:37 ora11p

775 – still no surprise. So we change them to 775 as required by SSH:

[root@rac1 ~]# chmod 755 /u01/app/oracle/product/11.2.0/ora11p
[root@rac1 ~]# ll /u01/app/oracle/product/11.2.0/
total 16
drwxr-xr-x 83 ora11p dba 12288 Sep 25 10:37 ora11p

The directory show required permissions. We will now unmount and mount the file system again:

[root@rac1 ~]# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda3             26306612  12828996  12119724  52% /
/dev/sda1               101086     11683     84184  13% /boot
tmpfs                  1933304    167016   1766288   9% /dev/shm
/dev/asm/ora11p_home-132
 10485760   4864328   5621432  47% /u01/app/oracle/product/11.2.0/ora11p
[root@rac1 ~]# umount /u01/app/oracle/product/11.2.0/ora11p
[root@rac1 ~]#
[root@rac1 ~]# umount /u01/app/oracle/product/11.2.0/ora11p
[root@rac1 ~]# mount /dev/asm/ora11p_home-132 /u01/app/oracle/product/11.2.0/ora11p/ -t acfs           
[root@rac1 ~]# ll /u01/app/oracle/product/11.2.0/
total 16
drwxr-xr-x 83 ora11p dba 12288 Sep 25 10:37 ora11p

There we are: Permission mode 775 is consistent accross mounts. So lets unmount everything and start acfs resource again:

[root@rac1 ~]# umount /u01/app/oracle/product/11.2.0/ora11p
[root@rac1 ~]#
[root@rac1 ~]# /u01/app/11.2.0/grid/bin/crsctl start resource  ora.data2.ora11p_home.acfs
CRS-2672: Attempting to start 'ora.data2.ora11p_home.acfs' on 'rac1'
CRS-2672: Attempting to start 'ora.data2.ora11p_home.acfs' on 'rac2'
CRS-2676: Start of 'ora.data2.ora11p_home.acfs' on 'rac2' succeeded
CRS-2676: Start of 'ora.data2.ora11p_home.acfs' on 'rac1' succeeded
[root@rac1 ~]# ll /u01/app/oracle/product/11.2.0/
total 16
drwxrwx--- 83 ora11p dba 12288 Sep 25 10:37 ora11p

The directory mode 770 was changed to 770…. but by which component and configuration?

Until now i did not find a solution for this behavior yet. I will open a SR and see what Oracle thinks about this “feature”.

In the meantime to get rid of this behavior you can set “StrictModes no” in sshd_config to prevent ssh from checking directory ownerships.