This is a short post about how to configure iSCSI multipathing using Nexenta. I let it is neccessary to document it since the documentation is rather spare about it.
In my lab setup i had a machine with two physically separated interfaces:
- e1000g0: 192.168.1.5/24 and
- e1000g1: 192.168.10.1/24
As you can see both interfaces are on different subnets. Note that it is a very good idea (i´d call it: “required”) to use a separate subnet for each interface. It will save you a lot of trouble and problems!
If you do not comply with this simple rule you will end up having problems with so called Arp Flux (also documented here, here, here and so on) which requires further modifications.
Note: If you modify and existing installation you only need to execute Step #2 and Step #3. Everything else should already be there.
Read more…
Starting with the Oracle Unbreakable Enterprise Linux Kernel (UEK) the ASMlib drivers are now included in the kernel. This means that updating the kernel does not require installing the appropriate kernel modules as well.
In addition to the kernel modules you need the following packages:
- oracleasm-support
- oracleasmlib
For Oracle Enterprise Linux “oracleasm-support” can be found on the installation media while “oracleasmlib” is not available on the installation media. “oracleasmlib” can be downloaded over the ULN network (more information here).
If you dont have a ULN account or using Red Hat you can get the freely available packages (for OEL and Red Hat) here.
When using Nexenta it is highly recommended to attach the syspool disks via AHCI instead of SATA. Attaching them via SATA might cause I/O errors leading to complete outages. The only way to solve this is to replace the driver with AHCI.
Normally changing the driver for the syspool requires a complete reinstallation. However if you want to avoid this simple follow this guide:
- Boot with Nexenta Setup CD; Choose “Rescue System”
- when booted import the syspool with:
zpool import -f syspool
- Install boot loader on ALL disks belonging to the syspool, e.g. for the first disk:
grub-install /dev/rdsk/c0t0d0s0
- Export the syspool:
zpool export syspool
- Reboot
Done. The system now uses AHCI drivers.
I just migrated my Ubuntu iSCSI storage server to Ubuntu 12.04 which ships with a new iSCSI implementation: LIO (linux-iscsi.org). Aside from the fact that this project lacks a HUGE amount of documentation I want to share this problem with you:
kernel: Exiting Time2Retain handler because session_reinstatement=1
This happens if you access LUNs from more multiple nodes (as required for RAC setups) while all nodes share the same initiatorname.
The fix is to use different and unique initiatornames for every node.