While you can also do quite some LUN mapping on the level of an FC switch I'd like to write here about what there are for possibilities on the level of the OpenSolaris / illumos STMF:
Last time I simply added a view to a LUN for everybody which isn't a good idea if you are not using a cluster-aware filesystem (i.e. GFS, OCFS, or NTFS as a cluster-shared volume). Now we need to restrict the access to a LUN: For this STMF allows us to create host groups (hg) and target groups (tg).
Last time I have mapped a LUN to everyone, now I want to restrict the access to this LUN to a node called 'akebono' so let's create a host group for all the HBAs installed in this akebono:
# stmfadm create-hg akebono # stmfadm list-hg Host Group: akebono
In the FC aread each HBA card has a unique WWNN (world wide node name) an since there are HBAs with more than one port, each port has its WWPN (worldwide port name), this is a 64-bit value like the MAC address that every network controller has. stmfadm allows to add different types of names to a host or target group it IQNs (iSCSI qualified name) or WWNs (worldwide names are also used in SAS). We have to know the WWPN from the initiating HBAs. There are several ways to get the WWPNs (and possibly others):
- Use vendor-provided tools for each platform (in this case this might be the Qlogic SANsurfer CLI)
- (Linux/BSD/Unix) Lookup dmesg when the HBA driver gets loaded
- Sometimes it's written on a lable on the HBA
But if you have a small FC environment, then you can cheat a little:
# stmfadm list-target -v Target: wwn.50060B0000655664 Operational Status: Online Provider Name : qlt Alias : qlt3,0 Protocol : Fibre Channel Sessions : 1 Initiator: wwn.210000E08B9BE2DF Alias: - Logged in since: Sat Sep 3 02:15:56 2011 Target: wwn.2101001B323FE743 Operational Status: Online Provider Name : qlt Alias : qlt2,0 Protocol : Fibre Channel Sessions : 0 Target: wwn.2100001B321FE743 Operational Status: Online Provider Name : qlt Alias : qlt1,0 Protocol : Fibre Channel Sessions : 1 Initiator: wwn.210000E08B9BF0E1 Alias: - Logged in since: Sat Sep 3 02:15:56 2011 Target: wwn.50060B000065566E Operational Status: Online Provider Name : qlt Alias : qlt0,0 Protocol : Fibre Channel Sessions : 1 Initiator: wwn.210000E08B9BF0E1 Alias: - Logged in since: Sat Sep 3 02:15:56 2011
The Nexenta box has 4 HBAs (3 Host Bus Adapters are connected, 2 to the same switch), so what we can now see are the WWNs of the targets and those of the (yet) single initiating node. Now we can add them to the host group - don't forget to have ww.<yourWWPN> because that's how STMF distinguishes between iSCSI IQNs, FC & SAS WWNs and EUI's (Extended Unique Identifier):
# stmfadm add-hg-member -g akebono wwn.210000E08B9BF0E1 # stmfadm add-hg-member -g akebono wwn.210000E08B9BE2DF # stmfadm list-hg -v Host Group: akebono Member: wwn.210000E08B9BF0E1 Member: wwn.210000E08B9BE2DF
Now we can delete our mapped LUN and re-map it properly so only HBAs in the host group akebono will see this LUN and be able to mount it:
# stmfadm list-view -l 600144f098680b0000004e632dc60004 View Entry: 0 Host group : All Target group : All LUN : 2 # stmfadm remove-view -a -l 600144f098680b0000004e632dc60004 # stmfadm list-view -l 600144f098680b0000004e632dc60004 stmfadm: 600144f098680b0000004e632dc60004: no views found # stmfadm add-view -h akebono 600144f098680b0000004e632dc60004 # stmfadm list-view -l 600144f098680b0000004e632dc60004 View Entry: 0 Host group : akebono Target group : All LUN : 2
Voilà - well that's it: If you want to further restrict a node to utilize only let's say 2 out of 4 HBAs you can also create target groups too - currently akebono will be able to connect to this LUN over every reachable target path (be it FC or every other target, i.e. iSCSI). There is also a possibility to group all FC ports together, but be aware that in order to add a any target to a target group, you will have to offline it for a short period (this is now problem if you have fully working multipathing):
# stmfadm create-tg fc-ports # stmfadm add-tg-member -g fc-ports wwn.50060B000065566E stmfadm: STMF target must be offline # stmfadm offline-target wwn.50060B000065566E # stmfadm add-tg-member -g fc-ports wwn.50060B000065566E # stmfadm online-target wwn.50060B000065566E
This offline-online procedure seems to be mandatory for every target added to a target group. Later on you can (if you want) add a view to a LUN by also adding '-t <targetGroupName>' including the host group like before. - It might also be good thing if you want to manually balance the load on all of your target-mode HBAs.
Next up: Setting up multipathing on Linux (Debian and Scientific Linux) and Windows (2008 R2).