Tuesday, 2 September 2014

V3700 multipath

Finally I've found some time to actually provision GPFS on the v3700 storage array, we've a couple of x3650m4 systems which are SAS attached which will be running GPFS server. Before setting up GPFS, I needed to look at multipath for the system for fail over between HBA's and controllers. Right at the moment each HBA is connected to one controller, I think we probably want to get a couple more SAS cables to that we can cross connect each HBA to each controller as well.

Multipath itself is pretty easy to get going, you need the device-mapper-multipath rpm installed. Create an /etc/multipath.conf file, I have a pretty basic one which should allow multiple paths to be active (see notes below on dual active though!)
defaults {
        user_friendly_names yes
        path_selector           "round-robin 0"
        path_grouping_policy    multibus

blacklist {

I've also configured the /etc/multipath/bindings file to have friendly names for each LUN id, e.g.:
v3700_ds01_md_lun01 360012345001234567800000000000000
v3700_ds01_md_lun02 360012345001234567800000000000001
v3700_ds01_nls_lun01 360012345001234567800000000000002
v3700_ds01_nls_lun02 360012345001234567800000000000003
v3700_ds01_nls_lun03 360012345001234567800000000000004
v3700_ds01_nls_lun04 360012345001234567800000000000005
v3700_ds01_nls_lun05 360012345001234567800000000000006
v3700_ds01_nls_lun06 360012345001234567800000000000007
v3700_ds01_nls_lun07 360012345001234567800000000000008
v3700_ds01_nls_lun08 360012345001234567800000000000009

But why aren't all the paths active?

When looking at the multipath config with it running (multipath -l), I noticed that all the paths weren't active - some were marked as status=enabled rather than status=active.

Of course, it turns out that the v3700 controllers are dual-active rather than active/active. i.e. they are both active, but a LUN is only active on one controller at a time, this means that when using multi path, we'll see that one port is in status active, and the second in status enabled:
v3700_ds01_md_lun01 (360012345001234567800000000000000) dm-2 IBM,2145
size=5.5T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=0 status=active
| `- 1:0:0:0 sdb 8:16  active undef running
`-+- policy='round-robin 0' prio=0 status=enabled

  `- 2:0:0:0 sdl 8:176 active undef running

So this is where I think some more SAS cables might be in order so that the second port on each SAS card can be cross connected to the other controller, this will mean that all cards will be attached to each controller once, and we'll have maximum connectivity and fault tolerance for the system that we can.

No comments:

Post a comment