SAN Protocols: Fibre Channel, iSCSI and NVMe over Fabrics

Initiators, targets and LUNs

A storage area network (SAN) moves block commands — "read 16 blocks starting at block 8,192" — between servers and storage arrays. Every SAN protocol uses the same three roles:

• The initiator is the side that sends commands: the server, through a host adapter or a software driver.
• The target is the side that executes them: a port on the storage array.
• Behind a target sit one or more LUNs (logical unit numbers) — the volumes the array presents (see LUNs). In NVMe the equivalent is a namespace.

For decades the command set was SCSI (Small Computer System Interface), the same language local SAS drives speak. Fibre Channel and iSCSI are two ways to carry SCSI across a network; NVMe over Fabrics swaps SCSI for the newer NVMe command set.

Fibre Channel

Fibre Channel (FC) is a network technology built only for storage. It has its own switches, adapters, addressing and protocol stack; it does not use Ethernet or IP. Its defining property is that it is lossless: a port may only send a frame when the receiver has signalled it has a free buffer (buffer-to-buffer credits), so frames are not dropped under load the way Ethernet drops packets. Storage traffic hates loss, because every lost frame means a retry and a latency spike. (The protocol is spelled "Fibre", and it runs over copper too; the cables in a data center are usually multimode optical fiber with LC connectors — see Cabling and the Physical Layer.)

Host bus adapters

A server connects to Fibre Channel through a host bus adapter (HBA): a PCIe card, usually with two ports, that handles the FC protocol in hardware and presents the LUNs it finds to the operating system as disks. Servers usually get two single- or dual-port HBAs, so that one failed card does not cut the server off. The term HBA is also used for SAS adapters without RAID, and iSCSI HBAs exist that offload iSCSI from the CPU; in FC it always means the FC card.

World Wide Names

Every FC device has a World Wide Name (WWN): a 64-bit identifier burned in at the factory, written as eight hexadecimal bytes such as 21:00:00:00:5e:00:53:01 (an example value). It plays the role a MAC address plays in Ethernet. There are two kinds:

• WWNN (World Wide Node Name): one per device, such as an HBA card or an array.
• WWPN (World Wide Port Name): one per port. This is the one you use in zoning and in the array's host definitions.

When a port connects, it logs in to the fabric (FLOGI) and receives a 24-bit FC address, then registers with the fabric's name server, so initiators can discover which targets exist. Virtualization adds virtual WWPNs (N_Port ID Virtualization, NPIV), which let one physical HBA port carry several WWPNs — for example one per VM or per blade.

Two fabrics, never connected

A fabric is a set of interconnected FC switches with shared services (name server, zoning). A properly built SAN has two fabrics, A and B, that are physically and logically separate: different switches, different power feeds where possible, and no inter-switch link between them. Every host has one HBA port in each fabric, and every array controller has ports in both. A fault in one fabric — a failed switch, a firmware bug, or a mistaken zoning change — can then only ever take down half the paths.

Zoning and LUN masking

By default every port in a fabric could talk to every other port. Zoning is the switch-level access list that prevents that: a zone is a set of WWPNs that may communicate, a zone set (zoning configuration) groups the zones of a fabric, and only the active zone set counts. The widely recommended practice is single-initiator zoning: each zone holds exactly one host port plus the array ports it needs. Hosts then never see each other, and a misbehaving HBA cannot disturb other servers.

zone name Z_DB01_HBA0__ARRAY1 vsan 10
  member pwwn 21:00:00:00:5e:00:53:01
  member pwwn 50:00:00:5e:00:53:00:10
  member pwwn 50:00:00:5e:00:53:00:20

zoneset name FABRIC_A vsan 10
  member Z_DB01_HBA0__ARRAY1

zoneset activate name FABRIC_A vsan 10

The zone holds host db01's hba0 (the initiator) and the fabric A ports of both array controllers (the targets); all WWPNs are example values. The VSAN number is Cisco's way of running several virtual fabrics on one switch. Brocade switches use different commands (zonecreate, cfgadd, cfgenable) for the same ideas.

Zoning decides which ports can see each other; LUN masking on the array decides which LUNs each host may see through those ports. Arrays implement it with host definitions (a host object listing its WWPNs or IQNs) that are mapped to volumes. You need both: zoning protects the fabric, masking protects the data. A boot LUN visible to the wrong server can be overwritten by an installer in seconds.

Speeds

FC generations are named by speed: 8GFC, 16GFC, 32GFC, 64GFC. As of 2026, new installations typically use 32GFC or 64GFC. The INCITS T11 committee completed the 128GFC standard (FC-PI-8) in 2023 and has moved on to 256GFC; 128GFC products are only emerging. Switches and HBAs negotiate down to older speeds, but usually only a couple of generations back, so check compatibility before mixing very old and new equipment.

FCoE

Fibre Channel over Ethernet (FCoE) puts FC frames directly into Ethernet frames — no IP, no TCP — so that one converged network adapter and one Ethernet network can carry both LAN and SAN traffic. Because FC assumes a lossless network, FCoE needs Ethernet switches with Data Center Bridging (DCB) features such as priority flow control, and it cannot be routed over IP. It saved cabling in blade and converged systems, but in practice it is far less common than native FC or iSCSI.

iSCSI

iSCSI (Internet Small Computer Systems Interface) carries SCSI commands inside TCP connections, over ordinary Ethernet and IP. Targets listen on TCP port 3260. The current specification is RFC 7143 (2014), which consolidated and replaced the original RFC 3720. Because it runs on standard network equipment, iSCSI is cheaper and easier to start with than FC, and with 25 or 100 Gigabit Ethernet it is fast enough for most workloads.

The initiator can be software (built into Windows, Linux, ESXi and others, using the normal network cards) or a hardware iSCSI HBA that offloads the work. The target side on the array exposes one or more portals (IP address and port). A host discovers the targets behind a portal, logs in to them, and gets a session per target and path.

iSCSI names

Initiators and targets have names that stay the same when IP addresses change. The common format is the iSCSI Qualified Name (IQN):

iqn.yyyy-mm.reversed-domain[:unique-name]

iqn.2026-10.com.example:db01            a host (initiator)
iqn.2026-10.com.example:array1.ct0      an array target

The date is a year and month in which the naming authority owned the domain, and the domain is written in reverse; everything after the colon is chosen by the owner. Two other formats exist: eui. followed by 16 hexadecimal digits (an IEEE EUI-64 identifier) and naa. (a Fibre Channel-style NAA identifier). Arrays grant access by initiator name, so a typo in an IQN simply means "no LUNs". A small check before names go into a host definition or a zone:

# Check storage identifiers before they go into an access list or a zone.
import re

IQN = re.compile(r"iqn\.(\d{4})-(\d{2})\.([a-z0-9-]+(?:\.[a-z0-9-]+)+)(?::(.+))?")
EUI = re.compile(r"eui\.[0-9A-F]{16}")
WWPN = re.compile(r"(?:[0-9a-f]{2}:){7}[0-9a-f]{2}")


def check(name):
    if m := IQN.fullmatch(name):
        year, month, domain, label = m.groups()
        if not 1 <= int(month) <= 12:
            return f"IQN with invalid month {month}"
        return f"IQN: authority {domain} ({year}-{month}), name {label}"
    if EUI.fullmatch(name):
        return "EUI-64 iSCSI name"
    if WWPN.fullmatch(name.lower()):
        return "WWN (8 bytes)"
    if name.lower().startswith("iqn."):
        return "not a valid IQN: expected iqn.yyyy-mm.reversed.domain[:name]"
    return "unknown format (normalize the separators first)"


names = [
    "iqn.2026-10.com.example:db01",
    "iqn.2026-10.com.example:array1.ct0",
    "iqn.2026-13.com.example:db02",
    "iqn.com.example:db03",
    "eui.02004567A425678D",
    "21:00:00:00:5e:00:53:01",
    "21:00:00:00:5E:00:53:02",
    "21-00-00-00-5e-00-53-03",
]
for name in names:
    print(f"{name:36} {check(name)}")

iqn.2026-10.com.example:db01         IQN: authority com.example (2026-10), name db01
iqn.2026-10.com.example:array1.ct0   IQN: authority com.example (2026-10), name array1.ct0
iqn.2026-13.com.example:db02         IQN with invalid month 13
iqn.com.example:db03                 not a valid IQN: expected iqn.yyyy-mm.reversed.domain[:name]
eui.02004567A425678D                 EUI-64 iSCSI name
21:00:00:00:5e:00:53:01              WWN (8 bytes)
21:00:00:00:5E:00:53:02              WWN (8 bytes)
21-00-00-00-5e-00-53-03              unknown format (normalize the separators first)

Switches, arrays and operating systems write WWNs with colons, with dashes, or without separators, and in either case. Normalize both sides before you compare an HBA's WWPN with a zone member.

Connecting a Linux host

On Linux, the open-iscsi tools do the work. The host's own IQN is in /etc/iscsi/initiatorname.iscsi; the array administrator adds it to the host definition first. Then the host discovers and logs in:

$ sudo iscsiadm -m discovery -t sendtargets -p 198.51.100.50
$ sudo iscsiadm -m node -T iqn.2026-10.com.example:array1.ct0 -p 198.51.100.50 --login

198.51.100.50:3260,1 iqn.2026-10.com.example:array1.ct0
198.51.100.51:3260,1 iqn.2026-10.com.example:array1.ct1
Logging in to [iface: default, target: iqn.2026-10.com.example:array1.ct0, portal: 198.51.100.50,3260]
Login to [iface: default, target: iqn.2026-10.com.example:array1.ct0, portal: 198.51.100.50,3260] successful.

After login, the LUNs appear as new SCSI disks (/dev/sdX) — one per path. That is why multipathing, below, is not optional.

Designing the iSCSI network

iSCSI on "ordinary" Ethernet does not mean iSCSI on the office network. The usual rules:

• Separate it. Dedicated switches, or at least dedicated VLANs (see VLANs), with no user traffic. iSCSI is not encrypted, and storage performance must not depend on someone's file download.
• Two paths, two networks. Mirror the FC A/B design: two NICs per host, two switches, often two IP subnets. The exact layout (one subnet or two, port binding or not) depends on the array and hypervisor; follow the vendor's guide.
• Do not route it. Keep initiators and targets in the same layer-2 segment; a router or firewall in the path adds latency and another failure point.
• Jumbo frames all the way or not at all. An MTU of 9000 bytes (see MTU and jumbo frames) can lower CPU load, but it must match on host NICs, every switch port and the array. A single mismatch makes small packets (logins, pings) work and large reads hang — a classic, confusing failure.
• Authenticate. Use CHAP (Challenge-Handshake Authentication Protocol), ideally mutual, in addition to the array's initiator-name access lists.

NVMe over Fabrics

SCSI was designed when storage meant spinning disks. NVMe (Non-Volatile Memory Express) was designed for flash: thousands of parallel command queues instead of one, and far less protocol overhead per I/O (see NVMe). NVMe over Fabrics (NVMe-oF) extends that command set across a network, so a server can use flash in a shared array with latency close to that of a local NVMe drive. Instead of LUNs it presents namespaces; hosts and subsystems are named by NVMe Qualified Names (NQNs), such as nqn.2026-10.com.example:array1.

As of October 2026, the current versions are NVM Express Base Specification 2.4 and the NVMe over TCP Transport Specification 1.3, both ratified at the end of July 2026. On Linux, the nvme-cli tool discovers and connects:

$ sudo nvme discover -t tcp -a 198.51.100.60 -s 8009
$ sudo nvme connect -t tcp -a 198.51.100.60 -s 4420 -n nqn.2026-10.com.example:array1

NVMe has its own multipath mechanism, Asymmetric Namespace Access (ANA), which plays the role ALUA plays for SCSI (below). Support depends on the operating system, hypervisor version and array, so NVMe-oF is still checked against compatibility lists more strictly than iSCSI or FC.

The protocols side by side

Multipathing

With two host ports, two fabrics and two array controllers, a host sees the same LUN over four paths — and without help, the operating system would see four separate disks. Writing to them as if they were independent would corrupt the data. Multipathing software recognizes that the paths lead to one LUN (by its unique identifier), presents a single device, spreads I/O over the healthy paths, and silently stops using a path that fails. Every SAN-attached host needs it: Microsoft MPIO (Multipath I/O) on Windows, dm-multipath on Linux, the native multipathing plug-in in ESXi, and native NVMe multipath for NVMe-oF.

ALUA and path policies

Many dual-controller arrays use ALUA (Asymmetric Logical Unit Access): paths through the controller that currently owns a LUN are active/optimized; paths through the partner controller work but are slower (active/non-optimized). Multipathing sends I/O over optimized paths and keeps the others for failover. The path selection policy decides how:

• Round robin: rotate I/O over all optimized paths — the usual recommendation for active/active arrays.
• Most recently used (MRU): stick with one working path and only move after it fails; it does not move back automatically.
• Fixed or failover only: one preferred path, others as standby.
• Least queue depth or service time: send each I/O to the path that is least busy.

The array vendor publishes which policy to use for each operating system; follow it rather than guessing. On Linux, multipath -ll shows the result — here a 2 TiB LUN with two optimized paths (priority 50) and two non-optimized ones (priority 10):

mpatha (36000000000000000000000000000001a) dm-2 EXAMPLE,FlashArray
size=2.0T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 3:0:0:1 sdb 8:16 active ready running
| `- 4:0:0:1 sdd 8:48 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 3:0:1:1 sdc 8:32 active ready running
  `- 4:0:1:1 sde 8:64 active ready running

What survives which failure

The design in the diagram above, as a model: which paths remain after each failure, and which of them are optimized. When the owning controller fails, the array moves ownership to its partner, whose paths become optimized.

# Paths from one host to one LUN over two fabrics, and what survives each failure.
HOST_PORTS = {"hba0": "A", "hba1": "B"}            # HBA port -> fabric
ARRAY_PORTS = {                                     # array port -> (controller, fabric)
    "ct0.fc0": ("ct0", "A"), "ct0.fc1": ("ct0", "B"),
    "ct1.fc0": ("ct1", "A"), "ct1.fc1": ("ct1", "B"),
}
LUN_OWNER = "ct0"   # with ALUA, paths to the owning controller are "optimized"


def paths(failed):
    owner = LUN_OWNER if LUN_OWNER not in failed else "ct1"   # array fails over
    result = []
    for hba, host_fabric in HOST_PORTS.items():
        for port, (controller, fabric) in ARRAY_PORTS.items():
            if fabric != host_fabric:
                continue            # the two fabrics are never connected
            if {hba, port, controller, "fabric " + fabric} & failed:
                continue
            kind = "optimized" if controller == owner else "non-optimized"
            result.append(f"{hba}->{port} ({kind})")
    return result


scenarios = [
    set(),
    {"fabric A"},
    {"ct0"},
    {"hba1"},
    {"fabric A", "ct1"},
    {"hba0", "fabric B"},
]
for failed in scenarios:
    alive = paths(failed)
    label = " + ".join(sorted(failed)) or "nothing"
    print(f"Failed: {label:17} {len(alive)} path(s)")
    for path in alive:
        print(f"    {path}")
    if not alive:
        print("    all paths down: the LUN is unreachable")

Failed: nothing           4 path(s)
    hba0->ct0.fc0 (optimized)
    hba0->ct1.fc0 (non-optimized)
    hba1->ct0.fc1 (optimized)
    hba1->ct1.fc1 (non-optimized)
Failed: fabric A          2 path(s)
    hba1->ct0.fc1 (optimized)
    hba1->ct1.fc1 (non-optimized)
Failed: ct0               2 path(s)
    hba0->ct1.fc0 (optimized)
    hba1->ct1.fc1 (optimized)
Failed: hba1              2 path(s)
    hba0->ct0.fc0 (optimized)
    hba0->ct1.fc0 (non-optimized)
Failed: ct1 + fabric A    1 path(s)
    hba1->ct0.fc1 (optimized)
Failed: fabric B + hba0   0 path(s)
    all paths down: the LUN is unreachable

Every single failure leaves at least two paths. Two failures can still leave a path, as long as one fabric keeps a working route to a working controller (ct1 + fabric A leaves one). If the two failures take out one path set in each fabric — a dead HBA on fabric A while fabric B is down for maintenance — none are left. That is why you check that all paths are healthy before starting planned work on one fabric: a path that has been silently down for weeks turns routine maintenance into an outage.

Troubleshooting SAN access