stafd.conf — stafd(8) configuration file
/etc/stas/stafd.conf
stafd.conf
is a plain text file divided into
sections, with configuration entries in the style
key
=value
.
Spaces immediately before or after the "=
" are
ignored. Empty lines are ignored as well as lines starting with
"#
", which may be used for commenting.
The following options are available in the
"[Global]
" section:
tron=
Trace ON. Takes a boolean argument. If true
,
enables full code tracing. The trace will be displayed in
the system log such as systemd's journal. Defaults to
false
.
hdr-digest=
Enable Protocol Data Unit (PDU) Header Digest. Takes a
boolean argument. NVMe/TCP facilitates an optional PDU
Header digest. Digests are calculated using the CRC32C
algorithm. If true
, Header Digests
are inserted in PDUs and checked for errors. Defaults to
false
.
data-digest=
Enable Protocol Data Unit (PDU) Data Digest. Takes a
boolean argument. NVMe/TCP facilitates an optional PDU
Data digest. Digests are calculated using the CRC32C
algorithm. If true
, Data Digests
are inserted in PDUs and checked for errors. Defaults to
false
.
kato=
Keep Alive Timeout (KATO) in seconds. Takes an unsigned integer. This field specifies the timeout value for the Keep Alive feature in seconds. Defaults to 30 seconds for Discovery Controller connections and 120 seconds for I/O Controller connections.
ip-family=
Takes a string argument. With this you can specify
whether IPv4, IPv6, or both are supported when
connecting to a Controller. Connections will not be
attempted to IP addresses (whether discovered or
manually configured with controller=
)
disabled by this option. If an invalid value
is entered, then the default (see below) will apply.
Choices are ipv4
, ipv6
, or ipv4+ipv6
.
Defaults to ipv4+ipv6
.
queue-size=
Takes a value in the range 16...1024.
Overrides the default number of elements in the I/O queues created by the driver. This option will be ignored for discovery, but will be passed on to the subsequent connect call.
Note: This parameter is identical to that provided by nvme-cli.
Defaults to 128
.
reconnect-delay=
Takes a value in the range 1 to N seconds.
Overrides the default delay before reconnect is attempted after a connect loss.
Note: This parameter is identical to that provided by nvme-cli.
Defaults to 10
. Retry to connect every 10 seconds.
ctrl-loss-tmo=
Takes a value in the range -1, 0, ..., N seconds. -1 means retry forever. 0 means do not retry.
Overrides the default controller loss timeout period (in seconds).
Note: This parameter is identical to that provided by nvme-cli.
Defaults to 600
seconds (10 minutes).
duplicate-connect=
Takes a boolean argument. Allows duplicated connections between same transport host and subsystem port.
Note: This parameter is identical to that provided by nvme-cli.
Defaults to false
.
disable-sqflow=
Takes a boolean argument. Disables SQ flow control to omit head doorbell update for submission queues when sending nvme completions.
Note: This parameter is identical to that provided by nvme-cli.
Defaults to false
.
ignore-iface=
Takes a boolean argument. This option controls how connections with Discovery Controllers (DC) are made.
DCs are automatically discovered using DNS-SD/mDNS. mDNS provides the DC's IP address and the interface on which the DC was discovered.
There is no guarantee that there will be a route to reach that DC. However, we can use the socket option SO_BINDTODEVICE to force the connection to be made on a specific interface instead of letting the routing tables decide where to make the connection.
This option determines whether stafd
will use SO_BINDTODEVICE to force connections on an
interface or just rely on the routing tables. The
default is to use SO_BINDTODEVICE, in other words,
stafd
does not ignore the interface by
default.
Defaults to false
.
pleo=
Port Local Entries Only. Takes a string argument
enabled
or
disabled
. This option is sent in
the LSP field (Log SPecific) of the Get Discovery Log
Page (DLP) command. It is used by stafd
to
tell Discovery Controllers (DC) whether the response to
a Get DLP command should contain all the NVM subsystems
or only those reachable by the host on the interface
where the Get DLP command was issued by the host.
This parameter was introduced in TP8010. When
pleo=
enabled
,
then the DC shall return records for only NVM subsystem
ports that are presented through the same NVM subsystem
port that received the Get Log Page command. When
pleo=
disabled
,
then the DC may return all the NVM subsystem ports
that it holds, even those that can only be reached
on NVM subsystem ports that did not receive the Get
Log Page command. In other words, the host may not
even be able to reach those subsystems.
Defaults to enabled
.
The following options are available in the
"[Service Discovery]
" section:
zeroconf=
Enable zeroconf provisioning using DNS-SD/mDNS.
Takes a string argument enabled
or
disabled
.
When enabled
, the default,
stafd
makes a request with the
Avahi daemon to locate Discovery Controllers using
DNS-SD/mDNS.
Discovery Controllers that support zeroconf advertize
themselves over mDNS with the service type
"_nvme-disc._tcp
".
Defaults to true
.
The following options are available in the
"[Discovery controller connection management]
" section:
persistent-connections=
Takes a boolean argument. Whether connections to
Discovery Controllers (DC) are persistent. When
true, connections initiated by stafd will persists
even when stafd is stopped. When
false
, stafd
will disconnect from all DCs it is connected to on
exit.
Defaults to false
.
zeroconf-connections-persistence=
Takes a unit-less value in seconds, or a time span value such as "72hours" or "5days". A value of 0 means no persistence. In other words, configuration acquired through zeroconf (mDNS service discovery) will be removed immediately when mDNS no longer reports the presence of a Discovery Controller (DC) and connectivity to that DC is lost. A value of -1 means that configuration acquired through zeroconf will persist forever.
This is used for the case where a DC that was discovered through mDNS service discovery no longer advertises itself through mDNS and can no longer be connected to. For example, the DC had some catastrophic failure (e.g. power surge) and needs to be replaced. In that case, the connection to that DC can never be restored and a replacement DC will be needed. The replacement DC will likely have a different NQN (or IP address). In that scenario, the host won't be able to determine that the old DC is not coming back. It won't know either that a newly discovered DC is really the replacement for the old one. For that reason, the host needs a way to "age" zeroconf-acquired configuration and remove it automatically after a certain amount of time. This is what this parameter is for.
Defaults to 72hours
.
The following options are available in the
"[Controllers]
" section:
controller=
Controllers are specified with the controller
option. This option may be specified more than once to specify
more than one controller. The format is one line per Controller
composed of a series of fields separated by semi-colons as follows:
controller=transport=[trtype];traddr=[traddr];trsvcid=[trsvcid];host-traddr=[traddr],host-iface=[iface];nqn=[nqn]
transport=
This is a mandatory field that specifies the
network fabric being used for a
NVMe-over-Fabrics network. Current
trtype
values understood
are:
Table 1. Transport type
trtype | Definition |
---|---|
rdma | The network fabric is an rdma network (RoCE, iWARP, Infiniband, basic rdma, etc) |
fc | The network fabric is a Fibre Channel network. |
tcp | The network fabric is a TCP/IP network. |
loop | Connect to a NVMe over Fabrics target on the local host |
traddr=
This is a mandatory field that specifies the network address of the Controller. For transports using IP addressing (e.g. rdma) this should be an IP-based address (ex. IPv4, IPv6). It could also be a resolvable host name (e.g. localhost).
trsvcid=
This is an optional field that specifies the transport service id. For transports using IP addressing (e.g. rdma, tcp) this field is the port number.
Depending on the transport type, this field will default to either 8009 or 4420 as follows.
UDP port 4420 and TCP port 4420 have been assigned by IANA for use by NVMe over Fabrics. NVMe/RoCEv2 controllers use UDP port 4420 by default. NVMe/iWARP controllers use TCP port 4420 by default.
TCP port 4420 has been assigned for use by NVMe over Fabrics and TCP port 8009 has been assigned by IANA for use by NVMe over Fabrics discovery. TCP port 8009 is the default TCP port for NVMe/TCP discovery controllers. There is no default TCP port for NVMe/TCP I/O controllers, the Transport Service Identifier (TRSVCID) field in the Discovery Log Entry indicates the TCP port to use.
The TCP ports that may be used for NVMe/TCP I/O controllers include TCP port 4420, and the Dynamic and/or Private TCP ports (i.e., ports in the TCP port number range from 49152 to 65535). NVMe/TCP I/O controllers should not use TCP port 8009. TCP port 4420 shall not be used for both NVMe/iWARP and NVMe/TCP at the same IP address on the same network.
nqn=
This field specifies the Controller's NVMe Qualified Name.
This field is mandatory for I/O Controllers, but is optional for
Discovery Controllers (DC). For the latter, the NQN will default
to the well-known DC NQN: "nqn.2014-08.org.nvmexpress.discovery
"
if left undefined.
host-traddr=
This is an optional field that specifies the network address used on the host to connect to the Controller. For TCP, this sets the source address on the socket.
host-iface=
This is an optional field that specifies the network interface used on the host to connect to the Controller (e.g. IP eth1, enp2s0, enx78e7d1ea46da). This forces the connection to be made on a specific interface instead of letting the system decide.
Examples:
controller = transport=tcp;traddr=localhost;trsvcid=8009 controller = transport=tcp;traddr=2001:db8::370:7334;host-iface=enp0s8 controller = transport=fc;traddr=nn-0x204600a098cbcac6:pn-0x204700a098cbcac6
exclude=
Controllers that should be excluded can be specified with the
exclude=
option. Using mDNS to
automatically discover and connect to controllers, can result
in unintentional connections being made. This keyword allows
configuring the controllers that should not be connected to.
The syntax is the same as for "controller", except that the parameter
host-traddr
does not apply. Multiple
exclude=
keywords may appear in the config
file to specify more than 1 excluded controller.
Note 1: A minimal match approach is used to eliminate unwanted
controllers. That is, you do not need to specify all the
parameters to identify a controller. Just specifying the
host-iface
, for example, can be used to
exclude all controllers on an interface.
Note 2: exclude=
takes precedence over
controller
. A controller specified by the
controller
keyword, can be eliminated by
the exclude=
keyword.
Examples:
exclude = transport=tcp;traddr=fe80::2c6e:dee7:857:26bb # Eliminate a specific address exclude = host-iface=enp0s8 # Eliminate everything on this interface