[PATCH rfc] nvme-cli: NVMf auto discovery systemd service

[PATCH rfc] nvme-cli: NVMf auto discovery systemd service

[Sagi Grimberg]

Run a one-shot discovery (and connect) at system boot time. This
allows to auto discover and establish nvmf sessions.

If the user is interested in using this, it should fill in
one or more discovery endpoints, for example:
-t rdma -a 192.168.1.1 -s 4420

This service relies on nvme-fabrics module being loaded, hence
we add it to modules load at boot list.

Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
Don't have a fedora station near-by, manually tested auto
discover during boot with systemd service, and it seem to work
(although I needed to manually get mlx5_ib modules to auto-load
as for some reason it fails to auto load at boot time).

We can also add some documentation into discovery.conf to help
the user add discovery endpoints.

 nvme.spec.in                   | 19 +++++++++++++++++++
 systemd/nvmf-discovery.service | 12 ++++++++++++
 2 files changed, 31 insertions(+)
 create mode 100644 systemd/nvmf-discovery.service

diff --git a/nvme.spec.in b/nvme.spec.in
index 0be61e398fd0..c22bdeec9138 100644
--- a/nvme.spec.in
+++ b/nvme.spec.in
@@ -43,12 +43,31 @@ if [ $1 = 1 ]; then # 1 : This package is being installed for the first time
         if [ ! -f /etc/nvme/hostid ]; then
                 uuidgen > /etc/nvme/hostid
         fi
+        if [ ! -f /etc/nvme/discovery.conf ]; then
+		touch /etc/nvme/discovery.conf
+        fi
 fi
 
+# NVMf auto-discovery service
+install -m 0755 systemd/nvmf-discovery.service /lib/systemd/system/
+systemctl enable nvmf-discovery.service
+
+# we rely on /dev/nvme-fabrics misc device, so auto load nvme-fabrics
+# at boot time
+echo nvme-fabrics >> /etc/modules-load.d/modules.conf
+
 %preun
 if [ "$1" = "remove" ]; then
+	sed -i 's/nvme-fabrics//e' /etc/modules-load.d/modules.conf
+
+	if [ -f /lib/systemd/system/nvmf-discovery.service]; then
+		systemctl disable nvmf-discovery.service
+		rm -f /lib/systemd/system/nvmf-discovery.service
+	fi
+
 	if [ -d /etc/nvme ]; then
 		rm -f /etc/nvme/hostnqn
+		rm -f /etc/nvme/discovery.conf
 		if [ ! -n "`ls -A /etc/nvme`" ]; then
 			rm -rf /etc/nvme
 		fi
diff --git a/systemd/nvmf-discovery.service b/systemd/nvmf-discovery.service
new file mode 100644
index 000000000000..25f7bf1f077b
--- /dev/null
+++ b/systemd/nvmf-discovery.service
@@ -0,0 +1,12 @@
+[Unit]
+Description=NVMf auto discovery service
+After=systemd-modules-load.service network-online.target
+
+[Service]
+Type=oneshot
+ExecStart=nvme connect-all
+StandardOutput=journal
+StandardError=journal
+
+[Install]
+WantedBy=multi-user.target
-- 
2.7.4

[PATCH rfc] nvme-cli: NVMf auto discovery systemd service

[Christoph Hellwig]

This looks reasoanable to me, although I'm really not the expert
for systemd stuff.

Does it handle the case where /etc/nvme/discovery.conf is non-existent
(yes, you created in the specfile, but not everyone uses rpms).

> We can also add some documentation into discovery.conf to help
> the user add discovery endpoints.

That would be great for the default one, e.g. some example RDMA
and FC configfs.

[PATCH rfc] nvme-cli: NVMf auto discovery systemd service

[Sagi Grimberg]

> This looks reasoanable to me, although I'm really not the expert
> for systemd stuff.

Me neither, Hence the RFC...

> Does it handle the case where /etc/nvme/discovery.conf is non-existent
> (yes, you created in the specfile, but not everyone uses rpms).

Well, it will fail the nvme connect-all command, which would appear
in the journal and disable the service, the user is expected to
add discovery.conf and re-enable the service at that point...

>> We can also add some documentation into discovery.conf to help
>> the user add discovery endpoints.
> 
> That would be great for the default one, e.g. some example RDMA
> and FC configfs.

I think James was pushing udev stuff for auto connection as FC
discovery is in-band, so not sure how this will apply to FC...

[PATCH rfc] nvme-cli: NVMf auto discovery systemd service

[Johannes Thumshirn]

On Sun, Sep 24, 2017@04:01:35PM +0300, Sagi Grimberg wrote:
> Run a one-shot discovery (and connect) at system boot time. This
> allows to auto discover and establish nvmf sessions.
> 
> If the user is interested in using this, it should fill in
> one or more discovery endpoints, for example:
> -t rdma -a 192.168.1.1 -s 4420
> 
> This service relies on nvme-fabrics module being loaded, hence
> we add it to modules load at boot list.
> 
> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
> ---
> Don't have a fedora station near-by, manually tested auto
> discover during boot with systemd service, and it seem to work
> (although I needed to manually get mlx5_ib modules to auto-load
> as for some reason it fails to auto load at boot time).
> 
> We can also add some documentation into discovery.conf to help
> the user add discovery endpoints.
> 
>  nvme.spec.in                   | 19 +++++++++++++++++++
>  systemd/nvmf-discovery.service | 12 ++++++++++++
>  2 files changed, 31 insertions(+)
>  create mode 100644 systemd/nvmf-discovery.service
> 
> diff --git a/nvme.spec.in b/nvme.spec.in
> index 0be61e398fd0..c22bdeec9138 100644
> --- a/nvme.spec.in
> +++ b/nvme.spec.in
> @@ -43,12 +43,31 @@ if [ $1 = 1 ]; then # 1 : This package is being installed for the first time
>          if [ ! -f /etc/nvme/hostid ]; then
>                  uuidgen > /etc/nvme/hostid
>          fi
> +        if [ ! -f /etc/nvme/discovery.conf ]; then
> +		touch /etc/nvme/discovery.conf
> +        fi

I don't think having an empty discovery.conf is a particularly good idea, the
contents should be either added by hand or by some configuration/installer
tool (YAST, Anaconda, Chef, Puppet, yada yada).


>  fi
>  
> +# NVMf auto-discovery service
> +install -m 0755 systemd/nvmf-discovery.service /lib/systemd/system/
> +systemctl enable nvmf-discovery.service

Unconditionally starting services will is (at least for SUSE RPMs) a no-go.

> +
> +# we rely on /dev/nvme-fabrics misc device, so auto load nvme-fabrics
> +# at boot time
> +echo nvme-fabrics >> /etc/modules-load.d/modules.conf

This should be done in the systemd unit file I think. Something like this:

> diff --git a/systemd/nvmf-discovery.service b/systemd/nvmf-discovery.service
> new file mode 100644
> index 000000000000..25f7bf1f077b
> --- /dev/null
> +++ b/systemd/nvmf-discovery.service
> @@ -0,0 +1,12 @@
> +[Unit]
> +Description=NVMf auto discovery service
> +After=systemd-modules-load.service network-online.target
> +
> +[Service]
> +Type=oneshot

ExecStartPre=-/sbin/modprobe nvme-fabrics


> +ExecStart=nvme connect-all
> +StandardOutput=journal
> +StandardError=journal
> +
> +[Install]
> +WantedBy=multi-user.target
> -- 
> 2.7.4
> 

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

[PATCH rfc] nvme-cli: NVMf auto discovery systemd service

[Sagi Grimberg]

>> +        if [ ! -f /etc/nvme/discovery.conf ]; then
>> +		touch /etc/nvme/discovery.conf
>> +        fi
> 
> I don't think having an empty discovery.conf is a particularly good idea, the
> contents should be either added by hand or by some configuration/installer
> tool (YAST, Anaconda, Chef, Puppet, yada yada).

Its just placing the file there to not fail the service by definition of
the conf not being there - if its empty it just silently exits. We can
skip it if you think its not a good idea.

>> +# NVMf auto-discovery service
>> +install -m 0755 systemd/nvmf-discovery.service /lib/systemd/system/
>> +systemctl enable nvmf-discovery.service
> 
> Unconditionally starting services will is (at least for SUSE RPMs) a no-go.

OK, we can lose it.

>> +
>> +# we rely on /dev/nvme-fabrics misc device, so auto load nvme-fabrics
>> +# at boot time
>> +echo nvme-fabrics >> /etc/modules-load.d/modules.conf
> 
> This should be done in the systemd unit file I think. Something like this:
> 
>> diff --git a/systemd/nvmf-discovery.service b/systemd/nvmf-discovery.service
>> new file mode 100644
>> index 000000000000..25f7bf1f077b
>> --- /dev/null
>> +++ b/systemd/nvmf-discovery.service
>> @@ -0,0 +1,12 @@
>> +[Unit]
>> +Description=NVMf auto discovery service
>> +After=systemd-modules-load.service network-online.target
>> +
>> +[Service]
>> +Type=oneshot
> 
> ExecStartPre=-/sbin/modprobe nvme-fabrics

looks better!