ibacm fixes/updates

ibacm fixes/updates

[Doug Ledford]

[-- Attachment #1.1: Type: text/plain, Size: 2532 bytes --]

Hi Sean,

We were asked to integrate this into our product for scalability
purposes.  While working on that, I ran across a number of issues that
needed fixed up (typical for a new package).  So, here's a number of
patches and fix ups for those things.

First, the package is ibacm but the binary is ib_acm.  Trust me, it's
best to pick one of the other and stick with it.  Especially since you
used ibacm for the log and pid files too.  In my build, I went ahead and
changed the binary name to match the package and log and pid file names.
 I patched the Makefile.am and reran the autoconf tools to make this
happen.  I've attached that patch.

In the package spec file don't differentiate between %{ver} and
%{version}, these should always be the same and you should never need to
use anything other than %{version}.  Your spec file had a srv component,
that I suspect was intended to hold the init script, but no init script
was ever written and the section was incomplete (so your spec file
wouldn't even build).  I rewrote the spec file so that the base
component is the service.  In truth, for this you really want it to be a
service and on by default if installed.  I've attached my spec file.
I've also attached the init script I made.

By default ibacm expects to find its configuration files in /etc/ibacm.
 This adds to the proliferation of directories in /etc/ needlessly.  We
already have a number of RDMA related directories to choose from
depending on your install (OFED == /etc/ofed or /etc/openib in the old
days, RHEL5 == /etc/openib from the old days, RHEL6 and Fedora are
/etc/rdma, don't know what SuSE uses).  Would be best if these files
were in the same place as the other RDMA related files.  And to make
that happen easily, it would be best if they picked up on %{sysconfdir}
from the ./configure invocation in order to set the directory.  I used a
static patch for now because I'm behind on my work, so updating the
configure script was more than I had time to do.  I'm not bothering to
include my patch as it needs done the other way so my patch is purely a
stopgap that should not see the time of day ;-)

OK, gotta run.  Would like to see this stuff picked up, especially the
init stuff.  Getting it in your release before we have one init script,
SuSE another, and OFED another will help keep things uniform.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD
	      http://people.redhat.com/dledford


[-- Attachment #1.2: ibacm.init --]
[-- Type: text/plain, Size: 1905 bytes --]

#!/bin/bash
#
# Bring up/down the ibacm daemon
#
# chkconfig: 2345 25 75
# description: Starts/Stops InfiniBand ACM service
#
### BEGIN INIT INFO
# Provides:       ibacm
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Required-Start: rdma $network
# Required-Stop: rdma $network
# Should-Start: opensm
# Should-Stop: opensm
# Short-Description: Starts and stops the InfiniBand ACM service
# Description: The InfiniBand ACM service provides a user space implementation
#	of someting resembling an ARP cache for InfiniBand SA queries and
#	host route lookups.
### END INIT INFO

pidfile=/var/run/ibacm.pid
subsys=/var/lock/subsys/ibacm
prog=/usr/sbin/ibacm

. /etc/rc.d/init.d/functions

start()
{
    echo -n "Starting ibacm daemon:"

    daemon $prog
    RC=$?
    [ $RC -eq 0 ] && touch $subsys
    echo
    return $RC
}

stop()
{
    echo -n "Stopping ibacm daemon:"

    killproc -p $pidfile $prog
    RC=$?
    rm -f $subsys
    echo
    return $RC
}

status()
{
    if [ ! -f $subsys -a ! -f $pidfile ]; then
    	return 3
    fi
    if [ -f $pidfile ]; then
	checkpid `cat $pidfile`
	return $?
    fi
    if [ -f $subsys ]; then
	return 2
    fi
}

restart ()
{
    stop
    start
}

condrestart ()
{
    [ -e $subsys ] && restart || return 0
}

usage ()
{
    echo
    echo "Usage: `basename $0` {start|stop|restart|condrestart|try-restart|force-reload|status}"
    echo
    return 2
}

case $1 in
    start|stop|restart|condrestart|try-restart|force-reload)
	[ `id -u` != "0" ] && exit 4 ;;
esac

case $1 in
    start) start; RC=$? ;;
    stop) stop; RC=$? ;;
    restart) restart; RC=$? ;;
    reload) RC=3 ;;
    condrestart) condrestart; RC=$? ;;
    try-restart) condrestart; RC=$? ;;
    force-reload) condrestart; RC=$? ;;
    status) status; RC=$? ;;
    *) usage; RC=$? ;;
esac

exit $RC

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.3: ibacm.spec --]
[-- Type: text/x-rpm-spec; name="ibacm.spec", Size: 3166 bytes --]

Name: ibacm
Version: 1.0.5
Release: 1%{?dist}
Summary: InfiniBand Communication Manager Assistant
Group: System Environment/Daemons
License: GPLv2 or BSD
Url: http://www.openfabrics.org/
Source: http://www.openfabrics.org/downloads/rdmacm/%{name}-%{version}.tar.gz
Source1: ibacm.init
Patch0: ibacm-1.0.5-make.patch
Patch1: ibacm-1.0.5-conf.patch
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
BuildRequires: libibverbs-devel >= 1.1-1, autoconf, libtool, libibumad-devel
Requires(post): chkconfig
Requires(preun): chkconfig
ExcludeArch: s390 s390x

%description
The ib_acm daemon helps reduce the load of managing path record lookups on
large InfiniBand fabrics by providing a user space implementation of what
is functionally similar to an ARP cache.  The use of ib_acm, when properly
configured, can reduce the SA packet load of a large IB cluster from O(n^2)
to O(n).  The ib_acm daemon is started and normally runs in the background,
user applications need not know about this daemon as long as their app
uses librdmacm to handle connection bring up/tear down.  The librdmacm
library knows how to talk directly to the ib_acm daemon to retrieve data.

%package devel
Summary: Headers file needed when building apps to talk directly to ib_acm
Requires: %{name} = %{version}-%{release}

%description devel
Most applications do not need to know how to talk directly to the ib_acm
daemon, but it does have a socket that it listens on, and it has a
specific protocol for incoming/outgoing data.  So if you wish to build
the ability to communicate directly with ib_acm into your own application,
the protocol used to communicate with it, and the data structures
involved, are in this header file.  Please note that this is an unsupported
method of using this daemon.  The only supported means of using this is
via librdmacm.  As such, even though this header file is provided, no
further documentation is available.  One must read the source if they
wish to make use of this header file.

%prep
%setup -q -n %{name}-%{version}
%patch0 -p1 -b .make
%patch1 -p1 -b .conf

%build
aclocal -I config && libtoolize --force --copy && autoheader && \
	automake --foreign --add-missing --copy && autoconf
%configure --sysconfdir=/etc/rdma CFLAGS="$CXXFLAGS -fno-strict-aliasing" LDFLAGS=-lpthread
mv man/ib_acm.1 man/ibacm.1
mv man/ib_acm.7 man/ibacm.7
make %{?_smp_mflags}

%install
rm -rf $RPM_BUILD_ROOT
make DESTDIR=%{buildroot} install
install -D -m 755 %{SOURCE1} %{buildroot}%{_initrddir}/ibacm

%clean
rm -rf $RPM_BUILD_ROOT

%post
if [ $1 = 1 ]; then
	chkconfig --add ibacm
fi

%preun
if [ $1 = 0 ]; then
	chkconfig --del ibacm
fi

%files
%defattr(-,root,root,-)
%doc AUTHORS COPYING README
%{_bindir}/ib_acme
%{_sbindir}/ibacm
%{_mandir}/man1/*
%{_mandir}/man7/*
%config(noreplace) %{_sysconfdir}/rdma/*
%{_initrddir}/ibacm

%files devel
%defattr(-,root,root,-)
%{_includedir}/infiniband/acm.h

%changelog
* Tue Feb 28 2012 Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> - 1.0.5-1
- Ininital version for rhel6
- Related: bz700285


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.4: ibacm-1.0.5-make.patch --]
[-- Type: text/x-patch; name="ibacm-1.0.5-make.patch", Size: 1330 bytes --]

diff -up ibacm-1.0.5/Makefile.am.dist ibacm-1.0.5/Makefile.am
--- ibacm-1.0.5/Makefile.am.dist	2010-12-10 15:05:18.000000000 -0500
+++ ibacm-1.0.5/Makefile.am	2012-02-28 13:25:36.662261951 -0500
@@ -3,10 +3,10 @@ INCLUDES = -I$(srcdir)/include -I$(srcdi
 AM_CFLAGS = -g -Wall -D_GNU_SOURCE
 
 bin_PROGRAMS = util/ib_acme
-sbin_PROGRAMS = svc/ib_acm
-svc_ib_acm_SOURCES = src/acm.c
+sbin_PROGRAMS = svc/ibacm
+svc_ibacm_SOURCES = src/acm.c
 util_ib_acme_SOURCES = src/acme.c linux/acme_linux.c src/libacm.c linux/libacm_linux.c src/parse.c
-svc_ib_acm_CFLAGS = $(AM_CFLAGS)
+svc_ibacm_CFLAGS = $(AM_CFLAGS)
 util_ib_acme_CFLAGS = $(AM_CFLAGS)
 
 ibacmincludedir = $(includedir)/infiniband
@@ -15,12 +15,19 @@ ibacminclude_HEADERS = include/infiniban
 
 man_MANS = \
 	man/ib_acme.1 \
-	man/ib_acm.1 \
-	man/ib_acm.7
+	man/ibacm.1 \
+	man/ibacm.7
 
 EXTRA_DIST = src/acm_mad.h src/libacm.h \
 	     linux/osd.h linux/dlist.h ibacm.spec.in $(man_MANS) acm_opts.cfg \
 	     acm_addr.cfg
 
+install-exec-hook:
+	if ! test -d $(DESTDIR)$(sysconfdir); then \
+		mkdir -p $(DESTDIR)$(sysconfdir); \
+	fi; \
+	install -m 644 acm_opts.cfg $(DESTDIR)$(sysconfdir)/acm_opts.cfg; \
+	install -m 644 acm_addr.cfg $(DESTDIR)$(sysconfdir)/acm_addr.cfg; 
+
 dist-hook: ibacm.spec
 	cp ibacm.spec $(distdir)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: ibacm fixes/updates

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 1597 bytes --]

On 02/28/2012 04:08 PM, Doug Ledford wrote:
> By default ibacm expects to find its configuration files in /etc/ibacm.
>  This adds to the proliferation of directories in /etc/ needlessly.  We
> already have a number of RDMA related directories to choose from
> depending on your install (OFED == /etc/ofed or /etc/openib in the old
> days, RHEL5 == /etc/openib from the old days, RHEL6 and Fedora are
> /etc/rdma, don't know what SuSE uses).  Would be best if these files
> were in the same place as the other RDMA related files.  And to make
> that happen easily, it would be best if they picked up on %{sysconfdir}
> from the ./configure invocation in order to set the directory.  I used a
> static patch for now because I'm behind on my work, so updating the
> configure script was more than I had time to do.  I'm not bothering to
> include my patch as it needs done the other way so my patch is purely a
> stopgap that should not see the time of day ;-)

Oh, and another issue, the pid and lock files are not in the LSB
standard locations:

Pid file: /var/run/
Lock file: /var/lock/subsys/  (this is created by the init script, and
it is doing the right thing here)

Right now the program creates the pid file in /var/lock (or
/var/run/lock, don't know which and they are linked so it shows up both
places), but not in /var/run which is the standard location for pid
files.  The port file can stay where it is.


-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD
	      http://people.redhat.com/dledford



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

RE: ibacm fixes/updates

[Hefty, Sean]

Thanks - I'll try to add these patches to the next release and work to get things fixed up.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: ibacm fixes/updates

[Hefty, Sean]

> By default ibacm expects to find its configuration files in /etc/ibacm.
>  This adds to the proliferation of directories in /etc/ needlessly.  We
> already have a number of RDMA related directories to choose from
> depending on your install (OFED == /etc/ofed or /etc/openib in the old
> days, RHEL5 == /etc/openib from the old days, RHEL6 and Fedora are
> /etc/rdma, don't know what SuSE uses).  Would be best if these files
> were in the same place as the other RDMA related files.  And to make
> that happen easily, it would be best if they picked up on %{sysconfdir}
> from the ./configure invocation in order to set the directory.  I used a
> static patch for now because I'm behind on my work, so updating the
> configure script was more than I had time to do.  I'm not bothering to
> include my patch as it needs done the other way so my patch is purely a
> stopgap that should not see the time of day ;-)

I've moved the config files to /etc/rdma.  I'm still trying to figure out how to make them use %{sysconfdir}, so I can avoid hard coding any paths.

Note that the config files are NOT needed by default.  If the option config file (acm_opts.cfg) is not found, the ibacm service will simply use the default settings.  The default values are the same as defined in the sample option file.

The acm_addr.cfg file included with the package is a sample file only.  The address file should be generated by running the ib_acme program on the same node.  Copying the acm_addr.cfg file included with the package will not work, since the addresses must be unique per node.  If no address config file is found, the ibacm service will automatically call ib_acme to generate one.

So... I think you want to remove the following install lines from your makefile.am patch.

+install-exec-hook:
+	if ! test -d $(DESTDIR)$(sysconfdir); then \
+		mkdir -p $(DESTDIR)$(sysconfdir); \
+	fi; \
+	install -m 644 acm_opts.cfg $(DESTDIR)$(sysconfdir)/acm_opts.cfg; \
+	install -m 644 acm_addr.cfg $(DESTDIR)$(sysconfdir)/acm_addr.cfg;

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ibacm fixes/updates

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 2312 bytes --]

On 04/02/2012 04:49 PM, Hefty, Sean wrote:
>> By default ibacm expects to find its configuration files in /etc/ibacm.
>>  This adds to the proliferation of directories in /etc/ needlessly.  We
>> already have a number of RDMA related directories to choose from
>> depending on your install (OFED == /etc/ofed or /etc/openib in the old
>> days, RHEL5 == /etc/openib from the old days, RHEL6 and Fedora are
>> /etc/rdma, don't know what SuSE uses).  Would be best if these files
>> were in the same place as the other RDMA related files.  And to make
>> that happen easily, it would be best if they picked up on %{sysconfdir}
>> from the ./configure invocation in order to set the directory.  I used a
>> static patch for now because I'm behind on my work, so updating the
>> configure script was more than I had time to do.  I'm not bothering to
>> include my patch as it needs done the other way so my patch is purely a
>> stopgap that should not see the time of day ;-)
> 
> I've moved the config files to /etc/rdma.  I'm still trying to figure out how to make them use %{sysconfdir}, so I can avoid hard coding any paths.
> 
> Note that the config files are NOT needed by default.  If the option config file (acm_opts.cfg) is not found, the ibacm service will simply use the default settings.  The default values are the same as defined in the sample option file.
> 
> The acm_addr.cfg file included with the package is a sample file only.  The address file should be generated by running the ib_acme program on the same node.  Copying the acm_addr.cfg file included with the package will not work, since the addresses must be unique per node.  If no address config file is found, the ibacm service will automatically call ib_acme to generate one.
> 
> So... I think you want to remove the following install lines from your makefile.am patch.
> 
> +install-exec-hook:
> +	if ! test -d $(DESTDIR)$(sysconfdir); then \
> +		mkdir -p $(DESTDIR)$(sysconfdir); \
> +	fi; \
> +	install -m 644 acm_opts.cfg $(DESTDIR)$(sysconfdir)/acm_opts.cfg; \
> +	install -m 644 acm_addr.cfg $(DESTDIR)$(sysconfdir)/acm_addr.cfg;

OK, will do.


-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD
	      http://people.redhat.com/dledford



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]