From mboxrd@z Thu Jan  1 00:00:00 1970
Content-Type: multipart/mixed; boundary="===============7556333131658133571=="
MIME-Version: 1.0
From: Harris, James R <james.r.harris at intel.com>
Subject: Re: [SPDK] __libc_message calling assert during delete_nvmf_subsystem
Date: Mon, 01 Apr 2019 16:02:59 +0000
Message-ID: <7DB2844B-D350-4D0B-A457-BC79C0470369@intel.com>
List-ID: <spdk@lists.01.org>
To: spdk@lists.01.org

--===============7556333131658133571==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable



=EF=BB=BFOn 4/1/19, 8:15 AM, "SPDK on behalf of Shahar Salzman" <spdk-bounc=
es(a)lists.01.org on behalf of shahar.salzman(a)kaminario.com> wrote:

    Hi,
    =

    I am running with spdk 18.10.3, and crashing during delete_nvmf_subsyst=
em.
    This happens pretty consistently (every 2-3 attempts) when I do the del=
ete while the initiator is still connected.
    =

    I saw that bug #235 has the same stack trace as I am seeing, but I alre=
ady have the fix.
    I am digging into this, but would be glad to get some pointers as to wh=
ere to start looking, or more specifically, how did you know to look into t=
he lib/nvmf controller code according to this error.

Hi Shahar,

Stack frames #8 and #9 indicate maybe an out-of-tree implementation of some=
 parts of the reactor framework?  If so, are you able to reproduce this iss=
ue with the stock SPDK NVMe-of target?

Can you go to frame #5 and print event->fn?  The callstack indicates this i=
s calling free() directly, but that doesn't really make sense here.  I woul=
d suggest you start looking there for clues.

Regards,

-Jim

    =

    This is my stack:
    (gdb) bt
    #0  0x00007f338ed46495 in raise () from /lib64/libc.so.6
    #1  0x00007f338ed47c75 in abort () from /lib64/libc.so.6
    #2  0x00007f338ed843a7 in __libc_message () from /lib64/libc.so.6
    #3  0x00007f338ed89dee in malloc_printerr () from /lib64/libc.so.6
    #4  0x00007f338ed8cc80 in _int_free () from /lib64/libc.so.6
    #5  0x000000000047ca07 in _spdk_event_queue_run_batch (arg=3D0x7f32dc00=
8640) at reactor.c:205
    #6  _spdk_reactor_run (arg=3D0x7f32dc008640) at reactor.c:502
    #7  0x000000000047ccac in spdk_reactors_start () at reactor.c:677
    #8  0x00000000007a942e in km_spdk_target_reactor_thread (arg=3D0x1da4b1=
60) at km_spdk_target_reactor.c:143
    #9  0x00000000007b28e1 in km_thread_log (ctxt=3D0x7f32e0001ea0) at km_t=
hread_pthreads.c:79
    #10 0x00007f338fb85aa1 in start_thread () from /lib64/libpthread.so.0
    #11 0x00007f338edfcbcd in clone () from /lib64/libc.so.6
    =

    =

    I will open a bug on github for this once I collect all the information.
    =

    Shahar
    _______________________________________________
    SPDK mailing list
    SPDK(a)lists.01.org
    https://lists.01.org/mailman/listinfo/spdk
    =



--===============7556333131658133571==--