All of lore.kernel.org
 help / color / mirror / Atom feed
* Kernel panic is seen while running iozone over multiple ports and toggling the link on TOT kernel
@ 2020-09-28 17:58 Dakshaja Uppalapati
  2020-09-29  8:05 ` Sagi Grimberg
  0 siblings, 1 reply; 2+ messages in thread
From: Dakshaja Uppalapati @ 2020-09-28 17:58 UTC (permalink / raw)
  To: linux-nvme, kbusch, Sagi Grimberg; +Cc: Potnuri Bharat Teja

[-- Attachment #1: Type: text/plain, Size: 2727 bytes --]

Hi all,
I observed the following trace with TOT linux kernel while running NVMF over multiple ports and toggling the link/interface. Issue is observed very intermittently.
Attached is the target configuration on actual disks.

On host, started iozone then toggling the multiport interface1 for 5 seconds and interface2 for 8 seconds one after the other in a loop
Observed the below kernel panic after couple of hours.

[142799.524961] BUG: kernel NULL pointer dereference, address: 0000000000000198
[142799.524965] #PF: supervisor write access in kernel mode
[142799.524966] #PF: error_code(0x0002) - not-present page
[142799.524967] PGD 0 P4D 0
[142799.524970] Oops: 0002 [#1] SMP PTI
[142799.524973] CPU: 1 PID: 16 Comm: ksoftirqd/1 Kdump: loaded Tainted: G S      W         5.9.0-rc6 #1
[142799.524974] Hardware name: Supermicro X10SRA-F/X10SRA-F, BIOS 2.0 01/28/2016
[142799.524980] RIP: 0010:blk_mq_free_request+0x80/0x110
[142799.524982] Code: 00 00 00 00 8b 53 18 b8 01 00 00 00 84 d2 74 0b 31 c0 81 e2 00 08 06 00 0f 95 c0 48 83 84 c5 80 00 00 00 01 f6 43 1c 40 74 08 <f0> 41 ff 8d 98 01 00 00 8b 05 5a 4a c4 01 85 c0 75 5e 49 8b 7c 24
[142799.524983] RSP: 0018:ffffbb96c0123dc0 EFLAGS: 00010202
[142799.524984] RAX: 0000000000000000 RBX: ffff9e6b70f60280 RCX: 0000000000000018
[142799.524986] RDX: 0000000000000000 RSI: 000000000000000a RDI: ffff9e6b70f60280
[142799.524987] RBP: ffffdb96be8fd400 R08: 0000000000000000 R09: 0000000000000000
[142799.524988] R10: 00513cf381a1baf1 R11: 0000000000000000 R12: ffff9e6b590f7698
[142799.524989] R13: 0000000000000000 R14: 0000000000000004 R15: ffffffff948050c0
[142799.524990] FS:  0000000000000000(0000) GS:ffff9e6befc40000(0000) knlGS:0000000000000000
[142799.524992] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[142799.524993] CR2: 0000000000000198 CR3: 00000001d6a0a001 CR4: 00000000003706e0
[142799.524994] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[142799.524995] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[142799.524995] Call Trace:
[142799.525006]  nvme_keep_alive_end_io+0x15/0x70 [nvme_core]
[142799.525011]  nvme_rdma_complete_rq+0x68/0xc0 [nvme_rdma]
[142799.525014]  ? set_next_entity+0xae/0x1f0
[142799.525016]  blk_done_softirq+0x95/0xc0
[142799.525021]  __do_softirq+0xde/0x2ec
[142799.525025]  ? sort_range+0x20/0x20
[142799.525029]  run_ksoftirqd+0x1a/0x20
[142799.525031]  smpboot_thread_fn+0xc5/0x160
[142799.525034]  kthread+0x116/0x130
[142799.525036]  ? kthread_park+0x80/0x80
[142799.525040]  ret_from_fork+0x22/0x30

Keepalive request structure is freed and is accessed by nvme_keep_alive_end_io. looks like a probable race to me.

Thanks,
Dakshaja


[-- Attachment #2: target_config_128 --]
[-- Type: application/octet-stream, Size: 78311 bytes --]

{
  "hosts": [],
  "ports": [
    {
      "addr": {
        "adrfam": "ipv4",
        "traddr": "102.2.2.146",
        "treq": "not specified",
        "trsvcid": "4420",
        "trtype": "rdma"
      },
      "portid": 2,
      "referrals": [],
      "subsystems": [
        "nvme-ram128",
        "nvme-ram127",
        "nvme-ram126",
        "nvme-ram125",
        "nvme-ram124",
        "nvme-ram123",
        "nvme-ram122",
        "nvme-ram121",
        "nvme-ram120",
        "nvme-ram119",
        "nvme-ram118",
        "nvme-ram117",
        "nvme-ram116",
        "nvme-ram115",
        "nvme-ram114",
        "nvme-ram113",
        "nvme-ram112",
        "nvme-ram111",
        "nvme-ram110",
        "nvme-ram109",
        "nvme-ram108",
        "nvme-ram107",
        "nvme-ram106",
        "nvme-ram105",
        "nvme-ram104",
        "nvme-ram103",
        "nvme-ram102",
        "nvme-ram101",
        "nvme-ram100",
        "nvme-ram99",
        "nvme-ram98",
        "nvme-ram97",
        "nvme-ram96",
        "nvme-ram95",
        "nvme-ram94",
        "nvme-ram93",
        "nvme-ram92",
        "nvme-ram91",
        "nvme-ram90",
        "nvme-ram89",
        "nvme-ram88",
        "nvme-ram87",
        "nvme-ram86",
        "nvme-ram85",
        "nvme-ram84",
        "nvme-ram83",
        "nvme-ram82",
        "nvme-ram81",
        "nvme-ram80",
        "nvme-ram79",
        "nvme-ram78",
        "nvme-ram77",
        "nvme-ram76",
        "nvme-ram75",
        "nvme-ram74",
        "nvme-ram73",
        "nvme-ram72",
        "nvme-ram71",
        "nvme-ram70",
        "nvme-ram69",
        "nvme-ram68",
        "nvme-ram67",
        "nvme-ram66",
        "nvme-ram65"
      ]
    },
    {
      "addr": {
        "adrfam": "ipv4",
        "traddr": "102.1.1.146",
        "treq": "not specified",
        "trsvcid": "4420",
        "trtype": "rdma"
      },
      "portid": 1,
      "referrals": [],
      "subsystems": [
        "nvme-ram64",
        "nvme-ram63",
        "nvme-ram62",
        "nvme-ram61",
        "nvme-ram60",
        "nvme-ram59",
        "nvme-ram58",
        "nvme-ram57",
        "nvme-ram56",
        "nvme-ram55",
        "nvme-ram54",
        "nvme-ram53",
        "nvme-ram52",
        "nvme-ram51",
        "nvme-ram50",
        "nvme-ram49",
        "nvme-ram48",
        "nvme-ram47",
        "nvme-ram46",
        "nvme-ram45",
        "nvme-ram44",
        "nvme-ram43",
        "nvme-ram42",
        "nvme-ram41",
        "nvme-ram40",
        "nvme-ram39",
        "nvme-ram38",
        "nvme-ram37",
        "nvme-ram36",
        "nvme-ram35",
        "nvme-ram34",
        "nvme-ram33",
        "nvme-ram32",
        "nvme-ram31",
        "nvme-ram30",
        "nvme-ram29",
        "nvme-ram28",
        "nvme-ram27",
        "nvme-ram26",
        "nvme-ram25",
        "nvme-ram24",
        "nvme-ram23",
        "nvme-ram22",
        "nvme-ram21",
        "nvme-ram20",
        "nvme-ram19",
        "nvme-ram18",
        "nvme-ram17",
        "nvme-ram16",
        "nvme-ram15",
        "nvme-ram14",
        "nvme-ram13",
        "nvme-ram12",
        "nvme-ram11",
        "nvme-ram10",
        "nvme-ram9",
        "nvme-ram8",
        "nvme-ram7",
        "nvme-ram6",
        "nvme-ram5",
        "nvme-ram4",
        "nvme-ram3",
        "nvme-ram2",
        "nvme-ram1"
      ]
    }
  ],
  "subsystems": [
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "6be47939fca6d912",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol128",
            "uuid": "af2dc4c0-04ed-4e2b-afdc-23828adebf4d"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram128"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "d6c1a7825f06603c",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol127",
            "uuid": "b36b55bc-96d7-4353-b0d4-6de23ad63b61"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram127"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "82d0373fb459a6ea",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol126",
            "uuid": "6d4c9a78-243f-4147-99ec-85be00e7d5b9"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram126"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "d934e90dfc4648e2",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol125",
            "uuid": "d99bfdb8-2295-40de-ab99-c30cca84b33b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram125"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "6614ca063e48ba43",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol124",
            "uuid": "b14f4d84-7b93-4646-b3af-cb11bc5c2e58"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram124"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "7e371588722ba510",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol123",
            "uuid": "93bc1f6c-15e7-4296-b8c6-b52fb338c09b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram123"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "1b9a41dc1d68e389",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol122",
            "uuid": "780af050-acc0-4102-a021-d9ce0fa720a0"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram122"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "47a3bfd6daa8cc83",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol121",
            "uuid": "aa0a503f-e4ee-4415-a5c1-7a21d2bc18c3"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram121"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "72b2a1c07001585a",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol120",
            "uuid": "f65b32cf-2124-4638-afbe-4ce4def8fbc6"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram120"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "7398fa12859039d8",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol119",
            "uuid": "9bcb9f11-684a-4b0f-a576-edea547d92a0"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram119"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "ae882e0976005f86",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol118",
            "uuid": "67097bc5-43b8-4869-8049-d3c5b1c411c2"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram118"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "2a85b6c4560f9fe8",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol117",
            "uuid": "384ea1f3-dddd-4930-a510-6aa14394229e"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram117"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "e79851c1568593a6",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol116",
            "uuid": "7706799f-6fca-4e54-8141-b83328d1b0ea"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram116"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "bf468dc28610fd89",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol115",
            "uuid": "99618a7c-6ff3-4d39-a083-c9c6b7a83730"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram115"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "c1d0f15451058720",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol114",
            "uuid": "0d929e08-fb87-4387-b792-c447aa7844e4"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram114"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "f2e8e448362c27ea",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol113",
            "uuid": "c723ae77-7711-493c-9746-24ec4c70520c"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram113"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "e4cd9259036d63cf",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol112",
            "uuid": "7181bcac-5ffb-46c6-9acb-1df314aaab9c"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram112"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "fdc50d4506a285b1",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol111",
            "uuid": "e5c108fb-8125-4115-8692-78a3438db10f"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram111"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "e9b0444a3a65bfca",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol110",
            "uuid": "3da5ff90-e8f9-48ec-9357-c95fc8bc652b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram110"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "6d6f6ccf079ba36a",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol109",
            "uuid": "67fcf8a3-2cf3-4626-b98b-328c0be27f41"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram109"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "5d47b9cac3524aa",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol108",
            "uuid": "1e084c0a-c3c6-4cc3-a3ff-03cb33403f7f"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram108"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "418f98376c841076",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol107",
            "uuid": "7ad38783-bc00-4bbd-ac91-7e5fa83eb1ec"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram107"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "55f2dc64181b9b4d",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol106",
            "uuid": "f13cc28e-e991-445c-9565-7d5d7acbac53"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram106"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "1c077196285b664e",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol105",
            "uuid": "1e8ca139-975e-4403-92d6-b5b27a5acff6"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram105"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "6ab5e12567f13b5a",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol104",
            "uuid": "0709544e-33ed-491c-8e25-8a92ec77560f"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram104"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "7bce563f1a8eb7ac",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol103",
            "uuid": "5d008ccc-e2df-498a-9af7-798f79073ff7"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram103"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "ab3a02457d4147ec",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol102",
            "uuid": "13faab56-2959-46b3-85a4-aad880e4df9b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram102"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "caf733158fbf51b6",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol101",
            "uuid": "802049da-3a49-4281-afb9-95dfe1ba8047"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram101"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "3fd80dde4fe6cda4",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol100",
            "uuid": "0e57dfaa-73b7-414f-9c34-434b3c51b379"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram100"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "8eb5d03c7b22696b",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol99",
            "uuid": "bbb98e25-7449-411b-93cb-b616de1e7be2"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram99"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "3e6e8eaedd55f051",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol98",
            "uuid": "db247179-5c1a-4d58-aebf-fe5e597ad820"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram98"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "3a7d65146de38cc1",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol97",
            "uuid": "d0465600-8c87-4720-8b7b-9c13846e8919"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram97"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "af4f9ce7fe07798c",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol96",
            "uuid": "845c9f93-4a32-4afa-9df3-00cf206d5284"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram96"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "61fde943a1ea7589",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol95",
            "uuid": "8fbb59b0-27e6-430c-8ffb-2731feac1ef8"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram95"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "8f3d0f37b163ef46",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol94",
            "uuid": "2eb0bce6-b10a-4cde-a54c-7854b30093ee"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram94"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "c6884a543d027d3a",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol93",
            "uuid": "43aa216c-b47a-4bb7-a0dc-72b221e81a05"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram93"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "cbea07e1f8ec5703",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol92",
            "uuid": "36f77d9c-bc39-40f0-b3d4-89e7b7d6506d"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram92"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "a31db63e0a7ca870",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol91",
            "uuid": "96c6bbf0-576d-4298-b4a2-ba021ec372b5"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram91"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "1198530d5168b12c",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol90",
            "uuid": "6509ebbf-812d-4d16-9fc5-0e8b3dc31bc0"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram90"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "ca395df44452333d",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol89",
            "uuid": "bd9e55d3-f1d4-4322-8cfd-db2a663c18dd"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram89"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "69207f3269dbf077",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol88",
            "uuid": "07a55e1f-0037-4b21-89d3-1b50a91fcc30"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram88"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "89bd59e7f68451d3",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol87",
            "uuid": "f8165307-7af7-4944-b74e-a10c397e3b9b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram87"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "15e87812dfae30",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol86",
            "uuid": "6c04ed76-40c1-473a-b7de-a8ea594b30c2"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram86"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "5167a47412630694",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol85",
            "uuid": "0ec5cc77-5878-4031-ad4e-74ace8e08158"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram85"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "e24d5242b687281",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol84",
            "uuid": "4d899284-7ccf-49f4-b0de-4aabf5ba8eca"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram84"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "3135c4cfe3433ded",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol83",
            "uuid": "734ebc60-062b-481c-bf7b-746887b77da3"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram83"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "aae327bd4df92d08",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol82",
            "uuid": "40127ce3-ca92-4760-9f8f-78e77e364b0a"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram82"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "2306ea714bf34aaf",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol81",
            "uuid": "62b4671a-8a26-4b1c-9665-9a4497dadb64"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram81"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "796352416947604f",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol80",
            "uuid": "5398431f-3af8-4390-ae4c-e9d0f81ea275"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram80"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "7688bed383db42d",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol79",
            "uuid": "558300e9-34f4-475b-906e-26ab62acfaed"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram79"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "50c06c25e914ef3",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol78",
            "uuid": "6bab923b-0789-4a4b-a327-8fc1634a397a"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram78"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "467c8dce484260fc",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol77",
            "uuid": "075b1e2c-a74d-42c5-94e1-fc04bd7ff1ff"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram77"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "7d7eacfea182ec46",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol76",
            "uuid": "f5fe0555-0e25-4c55-ab99-63628d68968b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram76"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "dbddb4c70c688bfe",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol75",
            "uuid": "e68f9798-d5cf-4e3e-bb1e-9d7f41dd8be3"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram75"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "c32b933022cc190",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol74",
            "uuid": "1efb9758-2b35-472b-b213-4c3487b2e8ae"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram74"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "80de5b3f5c7f6af9",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol73",
            "uuid": "9185454a-bbb9-4338-9dac-cf07cad5918b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram73"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "6ddf45ad229ca4d8",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol72",
            "uuid": "b4f61039-54e5-4d8b-ac48-611bf92f5bde"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram72"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "37a280025577033b",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol71",
            "uuid": "f3c505d5-bbf0-45d0-823e-3db7bf71d40f"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram71"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "61f64ac4b58bdf16",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol70",
            "uuid": "f396682f-58ec-42e1-9575-8603c412dbd1"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram70"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "1095927619bb0e7d",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol69",
            "uuid": "05bcc45f-271e-4099-a569-7306700e370d"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram69"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "931fc2912a470929",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol68",
            "uuid": "52754830-fc7b-4c9c-a38f-6e938d91ce87"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram68"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "32e7e94ab014a6da",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol67",
            "uuid": "08b1aa39-8860-424d-affe-49c20fae98d0"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram67"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "fa87abb314342012",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol66",
            "uuid": "016d92b4-a108-4ef9-a971-928c9349024d"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram66"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "90f7e6cbb245be8",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol65",
            "uuid": "dc56eb6c-a4c8-4c8b-b6a4-7586f00ce6bc"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram65"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "3e18037166c83dff",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol64",
            "uuid": "35aaef92-116a-4249-ada2-79de8ba03499"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram64"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "4a69397c4b60fe17",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol63",
            "uuid": "649b14b2-3074-4ddf-b8a1-d8f6a5598eda"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram63"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "6883fcac16f969fe",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol62",
            "uuid": "9bd33796-a858-469d-98d1-35d86501995a"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram62"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "606790e763892a8b",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol61",
            "uuid": "67fa0cdb-1f6d-48b8-8b20-293c2c3bb7c8"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram61"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "72305c21268626b3",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol60",
            "uuid": "f6ba8bf6-0b01-401f-83bc-6bae04dffb17"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram60"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "72423d887364112b",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol59",
            "uuid": "3685ad9d-7137-400a-89ea-2d2c2504ad9c"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram59"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "330975592162cc0a",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol58",
            "uuid": "0da3b574-bd0a-4f66-9283-190669fde96b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram58"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "d72c69a5620415d6",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol57",
            "uuid": "11ccbaef-9158-49ec-9ae7-9f008a6d18a4"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram57"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "f8dd01234330100a",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol56",
            "uuid": "eb378e44-3411-4a50-aca9-950b8a3df5ec"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram56"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "f54abaa89371e99a",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol55",
            "uuid": "ad869c54-c91f-4c55-ae32-e9443fbc04d9"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram55"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "25689f93774dba40",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol54",
            "uuid": "aad5b7b7-60ee-4db8-8bf3-38baf0c24c31"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram54"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "6967c1d90705148f",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol53",
            "uuid": "fff6b76b-4a7a-4199-899e-8c04a516201c"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram53"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "1e7354fafe80309b",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol52",
            "uuid": "51b44acc-f3ed-4e40-ace2-dbc6fc3e0dcd"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram52"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "56bba31716700961",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol51",
            "uuid": "309e51a1-2f88-4586-8d9c-80702552b6a5"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram51"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "1a89cf7d351d35e7",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol50",
            "uuid": "9e201256-b15b-4857-abc2-28388295b828"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram50"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "3885fcff4dc3079a",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol49",
            "uuid": "530e3296-1e34-4195-a089-3a42980bd1c2"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram49"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "be95922d784ea3c2",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol48",
            "uuid": "2cb86d60-4a38-4b34-b8b8-1709f15cf61c"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram48"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "6bbd307d6640fd6f",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol47",
            "uuid": "170c7a10-ec12-4752-898a-272dfaf7de8b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram47"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "974308bc0fc9d4cd",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol46",
            "uuid": "4e07810c-909e-415e-995a-ebbf3de4b4a2"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram46"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "b54e64cff16c7aa6",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol45",
            "uuid": "f15f9664-39b4-4fb7-99d8-2252ffae0667"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram45"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "d4c3223f3802a876",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol44",
            "uuid": "19f18236-e007-438f-b927-cd05765ccf40"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram44"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "cbf2207b9b14113e",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol43",
            "uuid": "85afc024-3425-476c-be97-e765e02fe817"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram43"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "333d366fa29c6edc",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol42",
            "uuid": "b11bb066-ed57-465f-9c9a-94674fcf89c0"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram42"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "4a809aef5fd4512f",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol41",
            "uuid": "3816659f-c962-4c7e-affe-0badcce0bc83"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram41"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "dd62487b5c71529b",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol40",
            "uuid": "64a31fb5-5388-4ac8-b1cd-49a8d5304bdf"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram40"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "e6dfd6b76c38534a",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol39",
            "uuid": "a178e79a-ad84-4d5b-b858-276655211f15"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram39"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "422f6c1bd72f0f06",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol38",
            "uuid": "111d3704-4811-44b0-9571-ced79b4ae708"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram38"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "cffa9b4291bd4ff8",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol37",
            "uuid": "8e1bbf49-0614-4d61-b16c-cbc6a297d42b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram37"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "b95f474d8628088d",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol36",
            "uuid": "b1673f6b-8e32-401e-922a-66e4a959552b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram36"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "7af00cd64323bc0f",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol35",
            "uuid": "149d563a-36da-4155-b6f4-a8f052b11d40"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram35"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "d84ba6a2781e980c",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol34",
            "uuid": "60f71027-3b12-4915-8d23-d69112297920"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram34"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "eb726ceeb5fe0015",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol33",
            "uuid": "636fb5ab-7172-4895-9f7d-8a74835e0c58"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram33"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "f4568a6db37bc2c5",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol32",
            "uuid": "adbf88aa-4b8d-4d8a-93c1-7b663fd2fab5"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram32"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "31a3e5df0f74ca0c",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol31",
            "uuid": "e7e3328c-688a-431b-affe-78fbe72ae99d"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram31"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "7ed11ea06e0e1c4b",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol30",
            "uuid": "1c47ce3f-1c11-4147-86bd-2803e28cb4d4"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram30"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "456c6f41c73c3092",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol29",
            "uuid": "24cf36b3-c179-4a22-8dd9-d8bcb82c8e9b"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram29"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "9897bf2b8ab93427",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol28",
            "uuid": "9e40d81c-fff7-42da-8e1b-4fa3c1b046e9"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram28"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "cd267024b557c8d8",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol27",
            "uuid": "83b17801-a99f-4d5e-b91a-a6eb74336f30"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram27"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "9959e8faba522e5e",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol26",
            "uuid": "56609da0-a1ad-4f75-9b7e-36a2fc8cbd2f"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram26"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "c9f29999f400705e",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol25",
            "uuid": "20b35bfd-9474-4afe-9e33-e5aa909b8f66"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram25"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "e565997b3c5120da",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol24",
            "uuid": "5d929b6d-7e5b-42b2-9982-7e733b9f3912"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram24"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "f91732793d1ede4b",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol23",
            "uuid": "1af9ba79-4d91-4c9a-9306-0510532689d8"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram23"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "5cba6566a34f148c",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol22",
            "uuid": "789163e0-d4f8-4a40-b0b4-8af50b7b03e2"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram22"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "7afd6337a1ab2ef5",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol21",
            "uuid": "7fe92823-6b5d-4662-826c-ee6a94202dd9"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram21"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "87146f04bffa9916",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol20",
            "uuid": "586270d1-b0fb-4ae3-869b-aa81526ea824"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram20"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "417a0cc4875e2374",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol19",
            "uuid": "89894744-f309-4f99-a5c5-1eff54fe8f09"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram19"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "b63d84e20f155c92",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol18",
            "uuid": "1632d025-8bd3-41e8-8ca6-3a9329f6cf87"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram18"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "ce1ac2ec04b47390",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol17",
            "uuid": "647e28f2-afbe-41f6-a69e-cd61f8e49011"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram17"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "3a23e8113bed423d",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol16",
            "uuid": "9811c6c8-fbd8-4489-ab11-cb25da1d0bf5"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram16"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "93ea941a56124f01",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol15",
            "uuid": "3b680292-be74-4eb7-8006-553cbd6e1ba5"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram15"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "6734594b0b8b18fd",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol14",
            "uuid": "66ebabc1-c108-4f8d-94ae-3835010dd998"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram14"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "b0a04804baf64d88",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol13",
            "uuid": "e2194858-af92-424c-8362-4f3d92781396"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram13"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "c5e82ec93a6244df",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol12",
            "uuid": "01bf072d-66fc-42d1-abdc-873db4a283a3"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram12"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "aa52c4f6263d3e99",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol11",
            "uuid": "51746c3f-5dce-48f0-9ec5-702f6f067b58"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram11"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "51dbbce1b4b48dc2",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol10",
            "uuid": "2f15ad6b-480b-41e3-9ced-44eb39b5d68c"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram10"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "9bb74efdca6bf2b8",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol9",
            "uuid": "1897628d-0980-4f6c-b78c-065b13c16558"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram9"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "d59ca8f39da9290d",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol8",
            "uuid": "3656ef1b-bc3f-4462-b396-9a51f65c5470"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram8"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "f94816b994a51489",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol7",
            "uuid": "a2502fe7-7e59-41c4-8785-13d356d111f7"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram7"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "f4f1718a790c3c30",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol6",
            "uuid": "80169770-aeac-4d95-b142-c0037c65cb4d"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram6"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "eb152b1fc3e8f9f3",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol5",
            "uuid": "c0cfb12a-e0cc-4713-acea-4b10db58d156"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram5"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "ffd321627b80cee8",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol4",
            "uuid": "f5e2085f-67da-4912-82e1-a96b80a3fe5c"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram4"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "dc2cb57d8475b57a",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol3",
            "uuid": "b88409e0-8b80-417f-87fa-b39fb2a6be3c"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram3"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "574057e3d365fc89",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol2",
            "uuid": "cda77d66-86e8-4849-8f02-c9afa41a17e2"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram2"
    },
    {
      "allowed_hosts": [],
      "attr": {
        "allow_any_host": "1",
        "cntlid_max": "65519",
        "cntlid_min": "1",
        "model": "Linux",
        "pi_enable": "0",
        "serial": "e6a9aca1d40e78ff",
        "version": "1.3"
      },
      "namespaces": [
        {
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/vgiser/vol1",
            "uuid": "bb2cabc7-436e-4e22-802a-500dd835ba9d"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn": "nvme-ram1"
    }
  ]
}

[-- Attachment #3: Type: text/plain, Size: 158 bytes --]

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: Kernel panic is seen while running iozone over multiple ports and toggling the link on TOT kernel
  2020-09-28 17:58 Kernel panic is seen while running iozone over multiple ports and toggling the link on TOT kernel Dakshaja Uppalapati
@ 2020-09-29  8:05 ` Sagi Grimberg
  0 siblings, 0 replies; 2+ messages in thread
From: Sagi Grimberg @ 2020-09-29  8:05 UTC (permalink / raw)
  To: Dakshaja Uppalapati, linux-nvme, kbusch; +Cc: Potnuri Bharat Teja


> Hi all,
> I observed the following trace with TOT linux kernel while running NVMF over multiple ports and toggling the link/interface. Issue is observed very intermittently.

What is TOT linux kernel?

> Attached is the target configuration on actual disks.
> 
> On host, started iozone then toggling the multiport interface1 for 5 seconds and interface2 for 8 seconds one after the other in a loop
> Observed the below kernel panic after couple of hours.
> 
> [142799.524961] BUG: kernel NULL pointer dereference, address: 0000000000000198
> [142799.524965] #PF: supervisor write access in kernel mode
> [142799.524966] #PF: error_code(0x0002) - not-present page
> [142799.524967] PGD 0 P4D 0
> [142799.524970] Oops: 0002 [#1] SMP PTI
> [142799.524973] CPU: 1 PID: 16 Comm: ksoftirqd/1 Kdump: loaded Tainted: G S      W         5.9.0-rc6 #1
> [142799.524974] Hardware name: Supermicro X10SRA-F/X10SRA-F, BIOS 2.0 01/28/2016
> [142799.524980] RIP: 0010:blk_mq_free_request+0x80/0x110
> [142799.524982] Code: 00 00 00 00 8b 53 18 b8 01 00 00 00 84 d2 74 0b 31 c0 81 e2 00 08 06 00 0f 95 c0 48 83 84 c5 80 00 00 00 01 f6 43 1c 40 74 08 <f0> 41 ff 8d 98 01 00 00 8b 05 5a 4a c4 01 85 c0 75 5e 49 8b 7c 24
> [142799.524983] RSP: 0018:ffffbb96c0123dc0 EFLAGS: 00010202
> [142799.524984] RAX: 0000000000000000 RBX: ffff9e6b70f60280 RCX: 0000000000000018
> [142799.524986] RDX: 0000000000000000 RSI: 000000000000000a RDI: ffff9e6b70f60280
> [142799.524987] RBP: ffffdb96be8fd400 R08: 0000000000000000 R09: 0000000000000000
> [142799.524988] R10: 00513cf381a1baf1 R11: 0000000000000000 R12: ffff9e6b590f7698
> [142799.524989] R13: 0000000000000000 R14: 0000000000000004 R15: ffffffff948050c0
> [142799.524990] FS:  0000000000000000(0000) GS:ffff9e6befc40000(0000) knlGS:0000000000000000
> [142799.524992] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [142799.524993] CR2: 0000000000000198 CR3: 00000001d6a0a001 CR4: 00000000003706e0
> [142799.524994] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [142799.524995] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [142799.524995] Call Trace:
> [142799.525006]  nvme_keep_alive_end_io+0x15/0x70 [nvme_core]
> [142799.525011]  nvme_rdma_complete_rq+0x68/0xc0 [nvme_rdma]
> [142799.525014]  ? set_next_entity+0xae/0x1f0
> [142799.525016]  blk_done_softirq+0x95/0xc0
> [142799.525021]  __do_softirq+0xde/0x2ec
> [142799.525025]  ? sort_range+0x20/0x20
> [142799.525029]  run_ksoftirqd+0x1a/0x20
> [142799.525031]  smpboot_thread_fn+0xc5/0x160
> [142799.525034]  kthread+0x116/0x130
> [142799.525036]  ? kthread_park+0x80/0x80
> [142799.525040]  ret_from_fork+0x22/0x30
> 
> Keepalive request structure is freed and is accessed by nvme_keep_alive_end_io. looks like a probable race to me.
> 
> Thanks,
> Dakshaja
> 

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2020-09-29  8:06 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-28 17:58 Kernel panic is seen while running iozone over multiple ports and toggling the link on TOT kernel Dakshaja Uppalapati
2020-09-29  8:05 ` Sagi Grimberg

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.