From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EEB04C07E95 for ; Wed, 14 Jul 2021 01:21:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC6BE61380 for ; Wed, 14 Jul 2021 01:21:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237310AbhGNBYS (ORCPT ); Tue, 13 Jul 2021 21:24:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237198AbhGNBYS (ORCPT ); Tue, 13 Jul 2021 21:24:18 -0400 Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com [IPv6:2a00:1450:4864:20::22b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64630C0613DD for ; Tue, 13 Jul 2021 18:21:27 -0700 (PDT) Received: by mail-lj1-x22b.google.com with SMTP id a6so868322ljq.3 for ; Tue, 13 Jul 2021 18:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=NRnxrOCgHG4wcC+kxAa3PYFsa1gETBXfv7Mz8LLaKFk=; b=hU+skSCwfNManDdL4EwLzayhRdhco92+/0uQYK5kGKUAQYZmRPJoyr+zPMnQbQL5jh 4eGItvWantJRBPsIGRULKSSGm3yEp8OsD10rnXC60SRwUpCuqDgurkEINStuvSnmN5Tm 5KiDGhACTqMvPk9PiCSRKPK3RYR7e1it0mU8CFADFvN2JPHgKnmRYRH7sXP38Aip4Cz1 C3pzcb3jG4kOrfvy7OwVrPVKwR62rD+RyuZZJHe4D35ZAMzjGZHWK4tkM2LQ/WLnvlYU fjo1mklcijJc1+/wCFxVsZRBtkQK8sibInZnom0k/UN01TjA47J1bkL4Vfn3gaNTXnuT ywQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=NRnxrOCgHG4wcC+kxAa3PYFsa1gETBXfv7Mz8LLaKFk=; b=jLxWK55PXvKkSfIpdhYj2uzlmJaXnYfI4OkANdv2klHHssp01V8aXXmrl2DhefuA6A EciqimA9e3c2RhBjMCJkMBMInR08UQHitnSAQVi5xilTUIic/JHHsNXzAwP+txYfrk4Z XYQen+zuKgSb98x4/O7kLlNGWDQ8B7mCGEPz7G2QB12A5YeaNyx+eDcvZBCL7l43Gv/t 5JeN/xm7AA5e5CTwDBi0JVfHa7qrnd7ycN+lObZGLNh9JTCp1r1zzIFZgb8OD/YGW8d/ Nc+35AJj4inCi4YqZ0RJSuRCXEqdXs+En1bnYZAGa9JUAanhFharZLgYttF//E+gQqyR oOaw== X-Gm-Message-State: AOAM530ShESD1Mo/7E6DJjaL1BP4MS7pawCjbI/L25miUuXr12VxA8hZ G2yxKLstuvMrToQwyNz7AgiDIsubUbFILJyEXlFL2oSzYELhvA== X-Google-Smtp-Source: ABdhPJzAYRlDq9/mV2UWB7nDZHYYz69K3K1aMmSIja9Ohj/C20Z8502aM/ztxfyT3rOsRzTZjO7FWH0neq496I+RM2A= X-Received: by 2002:a2e:6e09:: with SMTP id j9mr6642621ljc.319.1626225685442; Tue, 13 Jul 2021 18:21:25 -0700 (PDT) MIME-Version: 1.0 From: Bryce Evans Date: Tue, 13 Jul 2021 18:21:13 -0700 Message-ID: Subject: PROBLEM: sas_address sysfs attribute empty on E208i-p SR Gen10 To: don.brace@microsemi.com Cc: esc.storagedev@microsemi.com, linux-scsi@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The sas_address device attribute in sas_device sysfs in is 0x0, and missing (empty or null?) in scsi_disk sysfs on LTS kernel 5.10.x, and Stable 5.13.1 Specifically /sys/class/scsi_disk/X:X/device/sas_address and /sys/class/sas_device/end_device-X:X/sas_address are null and 0x0 respectively Using this device in hba mode (pass through) - Serial Attached SCSI controller: Adaptec Smart Storage PQI 12G SAS/PCIe 3 (rev 01) Subsystem: Hewlett-Packard Company Smart Array E208i-p SR Gen10 This appears to have regressed some time after 4.19 (expected results from 4.19 below) It seems that systemd udev depends on this attribute from sysfs to populate /dev/disk/by-path, and when it is unavailable the sas_address shows as 0x0, which breaks population of this directory, potentially breaking userspace tools that rely on this ================================================================== udevadm uses 0x0 for the address variable of ID_PATH, causing unique disks to have the same identifier used for ID_PATH, which breaks population of /dev/disk/by-path [bryce@myhost ~]$ udevadm info /dev/sdy | grep "ID_PATH="" E: ID_PATH=pci-0000:5c:00.0-sas-0x0000000000000000-lun-0 [bryce@myhost ~]$ udevadm info /dev/sdx | grep "ID_PATH="" E: ID_PATH=pci-0000:5c:00.0-sas-0x0000000000000000-lun-0 you can see the above 2 disks, should have a unique ID_PATH, and do not. ================================================================== In kernel sysfs, we see empty sas_address attribute for each disk attached to the hba in the scsi_disk sysfs [bryce@myhost ~]$ cat /sys/class/scsi_disk/*/device/sas_address cat: /sys/class/scsi_disk/0:0:0:0/device/sas_address: No such device cat: /sys/class/scsi_disk/0:0:1:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:0:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:10:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:1:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:11:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:12:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:13:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:14:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:15:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:16:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:17:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:18:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:19:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:20:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:2:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:21:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:22:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:23:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:3:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:4:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:5:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:6:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:7:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:8:0/device/sas_address: No such device cat: /sys/class/scsi_disk/1:0:9:0/device/sas_address: No such device ================================================================== See the following end device to pci device mapping [bryce@myhost ~]$ ls -al /sys/class/sas_device/ total 0 drwxr-xr-x 2 root root 0 Jul 13 16:06 . drwxr-xr-x 77 root root 0 Jul 13 16:06 .. lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-0:1 -> ../../devices/pci0000:36/0000:36:02.0/0000:38:00.0/host0/scsi_host/host0/port-0:1/end_device-0:1/sas_device/end_device-0:1 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-0:2 -> ../../devices/pci0000:36/0000:36:02.0/0000:38:00.0/host0/scsi_host/host0/port-0:2/end_device-0:2/sas_device/end_device-0:2 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-0:3 -> ../../devices/pci0000:36/0000:36:02.0/0000:38:00.0/host0/scsi_host/host0/port-0:3/end_device-0:3/sas_device/end_device-0:3 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:1 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:1/end_device-1:1/sas_device/end_device-1:1 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:10 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:10/end_device-1:10/sas_device/end_device-1:10 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:11 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:11/end_device-1:11/sas_device/end_device-1:11 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:12 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:12/end_device-1:12/sas_device/end_device-1:12 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:13 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:13/end_device-1:13/sas_device/end_device-1:13 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:14 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:14/end_device-1:14/sas_device/end_device-1:14 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:15 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:15/end_device-1:15/sas_device/end_device-1:15 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:16 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:16/end_device-1:16/sas_device/end_device-1:16 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:17 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:17/end_device-1:17/sas_device/end_device-1:17 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:18 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:18/end_device-1:18/sas_device/end_device-1:18 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:19 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:19/end_device-1:19/sas_device/end_device-1:19 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:2 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:2/end_device-1:2/sas_device/end_device-1:2 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:20 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:20/end_device-1:20/sas_device/end_device-1:20 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:21 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:21/end_device-1:21/sas_device/end_device-1:21 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:22 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:22/end_device-1:22/sas_device/end_device-1:22 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:23 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:23/end_device-1:23/sas_device/end_device-1:23 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:24 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:24/end_device-1:24/sas_device/end_device-1:24 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:25 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:25/end_device-1:25/sas_device/end_device-1:25 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:26 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:26/end_device-1:26/sas_device/end_device-1:26 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:27 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:27/end_device-1:27/sas_device/end_device-1:27 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:3 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:3/end_device-1:3/sas_device/end_device-1:3 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:4 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:4/end_device-1:4/sas_device/end_device-1:4 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:5 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:5/end_device-1:5/sas_device/end_device-1:5 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:6 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:6/end_device-1:6/sas_device/end_device-1:6 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:7 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:7/end_device-1:7/sas_device/end_device-1:7 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:8 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:8/end_device-1:8/sas_device/end_device-1:8 lrwxrwxrwx 1 root root 0 Jul 13 16:06 end_device-1:9 -> ../../devices/pci0000:5b/0000:5b:00.0/0000:5c:00.0/host1/scsi_host/host1/port-1:9/end_device-1:9/sas_device/end_device-1:9 ================================================================== and 0x0 in sysfs for sas_device/end_device sas_addresses (this is what breaks udev) [bryce@myhost ~]$ for d in $(ls /sys/class/sas_device/end_device*/sas_address); do echo $d; cat $d;done /sys/class/sas_device/end_device-0:1/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-0:2/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-0:3/sas_address 0x51402ec0130545a8 /sys/class/sas_device/end_device-1:10/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:11/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:12/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:13/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:14/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:15/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:16/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:17/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:18/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:19/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:1/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:20/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:21/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:22/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:23/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:24/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:25/sas_address 0x50014380436ed4bc /sys/class/sas_device/end_device-1:26/sas_address 0x50014380436ec7fc /sys/class/sas_device/end_device-1:27/sas_address 0x51402ec013612638 /sys/class/sas_device/end_device-1:2/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:3/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:4/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:5/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:6/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:7/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:8/sas_address 0x0000000000000000 /sys/class/sas_device/end_device-1:9/sas_address 0x0000000000000000 ================================================================== lspci -vvv output 38:00.0 Serial Attached SCSI controller: Adaptec Smart Storage PQI 12G SAS/PCIe 3 (rev 01) Subsystem: Hewlett-Packard Company Smart Array E208i-a SR Gen10 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- SERR-