USB storage no-boot regression (bisected)

USB storage no-boot regression (bisected)

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 1998 bytes --]


Once of the x86-64 machines I use for testing runs off of two 2GB USB 
flash drives, one for Fedora 10 userland, and one for kernel repository 
+ builds.

It boots correctly in 2.6.27, but fails with the same symptoms in 
2.6.28, 2.6.29 and 2.6.30-rc1:

	1) The kernel boots
	2) After time passes, kernel begins executing initramfs
	   userland
	3) the kernel prints out probe messages for the USB keyboard,
	   SCSI probe messages for the two USB flash drives

Or IOW, the keyboard and two SCSI drives appear after initramfs begins 
booting.  And this is for drivers built into the kernel (though same 
behavior with modules).

This no-boot regression is 100% reproducible, and neatly bisects down to

> commit 8520f38099ccfdac2147a0852f84ee7a8ee5e197
> Author: Alan Stern <stern@rowland.harvard.edu>
> Date:   Mon Sep 22 14:44:26 2008 -0400
> 
>     USB: change hub initialization sleeps to delayed_work
>     
>     This patch (as1137) changes the hub_activate() routine, replacing the
>     power-power-up and debounce delays with delayed_work calls.  The idea
>     is that on systems where the USB stack is compiled into the kernel
>     rather than built as modules, these delays will no longer block the
>     boot thread.  At least 100 ms is saved for each root hub, which can
>     add up to a significant savings in total boot time.
>     
>     Arjan van de Ven was very pleased to see that this shaved 700 ms off
>     his computer's boot time.  Since his total boot time is on the order
>     of two seconds, the improvement is considerable.
>     
>     Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
>     Tested-by: Arjan van de Ven <arjan@infradead.org>
>     Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>


My preliminary guess is that this made things --too-- asynchronous, and 
for some reason userland begins executing before the SCSI core 
initializes the USB storage as Linux block devices.

In any case, I cannot boot because of the above commit :)

	Jeff





[-- Attachment #2: lspci.txt --]
[-- Type: text/plain, Size: 1877 bytes --]

00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
00:02.0 VGA compatible controller: Intel Corporation 82G33/G31 Express Integrated Graphics Controller
00:19.0 Ethernet controller: Intel Corporation 82566DC-2 Gigabit Network Connection (rev 01)
00:1a.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 01)
00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 01)
00:1a.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 01)
00:1a.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 01)
00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 (rev 01)
00:1c.3 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 4 (rev 01)
00:1c.5 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 6 (rev 01)
00:1d.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 01)
00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 01)
00:1d.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 01)
00:1d.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 01)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 91)
00:1f.0 ISA bridge: Intel Corporation Device 2910 (rev 01)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA AHCI Controller (rev 01)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 01)
00:1f.6 Signal processing controller: Intel Corporation 82801I (ICH9 Family) Thermal Subsystem
02:00.0 SATA controller: Marvell Technology Group Ltd. 88SE6145 SATA II PCI-E controller (rev a1)
04:00.0 Ethernet controller: Lite-On Communications Inc LNE100TX (rev 20)

[-- Attachment #3: dmesg.txt --]
[-- Type: text/plain, Size: 28164 bytes --]

BIOS EBDA/lowmem at: 0009fc00/0009fc00
Linux version 2.6.27-bisect-05355-g3c4bb71 (jgarzik@localhost.localdomain) (gcc version 4.3.2 20081105 (Red Hat 4.3.2-7) (GCC) ) #14 SMP Tue Apr 14 16:55:03 EDT 2009
Command line: ro root=UUID=622e4b9b-68fd-4683-8a02-269fbce2263e nogui
KERNEL supported cpus:
  Intel GenuineIntel
  AMD AuthenticAMD
  Centaur CentaurHauls
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000003f5b0000 (usable)
 BIOS-e820: 000000003f5b0000 - 000000003f5be000 (ACPI data)
 BIOS-e820: 000000003f5be000 - 000000003f5f0000 (ACPI NVS)
 BIOS-e820: 000000003f5f0000 - 000000003f600000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000ffb00000 - 0000000100000000 (reserved)
DMI present.
last_pfn = 0x3f5b0 max_arch_pfn = 0x3ffffffff
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
init_memory_mapping
 0000000000 - 003f400000 page 2M
 003f400000 - 003f5b0000 page 4k
kernel direct mapping tables up to 3f5b0000 @ 8000-b000
last_map_addr: 3f5b0000 end: 3f5b0000
RAMDISK: 37c94000 - 37fef67b
ACPI: RSDP 000F94A0, 0024 (r2 ACPIAM)
ACPI: XSDT 3F5B0100, 0054 (r1 A M I  OEMXSDT  11000628 MSFT       97)
ACPI: FACP 3F5B0290, 00F4 (r3 A M I  OEMFACP  11000628 MSFT       97)
ACPI: DSDT 3F5B0440, 5B7A (r1 VVBLI9 VVBLI926       26 INTL 20051117)
ACPI: FACS 3F5BE000, 0040
ACPI: APIC 3F5B0390, 006C (r1 A M I  OEMAPIC  11000628 MSFT       97)
ACPI: MCFG 3F5B0400, 003C (r1 A M I  OEMMCFG  11000628 MSFT       97)
ACPI: OEMB 3F5BE040, 0065 (r1 A M I  AMI_OEM  11000628 MSFT       97)
ACPI: GSCI 3F5BE0B0, 2024 (r1 A M I  GMCHSCI  11000628 MSFT       97)
ACPI: DMAR 3F5B6060, 00D0 (r1 A M I  OEMDMAR         1 MSFT       97)
ACPI: Local APIC address 0xfee00000
(6 early reservations) ==> bootmem [0000000000 - 003f5b0000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0000006000 - 0000008000]       TRAMPOLINE ==> [0000006000 - 0000008000]
  #2 [0000200000 - 0000694f54]    TEXT DATA BSS ==> [0000200000 - 0000694f54]
  #3 [0037c94000 - 0037fef67b]          RAMDISK ==> [0037c94000 - 0037fef67b]
  #4 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
  #5 [0000008000 - 0000009000]          PGTABLE ==> [0000008000 - 0000009000]
found SMP MP-table at [ffff8800000ff780] 000ff780
 [ffffe20000000000-ffffe20000dfffff] PMD -> [ffff880001200000-ffff880001ffffff] on node 0
Zone PFN ranges:
  DMA      0x00000000 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   0x00100000 -> 0x00100000
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
    0: 0x00000000 -> 0x0000009f
    0: 0x00000100 -> 0x0003f5b0
On node 0 totalpages: 259407
  DMA zone: 2669 pages, LIFO batch:0
  DMA32 zone: 251916 pages, LIFO batch:31
BIOS BUG: DMAR advertised on Intel G31/G33 chipset -- ignoring
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x82] disabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x83] disabled)
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, version 0, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
SMP: Allowing 4 CPUs, 2 hotplug CPUs
Allocating PCI resources starting at 40000000 (gap: 3f600000:bf800000)
PERCPU: Allocating 43168 bytes of per cpu data
NR_CPUS: 8, nr_cpu_ids: 4, nr_node_ids 1
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 254585
Kernel command line: ro root=UUID=622e4b9b-68fd-4683-8a02-269fbce2263e nogui
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Extended CMOS year: 2000
Fast TSC calibration using PIT
Detected 2660.399 MHz processor.
Console: colour VGA+ 80x25
console [tty0] enabled
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
Checking aperture...
No AGP bridge found
Memory: 1012740k/1038016k available (2686k kernel code, 24304k reserved, 1177k data, 344k init)
Calibrating delay loop (skipped), value calculated using timer frequency.. 5320.79 BogoMIPS (lpj=10641596)
Mount-cache hash table entries: 256
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 4096K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
CPU0: Thermal monitoring enabled (TM2)
using mwait in idle threads.
ACPI: Core revision 20080609
Setting APIC routing to flat
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Intel(R) Core(TM)2 Duo CPU            @ 2.66GHz stepping 09
APIC timer calibration result 20784438
Detected 20.784 MHz APIC timer.
Booting processor 1/1 ip 6000
Initializing CPU#1
Calibrating delay using timer specific routine.. 5320.82 BogoMIPS (lpj=10641651)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 4096K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
CPU1: Thermal monitoring enabled (TM2)
x86 PAT enabled: cpu 1, old 0x7040600070406, new 0x7010600070106
CPU1: Intel(R) Core(TM)2 Duo CPU            @ 2.66GHz stepping 09
checking TSC synchronization [CPU#0 -> CPU#1]: passed.
Brought up 2 CPUs
Total of 2 processors activated (10641.62 BogoMIPS).
net_namespace: 888 bytes
NET: Registered protocol family 16
No dock devices found.
ACPI: bus type pci registered
PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
PCI: Not using MMCONFIG.
PCI: Using configuration type 1 for base access
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S1 S3 S5)
ACPI: Using IOAPIC for interrupt routing
PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
PCI: MCFG area at e0000000 reserved in ACPI motherboard resources
PCI: Using MMCONFIG at e0000000 - efffffff
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI: 0000:00:02.0 reg 10 32bit mmio: [ffa80000, ffafffff]
PCI: 0000:00:02.0 reg 14 io port: [ec00, ec07]
PCI: 0000:00:02.0 reg 18 32bit mmio: [d0000000, dfffffff]
PCI: 0000:00:02.0 reg 1c 32bit mmio: [ff900000, ff9fffff]
PCI: 0000:00:19.0 reg 10 32bit mmio: [ffa40000, ffa5ffff]
PCI: 0000:00:19.0 reg 14 32bit mmio: [ffa7a000, ffa7afff]
PCI: 0000:00:19.0 reg 18 io port: [e000, e01f]
pci 0000:00:19.0: PME# supported from D0 D3hot D3cold
pci 0000:00:19.0: PME# disabled
PCI: 0000:00:1a.0 reg 20 io port: [dc00, dc1f]
PCI: 0000:00:1a.1 reg 20 io port: [d880, d89f]
PCI: 0000:00:1a.2 reg 20 io port: [d800, d81f]
PCI: 0000:00:1a.7 reg 10 32bit mmio: [ffa7b400, ffa7b7ff]
pci 0000:00:1a.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1a.7: PME# disabled
pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.0: PME# disabled
pci 0000:00:1c.3: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.3: PME# disabled
pci 0000:00:1c.5: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.5: PME# disabled
PCI: 0000:00:1d.0 reg 20 io port: [d480, d49f]
PCI: 0000:00:1d.1 reg 20 io port: [d400, d41f]
PCI: 0000:00:1d.2 reg 20 io port: [d080, d09f]
PCI: 0000:00:1d.7 reg 10 32bit mmio: [ffa7b000, ffa7b3ff]
pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1d.7: PME# disabled
PCI: 0000:00:1f.2 reg 10 io port: [e880, e887]
PCI: 0000:00:1f.2 reg 14 io port: [e800, e803]
PCI: 0000:00:1f.2 reg 18 io port: [e480, e487]
PCI: 0000:00:1f.2 reg 1c io port: [e400, e403]
PCI: 0000:00:1f.2 reg 20 io port: [e080, e09f]
PCI: 0000:00:1f.2 reg 24 32bit mmio: [ffa7b800, ffa7bfff]
pci 0000:00:1f.2: PME# supported from D3hot
pci 0000:00:1f.2: PME# disabled
PCI: 0000:00:1f.3 reg 10 64bit mmio: [ffa79c00, ffa79cff]
PCI: 0000:00:1f.3 reg 20 io port: [400, 41f]
PCI: 0000:00:1f.6 reg 10 64bit mmio: [ffa78000, ffa78fff]
PCI: 0000:02:00.0 reg 10 io port: [bc00, bc07]
PCI: 0000:02:00.0 reg 14 io port: [b880, b883]
PCI: 0000:02:00.0 reg 18 io port: [b800, b807]
PCI: 0000:02:00.0 reg 1c io port: [b480, b483]
PCI: 0000:02:00.0 reg 20 io port: [b400, b40f]
PCI: 0000:02:00.0 reg 24 32bit mmio: [ff5ffc00, ff5fffff]
PCI: 0000:02:00.0 reg 30 32bit mmio: [ff580000, ff5bffff]
pci 0000:02:00.0: supports D1
pci 0000:02:00.0: PME# supported from D0 D1 D3hot
pci 0000:02:00.0: PME# disabled
PCI: bridge 0000:00:1c.3 io port: [b000, bfff]
PCI: bridge 0000:00:1c.3 32bit mmio: [ff500000, ff5fffff]
PCI: 0000:04:00.0 reg 10 io port: [c800, c8ff]
PCI: 0000:04:00.0 reg 14 32bit mmio: [ff6ffc00, ff6ffcff]
PCI: 0000:04:00.0 reg 30 32bit mmio: [ff680000, ff6bffff]
pci 0000:00:1e.0: transparent bridge
PCI: bridge 0000:00:1e.0 io port: [c000, cfff]
PCI: bridge 0000:00:1e.0 32bit mmio: [ff600000, ff6fffff]
bus 00 -> node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P4._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P7._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P9._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 12 14 *15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *10 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 *7 10 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 *5 6 7 10 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 12 *14 15)
ACPI: PCI Interrupt Link [LNKF] (IRQs *3 4 5 6 7 10 12 14 15)
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 *6 7 10 12 14 15)
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
DMAR:Host address width 40
DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed91000
DMAR:DRHD (flags: 0x00000001)base: 0x00000000fed93000
DMAR:RMRR base: 0x00000000000ed000 end: 0x00000000000effff
DMAR:RMRR base: 0x000000003f600000 end: 0x000000003fffffff
PCI-GART: No AMD northbridge found.
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 14 devices
ACPI: ACPI bus type pnp unregistered
system 00:01: iomem range 0xfed14000-0xfed19fff has been reserved
system 00:07: ioport range 0xa20-0xa3f has been reserved
system 00:07: ioport range 0xa00-0xa0f has been reserved
system 00:07: ioport range 0xa10-0xa1f has been reserved
system 00:07: ioport range 0xa40-0xa5f has been reserved
system 00:08: ioport range 0x4d0-0x4d1 has been reserved
system 00:08: ioport range 0x800-0x87f has been reserved
system 00:08: ioport range 0x480-0x4bf has been reserved
system 00:08: iomem range 0xfed1c000-0xfed1ffff has been reserved
system 00:08: iomem range 0xfed20000-0xfed8ffff has been reserved
system 00:0a: iomem range 0xffc00000-0xffefffff could not be reserved
system 00:0b: iomem range 0xfec00000-0xfec00fff has been reserved
system 00:0b: iomem range 0xfee00000-0xfee00fff could not be reserved
system 00:0c: iomem range 0xe0000000-0xefffffff has been reserved
system 00:0d: iomem range 0x0-0x9ffff could not be reserved
system 00:0d: iomem range 0xc0000-0xcffff has been reserved
system 00:0d: iomem range 0xe0000-0xfffff could not be reserved
system 00:0d: iomem range 0x100000-0x3f5fffff could not be reserved
pci 0000:00:1c.0: PCI bridge, secondary bus 0000:01
pci 0000:00:1c.0:   IO window: disabled
pci 0000:00:1c.0:   MEM window: disabled
pci 0000:00:1c.0:   PREFETCH window: disabled
pci 0000:00:1c.3: PCI bridge, secondary bus 0000:02
pci 0000:00:1c.3:   IO window: 0xb000-0xbfff
pci 0000:00:1c.3:   MEM window: 0xff500000-0xff5fffff
pci 0000:00:1c.3:   PREFETCH window: disabled
pci 0000:00:1c.5: PCI bridge, secondary bus 0000:03
pci 0000:00:1c.5:   IO window: disabled
pci 0000:00:1c.5:   MEM window: disabled
pci 0000:00:1c.5:   PREFETCH window: disabled
pci 0000:00:1e.0: PCI bridge, secondary bus 0000:04
pci 0000:00:1e.0:   IO window: 0xc000-0xcfff
pci 0000:00:1e.0:   MEM window: 0xff600000-0xff6fffff
pci 0000:00:1e.0:   PREFETCH window: 0x00000040000000-0x000000400fffff
pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.0: setting latency timer to 64
pci 0000:00:1c.3: PCI INT D -> GSI 19 (level, low) -> IRQ 19
pci 0000:00:1c.3: setting latency timer to 64
pci 0000:00:1c.5: PCI INT B -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:1c.5: setting latency timer to 64
pci 0000:00:1e.0: setting latency timer to 64
bus: 00 index 0 io port: [0, ffff]
bus: 00 index 1 mmio: [0, ffffffffffffffff]
bus: 01 index 0 mmio: [0, 0]
bus: 01 index 1 mmio: [0, 0]
bus: 01 index 2 mmio: [0, 0]
bus: 01 index 3 mmio: [0, 0]
bus: 02 index 0 io port: [b000, bfff]
bus: 02 index 1 mmio: [ff500000, ff5fffff]
bus: 02 index 2 mmio: [0, 0]
bus: 02 index 3 mmio: [0, 0]
bus: 03 index 0 mmio: [0, 0]
bus: 03 index 1 mmio: [0, 0]
bus: 03 index 2 mmio: [0, 0]
bus: 03 index 3 mmio: [0, 0]
bus: 04 index 0 io port: [c000, cfff]
bus: 04 index 1 mmio: [ff600000, ff6fffff]
bus: 04 index 2 mmio: [40000000, 400fffff]
bus: 04 index 3 io port: [0, ffff]
bus: 04 index 4 mmio: [0, ffffffffffffffff]
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 6, 262144 bytes)
TCP established hash table entries: 131072 (order: 9, 2097152 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
NET: Registered protocol family 1
checking if image is initramfs... it is
Freeing initrd memory: 3437k freed
alg: cipher: Test 1 failed on encryption for aes-asm
00000000: 00 01 02 03 04 05 06 07 08 08 08 08 08 08 08 08 
audit: initializing netlink socket (disabled)
type=2000 audit(1239742653.463:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
msgmni has been set to 1985
alg: No test for stdrng (krng)
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
pci 0000:00:02.0: Boot video device
pcieport-driver 0000:00:1c.0: setting latency timer to 64
pcieport-driver 0000:00:1c.0: found MSI capability
pci_express 0000:00:1c.0:pcie00: allocate port service
pci_express 0000:00:1c.0:pcie02: allocate port service
pci_express 0000:00:1c.0:pcie03: allocate port service
pcieport-driver 0000:00:1c.3: setting latency timer to 64
pcieport-driver 0000:00:1c.3: found MSI capability
pci_express 0000:00:1c.3:pcie00: allocate port service
pci_express 0000:00:1c.3:pcie02: allocate port service
pci_express 0000:00:1c.3:pcie03: allocate port service
pcieport-driver 0000:00:1c.5: setting latency timer to 64
pcieport-driver 0000:00:1c.5: found MSI capability
pci_express 0000:00:1c.5:pcie00: allocate port service
pci_express 0000:00:1c.5:pcie02: allocate port service
pci_express 0000:00:1c.5:pcie03: allocate port service
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
input: Power Button (FF) as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
ACPI: Power Button (FF) [PWRF]
input: Power Button (CM) as /devices/LNXSYSTM:00/device:00/PNP0C0C:00/input/input1
ACPI: Power Button (CM) [PWRB]
ACPI: SSDT 3F5B6130, 097D (r1    AMI   CPU1PM        1 INTL 20051117)
ACPI: CPU0 (power states: C1[C1] C2[C2])
processor ACPI0007:00: registered as cooling_device0
ACPI: Processor [CPU1] (supports 8 throttling states)
ACPI Warning (tbutils-0217): Incorrect checksum in table [SSDT] - E4, should be 0B [20080609]
ACPI: SSDT 3F5B6AB0, 097D (r1    AMI   CPU2PM        1 INTL 20051117)
ACPI: CPU1 (power states: C1[C1] C2[C2])
processor ACPI0007:01: registered as cooling_device1
ACPI: Processor [CPU2] (supports 8 throttling states)
Linux agpgart interface v0.103
agpgart-intel 0000:00:00.0: Intel G33 Chipset
agpgart-intel 0000:00:00.0: detected 6140K stolen memory
agpgart-intel 0000:00:00.0: AGP aperture is 256M @ 0xd0000000
Serial: 8250/16550 driver4 ports, IRQ sharing enabled
brd: module loaded
Driver 'sd' needs updating - please use bus_type methods
ehci_hcd 0000:00:1a.7: PCI INT D -> GSI 19 (level, low) -> IRQ 19
ehci_hcd 0000:00:1a.7: setting latency timer to 64
ehci_hcd 0000:00:1a.7: EHCI Host Controller
ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1a.7: debug port 1
ehci_hcd 0000:00:1a.7: cache line size of 32 is not supported
ehci_hcd 0000:00:1a.7: irq 19, io mem 0xffa7b400
ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 6 ports detected
Switched to high resolution mode on CPU 0
Switched to high resolution mode on CPU 1
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 2.6.27-bisect-05355-g3c4bb71 ehci_hcd
usb usb1: SerialNumber: 0000:00:1a.7
ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 23 (level, low) -> IRQ 23
ehci_hcd 0000:00:1d.7: setting latency timer to 64
ehci_hcd 0000:00:1d.7: EHCI Host Controller
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 2
ehci_hcd 0000:00:1d.7: debug port 1
ehci_hcd 0000:00:1d.7: cache line size of 32 is not supported
ehci_hcd 0000:00:1d.7: irq 23, io mem 0xffa7b000
ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 6 ports detected
usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: EHCI Host Controller
usb usb2: Manufacturer: Linux 2.6.27-bisect-05355-g3c4bb71 ehci_hcd
usb usb2: SerialNumber: 0000:00:1d.7
USB Universal Host Controller Interface driver v3.0
uhci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
uhci_hcd 0000:00:1a.0: setting latency timer to 64
uhci_hcd 0000:00:1a.0: UHCI Host Controller
uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:1a.0: irq 16, io base 0x0000dc00
usb usb3: configuration #1 chosen from 1 choice
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
usb 1-3: new high speed USB device using ehci_hcd and address 3
usb 1-3: configuration #1 chosen from 1 choice
usb 1-3: New USB device found, idVendor=090c, idProduct=1000
usb 1-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 1-3: Product: DISK 2.0
usb 1-3: Manufacturer: USB
usb 1-3: SerialNumber: 0Z0AYREUHJUQN3GH
usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb3: Product: UHCI Host Controller
usb usb3: Manufacturer: Linux 2.6.27-bisect-05355-g3c4bb71 uhci_hcd
usb usb3: SerialNumber: 0000:00:1a.0
uhci_hcd 0000:00:1a.1: PCI INT B -> GSI 21 (level, low) -> IRQ 21
uhci_hcd 0000:00:1a.1: setting latency timer to 64
uhci_hcd 0000:00:1a.1: UHCI Host Controller
uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 4
uhci_hcd 0000:00:1a.1: irq 21, io base 0x0000d880
usb usb4: configuration #1 chosen from 1 choice
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 2 ports detected
usb 1-4: new high speed USB device using ehci_hcd and address 4
usb 1-4: configuration #1 chosen from 1 choice
usb 1-4: New USB device found, idVendor=0930, idProduct=6545
usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 1-4: Product: USB Flash Memory
usb 1-4: Manufacturer:         
usb 1-4: SerialNumber: 5B861C0001B7
usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb4: Product: UHCI Host Controller
usb usb4: Manufacturer: Linux 2.6.27-bisect-05355-g3c4bb71 uhci_hcd
usb usb4: SerialNumber: 0000:00:1a.1
uhci_hcd 0000:00:1a.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
uhci_hcd 0000:00:1a.2: setting latency timer to 64
uhci_hcd 0000:00:1a.2: UHCI Host Controller
uhci_hcd 0000:00:1a.2: new USB bus registered, assigned bus number 5
uhci_hcd 0000:00:1a.2: irq 18, io base 0x0000d800
usb usb5: configuration #1 chosen from 1 choice
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
usb 3-1: new low speed USB device using uhci_hcd and address 2
usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb5: Product: UHCI Host Controller
usb usb5: Manufacturer: Linux 2.6.27-bisect-05355-g3c4bb71 uhci_hcd
usb usb5: SerialNumber: 0000:00:1a.2
uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
uhci_hcd 0000:00:1d.0: setting latency timer to 64
uhci_hcd 0000:00:1d.0: UHCI Host Controller
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 6
uhci_hcd 0000:00:1d.0: irq 23, io base 0x0000d480
usb usb6: configuration #1 chosen from 1 choice
hub 6-0:1.0: USB hub found
hub 6-0:1.0: 2 ports detected
usb usb6: New USB device found, idVendor=1d6b, idProduct=0001
usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb6: Product: UHCI Host Controller
usb usb6: Manufacturer: Linux 2.6.27-bisect-05355-g3c4bb71 uhci_hcd
usb usb6: SerialNumber: 0000:00:1d.0
uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
uhci_hcd 0000:00:1d.1: setting latency timer to 64
uhci_hcd 0000:00:1d.1: UHCI Host Controller
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 7
uhci_hcd 0000:00:1d.1: irq 19, io base 0x0000d400
usb usb7: configuration #1 chosen from 1 choice
hub 7-0:1.0: USB hub found
hub 7-0:1.0: 2 ports detected
usb 3-1: configuration #1 chosen from 1 choice
usb 3-1: New USB device found, idVendor=045e, idProduct=000b
usb 3-1: New USB device strings: Mfr=0, Product=1, SerialNumber=0
usb 3-1: Product: Microsoft Natural Keyboard Elite
usb usb7: New USB device found, idVendor=1d6b, idProduct=0001
usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb7: Product: UHCI Host Controller
usb usb7: Manufacturer: Linux 2.6.27-bisect-05355-g3c4bb71 uhci_hcd
usb usb7: SerialNumber: 0000:00:1d.1
uhci_hcd 0000:00:1d.2: PCI INT D -> GSI 16 (level, low) -> IRQ 16
uhci_hcd 0000:00:1d.2: setting latency timer to 64
uhci_hcd 0000:00:1d.2: UHCI Host Controller
uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 8
uhci_hcd 0000:00:1d.2: irq 16, io base 0x0000d080
usb usb8: configuration #1 chosen from 1 choice
hub 8-0:1.0: USB hub found
hub 8-0:1.0: 2 ports detected
usb usb8: New USB device found, idVendor=1d6b, idProduct=0001
usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb8: Product: UHCI Host Controller
usb usb8: Manufacturer: Linux 2.6.27-bisect-05355-g3c4bb71 uhci_hcd
usb usb8: SerialNumber: 0000:00:1d.2
Initializing USB Mass Storage driver...
scsi0 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 3
usb-storage: waiting for device to settle before scanning
scsi1 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 4
usb-storage: waiting for device to settle before scanning
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
usbcore: registered new interface driver libusual
PNP: PS/2 Controller [PNP0303:PS2K] at 0x60,0x64 irq 1
PNP: PS/2 appears to have AUX port disabled, if this is incorrect please boot with i8042.nopnp
Clocksource tsc unstable (delta = 9371902387 ns)
i8042.c: Can't read CTR while initializing i8042.
i8042: probe of i8042 failed with error -5
mice: PS/2 mouse device common for all mice
isa bounce pool size: 16 pages
scsi 0:0:0:0: Direct-Access     USB      DISK 2.0         0403 PQ: 0 ANSI: 0 CCS
scsi 1:0:0:0: Direct-Access              USB Flash Memory PMAP PQ: 0 ANSI: 0 CCS
sd 0:0:0:0: [sda] 3981312 512-byte hardware sectors: (2.03GB/1.89GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
sd 0:0:0:0: [sda] Assuming drive cache: write through
sd 1:0:0:0: [sdb] 3921920 512-byte hardware sectors: (2.00GB/1.87GiB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: 23 00 00 00
sd 1:0:0:0: [sdb] Assuming drive cache: write through
sd 0:0:0:0: [sda] 3981312 512-byte hardware sectors: (2.03GB/1.89GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
sd 0:0:0:0: [sda] Assuming drive cache: write through
 sda: sda1
sd 0:0:0:0: [sda] Attached SCSI removable disk
usb-storage: device scan complete
sd 1:0:0:0: [sdb] 3921920 512-byte hardware sectors: (2.00GB/1.87GiB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: 23 00 00 00
sd 1:0:0:0: [sdb] Assuming drive cache: write through
 sdb: sdb1
sd 1:0:0:0: [sdb] Attached SCSI removable disk
usb-storage: device scan complete
input: PC Speaker as /devices/platform/pcspkr/input/input2
cpuidle: using governor ladder
Marking TSC unstable due to TSC halts in idle
usbcore: registered new interface driver hiddev
input: Microsoft Natural Keyboard Elite as /devices/pci0000:00/0000:00:1a.0/usb3/3-1/3-1:1.0/input/input3
generic-usb 0003:045E:000B.0001: input,hidraw0: USB HID v1.10 Keyboard [Microsoft Natural Keyboard Elite] on usb-0000:00:1a.0-1/input0
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
TCP cubic registered
Freeing unused kernel memory: 344k freed
Write protecting the kernel read-only data: 3564k
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
udevd version 127 started
libata version 3.00 loaded.
i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18
pata_marvell 0000:02:00.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
pata_marvell 0000:02:00.0: setting latency timer to 64
scsi2 : pata_marvell
scsi3 : pata_marvell
ata1: PATA max UDMA/100 cmd 0xbc00 ctl 0xb880 bmdma 0xb400 irq 19
ata2: PATA max UDMA/133 cmd 0xb800 ctl 0xb480 bmdma 0xb408 irq 19
BAR5:00:04 01:7F 02:22 03:C8 04:00 05:00 06:00 07:00 08:00 09:00 0A:00 0B:00 0C:1F 0D:00 0E:00 0F:00 
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 1:0:0:0: Attached scsi generic sg1 type 0
ahci 0000:00:1f.2: version 3.0
ahci 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports ? Gbps 0x3f impl SATA mode
ahci 0000:00:1f.2: flags: 64bit stag pm led clo pmp pio slum part ems 
ahci 0000:00:1f.2: setting latency timer to 64
scsi4 : ahci
scsi5 : ahci
scsi6 : ahci
scsi7 : ahci
scsi8 : ahci
scsi9 : ahci
ata3: SATA max UDMA/133 abar m2048@0xffa7b800 port 0xffa7b900 irq 508
ata4: SATA max UDMA/133 abar m2048@0xffa7b800 port 0xffa7b980 irq 508
ata5: SATA max UDMA/133 abar m2048@0xffa7b800 port 0xffa7ba00 irq 508
ata6: SATA max UDMA/133 abar m2048@0xffa7b800 port 0xffa7ba80 irq 508
ata7: SATA max UDMA/133 abar m2048@0xffa7b800 port 0xffa7bb00 irq 508
ata8: SATA max UDMA/133 abar m2048@0xffa7b800 port 0xffa7bb80 irq 508
Linux Tulip driver version 1.1.15 (Feb 27, 2007)
tulip 0000:04:00.0: PCI INT A -> GSI 21 (level, low) -> IRQ 21
tulip0:  MII transceiver #1 config 3000 status 7829 advertising 01e1.
eth0: Lite-On 82c168 PNIC rev 32 at MMIO 0xff6ffc00, 00:a0:cc:60:b5:f4, IRQ 21.
ata3: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
ata5: SATA link down (SStatus 0 SControl 300)
ata6: SATA link down (SStatus 0 SControl 300)
ata7: SATA link down (SStatus 0 SControl 300)
ata8: SATA link down (SStatus 0 SControl 300)
EXT3 FS on sda1, internal journal
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdb1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
platform microcode: firmware: requesting intel-ucode/06-0f-09
platform microcode: firmware: requesting intel-ucode/06-0f-09
Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk> <peter.oruba@amd.com>
Microcode Update Driver: v2.00 removed.
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
NET: Registered protocol family 17
eth0: Setting full-duplex based on MII#1 link partner capability of cde1.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.

[-- Attachment #4: config.txt.gz --]
[-- Type: application/x-gzip, Size: 10112 bytes --]

Re: USB storage no-boot regression (bisected)

[Arjan van de Ven]

On Tue, 14 Apr 2009 17:06:14 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> 
> Once of the x86-64 machines I use for testing runs off of two 2GB USB 
> flash drives, one for Fedora 10 userland, and one for kernel
> repository 
> + builds.
> 
> It boots correctly in 2.6.27, but fails with the same symptoms in 
> 2.6.28, 2.6.29 and 2.6.30-rc1:
> 
> 	1) The kernel boots
> 	2) After time passes, kernel begins executing initramfs
> 	   userland
> 	3) the kernel prints out probe messages for the USB keyboard,
> 	   SCSI probe messages for the two USB flash drives
> 
> Or IOW, the keyboard and two SCSI drives appear after initramfs
> begins booting.  And this is for drivers built into the kernel
> (though same behavior with modules).
> 
> This no-boot regression is 100% reproducible, and neatly bisects down
> to
> 
> > commit 8520f38099ccfdac2147a0852f84ee7a8ee5e197
> > Author: Alan Stern <stern@rowland.harvard.edu>
> > Date:   Mon Sep 22 14:44:26 2008 -0400
> > 
> >     USB: change hub initialization sleeps to delayed_work
> >     
> >     This patch (as1137) changes the hub_activate() routine,
> > replacing the power-power-up and debounce delays with delayed_work
> > calls.  The idea is that on systems where the USB stack is compiled
> > into the kernel rather than built as modules, these delays will no
> > longer block the boot thread.  At least 100 ms is saved for each
> > root hub, which can add up to a significant savings in total boot
> > time. 
> >     Arjan van de Ven was very pleased to see that this shaved 700
> > ms off his computer's boot time.  Since his total boot time is on
> > the order of two seconds, the improvement is considerable.
> >     
> >     Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
> >     Tested-by: Arjan van de Ven <arjan@infradead.org>
> >     Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
> 
> 
> My preliminary guess is that this made things --too-- asynchronous,
> and for some reason userland begins executing before the SCSI core 
> initializes the USB storage as Linux block devices.
> 
> In any case, I cannot boot because of the above commit :)

the fundamental problem with USB is that you don'tm know when you're
done probing ... devices show up when they feel like or more or less.

This change just made it go faster enough for you to be out of luck;
fundamentally your userland needs to wait if the device it wants is not
there. 

Or you pass a kernel boot parameter to wait.. there is one (root_delay
or something, haven't used it in a while).


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: USB storage no-boot regression (bisected)

[Greg KH]

On Tue, Apr 14, 2009 at 05:06:14PM -0400, Jeff Garzik wrote:
> 
> Once of the x86-64 machines I use for testing runs off of two 2GB USB 
> flash drives, one for Fedora 10 userland, and one for kernel repository 
> + builds.
> 
> It boots correctly in 2.6.27, but fails with the same symptoms in 
> 2.6.28, 2.6.29 and 2.6.30-rc1:
> 
> 	1) The kernel boots
> 	2) After time passes, kernel begins executing initramfs
> 	   userland
> 	3) the kernel prints out probe messages for the USB keyboard,
> 	   SCSI probe messages for the two USB flash drives
> 
> Or IOW, the keyboard and two SCSI drives appear after initramfs begins 
> booting.  And this is for drivers built into the kernel (though same 
> behavior with modules).
> 
> This no-boot regression is 100% reproducible, and neatly bisects down to
> 
> > commit 8520f38099ccfdac2147a0852f84ee7a8ee5e197
> > Author: Alan Stern <stern@rowland.harvard.edu>
> > Date:   Mon Sep 22 14:44:26 2008 -0400
> > 
> >     USB: change hub initialization sleeps to delayed_work
> >     
> >     This patch (as1137) changes the hub_activate() routine, replacing the
> >     power-power-up and debounce delays with delayed_work calls.  The idea
> >     is that on systems where the USB stack is compiled into the kernel
> >     rather than built as modules, these delays will no longer block the
> >     boot thread.  At least 100 ms is saved for each root hub, which can
> >     add up to a significant savings in total boot time.
> >     
> >     Arjan van de Ven was very pleased to see that this shaved 700 ms off
> >     his computer's boot time.  Since his total boot time is on the order
> >     of two seconds, the improvement is considerable.
> >     
> >     Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
> >     Tested-by: Arjan van de Ven <arjan@infradead.org>
> >     Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
> 
> 
> My preliminary guess is that this made things --too-- asynchronous, and 
> for some reason userland begins executing before the SCSI core 
> initializes the USB storage as Linux block devices.
> 
> In any case, I cannot boot because of the above commit :)

Like Arjan said, this is because we are initializing faster now, and
things are a bit more asynchronous.  Use the root_delay boot option,
that's what I use for my USB-based systems, and have not had a problem
with that at all.

thanks,

greg k-h

Re: USB storage no-boot regression (bisected)

[Jeff Garzik]

Arjan van de Ven wrote:
> This change just made it go faster enough for you to be out of luck;
> fundamentally your userland needs to wait if the device it wants is not
> there. 

All these drivers are in-kernel, and the root device is passed via 
command line.  There is no userland at that point, that needs to wait.

If this change added an implicit initramfs requirement, that is a pretty 
major regression.

	Jeff

Re: USB storage no-boot regression (bisected)

[Jeff Garzik]

Greg KH wrote:
> On Tue, Apr 14, 2009 at 05:06:14PM -0400, Jeff Garzik wrote:
>> Once of the x86-64 machines I use for testing runs off of two 2GB USB 
>> flash drives, one for Fedora 10 userland, and one for kernel repository 
>> + builds.
>>
>> It boots correctly in 2.6.27, but fails with the same symptoms in 
>> 2.6.28, 2.6.29 and 2.6.30-rc1:
>>
>> 	1) The kernel boots
>> 	2) After time passes, kernel begins executing initramfs
>> 	   userland
>> 	3) the kernel prints out probe messages for the USB keyboard,
>> 	   SCSI probe messages for the two USB flash drives
>>
>> Or IOW, the keyboard and two SCSI drives appear after initramfs begins 
>> booting.  And this is for drivers built into the kernel (though same 
>> behavior with modules).
>>
>> This no-boot regression is 100% reproducible, and neatly bisects down to
>>
>>> commit 8520f38099ccfdac2147a0852f84ee7a8ee5e197
>>> Author: Alan Stern <stern@rowland.harvard.edu>
>>> Date:   Mon Sep 22 14:44:26 2008 -0400
>>>
>>>     USB: change hub initialization sleeps to delayed_work
>>>     
>>>     This patch (as1137) changes the hub_activate() routine, replacing the
>>>     power-power-up and debounce delays with delayed_work calls.  The idea
>>>     is that on systems where the USB stack is compiled into the kernel
>>>     rather than built as modules, these delays will no longer block the
>>>     boot thread.  At least 100 ms is saved for each root hub, which can
>>>     add up to a significant savings in total boot time.
>>>     
>>>     Arjan van de Ven was very pleased to see that this shaved 700 ms off
>>>     his computer's boot time.  Since his total boot time is on the order
>>>     of two seconds, the improvement is considerable.
>>>     
>>>     Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
>>>     Tested-by: Arjan van de Ven <arjan@infradead.org>
>>>     Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
>>
>> My preliminary guess is that this made things --too-- asynchronous, and 
>> for some reason userland begins executing before the SCSI core 
>> initializes the USB storage as Linux block devices.
>>
>> In any case, I cannot boot because of the above commit :)
> 
> Like Arjan said, this is because we are initializing faster now, and
> things are a bit more asynchronous.  Use the root_delay boot option,
> that's what I use for my USB-based systems, and have not had a problem
> with that at all.

Is that solution really scalable to every user with a regression severe 
enough it prevents them from booting?

When did regressions become an acceptable tradeoff for speed?

This system boots just fine under kernel 2.6.27, 2.6.26, 2.6.25, and so 
on.  Switch the kernel to 2.6.28, and it no longer boots.  A regression 
cannot get more clear than that.

Maybe this commit should have been accompanied by one that checks "root=" ?

	Jeff

Re: USB storage no-boot regression (bisected)

[Arjan van de Ven]

On Tue, 14 Apr 2009 22:30:28 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> Arjan van de Ven wrote:
> > This change just made it go faster enough for you to be out of luck;
> > fundamentally your userland needs to wait if the device it wants is
> > not there. 
> 
> All these drivers are in-kernel, and the root device is passed via 
> command line.  There is no userland at that point, that needs to wait.

ok fair; but that does not change that the kernel does not know if a
device is coming.
Yes that sucks; sadly USB is just this way, you don't know when no new
devices will come from a certain bus.

> 
> If this change added an implicit initramfs requirement, that is a
> pretty major regression.

it didn't; this is what root_wait is for.


Having said that, I have a patch that basically retries this stuff
inside the kernel if the mount-of-rootfs fails....

(but it's independent of the usb improvement)

-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: USB storage no-boot regression (bisected)

[Arjan van de Ven]

On Tue, 14 Apr 2009 22:35:59 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> Greg KH wrote:
> > On Tue, Apr 14, 2009 at 05:06:14PM -0400, Jeff Garzik wrote:
> >> Once of the x86-64 machines I use for testing runs off of two 2GB
> >> USB flash drives, one for Fedora 10 userland, and one for kernel
> >> repository 
> >> + builds.
> >>
> >> It boots correctly in 2.6.27, but fails with the same symptoms in 
> >> 2.6.28, 2.6.29 and 2.6.30-rc1:
> >>
> >> 	1) The kernel boots
> >> 	2) After time passes, kernel begins executing initramfs
> >> 	   userland
> >> 	3) the kernel prints out probe messages for the USB
> >> keyboard, SCSI probe messages for the two USB flash drives
> >>
> >> Or IOW, the keyboard and two SCSI drives appear after initramfs
> >> begins booting.  And this is for drivers built into the kernel
> >> (though same behavior with modules).
> >>
> >> This no-boot regression is 100% reproducible, and neatly bisects
> >> down to
> >>
> >>> commit 8520f38099ccfdac2147a0852f84ee7a8ee5e197
> >>> Author: Alan Stern <stern@rowland.harvard.edu>
> >>> Date:   Mon Sep 22 14:44:26 2008 -0400
> >>>
> >>>     USB: change hub initialization sleeps to delayed_work
> >>>     
> >>>     This patch (as1137) changes the hub_activate() routine,
> >>> replacing the power-power-up and debounce delays with
> >>> delayed_work calls.  The idea is that on systems where the USB
> >>> stack is compiled into the kernel rather than built as modules,
> >>> these delays will no longer block the boot thread.  At least 100
> >>> ms is saved for each root hub, which can add up to a significant
> >>> savings in total boot time. 
> >>>     Arjan van de Ven was very pleased to see that this shaved 700
> >>> ms off his computer's boot time.  Since his total boot time is on
> >>> the order of two seconds, the improvement is considerable.
> >>>     
> >>>     Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
> >>>     Tested-by: Arjan van de Ven <arjan@infradead.org>
> >>>     Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
> >>
> >> My preliminary guess is that this made things --too--
> >> asynchronous, and for some reason userland begins executing before
> >> the SCSI core initializes the USB storage as Linux block devices.
> >>
> >> In any case, I cannot boot because of the above commit :)
> > 
> > Like Arjan said, this is because we are initializing faster now, and
> > things are a bit more asynchronous.  Use the root_delay boot option,
> > that's what I use for my USB-based systems, and have not had a
> > problem with that at all.
> 
> Is that solution really scalable to every user with a regression
> severe enough it prevents them from booting?
> 
> When did regressions become an acceptable tradeoff for speed?
> 
> This system boots just fine under kernel 2.6.27, 2.6.26, 2.6.25, and
> so on.  Switch the kernel to 2.6.28, and it no longer boots.  A
> regression cannot get more clear than that.

You had pure luck though.

We used to wait 100 msec per USB bus.
A normal laptop has like 5 of these.
if your usb storage was in the first one, basically you got a "free"
500msec delay there. You are/were happy.

Now.. if you stuck your disk in the last port you would get a 100msec
delay. Probably not enough for what you want. But you didn't stick
your disk there....

In the new code all ports get their power turned on and THEN things
wait... so all ports get the 100 msec treatment, not the
500/400/300/200/100 staggering.


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: USB storage no-boot regression (bisected)

[Jeff Garzik]

Arjan van de Ven wrote:
> On Tue, 14 Apr 2009 22:30:28 -0400
> Jeff Garzik <jeff@garzik.org> wrote:
> 
>> Arjan van de Ven wrote:
>>> This change just made it go faster enough for you to be out of luck;
>>> fundamentally your userland needs to wait if the device it wants is
>>> not there. 
>> All these drivers are in-kernel, and the root device is passed via 
>> command line.  There is no userland at that point, that needs to wait.
> 
> ok fair; but that does not change that the kernel does not know if a
> device is coming.
> Yes that sucks; sadly USB is just this way, you don't know when no new
> devices will come from a certain bus.

Perhaps -- but I can say that kernels <= 2.6.27 booted with 100% 
reliability.

Now, Kernels >= 2.6.28 always fail.

The ONLY variable is the kernel.

	Jeff

Re: USB storage no-boot regression (bisected)

[Arjan van de Ven]

On Tue, 14 Apr 2009 22:48:50 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> Arjan van de Ven wrote:
> > On Tue, 14 Apr 2009 22:30:28 -0400
> > Jeff Garzik <jeff@garzik.org> wrote:
> > 
> >> Arjan van de Ven wrote:
> >>> This change just made it go faster enough for you to be out of
> >>> luck; fundamentally your userland needs to wait if the device it
> >>> wants is not there. 
> >> All these drivers are in-kernel, and the root device is passed via 
> >> command line.  There is no userland at that point, that needs to
> >> wait.
> > 
> > ok fair; but that does not change that the kernel does not know if a
> > device is coming.
> > Yes that sucks; sadly USB is just this way, you don't know when no
> > new devices will come from a certain bus.
> 
> Perhaps -- but I can say that kernels <= 2.6.27 booted with 100% 
> reliability.
> 
> Now, Kernels >= 2.6.28 always fail.
> d
> The ONLY variable is the kernel.

yes.
and you can get the old behavior back by just sticking an
msleep(100 * <number of USB ports you have minus one>); 
back in near the end of the boot.

that really is not the right answer.

by your argument if anything else gets faster and your system is then
booting too fast again that has to get reverted as well.

If there was some reasonable way to wait on USB scanning being done
we'd do that. Not a nanosecond doubt about that.

But since there isn't, this "race who's faster" is an unsolvable
problem other than by you saying "wait for root to be there please".

        rootwait        [KNL] Wait (indefinitely) for root device to show up.
                        Useful for devices that are detected asynchronously
                        (e.g. USB and MMC devices).

that option has been there for a really long time, and has technically be required
for your case. You got lucky so far by pure timing... until recently.
It's not nice to not boot suddenly, I sympathize with that. But there's nothing 
we can realistically do other than using the option that is there for exactly your case.



-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: USB storage no-boot regression (bisected)

[Greg KH]

On Tue, Apr 14, 2009 at 10:35:59PM -0400, Jeff Garzik wrote:
> > Like Arjan said, this is because we are initializing faster now, and
> > things are a bit more asynchronous.  Use the root_delay boot option,
> > that's what I use for my USB-based systems, and have not had a problem
> > with that at all.
> 
> Is that solution really scalable to every user with a regression severe 
> enough it prevents them from booting?
> 
> When did regressions become an acceptable tradeoff for speed?

So, we aren't allowed to go faster?

What happens when you buy a new box with more USB host controllers and a
faster processor?  Same problem.

> This system boots just fine under kernel 2.6.27, 2.6.26, 2.6.25, and so 
> on.  Switch the kernel to 2.6.28, and it no longer boots.  A regression 
> cannot get more clear than that.
> 
> Maybe this commit should have been accompanied by one that checks "root=" ?

How would that be accomplished?

The issue is that you were just lucky that your machine worked properly
previously.  My boxes with the same type of setup didn't, so I quickly
realized what the root delay boot option was for.  You need to just do
the same thing here, there's nothing else we can do.

thanks,

greg k-h

Re: USB storage no-boot regression (bisected)

[Alan Cox]

> If this change added an implicit initramfs requirement, that is a pretty 
> major regression.

USB root has always needed an initramfs. On some boxes it might happen to
work most of the time because of other implicit delays. You were just
lucky in the past.

If you are so initramfs averse you could try hacking the kernel to try
remounting the root every few seconds for a bit if it fails rather than
giving up.

Alan

Re: USB storage no-boot regression (bisected)

[Jeff Garzik]

Greg KH wrote:
> On Tue, Apr 14, 2009 at 10:35:59PM -0400, Jeff Garzik wrote:
>>> Like Arjan said, this is because we are initializing faster now, and
>>> things are a bit more asynchronous.  Use the root_delay boot option,
>>> that's what I use for my USB-based systems, and have not had a problem
>>> with that at all.
>> Is that solution really scalable to every user with a regression severe 
>> enough it prevents them from booting?
>>
>> When did regressions become an acceptable tradeoff for speed?
> 
> So, we aren't allowed to go faster?

Well, when the result fails to boot, you are only going faster to a point :)

Oh well...

	Jeff

Re: USB storage no-boot regression (bisected)

[Mark Lord]

Greg KH wrote:
> ..
> The issue is that you were just lucky that your machine worked properly
> previously.  My boxes with the same type of setup didn't, so I quickly
> realized what the root delay boot option was for.  You need to just do
> the same thing here, there's nothing else we can do.
..

Bad excuse.

SATA drives also take variable amounts of time to "show up" at boot.
Perhaps Jeff should customize libata for your and Arjan's exact setups,
just to help with understanding the point here.  :)

The speed ups are fine (and welcome), but we really now need
Arjan to follow-up with a patch to have the kernel *by default*
wait a little longer for the rootfs to show up.

Not forever, just a few seconds to compensate for the regression.

Re: USB storage no-boot regression (bisected)

[Arjan van de Ven]

On Wed, 15 Apr 2009 10:25:05 -0400
Mark Lord <lkml@rtr.ca> wrote:

> Greg KH wrote:
> > ..
> > The issue is that you were just lucky that your machine worked
> > properly previously.  My boxes with the same type of setup didn't,
> > so I quickly realized what the root delay boot option was for.  You
> > need to just do the same thing here, there's nothing else we can do.
> ..
> 
> Bad excuse.
> 
> SATA drives also take variable amounts of time to "show up" at boot.
> Perhaps Jeff should customize libata for your and Arjan's exact
> setups, just to help with understanding the point here.  :)

the difference is that with sata you know when you are done and have all
possible drives. No so much much with USB. So with SATA we can, and do,
wait for the scan to complete at the right point in the boot.

> 
> The speed ups are fine (and welcome), but we really now need
> Arjan to follow-up with a patch to have the kernel *by default*
> wait a little longer for the rootfs to show up.
> 
> Not forever, just a few seconds to compensate for the regression.

seconds!!!!!
The whole kernel boots in half a second!

for this case, which is unfortunately not detectable by the kernel,
there is the rootwait option.

-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: USB storage no-boot regression (bisected)

[Alan Cox]

> SATA drives also take variable amounts of time to "show up" at boot.
> Perhaps Jeff should customize libata for your and Arjan's exact setups,
> just to help with understanding the point here.  :)

Actually I would argue the reverse. The sooner we can push this so that
libata isn't blocking mounting the rootfs the better.

> The speed ups are fine (and welcome), but we really now need
> Arjan to follow-up with a patch to have the kernel *by default*
> wait a little longer for the rootfs to show up.
> 
> Not forever, just a few seconds to compensate for the regression.

Why should every user suffer a slower boot and a poorer resume time ?

Instead make the root fs mounting look like this


	while(my_rootfs_hasnt_appeared_and_i_am_sad()) {
		wait_on(&new_disk_discovery);
	}

and poke the queue whenever we add a relevant device.

That way if you are booting off an initrd you can finish the SATA probe
in parallel to getting userspace ticking over.

On what is nowdays essentially a hot plug system it all needs turning
this way up - eg RAID volumes should assemble and come online as the
drives are discovered not at some fixed point later in userspace.

Alan

Re: USB storage no-boot regression (bisected)

[Mark Lord]

Arjan van de Ven wrote:
> On Wed, 15 Apr 2009 10:25:05 -0400
> Mark Lord <lkml@rtr.ca> wrote:
> 
>> Greg KH wrote:
>>> ..
>>> The issue is that you were just lucky that your machine worked
>>> properly previously.  My boxes with the same type of setup didn't,
>>> so I quickly realized what the root delay boot option was for.  You
>>> need to just do the same thing here, there's nothing else we can do.
>> ..
>>
>> Bad excuse.
>>
>> SATA drives also take variable amounts of time to "show up" at boot.
>> Perhaps Jeff should customize libata for your and Arjan's exact
>> setups, just to help with understanding the point here.  :)
> 
> the difference is that with sata you know when you are done and have all
> possible drives. No so much much with USB. So with SATA we can, and do,
> wait for the scan to complete at the right point in the boot.
> 
>> The speed ups are fine (and welcome), but we really now need
>> Arjan to follow-up with a patch to have the kernel *by default*
>> wait a little longer for the rootfs to show up.
>>
>> Not forever, just a few seconds to compensate for the regression.
> 
> seconds!!!!!
> The whole kernel boots in half a second!
..

Oh, absolutely I agree.

That's why I'm not suggesting a DELAY,
but rather a TIMEOUT (where it keeps trying up until the timeout).

For desktop, it should really just wait forever,
but I can understand situations (server room)
where that would be a Really Bad Idea.

So just have it sit there and retry the rootfs for a few seconds,
to compensate for this regression and also for others that have
yet to be discovered / reported.

Everyone's life will be easier that way.

Cheers

Re: USB storage no-boot regression (bisected)

[Alan Stern]

On Wed, 15 Apr 2009, Alan Cox wrote:

> Why should every user suffer a slower boot and a poorer resume time ?
> 
> Instead make the root fs mounting look like this
> 
> 
> 	while(my_rootfs_hasnt_appeared_and_i_am_sad()) {
> 		wait_on(&new_disk_discovery);
> 	}
> 
> and poke the queue whenever we add a relevant device.
> 
> That way if you are booting off an initrd you can finish the SATA probe
> in parallel to getting userspace ticking over.
> 
> On what is nowdays essentially a hot plug system it all needs turning
> this way up - eg RAID volumes should assemble and come online as the
> drives are discovered not at some fixed point later in userspace.

Indeed, something like this should also be used for 
resume-from-hibernation, to wait for the swap device.

Alan Stern

Re: USB storage no-boot regression (bisected)

[Mark Lord]

Alan Stern wrote:
> On Wed, 15 Apr 2009, Alan Cox wrote:
> 
>> Why should every user suffer a slower boot and a poorer resume time ?
>>
>> Instead make the root fs mounting look like this
>>
>>
>> 	while(my_rootfs_hasnt_appeared_and_i_am_sad()) {
>> 		wait_on(&new_disk_discovery);
>> 	}
>>
>> and poke the queue whenever we add a relevant device.
>>
>> That way if you are booting off an initrd you can finish the SATA probe
>> in parallel to getting userspace ticking over.
>>
>> On what is nowdays essentially a hot plug system it all needs turning
>> this way up - eg RAID volumes should assemble and come online as the
>> drives are discovered not at some fixed point later in userspace.
> 
> Indeed, something like this should also be used for 
> resume-from-hibernation, to wait for the swap device.
..

It just needs a way to set a finite timeout, so that server room
equipment can auto-panic-reboot and try again if a device has died.

-ml

Re: USB storage no-boot regression (bisected)

[VomLehn]

On Wed, Apr 15, 2009 at 11:49:45AM -0400, Mark Lord wrote:
> Alan Stern wrote:
>> On Wed, 15 Apr 2009, Alan Cox wrote:
>>
>>> Why should every user suffer a slower boot and a poorer resume time ?
>>>
>>> Instead make the root fs mounting look like this
>>>
>>>
>>> 	while(my_rootfs_hasnt_appeared_and_i_am_sad()) {
>>> 		wait_on(&new_disk_discovery);
>>> 	}
>>>
>>> and poke the queue whenever we add a relevant device.
>>>
>>> That way if you are booting off an initrd you can finish the SATA probe
>>> in parallel to getting userspace ticking over.
>>>
>>> On what is nowdays essentially a hot plug system it all needs turning
>>> this way up - eg RAID volumes should assemble and come online as the
>>> drives are discovered not at some fixed point later in userspace.
>>
>> Indeed, something like this should also be used for  
>> resume-from-hibernation, to wait for the swap device.
> ..
>
> It just needs a way to set a finite timeout, so that server room
> equipment can auto-panic-reboot and try again if a device has died.

The problem with USB root devices is the same one I brought up a couple of
weeks ago--faster booting means that USB boot devices fail. We now have
problems with three different classes of devices:

o	Disks
o	Network devices
o	Serial consoles

Saying that we were "lucky" that things worked before is no help and
you should be aware that it ticks people off. I agree that this is not a
USB problem, but there is a *very* real problem:

     The work to decrease boot time has exposed race conditions that
     always existed, but are now making the kernel less usable.

So, instead of spending time denying that there is a USB problem, let's
focus on solve the boot device synchronization problem.  I have already
posted a (probably incomplete, possibly wrong) patch to synchronize console
initialization. We need to do the same for other boot devices, too.

David VomLehn

Re: USB storage no-boot regression (bisected)

[Alan Cox]

> Saying that we were "lucky" that things worked before is no help and
> you should be aware that it ticks people off. 

No doubt those who wrote the code and never expected that timing to
reliably work will take affront at those who blame them for it not
working. Those who expected it to work and never knew it was not
guaranteed will feel likewise in reverse.

The real questions are should it be fixed and is dumping the problem on
the initrd the right answer. The concensus appears to yes and no.

The console one is less clear but if we are doing one we should probably
do both.

Alan

Re: USB storage no-boot regression (bisected)

[Arjan van de Ven]

On Wed, 15 Apr 2009 11:37:44 -0400
Mark Lord <lkml@rtr.ca> wrote:

> Arjan van de Ven wrote:
> > On Wed, 15 Apr 2009 10:25:05 -0400
> > Mark Lord <lkml@rtr.ca> wrote:
> > 
> >> Greg KH wrote:
> >>> ..
> >>> The issue is that you were just lucky that your machine worked
> >>> properly previously.  My boxes with the same type of setup didn't,
> >>> so I quickly realized what the root delay boot option was for.
> >>> You need to just do the same thing here, there's nothing else we
> >>> can do.
> >> ..
> >>
> >> Bad excuse.
> >>
> >> SATA drives also take variable amounts of time to "show up" at
> >> boot. Perhaps Jeff should customize libata for your and Arjan's
> >> exact setups, just to help with understanding the point here.  :)
> > 
> > the difference is that with sata you know when you are done and
> > have all possible drives. No so much much with USB. So with SATA we
> > can, and do, wait for the scan to complete at the right point in
> > the boot.
> > 
> >> The speed ups are fine (and welcome), but we really now need
> >> Arjan to follow-up with a patch to have the kernel *by default*
> >> wait a little longer for the rootfs to show up.
> >>
> >> Not forever, just a few seconds to compensate for the regression.
> > 
> > seconds!!!!!
> > The whole kernel boots in half a second!
> ..
> 
> Oh, absolutely I agree.
> 
> That's why I'm not suggesting a DELAY
> but rather a TIMEOUT (where it keeps trying up until the timeout).

This exists today. It's just not something Jeff chose to use ;)
(because he didn't need to)

> 
> For desktop, it should really just wait forever,
> but I can understand situations (server room)
> where that would be a Really Bad Idea.

it's called rootwait and such :)


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: USB storage no-boot regression (bisected)

[Arjan van de Ven]

On Wed, 15 Apr 2009 18:32:52 +0100
Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> > Saying that we were "lucky" that things worked before is no help and
> > you should be aware that it ticks people off. 
> 
> No doubt those who wrote the code and never expected that timing to
> reliably work will take affront at those who blame them for it not
> working. 

It's even more than that; it's OTHER code that gets faster that
can/will break this.

> Those who expected it to work and never knew it was not
> guaranteed will feel likewise in reverse.

For storage at least the solution already existed (root_wait);
it just hasn't been used consistently.

For other pieces it's hard. Non-enumeratable busses just suck;
at some point all you can do is just wait (which we have already 
available today for anyone to do). I realize people don't want to
just wait 4 seconds (the people who first objected to boot time
improvements then suddenly care about boot time ;-)...

For root fs there's some options, and I have patches to basically retry
on fail. (The patches have a bug and I don't have time to solve it this
week, so I'm not submitting them)
For other devices it is hard. Realistically we need hotplug to work
well enough so that when a device shows up, we can just hook it up when
it does.


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: USB storage no-boot regression (bisected)

[VomLehn]

On Wed, Apr 15, 2009 at 01:06:14PM -0700, Arjan van de Ven wrote:
> For other pieces it's hard. Non-enumeratable busses just suck;
> at some point all you can do is just wait (which we have already 
> available today for anyone to do). I realize people don't want to
> just wait 4 seconds (the people who first objected to boot time
> improvements then suddenly care about boot time ;-)...

We can do better than just waiting. What we want to do is to wait
until a suitable device becomes available. For USB console, the patch
I submitted yesterday waits for the first console to be registered.
That might not be quite right, but it's a lot better than no console
at all. For network devices, we have to wait for a specific device and
I haven't had a chance to look at this. For both types of devices, we
can timeout after a reasonable interval, for some definition of reasonable.

Waiting for a suitable device preserves fast boot times, timing out allows
the system to come up enough to try to diagnose the problem.

David VomLehn

Re: USB storage no-boot regression (bisected)

[Mark Lord]

Arjan van de Ven wrote:
> On Wed, 15 Apr 2009 11:37:44 -0400
> Mark Lord <lkml@rtr.ca> wrote:
> 
>> Arjan van de Ven wrote:
>>> On Wed, 15 Apr 2009 10:25:05 -0400
>>> Mark Lord <lkml@rtr.ca> wrote:
..
>>>> Not forever, just a few seconds to compensate for the regression.
>>> seconds!!!!!
>>> The whole kernel boots in half a second!
>> ..
>>
>> Oh, absolutely I agree.
>>
>> That's why I'm not suggesting a DELAY
>> but rather a TIMEOUT (where it keeps trying up until the timeout).
> 
> This exists today. It's just not something Jeff chose to use ;)
> (because he didn't need to)
> 
>> For desktop, it should really just wait forever,
>> but I can understand situations (server room)
>> where that would be a Really Bad Idea.
> 
> it's called rootwait and such :)
..

No, that's not the same thing.
rootwait has no timeout -- it waits *forever*,
which will break auto-recovery on servers.

We just need it to wait/retry up to a timeout (parameter?).

Thanks

Re: USB storage no-boot regression (bisected)

[Arjan van de Ven]

On Wed, 15 Apr 2009 17:55:08 -0400
Mark Lord <lkml@rtr.ca> wrote:

> Arjan van de Ven wrote:
> > On Wed, 15 Apr 2009 11:37:44 -0400
> > Mark Lord <lkml@rtr.ca> wrote:
> > 
> >> Arjan van de Ven wrote:
> >>> On Wed, 15 Apr 2009 10:25:05 -0400
> >>> Mark Lord <lkml@rtr.ca> wrote:
> ..
> >>>> Not forever, just a few seconds to compensate for the regression.
> >>> seconds!!!!!
> >>> The whole kernel boots in half a second!
> >> ..
> >>
> >> Oh, absolutely I agree.
> >>
> >> That's why I'm not suggesting a DELAY
> >> but rather a TIMEOUT (where it keeps trying up until the timeout).
> > 
> > This exists today. It's just not something Jeff chose to use ;)
> > (because he didn't need to)
> > 
> >> For desktop, it should really just wait forever,
> >> but I can understand situations (server room)
> >> where that would be a Really Bad Idea.
> > 
> > it's called rootwait and such :)
> ..
> 
> No, that's not the same thing.
> rootwait has no timeout -- it waits *forever*,
> which will break auto-recovery on servers.

btw while I don't disagree that something like this is nice...

.... who would boot their server from a USB stick for production use?


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: USB storage no-boot regression (bisected)

[Mark Lord]

Arjan van de Ven wrote:
> On Wed, 15 Apr 2009 17:55:08 -0400
..
>> No, that's not the same thing.
>> rootwait has no timeout -- it waits *forever*,
>> which will break auto-recovery on servers.
> 
> btw while I don't disagree that something like this is nice...
> 
> .... who would boot their server from a USB stick for production use?
..

At least one MAJOR storage vendor that I know of.
Probably more.

Though it's an onboard USB SSD, not a pluggable stick.

Cheers

Re: USB storage no-boot regression (bisected)

[Hal Murray]

> .... who would boot their server from a USB stick for production use?

I've worked on a project that was doing that.  Think of embedded computing 
rather than a typical box in a server farm.

If you want to do something slightly more complicated than monitor 
temperature, it's sometimes simpler/cheaper to trim down a real OS than it is 
to build up an environment that has everything you need, especially if you 
want to do any networking.  With enough RAM, LInux runs fine without a disk, 
and if you don't need a GUI, "enough RAM" isn't much by modern standards.

There are several distributions targeted at running out of RAM after booting 
from an USB thumb drive or Compact Flash or CD or whatever.

Google for >Linux on a Stick< or >Puppy Linux< for a few starters.


-- 
These are my opinions, not necessarily my employer's.  I hate spam.

Re: USB storage no-boot regression (bisected)

[Greg KH]

On Wed, Apr 15, 2009 at 10:14:38PM -0400, Mark Lord wrote:
> Arjan van de Ven wrote:
> > On Wed, 15 Apr 2009 17:55:08 -0400
> ..
> >> No, that's not the same thing.
> >> rootwait has no timeout -- it waits *forever*,
> >> which will break auto-recovery on servers.
> > 
> > btw while I don't disagree that something like this is nice...
> > 
> > .... who would boot their server from a USB stick for production use?
> ..
> 
> At least one MAJOR storage vendor that I know of.
> Probably more.
> 
> Though it's an onboard USB SSD, not a pluggable stick.

Wow, that's insane.  Leave it to a hardware designer to use a bus that
you never know if you have discovered all the devices to be the primary
device to boot from.

greg k-h

Re: USB storage no-boot regression (bisected)

[Jeff Garzik]

Arjan van de Ven wrote:
> .... who would boot their server from a USB stick for production use?

Whole distros have chosen to do so.  the real world doesn't care about 
USB enumeration details, they just want to boot off their $9 Wal-Mart 
USB sticks... :)

	Jeff

Re: USB storage no-boot regression (bisected)

[Alan Cox]

> > Though it's an onboard USB SSD, not a pluggable stick.
> 
> Wow, that's insane.  Leave it to a hardware designer to use a bus that
> you never know if you have discovered all the devices to be the primary
> device to boot from.

Let me see:

- Standardised small component
- Low pin count and low wire count bus
- Cheap

In an embedded environment where you know the device is wired in at
software level that strikes me not as insane but very sensible.

Re: USB storage no-boot regression (bisected)

[Arjan van de Ven]

On Thu, 16 Apr 2009 11:51:53 +0100
Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> > > Though it's an onboard USB SSD, not a pluggable stick.
> > 
> > Wow, that's insane.  Leave it to a hardware designer to use a bus
> > that you never know if you have discovered all the devices to be
> > the primary device to boot from.
> 
> Let me see:
> 
> - Standardised small component
> - Low pin count and low wire count bus
> - Cheap
> 
> In an embedded environment where you know the device is wired in at
> software level that strikes me not as insane but very sensible.

I'm not surprised usb is used for storage. What I am surprised at is
that USB is used in such high end environments that rootwait is not
sufficient and that the panic-rebooter is needed for the reliability.



-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: USB storage no-boot regression (bisected)

[Mark Lord]

Arjan van de Ven wrote:
> On Thu, 16 Apr 2009 11:51:53 +0100
> Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> 
>>>> Though it's an onboard USB SSD, not a pluggable stick.
>>> Wow, that's insane.  Leave it to a hardware designer to use a bus
>>> that you never know if you have discovered all the devices to be
>>> the primary device to boot from.
>> Let me see:
>>
>> - Standardised small component
>> - Low pin count and low wire count bus
>> - Cheap
>>
>> In an embedded environment where you know the device is wired in at
>> software level that strikes me not as insane but very sensible.
> 
> I'm not surprised usb is used for storage. What I am surprised at is
> that USB is used in such high end environments that rootwait is not
> sufficient and that the panic-rebooter is needed for the reliability.
..

Failover and redundancy are standard requirements for really reliable systems.
As good as a component may be, there's always anticipation that it will fail
someday, and multiple ways of detecting/overcoming failures must be built-in.

In this case, imagine *two* such SSDs.