[Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Gerd Hoffmann]

  Hi,

This pull finally fixes the efi boot support.  ipxe is updated to the
latest master, two non-upstream commits needed to make efi work are
added on top, and the build process is tweaked a bit.

The ipxe changes are pushed to git://git.kraxel.org/ipxe (branch qemu,
tag qemu-2.4).  They should be mirrored to git://git.qemu.org/ipxe.git
before merging this pull request.

please pull,
  Gerd

The following changes since commit 2d5ee9e7a7dd495d233cf9613a865f63f88e3375:

  Merge remote-tracking branch 'remotes/lalrae/tags/mips-20150716' into staging (2015-07-16 10:40:23 +0100)

are available in the git repository at:


  git://git.kraxel.org/qemu tags/pull-ipxe-20150717-1

for you to fetch changes up to 4f0c601b71e53a7d225a1913b784242400788991:

  ipxe: update binaries (2015-07-16 17:39:12 +0200)

----------------------------------------------------------------
update ipxe roms, fix efi support

----------------------------------------------------------------
Gerd Hoffmann (7):
      ipxe: update from 35c53797 to 24112d9 (upstream/master)
      ipxe: update to 87981bb (qemu)
      ipxe: rm local config in cleanup
      ipxe: disable load file protocol
      ipxe: add qemu branding
      ipxe: don't override GITVERSION
      ipxe: update binaries

 pc-bios/efi-e1000.rom       | Bin 197120 -> 192512 bytes
 pc-bios/efi-eepro100.rom    | Bin 197632 -> 192512 bytes
 pc-bios/efi-ne2k_pci.rom    | Bin 195584 -> 190976 bytes
 pc-bios/efi-pcnet.rom       | Bin 195584 -> 190976 bytes
 pc-bios/efi-rtl8139.rom     | Bin 200192 -> 194560 bytes
 pc-bios/efi-virtio.rom      | Bin 194048 -> 188928 bytes
 roms/Makefile               |   9 +++++----
 roms/config.ipxe.branding.h |   2 ++
 roms/config.ipxe.general.h  |   1 +
 roms/ipxe                   |   2 +-
 10 files changed, 9 insertions(+), 5 deletions(-)
 create mode 100644 roms/config.ipxe.branding.h

[Qemu-devel] [PULL 1/7] ipxe: update from 35c53797 to 24112d9 (upstream/master)

[Gerd Hoffmann]

git shortlog
============

Alex Williamson (1):
      [dhcp] Extract timing parameters out to config/dhcp.h

Bernd Wiebelt (1):
      [tg3] Add support for BCM57766

Christian Hesse (3):
      [intel] Add PCI device IDs for Intel I218-LM and I218-V
      [build] Add missing "const" qualifiers
      [ath9k] Remove confusing logic inversion in an ANI variable

Ed Swierk (1):
      [intel] Update PCI device IDs for Intel 82599 and X540 10G NICs

Laszlo Ersek (1):
      [virtio] Downgrade per-iobuf debug messages to DBGC2

Michael Brown (228):
      [device] Provide a driver-private data field for root devices
      [iobuf] Add iob_split() to split an I/O buffer into portions
      [rndis] Add generic RNDIS device abstraction
      [hyperv] Add support for Hyper-V hypervisor
      [hyperv] Add support for VMBus devices
      [hyperv] Add support for NetVSC paravirtual network devices
      [rndis] Send RNDIS_INITIALISE_MSG
      [rndis] Send RNDIS_HALT_MSG
      [hyperv] Tear down NetVSC RX buffer GPADL after closing VMBus device
      [rndis] Clear receive filter when closing the device
      [hyperv] Receive all VMBus messages in a poll
      [hyperv] Increase TX ring size
      [hyperv] Assume that VMBus xfer page ranges correspond to RNDIS messages
      [rndis] Ignore start-of-day RNDIS_INDICATE_STATUS_MSG with status 0x40020006
      [hyperv] Tidy up debug output
      [hyperv] Require support for VMBus version 3.0 or newer
      [build] Include Hyper-V driver in the all-drivers build
      [pci] Allow drivers to specify a PCI class
      [romprefix] Ensure UNDI loader can be included by all ROM types
      [usb] Add basic support for USB devices
      [usb] Add basic support for USB hubs
      [usb] Add support for xHCI host controllers
      [ncm] Add support for CDC-NCM USB Ethernet devices
      [usb] Report xHCI host controller events
      [ncm] Use large multi-packet buffers by default
      [tftp] Explicitly abort connection whenever parent interface is closed
      [uri] Allow tftp_uri() to construct a URI with a custom port
      [pxe] Use tftp_uri() to construct PXE TFTP URIs
      [pxe] Maintain a queue for received PXE UDP packets
      [ncm] Reserve headroom in received packets
      [usb] Try multiple USB device configurations
      [usb] Handle CDC union functional descriptors
      [usb] Parse endpoint descriptor bInterval field
      [usb] Allow usb_stream() to enforce a terminating short packet
      [ecm] Add support for CDC-ECM USB Ethernet devices
      [xhci] Delay after (possibly) forcing port link state to RxDetect
      [build] Move branding information to config/branding.h
      [build] Use PRODUCT_SHORT_NAME for end-user visible strings
      [build] Allow product URI to be customised via config/branding.h
      [build] Allow error message URI to be customised via config/branding.h
      [build] Allow command help text URI to be customised via config/branding.h
      [build] Allow setting help text URI to be customised via config/branding.h
      [build] Allow product tag line to be customised via config/branding.h
      [rndis] Add rndis_rx_err()
      [usb] Handle port status changes received after failing to find a driver
      [efi] Disallow R_X86_64_32 relocations
      [build] Apply the "-fno-PIE -nopie" workaround only to i386 builds
      [usb] Provide generic framework for refilling receive endpoints
      [usb] Use generic refill framework for USB hub interrupt endpoints
      [ecm] Use generic refill framework for bulk IN and interrupt endpoints
      [ncm] Use generic refill framework for bulk IN and interrupt endpoints
      [libc] Remove unused string functions
      [libc] Rewrite string functions
      [test] Add self-tests for more string functions
      [test] Add constant-length memset() self-tests
      [libc] Reduce size of memset()
      [usb] Add generic USB network device framework
      [ecm] Use generic USB network device framework
      [ncm] Use generic USB network device framework
      [timer] Rewrite the 8254 Programmable Interval Timer support
      [xhci] Leak memory if controller fails to disable slot
      [xhci] Abort commands on timeout
      [test] Add IPv4 self-tests
      [legal] Add missing copyright header to net/ipv4.c
      [ipv4] Rewrite inet_aton()
      [libc] Rewrite strtoul()
      [hyperv] Check for required features
      [prefix] Use .bss16 as temporary stack space for calls to install_block
      [zbin] Use LZMA compression
      [zbin] Perform extra normalisation after completing decompression
      [prefix] Call decompressor in flat real mode when DEBUG=libprefix is enabled
      [zbin] Allow decompressor to generate debug output via BIOS console
      [zbin] Fix check for existence of most recent output byte
      [zbin] Remove now-unused unnrv2b.S decompressor
      [legal] Update GPLv2 licence text
      [legal] Include full licence text for all GPL2_OR_LATER files
      [mucurses] Add missing FILE_LICENCE declarations
      [legal] Add support for the Unmodified Binary Distribution Licence
      [legal] Add UBDL relicensing tool
      [legal] Relicense files under GPL2_OR_LATER_OR_UBDL
      [legal] Relicense files under GPL2_OR_LATER_OR_UBDL
      [legal] Relicense files under GPL2_OR_LATER_OR_UBDL
      [legal] Relicense files under GPL2_OR_LATER_OR_UBDL
      [libc] Rewrite unrelicensable portions of stddef.h
      [libc] Rewrite unrelicensable portions of ctype.h
      [libc] Rewrite setjmp() and longjmp()
      [libc] Rewrite byte-swapping code
      [elf] Rewrite ELF header
      [list] Relicense list.h
      [iscsi] Rewrite unrelicensable portions of iscsi.c
      [pci] Remove outdated and mostly-unused pci_ids.h file
      [pci] Rewrite unrelicensable portions of pci.h
      [settings] Use list_first_entry() when unregistering child settings
      [settings] Rewrite unrelicensable portions of settings.c
      [menu] Abstract out the generic concept of a jump scroller
      [settings] Use generic jump scrolling abstraction
      [malloc] Move valgrind headers out of arch/x86
      [malloc] Rewrite unrelicensable portions of malloc.c
      [build] Remove unused IMPORT_SYMBOL() and EXPORT_SYMBOL() macros
      [build] Remove unused __keepme macro
      [pxe] Remove obsolete references to pxeparent_dhcp
      [build] Remove obsolete and unused portions of config.c
      [build] Use REQUIRE_OBJECT() to drag in per-object configuration
      [build] Fix the REQUIRE_SYMBOL mechanism
      [i386] Move real_to_user() to realmode.h
      [linux] Rewrite headers included in all builds
      [retry] Rewrite unrelicensable portions of retry.c
      [retry] Colourise debug output
      [legal] Relicense files under GPL2_OR_LATER_OR_UBDL
      [xhci] Enable USB3 ports on Intel PCH8/PCH9 controllers
      [xhci] Undo PCH-specific quirk fixes when removing device
      [xen] Set the "feature-rx-notify" flag for netfront devices
      [http] Abstract out HTTP Digest hash algorithm operations
      [http] Support MD5-sess Digest authentication
      [dm96xx] Add driver for Davicom DM96xx USB Ethernet NICs
      [legal] Relicense Davicom DM96xx drivers
      [mii] Add generic mii_check_link() function
      [smsc75xx] Add driver for SMSC/Microchip LAN75xx USB Ethernet NICs
      [legal] Relicense files under GPL2_OR_LATER_OR_UBDL
      [tcp] Implement support for TCP Selective Acknowledgements (SACK)
      [smsc75xx] Move RX FIFO overflow message to DBGLVL_EXTRA
      [tcpip] Fix dubious calculation of min_port
      [libc] Add ffs(), ffsl(), and ffsll()
      [usb] Add the concept of a USB bus maximum transfer size
      [ncm] Respect maximum transfer size of the bus
      [usb] Add functions for manual device address assignment
      [xhci] Forcibly disable SMIs if BIOS fails to release ownership
      [autoboot] Match against parent devices when matching by bus type and location
      [usb] Add config/usb.h for USB configuration options
      [xhci] Do not release ownership back to BIOS when booting an OS
      [ehci] Add support for EHCI host controllers
      [netdevice] Add missing bus types to netdev_fetch_bustype()
      [usb] Fix USB timeouts to match specification
      [libprefix] Fix building on 64-bit FreeBSD 8.4
      [xhci] Ring doorbell as part of endpoint reset
      [usb] Reset endpoints without waiting for a new transfer to be enqueued
      [usb] Add clear_tt() hub method to clear transaction translator buffer
      [usb] Clear transaction translator buffers when applicable
      [ehci] Support USB1 devices attached via transaction translators
      [usb] Improve debug messages for failed control transactions
      [xhci] Support USB1 devices attached via transaction translators
      [libc] Fix typo in longjmp()
      [libc] Add x86_64 versions of setjmp() and longjmp()
      [test] Add setjmp()/longjmp() self-tests
      [test] Simplify digest algorithm self-tests
      [crypto] Add SHA-224 algorithm
      [crypto] Add SHA-512 algorithm
      [crypto] Add SHA-384 algorithm
      [crypto] Add SHA-512/256 algorithm
      [crypto] Add SHA-512/224 algorithm
      [efi] Ensure drivers are disconnected when ExitBootServices() is called
      [peerdist] Add support for decoding PeerDist Content Information
      [xhci] Always reset root hub ports
      [romprefix] Allow autoboot device filter to be disabled
      [util] Add ability to dump PCI device ID list
      [efi] Add EFI entropy source
      [efi] Add EFI time source
      [efi] Provide a dummy data block in nii_initialise()
      [efi] Poll media status only if advertised as supported
      [efi] Poll for TX completions only when there is an outstanding TX buffer
      [efi] Use the EFI_RNG_PROTOCOL as an entropy source if available
      [eepro100] Remove duplicate PCI_ROM() line
      [prism2] Remove duplicate PCI_ROM() lines
      [build] Allow building PCI ROMs with device ID lists
      [build] Fix compiler warning on OpenBSD 5.7
      [build] Work around binutils quirk on OpenBSD 5.7
      [build] Use a single call to parserom.pl to speed up building
      [intel] Report any unexpected interrupt causes
      [intel] Force RX polling on VMware emulated 82545em
      [realtek] Do not attempt to access EEPROM on RTL8169 chips
      [rtl818x] Obviate RTL_ROM() hack
      [build] Construct all-drivers list based on driver class
      [test] Include IPv6 support when performing settings self-tests
      [base16] Add buffer size parameter to base16_encode() and base16_decode()
      [base64] Add buffer size parameter to base64_encode() and base64_decode()
      [settings] Add "base64" setting type
      [vram] Add "vram" built-in setting to dump video RAM
      [usb] Include setup packet within I/O buffer for message transfers
      [pci] Provide PCI_CLASS() to calculate a scalar PCI class value
      [usb] Detect missed disconnections
      [usb] Maintain a list of all USB buses
      [usb] Maintain single lists of halted endpoints and changed ports
      [ehci] Poll child companion controllers after disowning port
      [usb] Add find_usb_bus_by_location() helper function
      [ehci] Allow UHCI/OHCI controllers to locate the EHCI companion controller
      [uhci] Add support for UHCI host controllers
      [usb] Provide usb_endpoint_name() for use by host controller drivers
      [xhci] Use meaningful device names in debug messages
      [ehci] Use meaningful device names in debug messages
      [uhci] Use meaningful device names in debug messages
      [ipv6] Disambiguate received ICMPv6 errors
      [usb] Add USB_INTERRUPT_OUT internal type
      [usb] Add generic USB human interface device (HID) framework
      [usb] Add basic support for USB keyboards
      [usb] Do not call usb_hotplug() when registering a new hub
      [usb] Always clear recorded disconnections after performing hotplug actions
      [intel] Expose intel_diag() for use by other Intel NIC drivers
      [intel] Allow for the use of advanced TX descriptors
      [intel] Add support for mailbox used by virtual functions
      [intel] Add intelxvf driver for Intel 10 GigE virtual function NICs
      [int13con] Add basic ability to log to a local disk via INT 13
      [intel] Add intelxvf_stats() to dump packet statistics registers
      [intel] Fix operation when physical function has jumbo frames enabled
      [neighbour] Return success when deferring a packet
      [xhci] Fix length of allocated slot array
      [build] Fix .ids.o creation for drivers not in the all-drivers build
      [xhci] Fix comparison of signed and unsigned integers
      [ipoib] Fix REMAC cache discarder
      [xhci] Record device-specific quirks in xHCI device structure
      [xhci] Ignore invalid protocol speed ID values on Intel Skylake platforms
      [pci] Use flat real mode to call INT 1a,b101
      [tcp] Do not shrink window when discarding received packets
      [mromprefix] Report a dummy size at offset 0x02 of .mrom payload
      [ethernet] Add minimal support for receiving LLC frames
      [netdevice] Add a generic concept of a "blocked link"
      [stp] Add support for detecting Spanning Tree Protocol non-forwarding ports
      [stp] Fix interpretaton of hello time
      [dhcp] Defer discovery if link is blocked
      [pxe] Always reconstruct packet for PXENV_GET_CACHED_INFO
      [serial] Add general abstraction of a 16550-compatible UART
      [gdb] Use new UART abstraction in GDB serial transport
      [serial] Use new UART abstraction in serial console driver
      [ipoib] Mark REMAC cache as expensive
      [ipoib] Attempt to generate ARPs as needed to repopulate REMAC cache
      [gdb] Allow gdbstub to be started on an arbitrary serial port
      [xen] Wait for and clear XenStore event before receiving data
      [tcp] Gracefully close connections during shutdown
      [ipoib] Transmit multicast packets as broadcasts

Olaf Hering (1):
      [build] Sort objects in blib.a

Robin Smidsrød (2):
      [vbox] Enable some more features now that we have LZMA compression
      [build] Rewrite parserom.pl to support multiple source files

Thomas Miletich (1):
      [intel] Add PCI ID for I218-LM

Wissam Shoukair (1):
      [comboot] Implement INT22,0x000c

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
---
 roms/ipxe | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/roms/ipxe b/roms/ipxe
index 35c5379..24112d9 160000
--- a/roms/ipxe
+++ b/roms/ipxe
@@ -1 +1 @@
-Subproject commit 35c5379760aa1fea5e38f7a78b090f92bb7813ee
+Subproject commit 24112d91a0ab4c29794bc7750cbfc4166bc9026a
-- 
1.8.3.1

[Qemu-devel] [PULL 2/7] ipxe: update to 87981bb (qemu)

[Gerd Hoffmann]

Add two patches we've been struggling to get upstream.
They are available from "git://git.qemu.org/ipxe.git qemu"

git shortlog
------------

Gerd Hoffmann (1):
      [efi] make load file protocol optional

Laszlo Ersek (1):
      efi_snp: improve compliance with the EFI_SIMPLE_NETWORK_PROTOCOL spec

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
---
 roms/ipxe | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/roms/ipxe b/roms/ipxe
index 24112d9..87981bb 160000
--- a/roms/ipxe
+++ b/roms/ipxe
@@ -1 +1 @@
-Subproject commit 24112d91a0ab4c29794bc7750cbfc4166bc9026a
+Subproject commit 87981bb66f5f496f124469fb6c13aa13ee00061e
-- 
1.8.3.1

[Qemu-devel] [PULL 3/7] ipxe: rm local config in cleanup

[Gerd Hoffmann]

ipxe build now generates empty local header files in case they are
not present.  Let's remove them on cleanup to make sure we store a
fresh copy on the next build.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
---
 roms/Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/roms/Makefile b/roms/Makefile
index 7b3f156..c0153c7 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -153,5 +153,6 @@ clean:
 	$(MAKE) -C sgabios clean
 	rm -f sgabios/.depend
 	$(MAKE) -C ipxe/src veryclean
+	(cd ipxe; rm -f src/config/local/*.h)
 	$(MAKE) -C SLOF clean
 	rm -rf u-boot/build.e500
-- 
1.8.3.1

[Qemu-devel] [PULL 4/7] ipxe: disable load file protocol

[Gerd Hoffmann]

Activate the opt-out added by one not-upstream ipxe patch:

    [efi] make load file protocol optional

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
---
 roms/config.ipxe.general.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/roms/config.ipxe.general.h b/roms/config.ipxe.general.h
index 619ee4c..2df042a 100644
--- a/roms/config.ipxe.general.h
+++ b/roms/config.ipxe.general.h
@@ -2,3 +2,4 @@
 #define BANNER_TIMEOUT 30
 #undef ROM_BANNER_TIMEOUT
 #define ROM_BANNER_TIMEOUT 0
+#undef EFI_PROTO_LOAD_FILE
-- 
1.8.3.1

[Qemu-devel] [PULL 5/7] ipxe: add qemu branding

[Gerd Hoffmann]

Apply qemu-project.org branding, so the official builds can easily be
identified in the banner.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
---
 roms/Makefile               | 4 ++--
 roms/config.ipxe.branding.h | 2 ++
 2 files changed, 4 insertions(+), 2 deletions(-)
 create mode 100644 roms/config.ipxe.branding.h

diff --git a/roms/Makefile b/roms/Makefile
index c0153c7..4971f6b 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -120,12 +120,12 @@ efi-rom-%: build-pxe-roms build-efi-roms
 		-ec ipxe/src/bin-x86_64-efi/$(VID)$(DID).efidrv \
 		-o ../pc-bios/efi-$*.rom
 
-build-pxe-roms: ipxe/src/config/local/general.h
+build-pxe-roms: ipxe/src/config/local/general.h ipxe/src/config/local/branding.h
 	$(MAKE) -C ipxe/src GITVERSION="" \
 		CROSS_COMPILE=$(x86_64_cross_prefix) \
 		$(patsubst %,bin/%.rom,$(pxerom_targets))
 
-build-efi-roms: build-pxe-roms ipxe/src/config/local/general.h
+build-efi-roms: build-pxe-roms ipxe/src/config/local/general.h ipxe/src/config/local/branding.h
 	$(MAKE) -C ipxe/src GITVERSION="" \
 		CROSS_COMPILE=$(x86_64_cross_prefix) \
 		$(patsubst %,bin-i386-efi/%.efidrv,$(pxerom_targets)) \
diff --git a/roms/config.ipxe.branding.h b/roms/config.ipxe.branding.h
new file mode 100644
index 0000000..324995e
--- /dev/null
+++ b/roms/config.ipxe.branding.h
@@ -0,0 +1,2 @@
+#undef PRODUCT_NAME
+#define PRODUCT_NAME "iPXE for qemu-project.org"
-- 
1.8.3.1

[Qemu-devel] [PULL 6/7] ipxe: don't override GITVERSION

[Gerd Hoffmann]

We had build problems due to the git version checking in the ipxe build
system in the past.  Don't remember the details, but the problem seems
to be gone now, so lets remove the workaround.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

[ most likely ipxe commit 6153c09c41034250408f3596555fcaae715da46c:
  [build] Set GITVERSION only if there is a git repository ]

Reviewed-by: Laszlo Ersek <lersek@redhat.com>
---
 roms/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/roms/Makefile b/roms/Makefile
index 4971f6b..55fc45b 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -121,12 +121,12 @@ efi-rom-%: build-pxe-roms build-efi-roms
 		-o ../pc-bios/efi-$*.rom
 
 build-pxe-roms: ipxe/src/config/local/general.h ipxe/src/config/local/branding.h
-	$(MAKE) -C ipxe/src GITVERSION="" \
+	$(MAKE) -C ipxe/src \
 		CROSS_COMPILE=$(x86_64_cross_prefix) \
 		$(patsubst %,bin/%.rom,$(pxerom_targets))
 
 build-efi-roms: build-pxe-roms ipxe/src/config/local/general.h ipxe/src/config/local/branding.h
-	$(MAKE) -C ipxe/src GITVERSION="" \
+	$(MAKE) -C ipxe/src \
 		CROSS_COMPILE=$(x86_64_cross_prefix) \
 		$(patsubst %,bin-i386-efi/%.efidrv,$(pxerom_targets)) \
 		$(patsubst %,bin-x86_64-efi/%.efidrv,$(pxerom_targets))
-- 
1.8.3.1

[Qemu-devel] [PULL 7/7] ipxe: update binaries

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
---
 pc-bios/efi-e1000.rom    | Bin 197120 -> 192512 bytes
 pc-bios/efi-eepro100.rom | Bin 197632 -> 192512 bytes
 pc-bios/efi-ne2k_pci.rom | Bin 195584 -> 190976 bytes
 pc-bios/efi-pcnet.rom    | Bin 195584 -> 190976 bytes
 pc-bios/efi-rtl8139.rom  | Bin 200192 -> 194560 bytes
 pc-bios/efi-virtio.rom   | Bin 194048 -> 188928 bytes
 6 files changed, 0 insertions(+), 0 deletions(-)

diff --git a/pc-bios/efi-e1000.rom b/pc-bios/efi-e1000.rom
index 4e29d9d..203bbaf 100644
Binary files a/pc-bios/efi-e1000.rom and b/pc-bios/efi-e1000.rom differ
diff --git a/pc-bios/efi-eepro100.rom b/pc-bios/efi-eepro100.rom
index 2a92d6f..0df3135 100644
Binary files a/pc-bios/efi-eepro100.rom and b/pc-bios/efi-eepro100.rom differ
diff --git a/pc-bios/efi-ne2k_pci.rom b/pc-bios/efi-ne2k_pci.rom
index 6366017..91c0ebb 100644
Binary files a/pc-bios/efi-ne2k_pci.rom and b/pc-bios/efi-ne2k_pci.rom differ
diff --git a/pc-bios/efi-pcnet.rom b/pc-bios/efi-pcnet.rom
index a61f586..4d79e37 100644
Binary files a/pc-bios/efi-pcnet.rom and b/pc-bios/efi-pcnet.rom differ
diff --git a/pc-bios/efi-rtl8139.rom b/pc-bios/efi-rtl8139.rom
index c9c77ea..0d3287b 100644
Binary files a/pc-bios/efi-rtl8139.rom and b/pc-bios/efi-rtl8139.rom differ
diff --git a/pc-bios/efi-virtio.rom b/pc-bios/efi-virtio.rom
index eec2790..8f9a9b4 100644
Binary files a/pc-bios/efi-virtio.rom and b/pc-bios/efi-virtio.rom differ
-- 
1.8.3.1

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Peter Maydell]

On 17 July 2015 at 15:37, Gerd Hoffmann <kraxel@redhat.com> wrote:
>   Hi,
>
> This pull finally fixes the efi boot support.  ipxe is updated to the
> latest master, two non-upstream commits needed to make efi work are
> added on top, and the build process is tweaked a bit.
>
> The ipxe changes are pushed to git://git.kraxel.org/ipxe (branch qemu,
> tag qemu-2.4).  They should be mirrored to git://git.qemu.org/ipxe.git
> before merging this pull request.

Is this supposed to happen automatically, or does somebody
need to manually do something for that to happen?

thanks
-- PMM

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Peter Maydell]

On 20 July 2015 at 11:04, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 17 July 2015 at 15:37, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>   Hi,
>>
>> This pull finally fixes the efi boot support.  ipxe is updated to the
>> latest master, two non-upstream commits needed to make efi work are
>> added on top, and the build process is tweaked a bit.
>>
>> The ipxe changes are pushed to git://git.kraxel.org/ipxe (branch qemu,
>> tag qemu-2.4).  They should be mirrored to git://git.qemu.org/ipxe.git
>> before merging this pull request.
>
> Is this supposed to happen automatically, or does somebody
> need to manually do something for that to happen?

I need an answer to this question or this pull will miss rc2 :-(

-- PMM

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Paolo Bonzini]

On 21/07/2015 10:21, Peter Maydell wrote:
>>> >> The ipxe changes are pushed to git://git.kraxel.org/ipxe (branch qemu,
>>> >> tag qemu-2.4).  They should be mirrored to git://git.qemu.org/ipxe.git
>>> >> before merging this pull request.
>> >
>> > Is this supposed to happen automatically, or does somebody
>> > need to manually do something for that to happen?
> I need an answer to this question or this pull will miss rc2 :-(

Looks like it needs to be done manually.

Paolo

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Peter Maydell]

On 21 July 2015 at 09:51, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 21/07/2015 10:21, Peter Maydell wrote:
>>>> >> The ipxe changes are pushed to git://git.kraxel.org/ipxe (branch qemu,
>>>> >> tag qemu-2.4).  They should be mirrored to git://git.qemu.org/ipxe.git
>>>> >> before merging this pull request.
>>> >
>>> > Is this supposed to happen automatically, or does somebody
>>> > need to manually do something for that to happen?
>> I need an answer to this question or this pull will miss rc2 :-(
>
> Looks like it needs to be done manually.

I think we're mirroring git://git.ipxe.org/ipxe.git, so the ipxe
changes need to be merged into that repo first.

Gerd, have you submitted those to the ipxe upstream?

thanks
-- PMM

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Paolo Bonzini]

On 21/07/2015 15:27, Peter Maydell wrote:
>> > On 21/07/2015 10:21, Peter Maydell wrote:
>>>>>>> >>>> >> The ipxe changes are pushed to git://git.kraxel.org/ipxe (branch qemu,
>>>>>>> >>>> >> tag qemu-2.4).  They should be mirrored to git://git.qemu.org/ipxe.git
>>>>>>> >>>> >> before merging this pull request.
>>>>> >>> >
>>>>> >>> > Is this supposed to happen automatically, or does somebody
>>>>> >>> > need to manually do something for that to happen?
>>> >> I need an answer to this question or this pull will miss rc2 :-(
>> >
>> > Looks like it needs to be done manually.
> I think we're mirroring git://git.ipxe.org/ipxe.git, so the ipxe
> changes need to be merged into that repo first.
> 
> Gerd, have you submitted those to the ipxe upstream?

Multiple times in the last six months.

Paolo

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Peter Maydell]

On 21 July 2015 at 14:47, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 21/07/2015 15:27, Peter Maydell wrote:
>>> > On 21/07/2015 10:21, Peter Maydell wrote:
>>>>>>>> >>>> >> The ipxe changes are pushed to git://git.kraxel.org/ipxe (branch qemu,
>>>>>>>> >>>> >> tag qemu-2.4).  They should be mirrored to git://git.qemu.org/ipxe.git
>>>>>>>> >>>> >> before merging this pull request.
>>>>>> >>> >
>>>>>> >>> > Is this supposed to happen automatically, or does somebody
>>>>>> >>> > need to manually do something for that to happen?
>>>> >> I need an answer to this question or this pull will miss rc2 :-(
>>> >
>>> > Looks like it needs to be done manually.
>> I think we're mirroring git://git.ipxe.org/ipxe.git, so the ipxe
>> changes need to be merged into that repo first.
>>
>> Gerd, have you submitted those to the ipxe upstream?
>
> Multiple times in the last six months.

So is the proposal here that we ship a non-upstream IPXE?
That was less than entirely clear from the pull request...

-- PMM

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Paolo Bonzini]

On 21/07/2015 15:57, Peter Maydell wrote:
> On 21 July 2015 at 14:47, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> On 21/07/2015 15:27, Peter Maydell wrote:
>>>>> On 21/07/2015 10:21, Peter Maydell wrote:
>>>>>>>>>>>>>>> The ipxe changes are pushed to git://git.kraxel.org/ipxe (branch qemu,
>>>>>>>>>>>>>>> tag qemu-2.4).  They should be mirrored to git://git.qemu.org/ipxe.git
>>>>>>>>>>>>>>> before merging this pull request.
>>>>>>>>>>>
>>>>>>>>>>> Is this supposed to happen automatically, or does somebody
>>>>>>>>>>> need to manually do something for that to happen?
>>>>>>> I need an answer to this question or this pull will miss rc2 :-(
>>>>>
>>>>> Looks like it needs to be done manually.
>>> I think we're mirroring git://git.ipxe.org/ipxe.git, so the ipxe
>>> changes need to be merged into that repo first.
>>>
>>> Gerd, have you submitted those to the ipxe upstream?
>>
>> Multiple times in the last six months.
> 
> So is the proposal here that we ship a non-upstream IPXE?
> That was less than entirely clear from the pull request...

Well, it did say "This pull finally fixes the efi boot support.  ipxe is
updated to the latest master, two non-upstream commits needed to make
efi work are added on top, and the build process is tweaked a bit".

Paolo

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Peter Maydell]

On 21 July 2015 at 15:09, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 21/07/2015 15:57, Peter Maydell wrote:
>> On 21 July 2015 at 14:47, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> So is the proposal here that we ship a non-upstream IPXE?
>> That was less than entirely clear from the pull request...
>
> Well, it did say "This pull finally fixes the efi boot support.  ipxe is
> updated to the latest master, two non-upstream commits needed to make
> efi work are added on top, and the build process is tweaked a bit".

Right, but if you want us to switch from "we just mirror
upstream ipxe" to "we have our own ipxe" then it's probably
better to start with that, rather than by submitting a pullreq
that can't be applied until we switch our ipxe workflow.

Do we really need this for 2.4, or can we wait and sort
this out afterwards? It feels a bit late in the release
cycle to start doing this kind of thing to me.

-- PMM

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Paolo Bonzini]

On 21/07/2015 16:25, Peter Maydell wrote:
>> >
>> > Well, it did say "This pull finally fixes the efi boot support.  ipxe is
>> > updated to the latest master, two non-upstream commits needed to make
>> > efi work are added on top, and the build process is tweaked a bit".
> Right, but if you want us to switch from "we just mirror
> upstream ipxe" to "we have our own ipxe" then it's probably
> better to start with that, rather than by submitting a pullreq
> that can't be applied until we switch our ipxe workflow.
> 
> Do we really need this for 2.4, or can we wait and sort
> this out afterwards? It feels a bit late in the release
> cycle to start doing this kind of thing to me.

Well, it is a bugfix.  shim.efi doesn't work with upstream iPXE, and
hence doesn't work with the ROMs currently distributed by QEMU (Fedora
applies the patches already).

The patches have missed 2.3 and 2.4 because Gerd has been sending them
again upstream every month.

That said, I see your point.  It's probably not of utmost importance as
long as OVMF remains non-free and hence not shipped by most distributions.

Paolo

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Stefan Hajnoczi]

On Tue, Jul 21, 2015 at 3:28 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 21/07/2015 16:25, Peter Maydell wrote:
>>> >
>>> > Well, it did say "This pull finally fixes the efi boot support.  ipxe is
>>> > updated to the latest master, two non-upstream commits needed to make
>>> > efi work are added on top, and the build process is tweaked a bit".
>> Right, but if you want us to switch from "we just mirror
>> upstream ipxe" to "we have our own ipxe" then it's probably
>> better to start with that, rather than by submitting a pullreq
>> that can't be applied until we switch our ipxe workflow.
>>
>> Do we really need this for 2.4, or can we wait and sort
>> this out afterwards? It feels a bit late in the release
>> cycle to start doing this kind of thing to me.
>
> Well, it is a bugfix.  shim.efi doesn't work with upstream iPXE, and
> hence doesn't work with the ROMs currently distributed by QEMU (Fedora
> applies the patches already).
>
> The patches have missed 2.3 and 2.4 because Gerd has been sending them
> again upstream every month.
>
> That said, I see your point.  It's probably not of utmost importance as
> long as OVMF remains non-free and hence not shipped by most distributions.

With regards to git.qemu.org mirroring:
I could update the ipxe.git mirror URL on git.qemu.org to Gerd's
public repo and change the description to indicate that this includes
out-of-tree patches.  Please let me know if you'd like to go ahead.

I'd prefer it if we don't ship a patched ipxe since Paolo mentions
Fedora already has a fix in place.  Instead of propagating that fix
into QEMU, let's focus on ipxe upstream to solve the problem.

I've reviewed the ipxe-devel archives and see that patches have been
on the list for weeks/months without replies.  I didn't see ping
emails though so maybe it just requires a bit of poking via email or
IRC.

We either need to figure out how to get attention upstream or work
with others to add upstream maintainers.  I see that Hannes Reinecke
also has patches on ipxe-devel that look ignored, so Gred and Laszlo
are not the only ones struggling to get patches upstream into ipxe.

Stefan

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Peter Maydell]

On 21 July 2015 at 17:10, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> With regards to git.qemu.org mirroring:
> I could update the ipxe.git mirror URL on git.qemu.org to Gerd's
> public repo and change the description to indicate that this includes
> out-of-tree patches.  Please let me know if you'd like to go ahead.
>
> I'd prefer it if we don't ship a patched ipxe since Paolo mentions
> Fedora already has a fix in place.  Instead of propagating that fix
> into QEMU, let's focus on ipxe upstream to solve the problem.

I think this would be my preference too. I wouldn't absolutely
rule out shipping a patched ipxe, but I think it should be a
last resort and not something we do at this point in our
release cycle.

thanks
-- PMM

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Laszlo Ersek]

On 07/21/15 18:18, Peter Maydell wrote:
> On 21 July 2015 at 17:10, Stefan Hajnoczi <stefanha@gmail.com> wrote:
>> With regards to git.qemu.org mirroring:
>> I could update the ipxe.git mirror URL on git.qemu.org to Gerd's
>> public repo and change the description to indicate that this includes
>> out-of-tree patches.  Please let me know if you'd like to go ahead.
>>
>> I'd prefer it if we don't ship a patched ipxe since Paolo mentions
>> Fedora already has a fix in place.  Instead of propagating that fix
>> into QEMU, let's focus on ipxe upstream to solve the problem.
> 
> I think this would be my preference too. I wouldn't absolutely
> rule out shipping a patched ipxe, but I think it should be a
> last resort

Yes, it should be. And, we're already there.

> and not something we do at this point in our
> release cycle.

Not contesting that.

Laszlo

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Laszlo Ersek]

On 07/21/15 18:10, Stefan Hajnoczi wrote:
> On Tue, Jul 21, 2015 at 3:28 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> On 21/07/2015 16:25, Peter Maydell wrote:
>>>>>
>>>>> Well, it did say "This pull finally fixes the efi boot support.  ipxe is
>>>>> updated to the latest master, two non-upstream commits needed to make
>>>>> efi work are added on top, and the build process is tweaked a bit".
>>> Right, but if you want us to switch from "we just mirror
>>> upstream ipxe" to "we have our own ipxe" then it's probably
>>> better to start with that, rather than by submitting a pullreq
>>> that can't be applied until we switch our ipxe workflow.
>>>
>>> Do we really need this for 2.4, or can we wait and sort
>>> this out afterwards? It feels a bit late in the release
>>> cycle to start doing this kind of thing to me.
>>
>> Well, it is a bugfix.  shim.efi doesn't work with upstream iPXE, and
>> hence doesn't work with the ROMs currently distributed by QEMU (Fedora
>> applies the patches already).
>>
>> The patches have missed 2.3 and 2.4 because Gerd has been sending them
>> again upstream every month.
>>
>> That said, I see your point.  It's probably not of utmost importance as
>> long as OVMF remains non-free and hence not shipped by most distributions.
> 
> With regards to git.qemu.org mirroring:
> I could update the ipxe.git mirror URL on git.qemu.org to Gerd's
> public repo and change the description to indicate that this includes
> out-of-tree patches.  Please let me know if you'd like to go ahead.
> 
> I'd prefer it if we don't ship a patched ipxe since Paolo mentions
> Fedora already has a fix in place.

RHEL doesn't (yet). In fact these QEMU patches are the result of a long
(and mostly fruitless) effort to get the ipxe patches upstream -- for
RHEL we prefer *something* to point at as "upstream".

> Instead of propagating that fix
> into QEMU, let's focus on ipxe upstream to solve the problem.

How's this for focus:

(1) [PATCH] efi_snp: improve compliance with the
            EFI_SIMPLE_NETWORK_PROTOCOL spec
    http://thread.gmane.org/gmane.network.ipxe.devel/3799
    Date: 2015-Jan-27
    feedback: none

(2) [PATCH] efi_snp: improve compliance with the
            EFI_SIMPLE_NETWORK_PROTOCOL spec
    http://thread.gmane.org/gmane.network.ipxe.devel/3955
    Date: 2015-Feb-10
    feedback: zero

(3) [PATCH] [efi] make load file protocol optional
    http://thread.gmane.org/gmane.network.ipxe.devel/3815
    Date: 2015-Feb-11
    feedback: the patch destroys the user's ability to use
              most features of iPXE
    our point: we don't care about most features of iPXE, we just need
               it for EFI drivers (Simple Network Protocol instances)
    result: nothing

(4) [RESENT PATCH 0/2] efi boot fixes.
    http://thread.gmane.org/gmane.network.ipxe.devel/3854
    Date: 2015-Mar-10
    feedback: zilch

(5) [RESEND PATCH] efi_snp: improve compliance with the
                   EFI_SIMPLE_NETWORK_PROTOCOL spec
    http://thread.gmane.org/gmane.network.ipxe.devel/3934
    Date: 2015-Apr-14
    feedback: nada

(6) [PATCH] efi_snp: improve compliance with the
            EFI_SIMPLE_NETWORK_PROTOCOL spec
    http://thread.gmane.org/gmane.network.ipxe.devel/4096
    Date: 2015-Jun-10
    feedback: null

So, let us *not* focus on getting these into upstream ipxe. The mailing
list is a black hole. The upstream maintainer behaves completely
unpredictably. For example, look at this one example:

  [PATCH 0/2] virtio-net shutdown fix for ExitBootServices()
  http://thread.gmane.org/gmane.network.ipxe.devel/3918
  Date: 2015-Apr-10

For this submission, the maintainer provided good feedback, then decided
to rewrite the patches (which was a good call, no doubt about it), then
went ahead and committed that patch, and even gave credit:

commit 755d2b8f6be681a2e620534b237471b75f28ed8c
Author: Michael Brown <mcb30@ipxe.org>
Date:   Mon Apr 13 12:06:59 2015 +0100

    [efi] Ensure drivers are disconnected when ExitBootServices() is
    called
    [...]
    Originally-fixed-by: Laszlo Ersek <lersek@redhat.com>
    Tested-by: Laszlo Ersek <lersek@redhat.com>
    Signed-off-by: Michael Brown <mcb30@ipxe.org>

(Therefore this patch is actually covered by the upstream rebase in
[PULL 1/7] here.)

And then, *just one day* after this commit, Gerd submits (5) -- and
complete silence.

> I've reviewed the ipxe-devel archives and see that patches have been
> on the list for weeks/months without replies.  I didn't see ping
> emails though so maybe it just requires a bit of poking via email or
> IRC.

Rather than pings, Gerd kept resending the actual patches. That's a
strictly stronger kind of "reminder". So no, pings are useless.

Regarding IRC, let me quote the following from freenode | #ipxe, date
2015-Jan-23 (ie. four days before posting (1)). The first quote is about
the patch in (3) -- the implementation looked differently at that point,
but the behavior was identical:

<mcb30>  Patches to modify our LOAD_FILE_PROTOCOL to support loading
         arbitrary files would be fine
<lersek> mcb30, thanks. :)
<mcb30>  but there's no way I'm turning UEFI iPXE back into being a dumb
         SNP driver with the appalling UEFI UX
<mcb30>  Consider what happens when a user attempts normal network boot
         and something goes wrong (e.g. the file doesn't exist)
<mcb30>  iPXE displays a UI with a meaningful error message and a shell
         allowing you to troubleshoot
<lersek> the UEFI boot option fails and the boot option processing
         continues.
<mcb30>  UEFI simply displays a dot on an otherwise blank screen and
         then returns to the boot menu
<lersek> yes
<lersek> if there are no more UEFI boot options to try
<lersek> I understand your point fully
<mcb30>  A dot on a blank screen is not a meaningful error message
<lersek> we have different goals here
<mcb30>  Understood
<lersek> the dot is not meant as an error message, it's progress info
         AFAIR
<mcb30>  In which case, a total absence of an error message is not a
         meaningful error message
<lersek> correct
<lersek> but it's consistent with the handling of other (not necessarily
         network related) boot options when they fail
<lersek> your main goal is to provide a nice iPXE experience, while we
         need some UEFI NIC drivers (for OVMF) in qemu-kvm guests
<lersek> so I thought I'd try to check with upstream developers before
         keeping these patches downstream only
<lersek> thanks!
<mcb30>  I would suggest adding code to support downloading arbitrary
         files via LOAD_FILE_PROTOCOL
<mcb30>  which would avoid a UX downgrade when network-booting a
         qemu-kvm guest
<mcb30>  (i.e. when not going through grub)

So, this is consistent with the feedback we'd actually get later for
(3). It makes no sense to struggle more with this patch.

Then, regarding the other patch (the one in (1)):

<lersek> mcb30, what about the second patch?
<mcb30>  lersek: sorry; I mssed the second patch
<mcb30>  Looking...
<lersek> mcb30, thanks -- my first msg in this channel was quite long &
         crowded
<mcb30>  lersek: looks okish.  I need to check the code again; it's been
         a long time since I looked at that.  In particular, is there
         any functional difference between the existing
         tx_count_interrupts (used I think to determine how many TX
         completions to return) and the producer/consumer counters that
         you have added?  If tx_count_interrupts is now redundant
         because the producer/consumer already include that information
         then it should probably be
<mcb30>  removed
<lersek> mcb30, that's a good question. The UEFI spec emphasizes that
         clearing / retrieving a recycled txbuf does not clear interrupt
         status if the caller doesn't ask for that too, and vice versa
         -- if the caller only asks about interrupt status, then no
         recycled txbuf is returned
<lersek> so they should remain independent
<lersek> but how exactly you calculate the interrupt status when the
         caller *does* ask for it, that's anyone's best guess
<lersek> the spec is unclear about it
<lersek> in OVMF's builtin virtio-net driver I think I just check if
         there are any pending outgoing & incoming packets, and set the
         corresponding bits
<lersek> I don't use a counter
<lersek> but it's very obscure indeed and no caller I know of seems to
         care about interrupt status
<lersek> which is why I didn't touch that part of the code
<lersek> I just kept those two things independent, because their
         independence *is* specified
<mcb30>  ok
<mcb30>  That makes sense
<lersek> (more precisely, re OVMF's builtin virtio-net driver, the RX
         intr bit is reported when more stuff is available for
         reception, and the TX intr bit is reported if we have
         transmitted at least one buffer that the caller queued with
         Transmit)
<lersek> mcb30, so can you pick up the 2nd patch from bugzilla, or
         should I send it to the list? (Actually if you deem the patch
         appropriate I can only send it to the list, because current
         master iPXE looks a bit different from our fork in RHEL, and
         the 2nd patch takes different forms accordingly.)

I got no further replies here. Based on the apparently positive feedback
on IRC, four days later I posted (1). No feedback.

> We either need to figure out how to get attention upstream

Good luck with that. In my educated experience: completely unpredictable
maintainer behavior, not a real community project.

> or work
> with others to add upstream maintainers.

When we can't get the maintainer's attention for our patches, and when
the maintainer tends to rewrite even those patches he more or less
likes, how do you propose we convince him to give *push access* to
random people?

> I see that Hannes Reinecke
> also has patches on ipxe-devel that look ignored, so Gred and Laszlo
> are not the only ones struggling to get patches upstream into ipxe.

I've said it several times (on other lists too), and I'll say it again:
ipxe is not an "open process" community project at this point. The last
half year, as Paolo indicated, and as I proved above, has been ample
experience.

--*--

On the technical level, let me summarize: we needed three patches in
total to get UEFI boot working:

#1 efi_snp: improve compliance with the EFI_SIMPLE_NETWORK_PROTOCOL spec
#2 [efi] make load file protocol optional
#3 [efi] Ensure drivers are disconnected when ExitBootServices() is
   called

Wrt. #1, the maintainer expressed agreement on IRC, but never replied to
patch emails.

Wrt. #2, the maintainer expressed strong disagreement (due to "user
interface" concerns) on both IRC and later on the mailing list.
Therefore the idea of upstreaming this patch is dead in the water. The
maintainer would accept an alternative that would take a huge
development effort, and would be useless for virtualization purposes
(ie. PXE-booting with OVMF in QEMU).

Wrt. #3, the maintainer decided to look at the patch, rewrite it, and
commit it. For some unfathomable reason. Maybe because he was Cc'd
directly on the patch email. I don't know. (The ipxe-devel list has
absolutely minimal traffic, why wouldn't he read it without explicit Cc?!)

This PULL is about getting #3 via rebase, and #1 and #2 as downstream
patches.

Thanks
Laszlo

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Laszlo Ersek]

Oops, I got carried away a little bit, and forgot about a crucial detail:

On 07/22/15 00:58, Laszlo Ersek wrote:

[snip]

> (1) [PATCH] efi_snp: improve compliance with the
>             EFI_SIMPLE_NETWORK_PROTOCOL spec
>     http://thread.gmane.org/gmane.network.ipxe.devel/3799
>     Date: 2015-Jan-27
>     feedback: none
> 
> (2) [PATCH] efi_snp: improve compliance with the
>             EFI_SIMPLE_NETWORK_PROTOCOL spec
>     http://thread.gmane.org/gmane.network.ipxe.devel/3955
>     Date: 2015-Feb-10
>     feedback: zero
> 
> (3) [PATCH] [efi] make load file protocol optional
>     http://thread.gmane.org/gmane.network.ipxe.devel/3815
>     Date: 2015-Feb-11
>     feedback: the patch destroys the user's ability to use
>               most features of iPXE
>     our point: we don't care about most features of iPXE, we just need
>                it for EFI drivers (Simple Network Protocol instances)
>     result: nothing

Mark this ^^^

> 
> (4) [RESENT PATCH 0/2] efi boot fixes.
>     http://thread.gmane.org/gmane.network.ipxe.devel/3854
>     Date: 2015-Mar-10
>     feedback: zilch
> 
> (5) [RESEND PATCH] efi_snp: improve compliance with the
>                    EFI_SIMPLE_NETWORK_PROTOCOL spec
>     http://thread.gmane.org/gmane.network.ipxe.devel/3934
>     Date: 2015-Apr-14
>     feedback: nada
> 
> (6) [PATCH] efi_snp: improve compliance with the
>             EFI_SIMPLE_NETWORK_PROTOCOL spec
>     http://thread.gmane.org/gmane.network.ipxe.devel/4096
>     Date: 2015-Jun-10
>     feedback: null

[snip]

> On the technical level, let me summarize: we needed three patches in
> total to get UEFI boot working:
> 
> #1 efi_snp: improve compliance with the EFI_SIMPLE_NETWORK_PROTOCOL spec
> #2 [efi] make load file protocol optional
> #3 [efi] Ensure drivers are disconnected when ExitBootServices() is
>    called
> 
> Wrt. #1, the maintainer expressed agreement on IRC, but never replied to
> patch emails.
> 
> Wrt. #2, the maintainer expressed strong disagreement (due to "user
> interface" concerns) on both IRC and later on the mailing list.
> Therefore the idea of upstreaming this patch is dead in the water. The
> maintainer would accept an alternative that would take a huge
> development effort, and would be useless for virtualization purposes
> (ie. PXE-booting with OVMF in QEMU).

So, the important detail that I forgot is that Gerd's patch, listed as
#2 here, and as (3) above in the list of earlier submissions, actually
does not change the behavior of ipxe *at all*. It just introduces a
config option, a knob if you like, that allows downstreams to rebuild
ipxe *that specific way* easily.

Therefore, absolutely no user interface concerns are valid in this case
-- those concerns may have been correct for my very first patch, but
Gerd reworked the patch to depend on a new config option (with unchanged
default behavior), and the maintainer refused to take even that.
(Despite the fact that upstream iPXE users would see no change in behavior.)

Laszlo

> 
> Wrt. #3, the maintainer decided to look at the patch, rewrite it, and
> commit it. For some unfathomable reason. Maybe because he was Cc'd
> directly on the patch email. I don't know. (The ipxe-devel list has
> absolutely minimal traffic, why wouldn't he read it without explicit Cc?!)
> 
> This PULL is about getting #3 via rebase, and #1 and #2 as downstream
> patches.
> 
> Thanks
> Laszlo
> 

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1588 bytes --]

On Wed, Jul 22, 2015 at 12:58:59AM +0200, Laszlo Ersek wrote:
> On 07/21/15 18:10, Stefan Hajnoczi wrote:
> > On Tue, Jul 21, 2015 at 3:28 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >> On 21/07/2015 16:25, Peter Maydell wrote:
> > or work
> > with others to add upstream maintainers.
> 
> When we can't get the maintainer's attention for our patches, and when
> the maintainer tends to rewrite even those patches he more or less
> likes, how do you propose we convince him to give *push access* to
> random people?
> 
> > I see that Hannes Reinecke
> > also has patches on ipxe-devel that look ignored, so Gred and Laszlo
> > are not the only ones struggling to get patches upstream into ipxe.
> 
> I've said it several times (on other lists too), and I'll say it again:
> ipxe is not an "open process" community project at this point. The last
> half year, as Paolo indicated, and as I proved above, has been ample
> experience.

I understand the frustration with upstream.  Thanks for posting a
summary of stranded patch series, it helped explain that.

The reason I'm suggesting reaching out to Michael Brown is that the
downstream repo will only be an "open process" for us virtualization
developers.  It won't have a user community, support, or help improve
the situation for non-virtualization developers - all things which
matter for a healthy long-term open source project.

It may be simplest if Gerd maintains a QEMU downstream repository.  I'm
not against that.  But let's notify Michael Brown so he has a chance to
consider the problem.

[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Laszlo Ersek]

On 07/22/15 11:05, Stefan Hajnoczi wrote:
> On Wed, Jul 22, 2015 at 12:58:59AM +0200, Laszlo Ersek wrote:
>> On 07/21/15 18:10, Stefan Hajnoczi wrote:
>>> On Tue, Jul 21, 2015 at 3:28 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>> On 21/07/2015 16:25, Peter Maydell wrote:
>>> or work
>>> with others to add upstream maintainers.
>>
>> When we can't get the maintainer's attention for our patches, and when
>> the maintainer tends to rewrite even those patches he more or less
>> likes, how do you propose we convince him to give *push access* to
>> random people?
>>
>>> I see that Hannes Reinecke
>>> also has patches on ipxe-devel that look ignored, so Gred and Laszlo
>>> are not the only ones struggling to get patches upstream into ipxe.
>>
>> I've said it several times (on other lists too), and I'll say it again:
>> ipxe is not an "open process" community project at this point. The last
>> half year, as Paolo indicated, and as I proved above, has been ample
>> experience.
> 
> I understand the frustration with upstream.  Thanks for posting a
> summary of stranded patch series, it helped explain that.
> 
> The reason I'm suggesting reaching out to Michael Brown is that the
> downstream repo will only be an "open process" for us virtualization
> developers.  It won't have a user community, support, or help improve
> the situation for non-virtualization developers - all things which
> matter for a healthy long-term open source project.

All the things upstream ipxe has been lacking for at least half a year
now, without much indication that it could improve.

> It may be simplest if Gerd maintains a QEMU downstream repository.  I'm
> not against that.  But let's notify Michael Brown so he has a chance to
> consider the problem.

If you can reach out to Michael Brown, that would be highly appreciated.
Personally I lost all hope.

Thanks
Laszlo

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Stefan Hajnoczi]

On Wed, Jul 22, 2015 at 11:05 AM, Laszlo Ersek <lersek@redhat.com> wrote:
> On 07/22/15 11:05, Stefan Hajnoczi wrote:
>> On Wed, Jul 22, 2015 at 12:58:59AM +0200, Laszlo Ersek wrote:
>>> On 07/21/15 18:10, Stefan Hajnoczi wrote:
>>>> On Tue, Jul 21, 2015 at 3:28 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>>> On 21/07/2015 16:25, Peter Maydell wrote:
>>>> or work
>>>> with others to add upstream maintainers.
>>>
>>> When we can't get the maintainer's attention for our patches, and when
>>> the maintainer tends to rewrite even those patches he more or less
>>> likes, how do you propose we convince him to give *push access* to
>>> random people?
>>>
>>>> I see that Hannes Reinecke
>>>> also has patches on ipxe-devel that look ignored, so Gred and Laszlo
>>>> are not the only ones struggling to get patches upstream into ipxe.
>>>
>>> I've said it several times (on other lists too), and I'll say it again:
>>> ipxe is not an "open process" community project at this point. The last
>>> half year, as Paolo indicated, and as I proved above, has been ample
>>> experience.
>>
>> I understand the frustration with upstream.  Thanks for posting a
>> summary of stranded patch series, it helped explain that.
>>
>> The reason I'm suggesting reaching out to Michael Brown is that the
>> downstream repo will only be an "open process" for us virtualization
>> developers.  It won't have a user community, support, or help improve
>> the situation for non-virtualization developers - all things which
>> matter for a healthy long-term open source project.
>
> All the things upstream ipxe has been lacking for at least half a year
> now, without much indication that it could improve.
>
>> It may be simplest if Gerd maintains a QEMU downstream repository.  I'm
>> not against that.  But let's notify Michael Brown so he has a chance to
>> consider the problem.
>
> If you can reach out to Michael Brown, that would be highly appreciated.
> Personally I lost all hope.

Done.

Stefan

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Laszlo Ersek]

On 07/22/15 13:42, Stefan Hajnoczi wrote:
> On Wed, Jul 22, 2015 at 11:05 AM, Laszlo Ersek <lersek@redhat.com> wrote:
>> On 07/22/15 11:05, Stefan Hajnoczi wrote:
>>> On Wed, Jul 22, 2015 at 12:58:59AM +0200, Laszlo Ersek wrote:
>>>> On 07/21/15 18:10, Stefan Hajnoczi wrote:
>>>>> On Tue, Jul 21, 2015 at 3:28 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>>>> On 21/07/2015 16:25, Peter Maydell wrote:
>>>>> or work
>>>>> with others to add upstream maintainers.
>>>>
>>>> When we can't get the maintainer's attention for our patches, and when
>>>> the maintainer tends to rewrite even those patches he more or less
>>>> likes, how do you propose we convince him to give *push access* to
>>>> random people?
>>>>
>>>>> I see that Hannes Reinecke
>>>>> also has patches on ipxe-devel that look ignored, so Gred and Laszlo
>>>>> are not the only ones struggling to get patches upstream into ipxe.
>>>>
>>>> I've said it several times (on other lists too), and I'll say it again:
>>>> ipxe is not an "open process" community project at this point. The last
>>>> half year, as Paolo indicated, and as I proved above, has been ample
>>>> experience.
>>>
>>> I understand the frustration with upstream.  Thanks for posting a
>>> summary of stranded patch series, it helped explain that.
>>>
>>> The reason I'm suggesting reaching out to Michael Brown is that the
>>> downstream repo will only be an "open process" for us virtualization
>>> developers.  It won't have a user community, support, or help improve
>>> the situation for non-virtualization developers - all things which
>>> matter for a healthy long-term open source project.
>>
>> All the things upstream ipxe has been lacking for at least half a year
>> now, without much indication that it could improve.
>>
>>> It may be simplest if Gerd maintains a QEMU downstream repository.  I'm
>>> not against that.  But let's notify Michael Brown so he has a chance to
>>> consider the problem.
>>
>> If you can reach out to Michael Brown, that would be highly appreciated.
>> Personally I lost all hope.
> 
> Done.

Thanks. Looks like you got through. Obviously, that cannot be ascribed
to anything else than blind luck, or a personal relationship from the
past. Ie. things that should not matter in open source.

In any case, I asked Michael Brown what we should do differently next time.

Laszlo

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Stefan Hajnoczi]

On Wed, Jul 22, 2015 at 9:08 PM, Laszlo Ersek <lersek@redhat.com> wrote:
> On 07/22/15 13:42, Stefan Hajnoczi wrote:
>> On Wed, Jul 22, 2015 at 11:05 AM, Laszlo Ersek <lersek@redhat.com> wrote:
>>> On 07/22/15 11:05, Stefan Hajnoczi wrote:
>>>> On Wed, Jul 22, 2015 at 12:58:59AM +0200, Laszlo Ersek wrote:
>>>>> On 07/21/15 18:10, Stefan Hajnoczi wrote:
>>>>>> On Tue, Jul 21, 2015 at 3:28 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>>>>> On 21/07/2015 16:25, Peter Maydell wrote:
>>>>>> or work
>>>>>> with others to add upstream maintainers.
>>>>>
>>>>> When we can't get the maintainer's attention for our patches, and when
>>>>> the maintainer tends to rewrite even those patches he more or less
>>>>> likes, how do you propose we convince him to give *push access* to
>>>>> random people?
>>>>>
>>>>>> I see that Hannes Reinecke
>>>>>> also has patches on ipxe-devel that look ignored, so Gred and Laszlo
>>>>>> are not the only ones struggling to get patches upstream into ipxe.
>>>>>
>>>>> I've said it several times (on other lists too), and I'll say it again:
>>>>> ipxe is not an "open process" community project at this point. The last
>>>>> half year, as Paolo indicated, and as I proved above, has been ample
>>>>> experience.
>>>>
>>>> I understand the frustration with upstream.  Thanks for posting a
>>>> summary of stranded patch series, it helped explain that.
>>>>
>>>> The reason I'm suggesting reaching out to Michael Brown is that the
>>>> downstream repo will only be an "open process" for us virtualization
>>>> developers.  It won't have a user community, support, or help improve
>>>> the situation for non-virtualization developers - all things which
>>>> matter for a healthy long-term open source project.
>>>
>>> All the things upstream ipxe has been lacking for at least half a year
>>> now, without much indication that it could improve.
>>>
>>>> It may be simplest if Gerd maintains a QEMU downstream repository.  I'm
>>>> not against that.  But let's notify Michael Brown so he has a chance to
>>>> consider the problem.
>>>
>>> If you can reach out to Michael Brown, that would be highly appreciated.
>>> Personally I lost all hope.
>>
>> Done.
>
> Thanks. Looks like you got through. Obviously, that cannot be ascribed
> to anything else than blind luck, or a personal relationship from the
> past. Ie. things that should not matter in open source.

Maybe a bit of both.

I sent a link to the email thread via IRC.

Stefan

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Michael Brown]

On 21/07/15 23:58, Laszlo Ersek wrote:
>> Instead of propagating that fix
>> into QEMU, let's focus on ipxe upstream to solve the problem.
>
> How's this for focus:
>
> (1) [PATCH] efi_snp: improve compliance with the
>              EFI_SIMPLE_NETWORK_PROTOCOL spec
>      http://thread.gmane.org/gmane.network.ipxe.devel/3799
>
> (2) [PATCH] efi_snp: improve compliance with the
>              EFI_SIMPLE_NETWORK_PROTOCOL spec
>      http://thread.gmane.org/gmane.network.ipxe.devel/3955
>
> (3) [PATCH] [efi] make load file protocol optional
>      http://thread.gmane.org/gmane.network.ipxe.devel/3815
>
> (4) [RESENT PATCH 0/2] efi boot fixes.
>      http://thread.gmane.org/gmane.network.ipxe.devel/3854
>
> (5) [RESEND PATCH] efi_snp: improve compliance with the
>                     EFI_SIMPLE_NETWORK_PROTOCOL spec
>      http://thread.gmane.org/gmane.network.ipxe.devel/3934
>
> (6) [PATCH] efi_snp: improve compliance with the
>              EFI_SIMPLE_NETWORK_PROTOCOL spec
>      http://thread.gmane.org/gmane.network.ipxe.devel/4096

Both (sic) of these patches now have upstream implementations:

   http://git.ipxe.org/ipxe.git/commitdiff/88a5f56
   http://git.ipxe.org/ipxe.git/commitdiff/a15c0d7

Unless I'm missing something, there are only two patches involved here.

I've also upstreamed a named configuration for qemu which incorporates 
your existing config changes (based on roms/config.ipxe.general.h) and 
adds the option to disable the EFI_LOAD_FILE_PROTOCOL installation:

   http://git.ipxe.org/ipxe.git/commitdiff/a200ad4

To build with this named configuration, just add "CONFIG=qemu" to the 
make command line when building iPXE.

Michael

Re: [Qemu-devel] [PULL for-2.4 0/7] update ipxe roms, fix efi support

[Laszlo Ersek]

On 07/22/15 22:06, Michael Brown wrote:
> On 21/07/15 23:58, Laszlo Ersek wrote:
>>> Instead of propagating that fix
>>> into QEMU, let's focus on ipxe upstream to solve the problem.
>>
>> How's this for focus:
>>
>> (1) [PATCH] efi_snp: improve compliance with the
>>              EFI_SIMPLE_NETWORK_PROTOCOL spec
>>      http://thread.gmane.org/gmane.network.ipxe.devel/3799
>>
>> (2) [PATCH] efi_snp: improve compliance with the
>>              EFI_SIMPLE_NETWORK_PROTOCOL spec
>>      http://thread.gmane.org/gmane.network.ipxe.devel/3955
>>
>> (3) [PATCH] [efi] make load file protocol optional
>>      http://thread.gmane.org/gmane.network.ipxe.devel/3815
>>
>> (4) [RESENT PATCH 0/2] efi boot fixes.
>>      http://thread.gmane.org/gmane.network.ipxe.devel/3854
>>
>> (5) [RESEND PATCH] efi_snp: improve compliance with the
>>                     EFI_SIMPLE_NETWORK_PROTOCOL spec
>>      http://thread.gmane.org/gmane.network.ipxe.devel/3934
>>
>> (6) [PATCH] efi_snp: improve compliance with the
>>              EFI_SIMPLE_NETWORK_PROTOCOL spec
>>      http://thread.gmane.org/gmane.network.ipxe.devel/4096
> 
> Both (sic) of these patches now have upstream implementations:
> 
>   http://git.ipxe.org/ipxe.git/commitdiff/88a5f56
>   http://git.ipxe.org/ipxe.git/commitdiff/a15c0d7

Thank you.

> Unless I'm missing something, there are only two patches involved here.

No, you are right, it's about two patches.

> I've also upstreamed a named configuration for qemu which incorporates
> your existing config changes (based on roms/config.ipxe.general.h) and
> adds the option to disable the EFI_LOAD_FILE_PROTOCOL installation:
> 
>   http://git.ipxe.org/ipxe.git/commitdiff/a200ad4
> 
> To build with this named configuration, just add "CONFIG=qemu" to the
> make command line when building iPXE.

Much appreciated. Hopefully Gerd can cover this when he returns from his
vacation.

Laszlo