Features and bug-fixes that went in Linux 3.2

Features and bug-fixes that went in Linux 3.2

[Konrad Rzeszutek Wilk]

Linux 3.2 was released on Jan 4th and this release saw a lot of 
interesting work. 

For the full credit list, please see the bottom of the email.

We did not have any new drivers in this release, but we did have some
awesome goodness:

 - Remove the XEN_PLATFORM_PCI config option. We need it 99% of time
   and not having it enabled caused tons of headaches. Squashing it in
   fixed a lot of distro bugs were the xen-platform-pci.ko module was not
   present in the installer.
 - Rework the E820 parser. It also for proper accounting of memory in
   the "low" (so below 4GB) and "high" (>4GB) memory that can be used for
   ballooning. In other words, what you see in 'xm list' vs 'xl list' is more
   truthfull.
 - Support "big-iron" machines with PCI segments (another layer in case you
   run out of PCI buses, so more than 256 devices).
 - PVonHVM kexec support. It all worked until we found out that under Amazon
   EC2 (and Xen 3.4.x) it would lock up the guest so had to revert the new
   functionality . Will revisit this for 3.4 with a 'feature-XX' flag.
 - MMU improvements by not using the 'vmalloc_sync_all' slow call but being
   more selective.
 - Initial work to make HVM domains be capable of being device drivers or
   even host the XenBus daemon.
 - Initial work laid out for netback page-flipping (also called zero-copying).
 - Block can pass in discard (TRIM/UNMAP) requests to the backends.
 - Block can support the old-style 'feature-barrier' that was removed in 2.6.36 kernels.
 - Tons of cleanup to make the code easier to read.
 - Security fixes in the gnt/gnt_alloc ioctl calls.
 - Work with more than 32 VCPUs.
 - Tons of bug-fixes in various layers. The most critical were back-ported to
   the stable kernels.


Dan Carpenter (4):
      xen/pciback: double lock typo
      xen-gntdev: integer overflow in gntdev_alloc_map()
      xen-gntalloc: integer overflow in gntalloc_ioctl_alloc()
      xen-gntalloc: signedness bug in add_grefs()

Dan Magenheimer (1):
      xen: Fix selfballooning and ensure it doesn't go too far

Daniel De Graaf (5):
      xenbus: Fix loopback event channel assuming domain 0
      xenbus: don't rely on xen_initial_domain to detect local xenstore
      xen/gntdev: Fix sleep-inside-spinlock
      xen: Remove hanging references to CONFIG_XEN_PLATFORM_PCI
      xen/balloon: Avoid OOM when requesting highmem

David Vrabel (9):
      xen/balloon: account for pages released during memory setup
      xen/balloon: simplify test for the end of usable RAM
      xen: allow balloon driver to use more than one memory region
      xen: allow extra memory to be in multiple regions
      xen: release all pages within 1-1 p2m mappings
      xen: use generic functions instead of xen_{alloc, free}_vm_area()
      block: xen-blkback: use API provided by xenbus module to map rings
      net: xen-netback: use API provided by xenbus module to map rings
      xen: map foreign pages for shared rings by updating the PTEs directly


Ian Campbell (12):
      atm: convert to SKB paged frag API.
      IB: amso1100: convert to SKB paged frag API.
      IB: nes: convert to SKB paged frag API.
      IPoIB: convert to SKB paged frag API.
      tg3: convert to SKB paged frag API.
      bnx2: convert to SKB paged frag API.
      bnx2x: convert to SKB paged frag API.
      bnx2fc: convert to SKB paged frag API.
      fcoe: convert to SKB paged frag API.
      xen: netfront: convert to SKB paged frag API.
      genirq: Add IRQF_RESUME_EARLY and resume such IRQs earlier
      xen: only limit memory map to maximum reservation for domain 0.


Jan Beulich (5):
      xen/pci: make bus notifier handler return sane values
      xen/pciback: use mutex rather than spinlock in passthrough backend
      xen/pciback: miscellaneous adjustments
      xen/pci: support multi-segment systems
      xen-blkback: use kzalloc() in favor of kmalloc()+memset()

Jeremy Fitzhardinge (1):
      stop_machine: make stop_machine safe and efficient to call early

Joe Jin (1):
      xen-blkback: fixed indentation and comments

Konrad Rzeszutek Wilk (30):
      Revert "xen/debug: WARN_ON when identity PFN has no _PAGE_IOMAP flag set."
      xen-pcifront: Update warning comment to use 'e820_host' option.
      xen-swiotlb: Retry up three times to allocate Xen-SWIOTLB
      xen-swiotlb: Fix wrong panic.
      xen-swiotlb: When doing coherent alloc/dealloc check before swizzling the MFNs.
      xen/pciback: Use mutexes when working with Xenbus state transitions.
      xen/pciback: use mutex rather than spinlock in vpci backend
      xen/p2m: Make debug/xen/mmu/p2m visible again.
      xen/p2m: Use SetPagePrivate and its friends for M2P overrides.
      xen/pv-on-hvm:kexec: Fix implicit declaration of function 'xen_hvm_domain'
      xen/pciback: Add flag indicating device has been assigned by Xen
      xen-blkfront: If no barrier or flush is supported, use invalid operation.
      xen/blkback: Support 'feature-barrier' aka old-style BARRIER requests.
      xen/blkback: Report VBD_WSECT (wr_sect) properly.
      xen/blkback: Fix the inhibition to map pages when discarding sector ranges.
      xen/blkback: Check for proper operation.
      xen/blkback: Fix two races in the handling of barrier requests.
      xen/pciback: Do not dereference psdev during printk when it is NULL.
      xen/pciback: Check if the device is found instead of blindly assuming so.
      xen/events: BUG() when we can't allocate our event->irq array.
      xen/events: Don't check the info for NULL as it is already done.
      xen/irq: If we fail during msi_capability_init return proper error code.
      xen/xenbus: Remove the unnecessary check.
      xen/enlighten: Fix compile warnings and set cx to known value.
      xen/p2m/debugfs: Fix potential pointer exception.
      xen/p2m/debugfs: Make type_name more obvious.
      xen/pm_idle: Make pm_idle be default_idle under Xen.
      x86/paravirt: PTE updates in k(un)map_atomic need to be synchronous, regardless of lazy_mmu mode
      xen/swiotlb: Use page alignment for early buffer allocation.
      Revert "xen/pv-on-hvm kexec: add xs_reset_watches to shutdown watches from old kernel"

Laszlo Ersek (1):
      xen-blkfront: plug device number leak in xlblk_init() error path

Li Dongyang (4):
      xen-blkfront: add BLKIF_OP_DISCARD and discard request struct
      xen-blkback: Implement discard requests ('feature-discard')
      xen-blkfront: Handle discard requests.
      xen-blkfront: fix a deadlock while handling discard response

Linus Torvalds (2):
      Revert "hvc_console: display printk messages on console."
      Revert "rtc: Expire alarms after the time is set."

Olaf Hering (6):
      xen: use static initializers in xen-balloon.c
      xen/pv-on-hvm kexec: prevent crash in xenwatch_thread() when stale watch events arrive
      xen/pv-on-hvm kexec: rebind virqs to existing eventchannel ports
      xen/pv-on-hvm kexec+kdump: reset PV devices in kexec or crash kernel
      xen/pv-on-hvm kexec: update xs_wire.h:xsd_sockmsg_type from xen-unstable
      xen/pv-on-hvm kexec: add xs_reset_watches to shutdown watches from old kernel

Randy Dunlap (1):
      xen-swiotlb: fix printk and panic args

Ruslan Pisarev (6):
      Xen: fix whitespaces,tabs coding style issue in drivers/xen/balloon.c
      Xen: fix whitespaces,tabs coding style issue in drivers/xen/events.c
      Xen: fix braces coding style issue in gntdev.c and grant-table.c
      Xen: fix whitespaces,tabs coding style issue in drivers/xen/pci.c
      Xen: fix braces coding style issue in xenbus_probe.h
      Xen: fix braces and tabs coding style issue in xenbus_probe.c

Stefano Stabellini (4):
      xen: add an "highmem" parameter to alloc_xenballooned_pages
      xen: modify kernel mappings corresponding to granted pages
      xen: XEN_PVHVM depends on PCI
      xen: remove XEN_PLATFORM_PCI config option

Thomas Meyer (1):
      xen/pciback: use resource_size()

Yu Ke (1):
      xen/acpi: Domain0 acpi parser related platform hypercall

Zhenzhong Duan (1):
      xen:pvhvm: enable PVHVM VCPU placement when using more than 32 CPUs.

Re: Features and bug-fixes that went in Linux 3.2

[James Harper]

>  - Initial work laid out for netback page-flipping (also called zero-copying).

Isn't this how it used to work originally?

James

Re: Features and bug-fixes that went in Linux 3.2

[Ian Campbell]

On Mon, 2012-01-09 at 22:26 +0000, James Harper wrote:
> >  - Initial work laid out for netback page-flipping (also called zero-copying).
> 
> Isn't this how it used to work originally?

Some of the original infrastructure for doing this was not upstreamable
(the PageForeign stuff) so while upstream netback I decided to go with a
simpler/less-intrusive copying mode so we could have some sort of
networking support in mainline. 

I've been working on re-laying the necessary infrastructure to allow for
page flipping/mapping mode in upstream (as well as fixing another
generic class of bug) -- you can see the "skb frag destructor" patches
on the netdev list.

Ian.

Re: Features and bug-fixes that went in Linux 3.2

[Fantu]

Thanks to all for good work, I tried Precise as domU pv (with kernel 3.2) and
I have found critical regression, probably there is a bug or unexpected
cases in some blk commit
I have already report on launchpad, this is the link:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/913760
Dom0 have Squeeze kernel build 2.6.32-35squeeze2, if you need more
information just ask I hope to be helpful.
Thanks and sorry for bad english.

--
View this message in context: http://xen.1045712.n5.nabble.com/Features-and-bug-fixes-that-went-in-Linux-3-2-tp5132575p5133661.html
Sent from the Xen - Dev mailing list archive at Nabble.com.

Re: Features and bug-fixes that went in Linux 3.2

[Laszlo Ersek]

On 01/09/12 23:37, Ian Campbell wrote:
> On Mon, 2012-01-09 at 22:26 +0000, James Harper wrote:
>>>   - Initial work laid out for netback page-flipping (also called zero-copying).
>>
>> Isn't this how it used to work originally?
>
> Some of the original infrastructure for doing this was not upstreamable
> (the PageForeign stuff) so while upstream netback I decided to go with a
> simpler/less-intrusive copying mode so we could have some sort of
> networking support in mainline.
>
> I've been working on re-laying the necessary infrastructure to allow for
> page flipping/mapping mode in upstream (as well as fixing another
> generic class of bug) -- you can see the "skb frag destructor" patches
> on the netdev list.

(Ultimately I found it here:

     http://lwn.net/Articles/474791/

.)

Ian, do you think the NFS fix in

     http://article.gmane.org/gmane.linux.nfs/45955

for problem

     http://marc.info/?l=linux-nfs&m=122424132729720&w=2

would be technically feasible to port to 2.6.18, based on the existing 
PageForeign stuff instead of parts 1-5 of the series?

Thanks
Laszlo

Re: Features and bug-fixes that went in Linux 3.2

[Ian Campbell]

On Thu, 2012-01-12 at 17:02 +0000, Laszlo Ersek wrote:
> On 01/09/12 23:37, Ian Campbell wrote:
> > On Mon, 2012-01-09 at 22:26 +0000, James Harper wrote:
> >>>   - Initial work laid out for netback page-flipping (also called zero-copying).
> >>
> >> Isn't this how it used to work originally?
> >
> > Some of the original infrastructure for doing this was not upstreamable
> > (the PageForeign stuff) so while upstream netback I decided to go with a
> > simpler/less-intrusive copying mode so we could have some sort of
> > networking support in mainline.
> >
> > I've been working on re-laying the necessary infrastructure to allow for
> > page flipping/mapping mode in upstream (as well as fixing another
> > generic class of bug) -- you can see the "skb frag destructor" patches
> > on the netdev list.
> 
> (Ultimately I found it here:
> 
>      http://lwn.net/Articles/474791/
> 
> .)


The most recent posting starts at: 
http://thread.gmane.org/gmane.linux.network/217006

> Ian, do you think the NFS fix in
> 
>      http://article.gmane.org/gmane.linux.nfs/45955
> 
> for problem
> 
>      http://marc.info/?l=linux-nfs&m=122424132729720&w=2
> 
> would be technically feasible to port to 2.6.18, based on the existing 
> PageForeign stuff instead of parts 1-5 of the series?

I don't think so -- PageForeign is triggered by the last core page
reference getting dropped, but in the NFS case the running process holds
at least one so you won't actually complete until the process exits...

This is similar to the sorts of issue I described in:
http://thread.gmane.org/gmane.linux.network/217006

Ian.