From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
To: qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] [Bug 1769053] Re: Cannot start a guest with more than 1TB of RAM
Date: Tue, 08 May 2018 10:26:46 -0000 [thread overview]
Message-ID: <20180508102645.GG2500@work-vm> (raw)
In-Reply-To: CAATJJ0LEGg1AZbfDjfA_SvWonA5VO-xd5XtG4t4cDTRz4ZyR8g@mail.gmail.com
* ChristianEhrhardt (1769053@bugs.launchpad.net) wrote:
> On Tue, May 8, 2018 at 10:37 AM, Dr. David Alan Gilbert <dgilbert@redhat.com
> > wrote:
>
> > * ChristianEhrhardt (1769053@bugs.launchpad.net) wrote:
> > > > Interesting; I thought this was supposed to work.
> > >
> > > Exactly that was my thought when triaging it initially
> > > Furthermore I assume people working la57 (https://lwn.net/Articles/
> > 730925/) and such ran tests on much bigger sizes.
> >
> > I assume so, but I've not looked at the detail of that.
> >
> > > > Ah right Dan, if you're seeing the 40 bits physical in the guest you
> > > definitely need to try the flags I suggest in comment 6; host-phys-
> > > bits=true should work for you.
> > >
> > > I tested Bionic to be at least on libvirt 4.0 / qemu 2.11.1 when we want
> > > to check things under the "supposed to work now" flag.
> > >
> > > Defaults:
> > > Host: address sizes : 46 bits physical, 48 bits virtual
> > > Guest: address sizes : 40 bits physical, 48 bits virtual
> > >
> > > I ensured that with option -cpu host,host-phys-bits=true set I
> > successfully get what my host can provide in the guest:
> > > Guest: address sizes : 46 bits physical, 48 bits virtual
> > >
> > > Starting a guest with that >1TB (that would be mostly on swap if needed)
> > works just fine as expected. Here ~1063 GB from /proc/meminfo
> > > MemTotal: 1114676492 kB
> >
> > OK, good - that suggests there's nothing missing.
> > We enable host-phys-bits=true by default I think (in our machine type?)
> >
>
> Interesting approach, I see your comment about that already in [1] when it
> was added.
> I didn't realize some machine types were setting this already - I assume it
> isn't the general default for migratebility to other hosts (like our 36/39
> bit laptops).
>
> I assume "we" in this context are RedHat downstream changes to the (some)
> machine type(s)?
That's right; you sohuld be able to find them if you dig around CentOS's
set.
> I see the benefit for huge guests to work without setting those properties,
> but I wonder if that caused you trouble in regard to migrations?
It could, although I don't remember any reports of people hitting it.
The problem is finding a better solution; that's why I added both the
host-phys-bits and the ability to set phys-bits= so that you can make
a smarter choice based on what hardware you actually have. Who or what
should make that smarter choice hasn't really ever been answered.
> [1]: https://patchwork.kernel.org/patch/9223999/
Prior to that patch set, QEMU had always been a fixed 40 bits, so I
didn't change the default behaviour with that set; I just let you change
it by adding the flags.
(As I remember TCG was hard coded as 40 bits in some places so didn't
want to break that either).
Dave
>
>
> > > I also checked a more compatible approach like -cpu qemu64,phys-bits=42
> > > and that works as well.
> > >
> > > IMHO - if anything - one could argue that libvirt/qemu could be smarter
> > > about e.g. auto adding those arguments (or print a warning) when
> > > crossing a certain memory size.
> >
> > The problem is there are a whole bunch of things that are hard to deal
> > with:
> > a) Cheaper CPUs tend to have smaller phys-bits even in the same
> > generation; e.g. my laptop is still 36 bits, a lot are 39 bits. I think
> > the same is true of the Xeon E3-.... family. It makes it hard to know
> > what to pick when you're going to allow migration.
> >
> > b) Reasoning about the total address size range is difficult; you've
> > got to take into account PCI address space and hot plug space etc
> > to know where the upper edge is.
> >
>
> I agree that checking the total address size might have too much false
> positives for all the complexities around "estimating" that size.
> /me is giving up this idea :-)
>
>
> > Dave
> >
> > > So for now I'd stick to the "actually works" summary and keep the status
> > > to incomplete.
> > >
> > > --
> > > You received this bug notification because you are subscribed to the bug
> > > report.
> > > https://bugs.launchpad.net/bugs/1769053
> > >
> > > Title:
> > > Cannot start a guest with more than 1TB of RAM
> > >
> > > Status in QEMU:
> > > Incomplete
> > > Status in qemu package in Ubuntu:
> > > Incomplete
> > >
> > > Bug description:
> > > Attempting to start a KVM guest with more than 1TB of RAM fails.
> > >
> > > It looks like we might need some extra patches:
> > > https://lists.gnu.org/archive/html/qemu-discuss/2017-12/msg00005.html
> > >
> > > ProblemType: Bug
> > > DistroRelease: Ubuntu 18.04
> > > Package: qemu-system-x86 1:2.11+dfsg-1ubuntu7
> > > ProcVersionSignature: Ubuntu 4.15.0-20.21-generic 4.15.17
> > > Uname: Linux 4.15.0-20-generic x86_64
> > > ApportVersion: 2.20.9-0ubuntu7
> > > Architecture: amd64
> > > CurrentDesktop: Unity:Unity7:ubuntu
> > > Date: Fri May 4 16:21:14 2018
> > > InstallationDate: Installed on 2017-04-05 (393 days ago)
> > > InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64
> > (20161012.2)
> > > MachineType: Dell Inc. XPS 13 9360
> > > ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-20-generic
> > root=/dev/mapper/ubuntu--vg-root ro quiet splash
> > transparent_hugepage=madvise vt.handoff=1
> > > SourcePackage: qemu
> > > UpgradeStatus: Upgraded to bionic on 2018-04-30 (3 days ago)
> > > dmi.bios.date: 02/26/2018
> > > dmi.bios.vendor: Dell Inc.
> > > dmi.bios.version: 2.6.2
> > > dmi.board.name: 0PF86Y
> > > dmi.board.vendor: Dell Inc.
> > > dmi.board.version: A00
> > > dmi.chassis.type: 9
> > > dmi.chassis.vendor: Dell Inc.
> > > dmi.modalias: dmi:bvnDellInc.:bvr2.6.2:bd02/26/2018:svnDellInc.:
> > pnXPS139360:pvr:rvnDellInc.:rn0PF86Y:rvrA00:cvnDellInc.:ct9:cvr:
> > > dmi.product.family: XPS
> > > dmi.product.name: XPS 13 9360
> > > dmi.sys.vendor: Dell Inc.
> > >
> > > To manage notifications about this bug go to:
> > > https://bugs.launchpad.net/qemu/+bug/1769053/+subscriptions
> > --
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> >
> > --
> > You received this bug notification because you are a member of Ubuntu
> > Virtualisation team, which is subscribed to qemu in Ubuntu.
> > https://bugs.launchpad.net/bugs/1769053
> >
> > Title:
> > Cannot start a guest with more than 1TB of RAM
> >
> > Status in QEMU:
> > Incomplete
> > Status in qemu package in Ubuntu:
> > Incomplete
> >
> > Bug description:
> > Attempting to start a KVM guest with more than 1TB of RAM fails.
> >
> > It looks like we might need some extra patches:
> > https://lists.gnu.org/archive/html/qemu-discuss/2017-12/msg00005.html
> >
> > ProblemType: Bug
> > DistroRelease: Ubuntu 18.04
> > Package: qemu-system-x86 1:2.11+dfsg-1ubuntu7
> > ProcVersionSignature: Ubuntu 4.15.0-20.21-generic 4.15.17
> > Uname: Linux 4.15.0-20-generic x86_64
> > ApportVersion: 2.20.9-0ubuntu7
> > Architecture: amd64
> > CurrentDesktop: Unity:Unity7:ubuntu
> > Date: Fri May 4 16:21:14 2018
> > InstallationDate: Installed on 2017-04-05 (393 days ago)
> > InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64
> > (20161012.2)
> > MachineType: Dell Inc. XPS 13 9360
> > ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-20-generic
> > root=/dev/mapper/ubuntu--vg-root ro quiet splash
> > transparent_hugepage=madvise vt.handoff=1
> > SourcePackage: qemu
> > UpgradeStatus: Upgraded to bionic on 2018-04-30 (3 days ago)
> > dmi.bios.date: 02/26/2018
> > dmi.bios.vendor: Dell Inc.
> > dmi.bios.version: 2.6.2
> > dmi.board.name: 0PF86Y
> > dmi.board.vendor: Dell Inc.
> > dmi.board.version: A00
> > dmi.chassis.type: 9
> > dmi.chassis.vendor: Dell Inc.
> > dmi.modalias: dmi:bvnDellInc.:bvr2.6.2:bd02/26/2018:svnDellInc.:
> > pnXPS139360:pvr:rvnDellInc.:rn0PF86Y:rvrA00:cvnDellInc.:ct9:cvr:
> > dmi.product.family: XPS
> > dmi.product.name: XPS 13 9360
> > dmi.sys.vendor: Dell Inc.
> >
> > To manage notifications about this bug go to:
> > https://bugs.launchpad.net/qemu/+bug/1769053/+subscriptions
> >
>
>
> --
> Christian Ehrhardt
> Software Engineer, Ubuntu Server
> Canonical Ltd
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1769053
>
> Title:
> Cannot start a guest with more than 1TB of RAM
>
> Status in QEMU:
> Incomplete
> Status in qemu package in Ubuntu:
> Incomplete
>
> Bug description:
> Attempting to start a KVM guest with more than 1TB of RAM fails.
>
> It looks like we might need some extra patches:
> https://lists.gnu.org/archive/html/qemu-discuss/2017-12/msg00005.html
>
> ProblemType: Bug
> DistroRelease: Ubuntu 18.04
> Package: qemu-system-x86 1:2.11+dfsg-1ubuntu7
> ProcVersionSignature: Ubuntu 4.15.0-20.21-generic 4.15.17
> Uname: Linux 4.15.0-20-generic x86_64
> ApportVersion: 2.20.9-0ubuntu7
> Architecture: amd64
> CurrentDesktop: Unity:Unity7:ubuntu
> Date: Fri May 4 16:21:14 2018
> InstallationDate: Installed on 2017-04-05 (393 days ago)
> InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64 (20161012.2)
> MachineType: Dell Inc. XPS 13 9360
> ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-20-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash transparent_hugepage=madvise vt.handoff=1
> SourcePackage: qemu
> UpgradeStatus: Upgraded to bionic on 2018-04-30 (3 days ago)
> dmi.bios.date: 02/26/2018
> dmi.bios.vendor: Dell Inc.
> dmi.bios.version: 2.6.2
> dmi.board.name: 0PF86Y
> dmi.board.vendor: Dell Inc.
> dmi.board.version: A00
> dmi.chassis.type: 9
> dmi.chassis.vendor: Dell Inc.
> dmi.modalias: dmi:bvnDellInc.:bvr2.6.2:bd02/26/2018:svnDellInc.:pnXPS139360:pvr:rvnDellInc.:rn0PF86Y:rvrA00:cvnDellInc.:ct9:cvr:
> dmi.product.family: XPS
> dmi.product.name: XPS 13 9360
> dmi.sys.vendor: Dell Inc.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/qemu/+bug/1769053/+subscriptions
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1769053
Title:
Cannot start a guest with more than 1TB of RAM
Status in QEMU:
Incomplete
Status in qemu package in Ubuntu:
Incomplete
Bug description:
Attempting to start a KVM guest with more than 1TB of RAM fails.
It looks like we might need some extra patches:
https://lists.gnu.org/archive/html/qemu-discuss/2017-12/msg00005.html
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: qemu-system-x86 1:2.11+dfsg-1ubuntu7
ProcVersionSignature: Ubuntu 4.15.0-20.21-generic 4.15.17
Uname: Linux 4.15.0-20-generic x86_64
ApportVersion: 2.20.9-0ubuntu7
Architecture: amd64
CurrentDesktop: Unity:Unity7:ubuntu
Date: Fri May 4 16:21:14 2018
InstallationDate: Installed on 2017-04-05 (393 days ago)
InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64 (20161012.2)
MachineType: Dell Inc. XPS 13 9360
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-20-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash transparent_hugepage=madvise vt.handoff=1
SourcePackage: qemu
UpgradeStatus: Upgraded to bionic on 2018-04-30 (3 days ago)
dmi.bios.date: 02/26/2018
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.6.2
dmi.board.name: 0PF86Y
dmi.board.vendor: Dell Inc.
dmi.board.version: A00
dmi.chassis.type: 9
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvr2.6.2:bd02/26/2018:svnDellInc.:pnXPS139360:pvr:rvnDellInc.:rn0PF86Y:rvrA00:cvnDellInc.:ct9:cvr:
dmi.product.family: XPS
dmi.product.name: XPS 13 9360
dmi.sys.vendor: Dell Inc.
To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1769053/+subscriptions
next prev parent reply other threads:[~2018-05-08 10:41 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <152541524728.557.4600864098110042577.malonedeb@gac.canonical.com>
2018-05-04 6:46 ` [Qemu-devel] [Bug 1769053] Re: Cannot start a guest with more than 1TB of RAM Daniel Axtens
2018-05-04 7:44 ` ChristianEhrhardt
2018-05-04 8:04 ` Daniel Axtens
2018-05-04 8:12 ` ChristianEhrhardt
2018-05-04 16:10 ` Dr. David Alan Gilbert
2018-05-04 16:14 ` Dan Streetman
2018-05-04 16:15 ` Dan Streetman
2018-05-04 16:19 ` Dan Streetman
2018-05-04 16:33 ` Dr. David Alan Gilbert
2018-05-07 10:33 ` ChristianEhrhardt
2018-05-08 8:37 ` Dr. David Alan Gilbert
2018-05-08 9:57 ` ChristianEhrhardt
2018-05-08 10:26 ` Dr. David Alan Gilbert [this message]
2018-05-08 10:48 ` Daniel Berrange
2018-05-08 11:23 ` Dr. David Alan Gilbert
2018-05-08 13:22 ` Gerd Hoffmann
2018-05-08 14:00 ` Dr. David Alan Gilbert
2018-05-15 1:16 ` David Coronel
2018-05-15 7:17 ` Christian Ehrhardt
2018-05-15 7:19 ` Christian Ehrhardt
2018-05-15 7:45 ` Christian Ehrhardt
2018-06-11 9:30 ` Christian Ehrhardt
2018-06-12 7:02 ` [Qemu-devel] [Bug 1769053] Re: Ability to control phys-bits through libvirt Launchpad Bug Tracker
2018-06-12 7:04 ` Christian Ehrhardt
2018-06-12 7:18 ` Launchpad Bug Tracker
2018-06-12 7:18 ` Christian Ehrhardt
2018-06-12 7:46 ` Launchpad Bug Tracker
2018-06-12 8:27 ` Christian Ehrhardt
2018-06-12 11:14 ` Launchpad Bug Tracker
2019-08-28 2:55 ` [Bug 1769053] sirswa
2019-08-28 14:16 ` ehabkost
2020-10-29 15:30 ` raistlin
2023-04-19 5:43 ` [Bug 1769053] Re: Ability to control phys-bits through libvirt Christian Ehrhardt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180508102645.GG2500@work-vm \
--to=dgilbert@redhat.com \
--cc=1769053@bugs.launchpad.net \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).