All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
To: Peter Maydell <peter.maydell@linaro.org>
Cc: Andrea Arcangeli <aarcange@redhat.com>,
	yamahata@private.email.ne.jp, Juan Quintela <quintela@redhat.com>,
	cristian.klein@cs.umu.se, QEMU Developers <qemu-devel@nongnu.org>,
	Amit Shah <amit.shah@redhat.com>,
	yanghy@cn.fujitsu.com
Subject: Re: [Qemu-devel] [PATCH v4 43/47] Host page!=target page: Cleanup bitmaps
Date: Tue, 27 Jan 2015 11:50:25 +0000	[thread overview]
Message-ID: <20150127115025.GC2425@work-vm> (raw)
In-Reply-To: <CAFEAcA_UA2LA9u3SEocF5P9NKO3tNexEk7N3j0rLKr7eru8e8Q@mail.gmail.com>

* Peter Maydell (peter.maydell@linaro.org) wrote:
> On 3 October 2014 at 18:47, Dr. David Alan Gilbert (git)
> <dgilbert@redhat.com> wrote:
> 
> Just noticed after writing most of this email that this is an
> old patch with new review comments. I'm sending the below
> anyway in case some of it is still valid...

Still useful; I'm just looking at doing a new post soon.

> >  /*
> > + * Utility for the outgoing postcopy code.
> > + *
> > + * Discard any partially sent host-page size chunks, mark any partially
> > + * dirty host-page size chunks as all dirty.
> > + *
> > + * Returns: 0 on success
> > + */
> > +static int postcopy_chunk_hostpages(MigrationState *ms)
> > +{
> > +    struct RAMBlock *block;
> > +    unsigned int host_bits = sysconf(_SC_PAGESIZE) / TARGET_PAGE_SIZE;
> 
> I'm guessing this won't build on Win32. Can you use getpagesize() ?
> We provide a compat wrapper for that in util/ as necessary.

I'd used sysconf since the manpage of getpagesize() says 'Portable
applications should employ sysconf(_SC_PAGESIZE) instead of getpagesize()'
and I can see there are a couple of other places in qemu that use the
same sysconf; however if it's not on Win32 then yes I'm happy to change
over.

> What happens if the TARGET_PAGE_SIZE is larger than the
> host page size? (If you want MIN(host page size, TARGET_PAGE_SIZE)
> try qemu_host_page_size, see page_size_init()).

Thanks; I hadn't realised that was possible - but yes I should probably
just use qemu_host_page_size  instead of my sysconf in most places.

What happens where the target wants to map a RAMBlock with Target-page-size
alignment?

> > +    uint32_t host_mask;
> > +
> > +    /* Should be a power of 2 */
> > +    assert(host_bits && !(host_bits & (host_bits - 1)));
> 
> assert(is_power_of_2(host_bits));

Thanks; fixed.

> > +    /*
> > +     * If the host_bits isn't a division of 32 (the minimum long size)
> > +     * then the code gets a lot more complex; disallow for now
> > +     * (I'm not aware of a system where it's true anyway)
> > +     */
> > +    assert((32 % host_bits) == 0);
> > +
> > +    /* A mask, starting at bit 0, containing host_bits continuous set bits */
> > +    host_mask =  (1u << host_bits) - 1;
> 
> If the host has 64K pages and the guest TARGET_PAGE_SIZE is 1K
> (eg ARM) then this will shift off the end of your uint32_t.

Gah! That's going to make things a lot hairier; OK, that's going to take
some rework, I'll have a think how.

Note, keep an eye out for the RAM_SAVE_FLAG definitions in arch_init,
they're one-bit-per-type of message (for no good reason) and with a
TPS of 1K there are only a couple spare.

Thanks for the comments,

Dave
> 
> thanks
> -- PMM
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

  reply	other threads:[~2015-01-27 11:51 UTC|newest]

Thread overview: 204+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-10-03 17:47 [Qemu-devel] [PATCH v4 00/47] Postcopy implementation Dr. David Alan Gilbert (git)
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 01/47] QEMUSizedBuffer based QEMUFile Dr. David Alan Gilbert (git)
2014-10-08  2:10   ` zhanghailiang
2014-11-03  0:53   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 02/47] Tests: QEMUSizedBuffer/QEMUBuffer Dr. David Alan Gilbert (git)
2014-11-03  1:02   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 03/47] Start documenting how postcopy works Dr. David Alan Gilbert (git)
2014-11-03  1:31   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 04/47] qemu_ram_foreach_block: pass up error value, and down the ramblock name Dr. David Alan Gilbert (git)
2014-11-03  2:34   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 05/47] improve DPRINTF macros, add to savevm Dr. David Alan Gilbert (git)
2014-11-03  2:35   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 06/47] Add qemu_get_counted_string to read a string prefixed by a count byte Dr. David Alan Gilbert (git)
2014-11-03  2:39   ` David Gibson
2014-11-25 16:13     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 07/47] Create MigrationIncomingState Dr. David Alan Gilbert (git)
2014-11-03  2:45   ` David Gibson
2014-11-04 19:06     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 08/47] socket shutdown Dr. David Alan Gilbert (git)
2014-10-04 18:09   ` Paolo Bonzini
2014-10-07 10:00     ` Dr. David Alan Gilbert
2014-10-07 11:10       ` Paolo Bonzini
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 09/47] Provide runtime Target page information Dr. David Alan Gilbert (git)
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 10/47] Return path: Open a return path on QEMUFile for sockets Dr. David Alan Gilbert (git)
2014-11-03  3:05   ` David Gibson
2014-11-03 19:04     ` Dr. David Alan Gilbert
2014-11-18  4:34       ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 11/47] Return path: socket_writev_buffer: Block even on non-blocking fd's Dr. David Alan Gilbert (git)
2014-11-03  3:10   ` David Gibson
2014-11-03 18:59     ` Dr. David Alan Gilbert
2014-11-18  3:54       ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 12/47] Handle bi-directional communication for fd migration Dr. David Alan Gilbert (git)
2014-11-03  3:12   ` David Gibson
2014-11-03 13:53     ` Cristian Klein
2014-11-18  3:53       ` David Gibson
2014-11-19 17:27         ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 13/47] Migration commands Dr. David Alan Gilbert (git)
2014-11-03  3:14   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 14/47] Return path: Control commands Dr. David Alan Gilbert (git)
2014-10-04 18:08   ` Paolo Bonzini
2014-10-23 16:23     ` Dr. David Alan Gilbert
2014-10-23 20:15       ` Paolo Bonzini
2014-11-03  3:20         ` David Gibson
2014-11-04 18:58         ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 15/47] Return path: Send responses from destination to source Dr. David Alan Gilbert (git)
2014-11-03  3:22   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 16/47] Return path: Source handling of return path Dr. David Alan Gilbert (git)
2014-10-04 18:14   ` Paolo Bonzini
2014-10-23 18:00     ` Dr. David Alan Gilbert
2014-10-24 10:04       ` Paolo Bonzini
2014-10-16  8:26   ` zhanghailiang
2014-10-16  8:35     ` Dr. David Alan Gilbert
2014-10-16  9:09       ` zhanghailiang
2014-11-03  3:47     ` David Gibson
2014-11-25 15:44       ` Dr. David Alan Gilbert
2014-11-03  3:46   ` David Gibson
2014-11-03 13:22     ` Dr. David Alan Gilbert
2014-11-18  3:52       ` David Gibson
2014-11-19 17:06         ` Dr. David Alan Gilbert
2014-11-19 21:12           ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 17/47] qemu_loadvm errors and debug Dr. David Alan Gilbert (git)
2014-11-03  3:49   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 18/47] ram_debug_dump_bitmap: Dump a migration bitmap as text Dr. David Alan Gilbert (git)
2014-11-03  3:58   ` David Gibson
2014-11-19 17:35     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 19/47] Rework loadvm path for subloops Dr. David Alan Gilbert (git)
2014-10-04 16:46   ` Paolo Bonzini
2014-10-07  8:58     ` Dr. David Alan Gilbert
2014-10-07 10:12       ` Paolo Bonzini
2014-10-07 10:21         ` Dr. David Alan Gilbert
2014-11-03  5:08   ` David Gibson
2014-11-19 17:50     ` Dr. David Alan Gilbert
2014-11-21  6:53       ` David Gibson
2014-12-11 14:47         ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 20/47] Add migration-capability boolean for postcopy-ram Dr. David Alan Gilbert (git)
2014-10-06 18:59   ` Eric Blake
2014-10-06 19:07     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 21/47] Add wrappers and handlers for sending/receiving the postcopy-ram migration messages Dr. David Alan Gilbert (git)
2014-11-03  5:51   ` David Gibson
2014-12-17 14:50     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 22/47] QEMU_VM_CMD_PACKAGED: Send a packaged chunk of migration stream Dr. David Alan Gilbert (git)
2014-11-04  1:28   ` David Gibson
2014-11-04 10:19     ` Dr. David Alan Gilbert
2014-11-18  4:36       ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 23/47] migrate_init: Call from savevm Dr. David Alan Gilbert (git)
2014-10-08  2:28   ` zhanghailiang
2014-11-04  1:29   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 24/47] Allow savevm handlers to state whether they could go into postcopy Dr. David Alan Gilbert (git)
2014-11-04  1:33   ` David Gibson
2014-11-19 17:53     ` Dr. David Alan Gilbert
2014-11-21  6:58       ` David Gibson
2014-11-25 19:58         ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 25/47] postcopy: OS support test Dr. David Alan Gilbert (git)
2014-11-04  1:40   ` David Gibson
2014-11-25 17:34     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 26/47] migrate_start_postcopy: Command to trigger transition to postcopy Dr. David Alan Gilbert (git)
2014-11-04  1:47   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 27/47] MIG_STATE_POSTCOPY_ACTIVE: Add new migration state Dr. David Alan Gilbert (git)
2014-11-04  1:49   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 28/47] qemu_savevm_state_complete: Postcopy changes Dr. David Alan Gilbert (git)
2014-11-04  2:18   ` David Gibson
2014-12-17 16:14     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 29/47] Postcopy page-map-incoming (PMI) structure Dr. David Alan Gilbert (git)
2014-11-04  3:09   ` David Gibson
2014-11-19 18:46     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 30/47] Postcopy: Maintain sentmap and calculate discard Dr. David Alan Gilbert (git)
2014-11-05  6:38   ` David Gibson
2014-12-17 16:48     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 31/47] postcopy: Incoming initialisation Dr. David Alan Gilbert (git)
2014-11-05  6:47   ` David Gibson
2014-12-17 17:21     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 32/47] postcopy: ram_enable_notify to switch on userfault Dr. David Alan Gilbert (git)
2014-10-04 16:42   ` Paolo Bonzini
2014-10-06 19:00     ` Dr. David Alan Gilbert
2014-11-05  6:49   ` David Gibson
2014-11-19 18:59     ` Dr. David Alan Gilbert
2014-11-19 21:17       ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 33/47] Postcopy: Postcopy startup in migration thread Dr. David Alan Gilbert (git)
2014-10-04 16:27   ` Paolo Bonzini
2014-11-20 11:45     ` Dr. David Alan Gilbert
2014-11-21 12:01       ` Paolo Bonzini
2014-11-21 12:07         ` Dr. David Alan Gilbert
2014-11-20 17:12     ` Dr. David Alan Gilbert
2014-11-20 17:19       ` Paolo Bonzini
2014-11-24 18:26     ` Dr. David Alan Gilbert
2014-11-10  6:05   ` David Gibson
2015-01-05 16:06     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 34/47] Postcopy: Create a fault handler thread before marking the ram as userfault Dr. David Alan Gilbert (git)
2014-11-10  6:10   ` David Gibson
2014-11-19 18:56     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 35/47] Page request: Add MIG_RPCOMM_REQPAGES reverse command Dr. David Alan Gilbert (git)
2014-11-10  6:19   ` David Gibson
2014-11-19 20:01     ` Dr. David Alan Gilbert
2014-11-19 21:48       ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 36/47] Page request: Process incoming page request Dr. David Alan Gilbert (git)
2014-10-08  2:31   ` zhanghailiang
2014-10-08  7:49     ` Dr. David Alan Gilbert
2014-10-08  8:07       ` Paolo Bonzini
2014-10-08  8:10       ` zhanghailiang
2014-10-08  8:18         ` Dr. David Alan Gilbert
2014-11-10  6:31   ` David Gibson
2014-11-17 19:07     ` Dr. David Alan Gilbert
2014-11-18  4:38       ` David Gibson
2014-11-19 19:37         ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 37/47] Page request: Consume pages off the post-copy queue Dr. David Alan Gilbert (git)
2014-10-04 18:04   ` Paolo Bonzini
2014-10-07 11:35     ` Dr. David Alan Gilbert
2014-11-11  1:13   ` David Gibson
2015-01-14 20:13     ` Dr. David Alan Gilbert
2015-01-27  4:38       ` David Gibson
2015-01-27  9:40         ` Dr. David Alan Gilbert
2015-01-28  5:33           ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 38/47] Add assertion to check migration_dirty_pages Dr. David Alan Gilbert (git)
2014-10-04 18:32   ` Paolo Bonzini
2014-10-06 18:51     ` Dr. David Alan Gilbert
2014-10-06 20:30       ` Paolo Bonzini
2014-11-11  1:14   ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 39/47] postcopy_ram.c: place_page and helpers Dr. David Alan Gilbert (git)
2014-11-11  1:39   ` David Gibson
2015-01-15 18:14     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 40/47] Postcopy: Use helpers to map pages during migration Dr. David Alan Gilbert (git)
2014-11-13  2:53   ` David Gibson
2014-11-25 18:14     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 41/47] qemu_ram_block_from_host Dr. David Alan Gilbert (git)
2014-11-13  2:59   ` David Gibson
2014-11-25 18:55     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 42/47] Don't sync dirty bitmaps in postcopy Dr. David Alan Gilbert (git)
2014-11-13  3:01   ` David Gibson
2014-11-25 16:25     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 43/47] Host page!=target page: Cleanup bitmaps Dr. David Alan Gilbert (git)
2014-11-13  3:10   ` David Gibson
2014-12-17 18:21     ` Dr. David Alan Gilbert
2015-01-27  4:50       ` David Gibson
2015-01-27 10:04         ` Dr. David Alan Gilbert
2015-01-28  5:36           ` David Gibson
2015-01-27 10:20   ` Peter Maydell
2015-01-27 11:50     ` Dr. David Alan Gilbert [this message]
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 44/47] Postcopy; Handle userfault requests Dr. David Alan Gilbert (git)
2014-11-13  3:23   ` David Gibson
2015-01-05 17:13     ` Dr. David Alan Gilbert
2015-01-27  4:33       ` David Gibson
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 45/47] Start up a postcopy/listener thread ready for incoming page data Dr. David Alan Gilbert (git)
2014-11-13  3:29   ` David Gibson
2014-11-19 19:40     ` Dr. David Alan Gilbert
2014-11-21  8:36       ` David Gibson
2014-11-21 10:17         ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 46/47] postcopy: Wire up loadvm_postcopy_ram_handle_{run, end} commands Dr. David Alan Gilbert (git)
2014-10-04 17:51   ` Paolo Bonzini
2014-10-23 12:18     ` Dr. David Alan Gilbert
2014-10-03 17:47 ` [Qemu-devel] [PATCH v4 47/47] End of migration for postcopy Dr. David Alan Gilbert (git)
2014-10-04 17:49   ` Paolo Bonzini
2014-10-23 14:24     ` Dr. David Alan Gilbert
2014-10-04 18:31   ` Paolo Bonzini
2014-10-07 10:29     ` Dr. David Alan Gilbert
2014-10-07 11:12       ` Paolo Bonzini
2014-10-03 19:21 ` [Qemu-devel] [PATCH v4 00/47] Postcopy implementation Dr. David Alan Gilbert
2014-10-07  2:27   ` Cristian Klein
2014-10-07  8:12     ` Dr. David Alan Gilbert
2014-10-08  8:36       ` Cristian Klein
2014-11-21  3:48 ` zhanghailiang
2014-11-21 10:14   ` Dr. David Alan Gilbert
2014-11-24  8:10     ` zhanghailiang
2014-11-21 18:56   ` Andrea Arcangeli
2014-11-24  8:25     ` zhanghailiang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150127115025.GC2425@work-vm \
    --to=dgilbert@redhat.com \
    --cc=aarcange@redhat.com \
    --cc=amit.shah@redhat.com \
    --cc=cristian.klein@cs.umu.se \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=quintela@redhat.com \
    --cc=yamahata@private.email.ne.jp \
    --cc=yanghy@cn.fujitsu.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.