qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes
@ 2021-04-06  8:01 David Hildenbrand
  2021-04-06  8:01 ` [PATCH for-6.0 v1 1/3] softmmu/physmem: Mark shared anonymous memory RAM_SHARED David Hildenbrand
                   ` (4 more replies)
  0 siblings, 5 replies; 7+ messages in thread
From: David Hildenbrand @ 2021-04-06  8:01 UTC (permalink / raw)
  To: qemu-devel
  Cc: David Hildenbrand, Igor Kotrasinski, Richard Henderson,
	Dr . David Alan Gilbert, Peter Xu, Paolo Bonzini

Fixes related to shared anonymous memory, previously sent as part of
    https://lore.kernel.org/r/20210319101230.21531-1-david@redhat.com

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Igor Kotrasinski <i.kotrasinsk@partner.samsung.com>
Cc: Richard Henderson <richard.henderson@linaro.org>

David Hildenbrand (3):
  softmmu/physmem: Mark shared anonymous memory RAM_SHARED
  softmmu/physmem: Fix ram_block_discard_range() to handle shared
    anonymous memory
  softmmu/physmem: Fix qemu_ram_remap() to handle shared anonymous
    memory

 softmmu/physmem.c | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)


base-commit: ee82c086baaa534d1af26cb8b86e86fb047af918
-- 
2.30.2



^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH for-6.0 v1 1/3] softmmu/physmem: Mark shared anonymous memory RAM_SHARED
  2021-04-06  8:01 [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes David Hildenbrand
@ 2021-04-06  8:01 ` David Hildenbrand
  2021-04-06  8:01 ` [PATCH for-6.0 v1 2/3] softmmu/physmem: Fix ram_block_discard_range() to handle shared anonymous memory David Hildenbrand
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 7+ messages in thread
From: David Hildenbrand @ 2021-04-06  8:01 UTC (permalink / raw)
  To: qemu-devel
  Cc: David Hildenbrand, Igor Kotrasinski, Richard Henderson,
	Dr . David Alan Gilbert, Peter Xu, Paolo Bonzini

Let's drop the "shared" parameter from ram_block_add() and properly
store it in the flags of the ram block instead, such that
qemu_ram_is_shared() properly succeeds on all ram blocks that were mapped
MAP_SHARED.

We'll use this information next to fix some cases with shared anonymous
memory.

Reviewed-by: Igor Kotrasinski <i.kotrasinsk@partner.samsung.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
---
 softmmu/physmem.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/softmmu/physmem.c b/softmmu/physmem.c
index 85034d9c11..76bb8e324e 100644
--- a/softmmu/physmem.c
+++ b/softmmu/physmem.c
@@ -1927,8 +1927,9 @@ static void dirty_memory_extend(ram_addr_t old_ram_size,
     }
 }
 
-static void ram_block_add(RAMBlock *new_block, Error **errp, bool shared)
+static void ram_block_add(RAMBlock *new_block, Error **errp)
 {
+    const bool shared = qemu_ram_is_shared(new_block);
     RAMBlock *block;
     RAMBlock *last_block = NULL;
     ram_addr_t old_ram_size, new_ram_size;
@@ -2064,7 +2065,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,
         return NULL;
     }
 
-    ram_block_add(new_block, &local_err, ram_flags & RAM_SHARED);
+    ram_block_add(new_block, &local_err);
     if (local_err) {
         g_free(new_block);
         error_propagate(errp, local_err);
@@ -2127,10 +2128,13 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
     if (host) {
         new_block->flags |= RAM_PREALLOC;
     }
+    if (share) {
+        new_block->flags |= RAM_SHARED;
+    }
     if (resizeable) {
         new_block->flags |= RAM_RESIZEABLE;
     }
-    ram_block_add(new_block, &local_err, share);
+    ram_block_add(new_block, &local_err);
     if (local_err) {
         g_free(new_block);
         error_propagate(errp, local_err);
-- 
2.30.2



^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH for-6.0 v1 2/3] softmmu/physmem: Fix ram_block_discard_range() to handle shared anonymous memory
  2021-04-06  8:01 [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes David Hildenbrand
  2021-04-06  8:01 ` [PATCH for-6.0 v1 1/3] softmmu/physmem: Mark shared anonymous memory RAM_SHARED David Hildenbrand
@ 2021-04-06  8:01 ` David Hildenbrand
  2021-04-06  8:01 ` [PATCH for-6.0 v1 3/3] softmmu/physmem: Fix qemu_ram_remap() " David Hildenbrand
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 7+ messages in thread
From: David Hildenbrand @ 2021-04-06  8:01 UTC (permalink / raw)
  To: qemu-devel
  Cc: David Hildenbrand, Igor Kotrasinski, Richard Henderson,
	Dr . David Alan Gilbert, Peter Xu, Paolo Bonzini

We can create shared anonymous memory via
    "-object memory-backend-ram,share=on,..."
which is, for example, required by PVRDMA for mremap() to work.

Shared anonymous memory is weird, though. Instead of MADV_DONTNEED, we
have to use MADV_REMOVE: MADV_DONTNEED will only remove / zap all
relevant page table entries of the current process, the backend storage
will not get removed, resulting in no reduced memory consumption and
a repopulation of previous content on next access.

Shared anonymous memory is internally really just shmem, but without a
fd exposed. As we cannot use fallocate() without the fd to discard the
backing storage, MADV_REMOVE gets the same job done without a fd as
documented in "man 2 madvise". Removing backing storage implicitly
invalidates all page table entries with relevant mappings - an additional
MADV_DONTNEED is not required.

Fixes: 06329ccecfa0 ("mem: add share parameter to memory-backend-ram")
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
---
 softmmu/physmem.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/softmmu/physmem.c b/softmmu/physmem.c
index 76bb8e324e..afff96a6dc 100644
--- a/softmmu/physmem.c
+++ b/softmmu/physmem.c
@@ -3506,6 +3506,7 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length)
         /* The logic here is messy;
          *    madvise DONTNEED fails for hugepages
          *    fallocate works on hugepages and shmem
+         *    shared anonymous memory requires madvise REMOVE
          */
         need_madvise = (rb->page_size == qemu_host_page_size);
         need_fallocate = rb->fd != -1;
@@ -3539,7 +3540,11 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length)
              * fallocate'd away).
              */
 #if defined(CONFIG_MADVISE)
-            ret =  madvise(host_startaddr, length, MADV_DONTNEED);
+            if (qemu_ram_is_shared(rb) && rb->fd < 0) {
+                ret = madvise(host_startaddr, length, MADV_REMOVE);
+            } else {
+                ret = madvise(host_startaddr, length, MADV_DONTNEED);
+            }
             if (ret) {
                 ret = -errno;
                 error_report("ram_block_discard_range: Failed to discard range "
-- 
2.30.2



^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH for-6.0 v1 3/3] softmmu/physmem: Fix qemu_ram_remap() to handle shared anonymous memory
  2021-04-06  8:01 [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes David Hildenbrand
  2021-04-06  8:01 ` [PATCH for-6.0 v1 1/3] softmmu/physmem: Mark shared anonymous memory RAM_SHARED David Hildenbrand
  2021-04-06  8:01 ` [PATCH for-6.0 v1 2/3] softmmu/physmem: Fix ram_block_discard_range() to handle shared anonymous memory David Hildenbrand
@ 2021-04-06  8:01 ` David Hildenbrand
  2021-04-06 16:46 ` [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes Peter Xu
  2021-05-25  8:06 ` David Hildenbrand
  4 siblings, 0 replies; 7+ messages in thread
From: David Hildenbrand @ 2021-04-06  8:01 UTC (permalink / raw)
  To: qemu-devel
  Cc: David Hildenbrand, Igor Kotrasinski, Richard Henderson,
	Dr . David Alan Gilbert, Peter Xu, Paolo Bonzini

RAM_SHARED now also properly indicates shared anonymous memory. Let's check
that flag for anonymous memory as well, to restore the proper mapping.

Fixes: 06329ccecfa0 ("mem: add share parameter to memory-backend-ram")
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
---
 softmmu/physmem.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/softmmu/physmem.c b/softmmu/physmem.c
index afff96a6dc..cc59f05593 100644
--- a/softmmu/physmem.c
+++ b/softmmu/physmem.c
@@ -2222,13 +2222,13 @@ void qemu_ram_remap(ram_addr_t addr, ram_addr_t length)
                 abort();
             } else {
                 flags = MAP_FIXED;
+                flags |= block->flags & RAM_SHARED ?
+                         MAP_SHARED : MAP_PRIVATE;
                 if (block->fd >= 0) {
-                    flags |= (block->flags & RAM_SHARED ?
-                              MAP_SHARED : MAP_PRIVATE);
                     area = mmap(vaddr, length, PROT_READ | PROT_WRITE,
                                 flags, block->fd, offset);
                 } else {
-                    flags |= MAP_PRIVATE | MAP_ANONYMOUS;
+                    flags |= MAP_ANONYMOUS;
                     area = mmap(vaddr, length, PROT_READ | PROT_WRITE,
                                 flags, -1, 0);
                 }
-- 
2.30.2



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes
  2021-04-06  8:01 [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes David Hildenbrand
                   ` (2 preceding siblings ...)
  2021-04-06  8:01 ` [PATCH for-6.0 v1 3/3] softmmu/physmem: Fix qemu_ram_remap() " David Hildenbrand
@ 2021-04-06 16:46 ` Peter Xu
  2021-04-07  8:34   ` David Hildenbrand
  2021-05-25  8:06 ` David Hildenbrand
  4 siblings, 1 reply; 7+ messages in thread
From: Peter Xu @ 2021-04-06 16:46 UTC (permalink / raw)
  To: David Hildenbrand
  Cc: Paolo Bonzini, Igor Kotrasinski, Richard Henderson, qemu-devel,
	Dr . David Alan Gilbert

On Tue, Apr 06, 2021 at 10:01:23AM +0200, David Hildenbrand wrote:
> Fixes related to shared anonymous memory, previously sent as part of

The series looks good, but maybe 6.1 material? Since 06329ccecfa ("mem: add
share parameter to memory-backend-ram", 2018-02-19) is introduced in 2.12.

-- 
Peter Xu



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes
  2021-04-06 16:46 ` [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes Peter Xu
@ 2021-04-07  8:34   ` David Hildenbrand
  0 siblings, 0 replies; 7+ messages in thread
From: David Hildenbrand @ 2021-04-07  8:34 UTC (permalink / raw)
  To: Peter Xu
  Cc: Paolo Bonzini, Igor Kotrasinski, Richard Henderson, qemu-devel,
	Dr . David Alan Gilbert

On 06.04.21 18:46, Peter Xu wrote:
> On Tue, Apr 06, 2021 at 10:01:23AM +0200, David Hildenbrand wrote:
>> Fixes related to shared anonymous memory, previously sent as part of
> 
> The series looks good, but maybe 6.1 material? Since 06329ccecfa ("mem: add
> share parameter to memory-backend-ram", 2018-02-19) is introduced in 2.12.

Thanks, but IIRC, there is no rule that requires bugfixes for fixes only 
introduced in this release.

"After the hard feature freeze, the master branch in git is no longer 
open for general development. Only bug fixes will be accepted until the 
next release."

-- 
Thanks,

David / dhildenb



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes
  2021-04-06  8:01 [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes David Hildenbrand
                   ` (3 preceding siblings ...)
  2021-04-06 16:46 ` [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes Peter Xu
@ 2021-05-25  8:06 ` David Hildenbrand
  4 siblings, 0 replies; 7+ messages in thread
From: David Hildenbrand @ 2021-05-25  8:06 UTC (permalink / raw)
  To: qemu-devel, Peter Maydell, Eduardo Habkost, Michael S. Tsirkin,
	Paolo Bonzini, Dr . David Alan Gilbert
  Cc: Igor Kotrasinski, Richard Henderson, Peter Xu

On 06.04.21 10:01, David Hildenbrand wrote:
> Fixes related to shared anonymous memory, previously sent as part of
>      https://lore.kernel.org/r/20210319101230.21531-1-david@redhat.com
> 

Peter, Eduardo, Michael, David, Paolo,

can someone please pick this up?

Thanks

> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Peter Xu <peterx@redhat.com>
> Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
> Cc: Igor Kotrasinski <i.kotrasinsk@partner.samsung.com>
> Cc: Richard Henderson <richard.henderson@linaro.org>
> 
> David Hildenbrand (3):
>    softmmu/physmem: Mark shared anonymous memory RAM_SHARED
>    softmmu/physmem: Fix ram_block_discard_range() to handle shared
>      anonymous memory
>    softmmu/physmem: Fix qemu_ram_remap() to handle shared anonymous
>      memory
> 
>   softmmu/physmem.c | 23 ++++++++++++++++-------
>   1 file changed, 16 insertions(+), 7 deletions(-)
> 
> 
> base-commit: ee82c086baaa534d1af26cb8b86e86fb047af918
> 


-- 
Thanks,

David / dhildenb



^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-05-25  8:10 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-06  8:01 [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes David Hildenbrand
2021-04-06  8:01 ` [PATCH for-6.0 v1 1/3] softmmu/physmem: Mark shared anonymous memory RAM_SHARED David Hildenbrand
2021-04-06  8:01 ` [PATCH for-6.0 v1 2/3] softmmu/physmem: Fix ram_block_discard_range() to handle shared anonymous memory David Hildenbrand
2021-04-06  8:01 ` [PATCH for-6.0 v1 3/3] softmmu/physmem: Fix qemu_ram_remap() " David Hildenbrand
2021-04-06 16:46 ` [PATCH for-6.0 v1 0/3] softmmu/physmem: shared anonymous memory fixes Peter Xu
2021-04-07  8:34   ` David Hildenbrand
2021-05-25  8:06 ` David Hildenbrand

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).