All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch
@ 2013-06-28 18:26 Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 01/30] memory: access FlatView from a local variable Paolo Bonzini
                   ` (29 more replies)
  0 siblings, 30 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

These are the next part in the unlocked address space dispatch work.
At the end of this series, address space dispatch runs under an RCU
critical section (RCU is hopefully well documented in patch 8).
It becomes then possible to add unlocked variants of address_space_rw.

Compared to Liu Ping Fan's approach posted last year, the path I followed
is more long-winded but requires much less changes in device models that
choose to adopt lockless MMIO dispatching.  This is nice, because it
produces an API that is easier to use.

In addition, I decided to sidestep complicated locking discussions by
using RCU.  RCU can bulk-protect the read side of all data structures
while remaining extremely light-weight.  The write side needs its own
lock, but in many cases it can simply be the BQL (very useful for things
that perform updates in callbacks!).

Patches 1-3 add reference counting to FlatView, which is independent
from the other patches but a necessary prerequisite to protect
as->current_map with RCU.

Patches 4-17 implement RCU and update existing threads to be RCU-friendly
(again, more information is in docs/rcu.txt) whenever necessary.
A lot of the code here comes from the URCU library.

Patches 18-30 do the rest of the work, using fresh copies of the dispatch
data structures on every update.  Several patches in this series are
based on Liu Ping Fan's work, which I made more fine-grained.

The diffstat looks daunting; however, note that ~50% of the new lines
are documentation and test cases.  Available from branch rcu of my
github repository.

I don't plan to send a pull request for this series anytime soon,
but reviews and testing are welcome.

Paolo

Liu Ping Fan (1):
  exec: change well-known physical sections to macros

Paolo Bonzini (29):
  memory: access FlatView from a local variable
  memory: use a new FlatView pointer on every topology update
  memory: add reference counting to FlatView
  add a header file for atomic operations
  exec: do not use qemu/tls.h
  qemu-thread: add TLS wrappers
  qemu-thread: add QemuEvent
  rcu: add rcu library
  qemu-thread: register threads with RCU
  rcu: add call_rcu
  rcu: add rcutorture
  rcu: allow nested calls to rcu_thread_offline/rcu_thread_online
  qemu-thread: report RCU quiescent states
  event loop: report RCU quiescent states
  cpus: report RCU quiescent states
  block: report RCU quiescent states
  migration: report RCU quiescent states
  memory: protect current_map by RCU
  memory: avoid ref/unref in memory_region_find
  exec: separate current memory map from the one being built
  memory: move MemoryListener declaration earlier
  exec: move listener from AddressSpaceDispatch to AddressSpace
  exec: separate current radix tree from the one being built
  exec: put memory map in AddressSpaceDispatch
  exec: remove cur_map
  exec: change some APIs to take AddressSpaceDispatch
  exec: change iotlb APIs to take AddressSpaceDispatch
  exec: add a reference to the region returned by
    address_space_translate
  exec: put address space dispatch under RCU critical section

 aio-posix.c                 |   8 +
 aio-win32.c                 |  11 +-
 block/raw-posix.c           |   3 +
 block/raw-win32.c           |   3 +
 configure                   |  21 +++
 cpus.c                      |   3 +
 cputlb.c                    |   9 +-
 docs/atomics.txt            | 345 ++++++++++++++++++++++++++++++++++
 docs/rcu.txt                | 440 ++++++++++++++++++++++++++++++++++++++++++++
 exec.c                      | 244 ++++++++++++++++--------
 hw/9pfs/virtio-9p-synth.c   |   1 +
 hw/display/qxl.c            |   3 +-
 hw/ppc/spapr_iommu.c        |  10 +-
 hw/virtio/vhost.c           |   9 +-
 include/exec/cpu-all.h      |  14 +-
 include/exec/cputlb.h       |   9 +-
 include/exec/memory.h       |  77 ++++----
 include/qemu/atomic.h       | 190 +++++++++++++++----
 include/qemu/queue.h        |  13 ++
 include/qemu/rcu-pointer.h  | 110 +++++++++++
 include/qemu/rcu.h          | 180 ++++++++++++++++++
 include/qemu/thread-posix.h |   8 +
 include/qemu/thread-win32.h |   4 +
 include/qemu/thread.h       |  10 +-
 include/qemu/tls.h          | 125 +++++++++++--
 kvm-all.c                   |   3 +
 libcacard/Makefile          |   3 +-
 main-loop.c                 |   6 +
 memory.c                    | 127 +++++++++----
 migration.c                 |   5 +-
 tests/Makefile              |  10 +-
 tests/rcutorture.c          | 439 +++++++++++++++++++++++++++++++++++++++++++
 tests/test-thread-pool.c    |   8 +-
 tests/test-tls.c            |  87 +++++++++
 util/Makefile.objs          |   1 +
 util/qemu-thread-posix.c    | 174 +++++++++++++++++-
 util/qemu-thread-win32.c    |  61 +++++-
 util/rcu.c                  | 312 +++++++++++++++++++++++++++++++
 38 files changed, 2857 insertions(+), 229 deletions(-)
 create mode 100644 docs/atomics.txt
 create mode 100644 docs/rcu.txt
 create mode 100644 include/qemu/rcu-pointer.h
 create mode 100644 include/qemu/rcu.h
 create mode 100644 tests/rcutorture.c
 create mode 100644 tests/test-tls.c
 create mode 100644 util/rcu.c

-- 
1.8.1.4

^ permalink raw reply	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 01/30] memory: access FlatView from a local variable
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 20:01   ` Anthony Liguori
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 02/30] memory: use a new FlatView pointer on every topology update Paolo Bonzini
                   ` (28 subsequent siblings)
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

We will soon require accesses to as->current_map to be placed under
a lock (with reference counting so as to keep the critical section
small).  To simplify this change, always fetch as->current_map into
a local variable and access it through that variable.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 memory.c | 31 +++++++++++++++++++++----------
 1 file changed, 21 insertions(+), 10 deletions(-)

diff --git a/memory.c b/memory.c
index 688c817..1f44cd1 100644
--- a/memory.c
+++ b/memory.c
@@ -578,13 +578,15 @@ static void address_space_add_del_ioeventfds(AddressSpace *as,
 
 static void address_space_update_ioeventfds(AddressSpace *as)
 {
+    FlatView *view;
     FlatRange *fr;
     unsigned ioeventfd_nb = 0;
     MemoryRegionIoeventfd *ioeventfds = NULL;
     AddrRange tmp;
     unsigned i;
 
-    FOR_EACH_FLAT_RANGE(fr, as->current_map) {
+    view = as->current_map;
+    FOR_EACH_FLAT_RANGE(fr, view) {
         for (i = 0; i < fr->mr->ioeventfd_nb; ++i) {
             tmp = addrrange_shift(fr->mr->ioeventfds[i].addr,
                                   int128_sub(fr->addr.start,
@@ -1142,7 +1144,8 @@ void memory_region_sync_dirty_bitmap(MemoryRegion *mr)
     FlatRange *fr;
 
     QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) {
-        FOR_EACH_FLAT_RANGE(fr, as->current_map) {
+        FlatView *view = as->current_map;
+        FOR_EACH_FLAT_RANGE(fr, view) {
             if (fr->mr == mr) {
                 MEMORY_LISTENER_UPDATE_REGION(fr, as, Forward, log_sync);
             }
@@ -1192,12 +1195,14 @@ void *memory_region_get_ram_ptr(MemoryRegion *mr)
 
 static void memory_region_update_coalesced_range_as(MemoryRegion *mr, AddressSpace *as)
 {
+    FlatView *view;
     FlatRange *fr;
     CoalescedMemoryRange *cmr;
     AddrRange tmp;
     MemoryRegionSection section;
 
-    FOR_EACH_FLAT_RANGE(fr, as->current_map) {
+    view = as->current_map;
+    FOR_EACH_FLAT_RANGE(fr, view) {
         if (fr->mr == mr) {
             section = (MemoryRegionSection) {
                 .address_space = as,
@@ -1488,9 +1493,9 @@ static int cmp_flatrange_addr(const void *addr_, const void *fr_)
     return 0;
 }
 
-static FlatRange *address_space_lookup(AddressSpace *as, AddrRange addr)
+static FlatRange *flatview_lookup(FlatView *view, AddrRange addr)
 {
-    return bsearch(&addr, as->current_map->ranges, as->current_map->nr,
+    return bsearch(&addr, view->ranges, view->nr,
                    sizeof(FlatRange), cmp_flatrange_addr);
 }
 
@@ -1501,6 +1506,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
     MemoryRegion *root;
     AddressSpace *as;
     AddrRange range;
+    FlatView *view;
     FlatRange *fr;
 
     addr += mr->addr;
@@ -1511,13 +1517,14 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
 
     as = memory_region_to_address_space(root);
     range = addrrange_make(int128_make64(addr), int128_make64(size));
-    fr = address_space_lookup(as, range);
+
+    view = as->current_map;
+    fr = flatview_lookup(view, range);
     if (!fr) {
         return ret;
     }
 
-    while (fr > as->current_map->ranges
-           && addrrange_intersects(fr[-1].addr, range)) {
+    while (fr > view->ranges && addrrange_intersects(fr[-1].addr, range)) {
         --fr;
     }
 
@@ -1537,9 +1544,11 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
 
 void address_space_sync_dirty_bitmap(AddressSpace *as)
 {
+    FlatView *view;
     FlatRange *fr;
 
-    FOR_EACH_FLAT_RANGE(fr, as->current_map) {
+    view = as->current_map;
+    FOR_EACH_FLAT_RANGE(fr, view) {
         MEMORY_LISTENER_UPDATE_REGION(fr, as, Forward, log_sync);
     }
 }
@@ -1559,6 +1568,7 @@ void memory_global_dirty_log_stop(void)
 static void listener_add_address_space(MemoryListener *listener,
                                        AddressSpace *as)
 {
+    FlatView *view;
     FlatRange *fr;
 
     if (listener->address_space_filter
@@ -1572,7 +1582,8 @@ static void listener_add_address_space(MemoryListener *listener,
         }
     }
 
-    FOR_EACH_FLAT_RANGE(fr, as->current_map) {
+    view = as->current_map;
+    FOR_EACH_FLAT_RANGE(fr, view) {
         MemoryRegionSection section = {
             .mr = fr->mr,
             .address_space = as,
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 02/30] memory: use a new FlatView pointer on every topology update
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 01/30] memory: access FlatView from a local variable Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 20:02   ` Anthony Liguori
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 03/30] memory: add reference counting to FlatView Paolo Bonzini
                   ` (27 subsequent siblings)
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

This is the first step towards converting as->current_map to
RCU-style updates, where the FlatView updates run concurrently
with uses of an old FlatView.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 memory.c | 34 ++++++++++++++++++----------------
 1 file changed, 18 insertions(+), 16 deletions(-)

diff --git a/memory.c b/memory.c
index 1f44cd1..319894e 100644
--- a/memory.c
+++ b/memory.c
@@ -276,6 +276,7 @@ static void flatview_destroy(FlatView *view)
         memory_region_unref(view->ranges[i].mr);
     }
     g_free(view->ranges);
+    g_free(view);
 }
 
 static bool can_merge(FlatRange *r1, FlatRange *r2)
@@ -512,17 +513,18 @@ static void render_memory_region(FlatView *view,
 }
 
 /* Render a memory topology into a list of disjoint absolute ranges. */
-static FlatView generate_memory_topology(MemoryRegion *mr)
+static FlatView *generate_memory_topology(MemoryRegion *mr)
 {
-    FlatView view;
+    FlatView *view;
 
-    flatview_init(&view);
+    view = g_new(FlatView, 1);
+    flatview_init(view);
 
     if (mr) {
-        render_memory_region(&view, mr, int128_zero(),
+        render_memory_region(view, mr, int128_zero(),
                              addrrange_make(int128_zero(), int128_2_64()), false);
     }
-    flatview_simplify(&view);
+    flatview_simplify(view);
 
     return view;
 }
@@ -610,8 +612,8 @@ static void address_space_update_ioeventfds(AddressSpace *as)
 }
 
 static void address_space_update_topology_pass(AddressSpace *as,
-                                               FlatView old_view,
-                                               FlatView new_view,
+                                               const FlatView *old_view,
+                                               const FlatView *new_view,
                                                bool adding)
 {
     unsigned iold, inew;
@@ -621,14 +623,14 @@ static void address_space_update_topology_pass(AddressSpace *as,
      * Kill ranges in the old map, and instantiate ranges in the new map.
      */
     iold = inew = 0;
-    while (iold < old_view.nr || inew < new_view.nr) {
-        if (iold < old_view.nr) {
-            frold = &old_view.ranges[iold];
+    while (iold < old_view->nr || inew < new_view->nr) {
+        if (iold < old_view->nr) {
+            frold = &old_view->ranges[iold];
         } else {
             frold = NULL;
         }
-        if (inew < new_view.nr) {
-            frnew = &new_view.ranges[inew];
+        if (inew < new_view->nr) {
+            frnew = &new_view->ranges[inew];
         } else {
             frnew = NULL;
         }
@@ -674,14 +676,14 @@ static void address_space_update_topology_pass(AddressSpace *as,
 
 static void address_space_update_topology(AddressSpace *as)
 {
-    FlatView old_view = *as->current_map;
-    FlatView new_view = generate_memory_topology(as->root);
+    FlatView *old_view = as->current_map;
+    FlatView *new_view = generate_memory_topology(as->root);
 
     address_space_update_topology_pass(as, old_view, new_view, false);
     address_space_update_topology_pass(as, old_view, new_view, true);
 
-    *as->current_map = new_view;
-    flatview_destroy(&old_view);
+    as->current_map = new_view;
+    flatview_destroy(old_view);
     address_space_update_ioeventfds(as);
 }
 
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 03/30] memory: add reference counting to FlatView
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 01/30] memory: access FlatView from a local variable Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 02/30] memory: use a new FlatView pointer on every topology update Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 20:07   ` Anthony Liguori
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 04/30] add a header file for atomic operations Paolo Bonzini
                   ` (26 subsequent siblings)
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

With this change, a FlatView can be used even after a concurrent
update has replaced it.  Because we do not have RCU, we use a
mutex to protect the small critical sections that read/write the
as->current_map pointer.  Accesses to the FlatView can be done
outside the mutex.

If a MemoryRegion will be used after the FlatView is unref-ed (or after
a MemoryListener callback is returned), a reference has to be added to
that MemoryRegion.  For example, memory_region_find adds a reference to
the MemoryRegion that it returns.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 memory.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 69 insertions(+), 10 deletions(-)

diff --git a/memory.c b/memory.c
index 319894e..bb92e17 100644
--- a/memory.c
+++ b/memory.c
@@ -29,12 +29,26 @@ static unsigned memory_region_transaction_depth;
 static bool memory_region_update_pending;
 static bool global_dirty_log = false;
 
+/* flat_view_mutex is taken around reading as->current_map; the critical
+ * section is extremely short, so I'm using a single mutex for every AS.
+ * We could also RCU for the read-side.
+ *
+ * The BQL is taken around transaction commits, hence both locks are taken
+ * while writing to as->current_map (with the BQL taken outside).
+ */
+static QemuMutex flat_view_mutex;
+
 static QTAILQ_HEAD(memory_listeners, MemoryListener) memory_listeners
     = QTAILQ_HEAD_INITIALIZER(memory_listeners);
 
 static QTAILQ_HEAD(, AddressSpace) address_spaces
     = QTAILQ_HEAD_INITIALIZER(address_spaces);
 
+static void memory_init(void)
+{
+    qemu_mutex_init(&flat_view_mutex);
+}
+
 typedef struct AddrRange AddrRange;
 
 /*
@@ -225,6 +239,7 @@ struct FlatRange {
  * order.
  */
 struct FlatView {
+    unsigned ref;
     FlatRange *ranges;
     unsigned nr;
     unsigned nr_allocated;
@@ -246,6 +261,7 @@ static bool flatrange_equal(FlatRange *a, FlatRange *b)
 
 static void flatview_init(FlatView *view)
 {
+    view->ref = 1;
     view->ranges = NULL;
     view->nr = 0;
     view->nr_allocated = 0;
@@ -279,6 +295,18 @@ static void flatview_destroy(FlatView *view)
     g_free(view);
 }
 
+static void flatview_ref(FlatView *view)
+{
+    __sync_fetch_and_add(&view->ref, 1);
+}
+
+static void flatview_unref(FlatView *view)
+{
+    if (__sync_fetch_and_sub(&view->ref, 1) == 1) {
+        flatview_destroy(view);
+    }
+}
+
 static bool can_merge(FlatRange *r1, FlatRange *r2)
 {
     return int128_eq(addrrange_end(r1->addr), r2->addr.start)
@@ -578,6 +606,17 @@ static void address_space_add_del_ioeventfds(AddressSpace *as,
     }
 }
 
+static FlatView *address_space_get_flatview(AddressSpace *as)
+{
+    FlatView *view;
+
+    qemu_mutex_lock(&flat_view_mutex);
+    view = as->current_map;
+    flatview_ref(view);
+    qemu_mutex_unlock(&flat_view_mutex);
+    return view;
+}
+
 static void address_space_update_ioeventfds(AddressSpace *as)
 {
     FlatView *view;
@@ -587,7 +626,7 @@ static void address_space_update_ioeventfds(AddressSpace *as)
     AddrRange tmp;
     unsigned i;
 
-    view = as->current_map;
+    view = address_space_get_flatview(as);
     FOR_EACH_FLAT_RANGE(fr, view) {
         for (i = 0; i < fr->mr->ioeventfd_nb; ++i) {
             tmp = addrrange_shift(fr->mr->ioeventfds[i].addr,
@@ -609,6 +648,7 @@ static void address_space_update_ioeventfds(AddressSpace *as)
     g_free(as->ioeventfds);
     as->ioeventfds = ioeventfds;
     as->ioeventfd_nb = ioeventfd_nb;
+    flatview_unref(view);
 }
 
 static void address_space_update_topology_pass(AddressSpace *as,
@@ -676,14 +716,25 @@ static void address_space_update_topology_pass(AddressSpace *as,
 
 static void address_space_update_topology(AddressSpace *as)
 {
-    FlatView *old_view = as->current_map;
+    FlatView *old_view = address_space_get_flatview(as);
     FlatView *new_view = generate_memory_topology(as->root);
 
     address_space_update_topology_pass(as, old_view, new_view, false);
     address_space_update_topology_pass(as, old_view, new_view, true);
 
+    qemu_mutex_lock(&flat_view_mutex);
+    flatview_unref(as->current_map);
     as->current_map = new_view;
-    flatview_destroy(old_view);
+    qemu_mutex_unlock(&flat_view_mutex);
+
+    /* Note that all the old MemoryRegions are still alive up to this
+     * point.  This relieves most MemoryListeners from the need to
+     * ref/unref the MemoryRegions they get---unless they use them
+     * outside the iothread mutex, in which case precise reference
+     * counting is necessary.
+     */
+    flatview_unref(old_view);
+
     address_space_update_ioeventfds(as);
 }
 
@@ -1146,12 +1197,13 @@ void memory_region_sync_dirty_bitmap(MemoryRegion *mr)
     FlatRange *fr;
 
     QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) {
-        FlatView *view = as->current_map;
+        FlatView *view = address_space_get_flatview(as);
         FOR_EACH_FLAT_RANGE(fr, view) {
             if (fr->mr == mr) {
                 MEMORY_LISTENER_UPDATE_REGION(fr, as, Forward, log_sync);
             }
         }
+        flatview_unref(view);
     }
 }
 
@@ -1203,7 +1255,7 @@ static void memory_region_update_coalesced_range_as(MemoryRegion *mr, AddressSpa
     AddrRange tmp;
     MemoryRegionSection section;
 
-    view = as->current_map;
+    view = address_space_get_flatview(as);
     FOR_EACH_FLAT_RANGE(fr, view) {
         if (fr->mr == mr) {
             section = (MemoryRegionSection) {
@@ -1229,6 +1281,7 @@ static void memory_region_update_coalesced_range_as(MemoryRegion *mr, AddressSpa
             }
         }
     }
+    flatview_unref(view);
 }
 
 static void memory_region_update_coalesced_range(MemoryRegion *mr)
@@ -1520,7 +1573,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
     as = memory_region_to_address_space(root);
     range = addrrange_make(int128_make64(addr), int128_make64(size));
 
-    view = as->current_map;
+    view = address_space_get_flatview(as);
     fr = flatview_lookup(view, range);
     if (!fr) {
         return ret;
@@ -1541,6 +1594,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
     ret.readonly = fr->readonly;
     memory_region_ref(ret.mr);
 
+    flatview_unref(view);
     return ret;
 }
 
@@ -1549,10 +1603,11 @@ void address_space_sync_dirty_bitmap(AddressSpace *as)
     FlatView *view;
     FlatRange *fr;
 
-    view = as->current_map;
+    view = address_space_get_flatview(as);
     FOR_EACH_FLAT_RANGE(fr, view) {
         MEMORY_LISTENER_UPDATE_REGION(fr, as, Forward, log_sync);
     }
+    flatview_unref(view);
 }
 
 void memory_global_dirty_log_start(void)
@@ -1584,7 +1639,7 @@ static void listener_add_address_space(MemoryListener *listener,
         }
     }
 
-    view = as->current_map;
+    view = address_space_get_flatview(as);
     FOR_EACH_FLAT_RANGE(fr, view) {
         MemoryRegionSection section = {
             .mr = fr->mr,
@@ -1598,6 +1653,7 @@ static void listener_add_address_space(MemoryListener *listener,
             listener->region_add(listener, &section);
         }
     }
+    flatview_unref(view);
 }
 
 void memory_listener_register(MemoryListener *listener, AddressSpace *filter)
@@ -1631,6 +1687,10 @@ void memory_listener_unregister(MemoryListener *listener)
 
 void address_space_init(AddressSpace *as, MemoryRegion *root, const char *name)
 {
+    if (QTAILQ_EMPTY(&address_spaces)) {
+        memory_init();
+    }
+
     memory_region_transaction_begin();
     as->root = root;
     as->current_map = g_new(FlatView, 1);
@@ -1652,9 +1712,8 @@ void address_space_destroy(AddressSpace *as)
     memory_region_transaction_commit();
     QTAILQ_REMOVE(&address_spaces, as, address_spaces_link);
     address_space_destroy_dispatch(as);
-    flatview_destroy(as->current_map);
+    flatview_unref(as->current_map);
     g_free(as->name);
-    g_free(as->current_map);
     g_free(as->ioeventfds);
 }
 
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (2 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 03/30] memory: add reference counting to FlatView Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 20:41   ` Anthony Liguori
  2013-07-03  2:24   ` liu ping fan
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h Paolo Bonzini
                   ` (25 subsequent siblings)
  29 siblings, 2 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

We're already using them in several places, but __sync builtins are just
too ugly to type, and do not provide seqcst load/store operations.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/atomics.txt         | 345 +++++++++++++++++++++++++++++++++++++++++++++++
 hw/display/qxl.c         |   3 +-
 hw/virtio/vhost.c        |   9 +-
 include/qemu/atomic.h    | 190 +++++++++++++++++++++-----
 migration.c              |   3 +-
 tests/test-thread-pool.c |   8 +-
 6 files changed, 514 insertions(+), 44 deletions(-)
 create mode 100644 docs/atomics.txt

diff --git a/docs/atomics.txt b/docs/atomics.txt
new file mode 100644
index 0000000..e2ce04b
--- /dev/null
+++ b/docs/atomics.txt
@@ -0,0 +1,345 @@
+CPUs perform independent memory operations effectively in random order.
+but this can be a problem for CPU-CPU interaction (including interactions
+between QEMU and the guest).  Multi-threaded programs use various tools
+to instruct the compiler and the CPU to restrict the order to something
+that is consistent with the expectations of the programmer.
+
+The most basic tool is locking.  Mutexes, condition variables and
+semaphores are used in QEMU, and should be the default approach to
+synchronization.  Anything else is considerably harder, but it's
+also justified more often than one would like.  The two tools that
+are provided by qemu/atomic.h are memory barriers and atomic operations.
+
+Macros defined by qemu/atomic.h fall in three camps:
+
+- compiler barriers: barrier();
+
+- weak atomic access and manual memory barriers: atomic_read(),
+  atomic_set(), smp_rmb(), smp_wmb(), smp_mb(), smp_read_barrier_depends();
+
+- sequentially consistent atomic access: everything else.
+
+
+COMPILER MEMORY BARRIER
+=======================
+
+barrier() prevents the compiler from moving the memory accesses either
+side of it to the other side.  The compiler barrier has no direct effect
+on the CPU, which may then reorder things however it wishes.
+
+barrier() is mostly used within qemu/atomic.h itself.  On some
+architectures, CPU guarantees are strong enough that blocking compiler
+optimizations already ensures the correct order of execution.  In this
+case, qemu/atomic.h will reduce stronger memory barriers to simple
+compiler barriers.
+
+Still, barrier() can be useful when writing code that can be interrupted
+by signal handlers.
+
+
+SEQUENTIALLY CONSISTENT ATOMIC ACCESS
+=====================================
+
+Most of the operations in the qemu/atomic.h header ensure *sequential
+consistency*, where "the result of any execution is the same as if the
+operations of all the processors were executed in some sequential order,
+and the operations of each individual processor appear in this sequence
+in the order specified by its program".
+
+qemu/atomic.h provides the following set of atomic read-modify-write
+operations:
+
+    typeof(*ptr) atomic_inc(ptr)
+    typeof(*ptr) atomic_dec(ptr)
+    typeof(*ptr) atomic_add(ptr, val)
+    typeof(*ptr) atomic_sub(ptr, val)
+    typeof(*ptr) atomic_and(ptr, val)
+    typeof(*ptr) atomic_or(ptr, val)
+    typeof(*ptr) atomic_xchg(ptr, val
+    typeof(*ptr) atomic_cmpxchg(ptr, old, new)
+
+all of which return the old value of *ptr.  These operations are
+polymorphic; they operate on any type that is as wide as an int.
+
+Sequentially consistent loads and stores can be done using:
+
+    atomic_add(ptr, 0) for loads
+    atomic_xchg(ptr, val) for stores
+
+However, they are quite expensive on some platforms, notably POWER and
+ARM.  Therefore, qemu/atomic.h provides two primitives with slightly
+weaker constraints:
+
+    typeof(*ptr) atomic_mb_read(ptr)
+    void         atomic_mb_set(ptr, val)
+
+The semantics of these primitives map to Java volatile variables,
+and are strongly related to memory barriers as used in the Linux
+kernel (see below).
+
+As long as you use atomic_mb_read and atomic_mb_set, accesses cannot
+be reordered with each other, and it is also not possible to reorder
+"normal" accesses around them.
+
+However, and this is the important difference between
+atomic_mb_read/atomic_mb_set and sequential consistency, it is important
+for both threads to access the same volatile variable.  It is not the
+case that everything visible to thread A when it writes volatile field f
+becomes visible to thread B after it reads volatile field g. The store
+and load have to "match" (i.e., be performed on the same volatile
+field) to achieve the right semantics.
+
+
+These operations operate on any type that is as wide as an int or smaller.
+
+
+WEAK ATOMIC ACCESS AND MANUAL MEMORY BARRIERS
+=============================================
+
+Compared to sequentially consistent atomic access, programming with
+weaker consistency models can be considerably more complicated.
+In general, if the algorithm you are writing includes both writes
+and reads on the same side, it is generally simpler to use sequentially
+consistent primitives.
+
+When using this model, variables are accessed with atomic_read() and
+atomic_set(), and restrictions to the ordering of accesses is enforced
+using the smp_rmb(), smp_wmb(), smp_mb() and smp_read_barrier_depends()
+memory barriers.
+
+atomic_read() and atomic_set() prevents the compiler from using
+optimizations that might otherwise optimize accesses out of existence
+on the one hand, or that might create unsolicited accesses on the other.
+In general this should not have any effect, because the same compiler
+barriers are already implied by memory barriers.  However, it is useful
+to do so, because it tells readers which variables are shared with
+other threads, and which are local to the current thread or protected
+by other, more mundane means.
+
+Memory barriers control the order of references to shared memory.
+They come in four kinds:
+
+- smp_rmb() guarantees that all the LOAD operations specified before
+  the barrier will appear to happen before all the LOAD operations
+  specified after the barrier with respect to the other components of
+  the system.
+
+  In other words, smp_rmb() puts a partial ordering on loads, but is not
+  required to have any effect on stores.
+
+- smp_wmb() guarantees that all the STORE operations specified before
+  the barrier will appear to happen before all the STORE operations
+  specified after the barrier with respect to the other components of
+  the system.
+
+  In other words, smp_wmb() puts a partial ordering on stores, but is not
+  required to have any effect on loads.
+
+- smp_mb() guarantees that all the LOAD and STORE operations specified
+  before the barrier will appear to happen before all the LOAD and
+  STORE operations specified after the barrier with respect to the other
+  components of the system.
+
+  smp_mb() puts a partial ordering on both loads and stores.  It is
+  stronger than both a read and a write memory barrier; it implies both
+  smp_rmb() and smp_wmb(), but it also prevents STOREs coming before the
+  barrier from overtaking LOADs coming after the barrier and vice versa.
+
+- smp_read_barrier_depends() is a weaker kind of read barrier.  On
+  most processors, whenever two loads are performed such that the
+  second depends on the result of the first (e.g., the first load
+  retrieves the address to which the second load will be directed),
+  the processor will guarantee that the first LOAD will appear to happen
+  before the second with respect to the other components of the system.
+  However, this is not always true---for example, it was not true on
+  Alpha processors.  Whenever this kind of access happens to shared
+  memory (that is not protected by a lock), a read barrier is needed,
+  and smp_read_barrier_depends() can be used instead of smp_rmb().
+
+  Note that the first load really has to have a _data_ dependency and not
+  a control dependency.  If the address for the second load is dependent
+  on the first load, but the dependency is through a conditional rather
+  than actually loading the address itself, then it's a _control_
+  dependency and a full read barrier or better is required.
+
+
+This is the set of barriers that is required *between* two atomic_read()
+and atomic_set() operations to achieve sequential consistency:
+
+                    |               2nd operation             |
+                    |-----------------------------------------|
+     1st operation  | (after last) | atomic_read | atomic_set |
+     ---------------+--------------+-------------+------------|
+     (before first) |              | none        | smp_wmb()  |
+     ---------------+--------------+-------------+------------|
+     atomic_read    | smp_rmb()    | smp_rmb()*  | **         |
+     ---------------+--------------+-------------+------------|
+     atomic_set     | none         | smp_mb()*** | smp_wmb()  |
+     ---------------+--------------+-------------+------------|
+
+       * Or smp_read_barrier_depends().
+
+      ** This requires a load-store barrier.  How to achieve this varies
+         depending on the machine, but in practice smp_rmb()+smp_wmb()
+         should have the desired effect.  For example, on PowerPC the
+         lwsync instruction is a combined load-load, load-store and
+         store-store barrier.
+
+     *** This requires a store-load barrier.  On most machines, the only
+         way to achieve this is a full barrier.
+
+
+You can see that the two possible definitions of atomic_mb_read()
+and atomic_mb_set() are the following:
+
+    1) atomic_mb_read(p)   = atomic_read(p); smp_rmb()
+       atomic_mb_set(p, v) = smp_wmb(); atomic_set(p, v); smp_mb()
+
+    2) atomic_mb_read(p)   = smp_mb() atomic_read(p); smp_rmb()
+       atomic_mb_set(p, v) = smp_wmb(); atomic_set(p, v);
+
+Usually the former is used, because smp_mb() is expensive and a program
+normally has more reads than writes.  Therefore it makes more sense to
+make atomic_mb_set() the more expensive operation.
+
+There are two common cases in which atomic_mb_read and atomic_mb_set
+generate too many memory barriers, and thus it can be useful to manually
+place barriers instead:
+
+- when a data structure has one thread that is always a writer
+  and one thread that is always a reader, manual placement of
+  memory barriers makes the write side faster.  Furthermore,
+  correctness is easy to check for in this case using the "pairing"
+  trick that is explained below:
+
+     thread 1                                thread 1
+     -------------------------               ------------------------
+     (other writes)
+                                             smp_wmb()
+     atomic_mb_set(&a, x)                    atomic_set(&a, x)
+                                             smp_wmb()
+     atomic_mb_set(&b, y)                    atomic_set(&b, y)
+
+                                       =>
+     thread 2                                thread 2
+     -------------------------               ------------------------
+     y = atomic_mb_read(&b)                  y = atomic_read(&b)
+                                             smp_rmb()
+     x = atomic_mb_read(&a)                  x = atomic_read(&a)
+                                             smp_rmb()
+
+- sometimes, a thread is accessing many variables that are otherwise
+  unrelated to each other (for example because, apart from the current
+  thread, exactly one other thread will read or write each of these
+  variables).  In this case, it is possible to "hoist" the implicit
+  barriers provided by atomic_mb_read() and atomic_mb_set() outside
+  a loop.  For example, the above definition atomic_mb_read() gives
+  the following transformation:
+
+     n = 0;                                  n = 0;
+     for (i = 0; i < 10; i++)          =>    for (i = 0; i < 10; i++)
+       n += atomic_mb_read(&a[i]);             n += atomic_read(&a[i]);
+                                             smp_rmb();
+
+  Similarly, atomic_mb_set() can be transformed as follows:
+  smp_mb():
+
+                                             smp_wmb();
+     for (i = 0; i < 10; i++)          =>    for (i = 0; i < 10; i++)
+       atomic_mb_set(&a[i], false);            atomic_set(&a[i], false);
+                                             smp_mb();
+
+
+The two tricks can be combined.  In this case, splitting a loop in
+two lets you hoist the barriers out of the loops _and_ eliminate the
+expensive smp_mb():
+
+                                             smp_wmb();
+     for (i = 0; i < 10; i++) {        =>    for (i = 0; i < 10; i++)
+       atomic_mb_set(&a[i], false);            atomic_set(&a[i], false);
+       atomic_mb_set(&b[i], false);          smb_wmb();
+     }                                       for (i = 0; i < 10; i++)
+                                               atomic_set(&a[i], false);
+                                             smp_mb();
+
+  The other thread can still use atomic_mb_read()/atomic_mb_set()
+
+
+Memory barrier pairing
+----------------------
+
+A useful rule of thumb is that memory barriers should always, or almost
+always, be paired with another barrier.  In the case of QEMU, however,
+note that the other barrier may actually be in a driver that runs in
+the guest!
+
+For the purposes of pairing, smp_read_barrier_depends() and smp_rmb()
+both count as read barriers.  A read barriers shall pair with a write
+barrier or a full barrier; a write barrier shall pair with a read
+barrier or a full barrier.  A full barrier can pair with anything.
+For example:
+
+        thread 1             thread 2
+        ===============      ===============
+        a = 1;
+        smp_wmb();
+        b = 2;               x = b;
+                             smp_rmb();
+                             y = a;
+
+Note that the "writing" thread are accessing the variables in the
+opposite order as the "reading" thread.  This is expected: stores
+before the write barrier will normally match the loads after the
+read barrier, and vice versa.  The same is true for more than 2
+access and for data dependency barriers:
+
+        thread 1             thread 2
+        ===============      ===============
+        b[2] = 1;
+        smp_wmb();
+        x->i = 2;
+        smp_wmb();
+        a = x;               x = a;
+                             smp_read_barrier_depends();
+                             y = x->i;
+                             smp_read_barrier_depends();
+                             z = b[y];
+
+smp_wmb() also pairs with atomic_mb_read(), and smp_rmb() also pairs
+with atomic_mb_set().
+
+
+COMPARISON WITH LINUX KERNEL MEMORY BARRIERS
+============================================
+
+Here is a list of differences between Linux kernel atomic operations
+and memory barriers, and the equivalents in QEMU:
+
+- atomic operations in Linux are always on a 32-bit int type and
+  use a boxed atomic_t type; atomic operations in QEMU are polymorphic
+  and use normal C types.
+
+- atomic_read and atomic_set in Linux give no guarantee at all;
+  atomic_read and atomic_set in QEMU include a compiler barrier
+  (similar to the ACCESS_ONCE macro in Linux).
+
+- most atomic read-modify-write operations in Linux return void;
+  in QEMU, all of them return the old value of the variable.
+
+- different atomic read-modify-write operations in Linux imply
+  a different set of memory barriers; in QEMU, all of them enforce
+  sequential consistency, which means they imply full memory barriers
+  before and after the operation.
+
+- Linux does not have an equivalent of atomic_mb_read() and
+  atomic_mb_set().  In particular, note that set_mb() is a little
+  weaker than atomic_mb_set().
+
+
+SOURCES
+=======
+
+* Documentation/memory-barriers.txt from the Linux kernel
+
+* "The JSR-133 Cookbook for Compiler Writers", available at
+  http://g.oswego.edu/dl/jmm/cookbook.html
diff --git a/hw/display/qxl.c b/hw/display/qxl.c
index 3862d7a..f24cb4e 100644
--- a/hw/display/qxl.c
+++ b/hw/display/qxl.c
@@ -23,6 +23,7 @@
 #include "qemu-common.h"
 #include "qemu/timer.h"
 #include "qemu/queue.h"
+#include "qemu/atomic.h"
 #include "monitor/monitor.h"
 #include "sysemu/sysemu.h"
 #include "trace.h"
@@ -1726,7 +1727,7 @@ static void qxl_send_events(PCIQXLDevice *d, uint32_t events)
         trace_qxl_send_events_vm_stopped(d->id, events);
         return;
     }
-    old_pending = __sync_fetch_and_or(&d->ram->int_pending, le_events);
+    old_pending = atomic_or(&d->ram->int_pending, le_events);
     if ((old_pending & le_events) == le_events) {
         return;
     }
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 96ab625..8f6ab13 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -16,6 +16,7 @@
 #include <sys/ioctl.h>
 #include "hw/virtio/vhost.h"
 #include "hw/hw.h"
+#include "qemu/atomic.h"
 #include "qemu/range.h"
 #include <linux/vhost.h>
 #include "exec/address-spaces.h"
@@ -47,11 +48,9 @@ static void vhost_dev_sync_region(struct vhost_dev *dev,
             addr += VHOST_LOG_CHUNK;
             continue;
         }
-        /* Data must be read atomically. We don't really
-         * need the barrier semantics of __sync
-         * builtins, but it's easier to use them than
-         * roll our own. */
-        log = __sync_fetch_and_and(from, 0);
+        /* Data must be read atomically. We don't really need barrier semantics
+         * but it's easier to use atomic_* than roll our own. */
+        log = atomic_xchg(from, 0);
         while ((bit = sizeof(log) > sizeof(int) ?
                 ffsll(log) : ffs(log))) {
             hwaddr page_addr;
diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index 10becb6..04d64d0 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -1,68 +1,194 @@
-#ifndef __QEMU_BARRIER_H
-#define __QEMU_BARRIER_H 1
+/*
+ * Simple interface for atomic operations.
+ *
+ * Copyright (C) 2013 Red Hat, Inc.
+ *
+ * Author: Paolo Bonzini <pbonzini@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
 
-/* Compiler barrier */
-#define barrier()   asm volatile("" ::: "memory")
+#ifndef __QEMU_ATOMIC_H
+#define __QEMU_ATOMIC_H 1
 
-#if defined(__i386__)
+#include "qemu/compiler.h"
 
-#include "qemu/compiler.h"        /* QEMU_GNUC_PREREQ */
+/* For C11 atomic ops */
 
-/*
- * Because of the strongly ordered x86 storage model, wmb() and rmb() are nops
- * on x86(well, a compiler barrier only).  Well, at least as long as
- * qemu doesn't do accesses to write-combining memory or non-temporal
- * load/stores from C code.
- */
-#define smp_wmb()   barrier()
-#define smp_rmb()   barrier()
+/* Compiler barrier */
+#define barrier()   ({ asm volatile("" ::: "memory"); (void)0; })
+
+#ifndef __ATOMIC_RELAXED
 
 /*
- * We use GCC builtin if it's available, as that can use
- * mfence on 32 bit as well, e.g. if built with -march=pentium-m.
- * However, on i386, there seem to be known bugs as recently as 4.3.
- * */
-#if QEMU_GNUC_PREREQ(4, 4)
-#define smp_mb() __sync_synchronize()
+ * We use GCC builtin if it's available, as that can use mfence on
+ * 32-bit as well, e.g. if built with -march=pentium-m. However, on
+ * i386 the spec is buggy, and the implementation followed it until
+ * 4.3 (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36793).
+ */
+#if defined(__i386__) || defined(__x86_64__)
+#if !QEMU_GNUC_PREREQ(4, 4)
+#if defined __x86_64__
+#define smp_mb()    ({ asm volatile("mfence" ::: "memory"); (void)0; })
 #else
-#define smp_mb() asm volatile("lock; addl $0,0(%%esp) " ::: "memory")
+#define smp_mb()    ({ asm volatile("lock; addl $0,0(%%esp) " ::: "memory"); (void)0; })
+#endif
+#endif
 #endif
 
-#elif defined(__x86_64__)
 
+#ifdef __alpha__
+#define smp_read_barrier_depends()   asm volatile("mb":::"memory")
+#endif
+
+#if defined(__i386__) || defined(__x86_64__) || defined(__s390x__)
+
+/*
+ * Because of the strongly ordered storage model, wmb() and rmb() are nops
+ * here (a compiler barrier only).  QEMU doesn't do accesses to write-combining
+ * qemu memory or non-temporal load/stores from C code.
+ */
 #define smp_wmb()   barrier()
 #define smp_rmb()   barrier()
-#define smp_mb() asm volatile("mfence" ::: "memory")
+
+/*
+ * __sync_lock_test_and_set() is documented to be an acquire barrier only,
+ * but it is a full barrier at the hardware level.  Add a compiler barrier
+ * to make it a full barrier also at the compiler level.
+ */
+#define atomic_xchg(ptr, i)    (barrier(), __sync_lock_test_and_set(ptr, i))
+
+/*
+ * Load/store with Java volatile semantics.
+ */
+#define atomic_mb_set(ptr, i)  ((void)atomic_xchg(ptr, i))
 
 #elif defined(_ARCH_PPC)
 
 /*
  * We use an eieio() for wmb() on powerpc.  This assumes we don't
  * need to order cacheable and non-cacheable stores with respect to
- * each other
+ * each other.
+ *
+ * smp_mb has the same problem as on x86 for not-very-new GCC
+ * (http://patchwork.ozlabs.org/patch/126184/, Nov 2011).
  */
-#define smp_wmb()   asm volatile("eieio" ::: "memory")
-
+#define smp_wmb()   ({ asm volatile("eieio" ::: "memory"); (void)0; })
 #if defined(__powerpc64__)
-#define smp_rmb()   asm volatile("lwsync" ::: "memory")
+#define smp_rmb()   ({ asm volatile("lwsync" ::: "memory"); (void)0; })
 #else
-#define smp_rmb()   asm volatile("sync" ::: "memory")
+#define smp_rmb()   ({ asm volatile("sync" ::: "memory"); (void)0; })
 #endif
+#define smp_mb()    ({ asm volatile("sync" ::: "memory"); (void)0; })
 
-#define smp_mb()   asm volatile("sync" ::: "memory")
+#endif /* _ARCH_PPC */
 
-#else
+#endif /* C11 atomics */
 
 /*
  * For (host) platforms we don't have explicit barrier definitions
  * for, we use the gcc __sync_synchronize() primitive to generate a
  * full barrier.  This should be safe on all platforms, though it may
- * be overkill for wmb() and rmb().
+ * be overkill for smp_wmb() and smp_rmb().
  */
+#ifndef smp_mb
+#define smp_mb()    __sync_synchronize()
+#endif
+
+#ifndef smp_wmb
+#ifdef __ATOMIC_RELEASE
+#define smp_wmb()   __atomic_thread_fence(__ATOMIC_RELEASE)
+#else
 #define smp_wmb()   __sync_synchronize()
-#define smp_mb()   __sync_synchronize()
+#endif
+#endif
+
+#ifndef smp_rmb
+#ifdef __ATOMIC_ACQUIRE
+#define smp_rmb()   __atomic_thread_fence(__ATOMIC_ACQUIRE)
+#else
 #define smp_rmb()   __sync_synchronize()
+#endif
+#endif
+
+#ifndef smp_read_barrier_depends
+#ifdef __ATOMIC_CONSUME
+#define smp_read_barrier_depends()   __atomic_thread_fence(__ATOMIC_CONSUME)
+#else
+#define smp_read_barrier_depends()   barrier()
+#endif
+#endif
 
+#ifndef atomic_read
+#define atomic_read(ptr)       (*(__typeof__(*ptr) *volatile) (ptr))
 #endif
 
+#ifndef atomic_set
+#define atomic_set(ptr, i)     ((*(__typeof__(*ptr) *volatile) (ptr)) = (i))
+#endif
+
+/* These have the same semantics as Java volatile variables.
+ * See http://gee.cs.oswego.edu/dl/jmm/cookbook.html:
+ * "1. Issue a StoreStore barrier (wmb) before each volatile store."
+ *  2. Issue a StoreLoad barrier after each volatile store.
+ *     Note that you could instead issue one before each volatile load, but
+ *     this would be slower for typical programs using volatiles in which
+ *     reads greatly outnumber writes. Alternatively, if available, you
+ *     can implement volatile store as an atomic instruction (for example
+ *     XCHG on x86) and omit the barrier. This may be more efficient if
+ *     atomic instructions are cheaper than StoreLoad barriers.
+ *  3. Issue LoadLoad and LoadStore barriers after each volatile load."
+ *
+ * If you prefer to think in terms of "pairing" of memory barriers,
+ * an atomic_mb_read pairs with an atomic_mb_set.
+ *
+ * And for the few ia64 lovers that exist, an atomic_mb_read is a ld.acq,
+ * while an atomic_mb_set is a st.rel followed by a memory barrier.
+ *
+ * These are a bit weaker than __atomic_load/store with __ATOMIC_SEQ_CST
+ * (see docs/atomics.txt), and I'm not sure that __ATOMIC_ACQ_REL is enough.
+ * Just always use the barriers manually by the rules above.
+ */
+#ifndef atomic_mb_read
+#define atomic_mb_read(ptr)    ({           \
+    typeof(*ptr) _val = atomic_read(ptr);   \
+    smp_rmb();                              \
+    _val;                                   \
+})
+#endif
+
+#ifndef atomic_mb_set
+#define atomic_mb_set(ptr, i)  do {         \
+    smp_wmb();                              \
+    atomic_set(ptr, i);                     \
+    smp_mb();                               \
+} while (0)
+#endif
+
+#ifndef atomic_xchg
+#ifdef __ATOMIC_SEQ_CST
+#define atomic_xchg(ptr, i)    ({                           \
+    typeof(*ptr) _new = (i), _old;                          \
+    __atomic_exchange(ptr, &_new, &_old, __ATOMIC_SEQ_CST); \
+    _old;                                                   \
+})
+#elif defined __clang__
+#define atomic_xchg(ptr, i)    __sync_exchange(ptr, i)
+#else
+/* __sync_lock_test_and_set() is documented to be an acquire barrier only.  */
+#define atomic_xchg(ptr, i)    (smp_mb(), __sync_lock_test_and_set(ptr, i))
+#endif
+#endif
+
+/* Provide shorter names for GCC atomic builtins.  */
+#define atomic_inc(ptr)        __sync_fetch_and_add(ptr, 1)
+#define atomic_dec(ptr)        __sync_fetch_and_add(ptr, -1)
+#define atomic_add             __sync_fetch_and_add
+#define atomic_sub             __sync_fetch_and_sub
+#define atomic_and             __sync_fetch_and_and
+#define atomic_or              __sync_fetch_and_or
+#define atomic_cmpxchg         __sync_val_compare_and_swap
+
 #endif
diff --git a/migration.c b/migration.c
index 058f9e6..83f5691 100644
--- a/migration.c
+++ b/migration.c
@@ -290,8 +290,7 @@ static void migrate_fd_cleanup(void *opaque)
 
 static void migrate_finish_set_state(MigrationState *s, int new_state)
 {
-    if (__sync_val_compare_and_swap(&s->state, MIG_STATE_ACTIVE,
-                                    new_state) == new_state) {
+    if (atomic_cmpxchg(&s->state, MIG_STATE_ACTIVE, new_state) == new_state) {
         trace_migrate_set_state(new_state);
     }
 }
diff --git a/tests/test-thread-pool.c b/tests/test-thread-pool.c
index 22915aa..dbf2fc8 100644
--- a/tests/test-thread-pool.c
+++ b/tests/test-thread-pool.c
@@ -17,15 +17,15 @@ typedef struct {
 static int worker_cb(void *opaque)
 {
     WorkerTestData *data = opaque;
-    return __sync_fetch_and_add(&data->n, 1);
+    return atomic_inc(&data->n);
 }
 
 static int long_cb(void *opaque)
 {
     WorkerTestData *data = opaque;
-    __sync_fetch_and_add(&data->n, 1);
+    atomic_inc(&data->n);
     g_usleep(2000000);
-    __sync_fetch_and_add(&data->n, 1);
+    atomic_inc(&data->n);
     return 0;
 }
 
@@ -169,7 +169,7 @@ static void test_cancel(void)
     /* Cancel the jobs that haven't been started yet.  */
     num_canceled = 0;
     for (i = 0; i < 100; i++) {
-        if (__sync_val_compare_and_swap(&data[i].n, 0, 3) == 0) {
+        if (atomic_cmpxchg(&data[i].n, 0, 3) == 0) {
             data[i].ret = -ECANCELED;
             bdrv_aio_cancel(data[i].aiocb);
             active--;
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (3 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 04/30] add a header file for atomic operations Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 20:43   ` Anthony Liguori
                     ` (2 more replies)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 06/30] qemu-thread: add TLS wrappers Paolo Bonzini
                   ` (24 subsequent siblings)
  29 siblings, 3 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

The next patch will change qemu/tls.h to support more platforms, but at
some performance cost.  Declare cpu_single_env directly instead of using
the tls.h abstractions.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c                 | 10 ++++++++--
 include/exec/cpu-all.h | 14 +++++++++++---
 include/qemu/tls.h     | 52 --------------------------------------------------
 3 files changed, 19 insertions(+), 57 deletions(-)
 delete mode 100644 include/qemu/tls.h

diff --git a/exec.c b/exec.c
index d28403b..a788981 100644
--- a/exec.c
+++ b/exec.c
@@ -70,9 +70,15 @@ static MemoryRegion io_mem_unassigned;
 #endif
 
 CPUArchState *first_cpu;
+
 /* current CPU in the current thread. It is only valid inside
-   cpu_exec() */
-DEFINE_TLS(CPUArchState *,cpu_single_env);
+ * cpu_exec().  See comment in include/exec/cpu-all.h.  */
+#if defined CONFIG_KVM || (defined CONFIG_USER_ONLY && defined CONFIG_USE_NPTL)
+__thread CPUArchState *cpu_single_env;
+#else
+CPUArchState *cpu_single_env;
+#endif
+
 /* 0 = Do not count executed instructions.
    1 = Precise instruction counting.
    2 = Adaptive rate instruction counting.  */
diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index e9c3717..2202ba3 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -20,7 +20,6 @@
 #define CPU_ALL_H
 
 #include "qemu-common.h"
-#include "qemu/tls.h"
 #include "exec/cpu-common.h"
 #include "qemu/thread.h"
 
@@ -368,8 +367,17 @@ void cpu_dump_statistics(CPUArchState *env, FILE *f, fprintf_function cpu_fprint
 void QEMU_NORETURN cpu_abort(CPUArchState *env, const char *fmt, ...)
     GCC_FMT_ATTR(2, 3);
 extern CPUArchState *first_cpu;
-DECLARE_TLS(CPUArchState *,cpu_single_env);
-#define cpu_single_env tls_var(cpu_single_env)
+
+/* This is thread-local depending on __linux__ because:
+ *  - the only -user mode supporting multiple VCPU threads is linux-user
+ *  - TCG system mode is single-threaded regarding VCPUs
+ *  - KVM system mode is multi-threaded but limited to Linux
+ */
+#if defined CONFIG_KVM || (defined CONFIG_USER_ONLY && defined CONFIG_USE_NPTL)
+extern __thread CPUArchState *cpu_single_env;
+#else
+extern CPUArchState *cpu_single_env;
+#endif
 
 /* Flags for use in ENV->INTERRUPT_PENDING.
 
diff --git a/include/qemu/tls.h b/include/qemu/tls.h
deleted file mode 100644
index b92ea9d..0000000
--- a/include/qemu/tls.h
+++ /dev/null
@@ -1,52 +0,0 @@
-/*
- * Abstraction layer for defining and using TLS variables
- *
- * Copyright (c) 2011 Red Hat, Inc
- * Copyright (c) 2011 Linaro Limited
- *
- * Authors:
- *  Paolo Bonzini <pbonzini@redhat.com>
- *  Peter Maydell <peter.maydell@linaro.org>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of
- * the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, see <http://www.gnu.org/licenses/>.
- */
-
-#ifndef QEMU_TLS_H
-#define QEMU_TLS_H
-
-/* Per-thread variables. Note that we only have implementations
- * which are really thread-local on Linux; the dummy implementations
- * define plain global variables.
- *
- * This means that for the moment use should be restricted to
- * per-VCPU variables, which are OK because:
- *  - the only -user mode supporting multiple VCPU threads is linux-user
- *  - TCG system mode is single-threaded regarding VCPUs
- *  - KVM system mode is multi-threaded but limited to Linux
- *
- * TODO: proper implementations via Win32 .tls sections and
- * POSIX pthread_getspecific.
- */
-#ifdef __linux__
-#define DECLARE_TLS(type, x) extern DEFINE_TLS(type, x)
-#define DEFINE_TLS(type, x)  __thread __typeof__(type) tls__##x
-#define tls_var(x)           tls__##x
-#else
-/* Dummy implementations which define plain global variables */
-#define DECLARE_TLS(type, x) extern DEFINE_TLS(type, x)
-#define DEFINE_TLS(type, x)  __typeof__(type) tls__##x
-#define tls_var(x)           tls__##x
-#endif
-
-#endif
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 06/30] qemu-thread: add TLS wrappers
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (4 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 07/30] qemu-thread: add QemuEvent Paolo Bonzini
                   ` (23 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Fast TLS is not available on some platforms, but it is always nice to
use it.  This wrapper implementation falls back to pthread_get/setspecific
on POSIX systems that lack __thread, but uses the dynamic linker's TLS
support on Linux and Windows.

The user shall call alloc_foo() in every thread that needs to access the
variable---exactly once and before any access.  foo is the name of the
variable as passed to DECLARE_TLS and DEFINE_TLS.  Then, get_foo() will
return the address of the variable.  It is guaranteed to remain the same
across the lifetime of a thread, so you can cache it.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                |  21 +++++++
 include/qemu/tls.h       | 139 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/Makefile           |   7 ++-
 tests/test-tls.c         |  87 +++++++++++++++++++++++++++++
 util/qemu-thread-win32.c |  17 ++++++
 5 files changed, 270 insertions(+), 1 deletion(-)
 create mode 100644 include/qemu/tls.h
 create mode 100644 tests/test-tls.c

diff --git a/configure b/configure
index 0e0adde..8ccdf2e 100755
--- a/configure
+++ b/configure
@@ -284,6 +284,7 @@ fi
 ar="${AR-${cross_prefix}ar}"
 as="${AS-${cross_prefix}as}"
 cpp="${CPP-$cc -E}"
+nm="${NM-${cross_prefix}nm}"
 objcopy="${OBJCOPY-${cross_prefix}objcopy}"
 ld="${LD-${cross_prefix}ld}"
 libtool="${LIBTOOL-${cross_prefix}libtool}"
@@ -3163,6 +3164,22 @@ if test "$trace_backend" = "dtrace"; then
 fi
 
 ##########################################
+# check for TLS runtime
+
+# Some versions of mingw include the "magic" definitions that make
+# TLS work, some don't.  Check for it.
+
+if test "$mingw32" = yes; then
+  cat > $TMPC << EOF
+int main(void) {}
+EOF
+  compile_prog "" ""
+  if $nm $TMPE | grep _tls_used > /dev/null 2>&1; then
+    mingw32_tls_runtime=yes
+  fi
+fi
+
+##########################################
 # check and set a backend for coroutine
 
 # We prefer ucontext, but it's not always possible. The fallback
@@ -3621,6 +3638,9 @@ if test "$mingw32" = "yes" ; then
   version_micro=0
   echo "CONFIG_FILEVERSION=$version_major,$version_minor,$version_subminor,$version_micro" >> $config_host_mak
   echo "CONFIG_PRODUCTVERSION=$version_major,$version_minor,$version_subminor,$version_micro" >> $config_host_mak
+  if test "$mingw32_tls_runtime" = yes; then
+    echo "CONFIG_MINGW32_TLS_RUNTIME=y" >> $config_host_mak
+  fi
 else
   echo "CONFIG_POSIX=y" >> $config_host_mak
 fi
@@ -4043,6 +4063,7 @@ echo "OBJCC=$objcc" >> $config_host_mak
 echo "AR=$ar" >> $config_host_mak
 echo "AS=$as" >> $config_host_mak
 echo "CPP=$cpp" >> $config_host_mak
+echo "NM=$nm" >> $config_host_mak
 echo "OBJCOPY=$objcopy" >> $config_host_mak
 echo "LD=$ld" >> $config_host_mak
 echo "WINDRES=$windres" >> $config_host_mak
diff --git a/include/qemu/tls.h b/include/qemu/tls.h
new file mode 100644
index 0000000..6a0f976
--- /dev/null
+++ b/include/qemu/tls.h
@@ -0,0 +1,139 @@
+/*
+ * Abstraction layer for defining and using TLS variables
+ *
+ * Copyright (c) 2011, 2013 Red Hat, Inc
+ * Copyright (c) 2011 Linaro Limited
+ *
+ * Authors:
+ *  Paolo Bonzini <pbonzini@redhat.com>
+ *  Peter Maydell <peter.maydell@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef QEMU_TLS_H
+#define QEMU_TLS_H
+
+#if defined(__linux__) || defined(__FreeBSD__)
+#define DECLARE_TLS(type, x)                     \
+extern __thread typeof(type) x;                  \
+                                                 \
+static inline typeof(type) *tls_get_##x(void)    \
+{                                                \
+    return &x;                                   \
+}                                                \
+                                                 \
+static inline typeof(type) *tls_alloc_##x(void)  \
+{                                                \
+    return &x;                                   \
+}                                                \
+                                                 \
+extern int dummy_##__LINE__
+
+#define DEFINE_TLS(type, x)                  \
+__thread typeof(type) x
+
+#elif defined CONFIG_POSIX
+typedef struct QEMUTLSValue {
+    pthread_key_t k;
+    pthread_once_t o;
+} QEMUTLSValue;
+
+#define DECLARE_TLS(type, x)                     \
+extern QEMUTLSValue x;                           \
+extern void init_##x(void);                      \
+                                                 \
+static inline typeof(type) *tls_get_##x(void)    \
+{                                                \
+    return pthread_getspecific(x.k);             \
+}                                                \
+                                                 \
+static inline typeof(type) *tls_alloc_##x(void)  \
+{                                                \
+    void *datum = g_malloc0(sizeof(type));       \
+    pthread_once(&x.o, tls_init_##x);            \
+    pthread_setspecific(x.k, datum);             \
+    return datum;                                \
+}                                                \
+                                                 \
+extern int dummy_##__LINE__
+
+#define DEFINE_TLS(type, x)                      \
+void tls_init_##x(void) {                        \
+    pthread_key_create(&x.k, g_free);            \
+}                                                \
+                                                 \
+QEMUTLSValue x = { .o = PTHREAD_ONCE_INIT }
+
+#elif defined CONFIG_WIN32
+
+/* The initial contents of TLS variables are placed in the .tls section.
+ * The linker takes all section starting with ".tls$", sorts them and puts
+ * the contents in a single ".tls" section.  qemu-thread-win32.c defines
+ * special symbols in .tls$000 and .tls$ZZZ that represent the beginning
+ * and end of TLS memory.  The linker and run-time library then cooperate
+ * to copy memory between those symbols in the TLS area of new threads.
+ *
+ * _tls_index holds the number of our module.  The executable should be
+ * zero, DLLs are numbered 1 and up.  The loader fills it in for us.
+ *
+ * Thus, Teb->ThreadLocalStoragePointer[_tls_index] is the base of
+ * the TLS segment for this (thread, module) pair.  Each segment has
+ * the same layout as this module's .tls segment and is initialized
+ * with the content of the .tls segment; 0 is the _tls_start variable.
+ * So, get_##x passes us the offset of the passed variable relative to
+ * _tls_start, and we return that same offset plus the base of segment.
+ */
+
+typedef struct _TEB {
+    NT_TIB NtTib;
+    void *EnvironmentPointer;
+    void *x[3];
+    char **ThreadLocalStoragePointer;
+} TEB, *PTEB;
+
+extern int _tls_index;
+extern int _tls_start;
+
+static inline void *tls_var(size_t offset)
+{
+    PTEB Teb = NtCurrentTeb();
+    return (char *)(Teb->ThreadLocalStoragePointer[_tls_index]) + offset;
+}
+
+#define DECLARE_TLS(type, x)                                         \
+extern typeof(type) tls_##x __attribute__((section(".tls$QEMU")));   \
+                                                                     \
+static inline typeof(type) *tls_get_##x(void)                        \
+{                                                                    \
+    return tls_var((ULONG_PTR)&(tls_##x) - (ULONG_PTR)&_tls_start);  \
+}                                                                    \
+                                                                     \
+static inline typeof(type) *tls_alloc_##x(void)                      \
+{                                                                    \
+    typeof(type) *addr = get_##x();                                  \
+    memset((void *)addr, 0, sizeof(type));                           \
+    return addr;                                                     \
+}                                                                    \
+                                                                     \
+extern int dummy_##__LINE__
+
+#define DEFINE_TLS(type, x)                                          \
+typeof(type) tls_##x __attribute__((section(".tls$QEMU")))
+
+#else
+#error No TLS abstraction available on this platform
+#endif
+
+#endif
diff --git a/tests/Makefile b/tests/Makefile
index 279d5f8..0400b39 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -44,6 +44,9 @@ check-unit-y += tests/test-cutils$(EXESUF)
 gcov-files-test-cutils-y += util/cutils.c
 check-unit-y += tests/test-mul64$(EXESUF)
 gcov-files-test-mul64-y = util/host-utils.c
+check-unit-y += tests/test-tls$(EXESUF)
+# all code tested by test-tls is inside tls.h
+gcov-files-test-tls-y =
 check-unit-y += tests/test-int128$(EXESUF)
 # all code tested by test-int128 is inside int128.h
 gcov-files-test-int128-y =
@@ -78,7 +81,8 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
 	tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
 	tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
 	tests/test-qmp-commands.o tests/test-visitor-serialization.o \
-	tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o
+	tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
+	tests/test-tls.o
 
 test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o
 
@@ -102,6 +106,7 @@ tests/test-x86-cpuid$(EXESUF): tests/test-x86-cpuid.o
 tests/test-xbzrle$(EXESUF): tests/test-xbzrle.o xbzrle.o page_cache.o libqemuutil.a
 tests/test-cutils$(EXESUF): tests/test-cutils.o util/cutils.o
 tests/test-int128$(EXESUF): tests/test-int128.o
+tests/test-tls$(EXESUF): tests/test-tls.o libqemuutil.a
 
 tests/test-qapi-types.c tests/test-qapi-types.h :\
 $(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py
diff --git a/tests/test-tls.c b/tests/test-tls.c
new file mode 100644
index 0000000..26a9ec7
--- /dev/null
+++ b/tests/test-tls.c
@@ -0,0 +1,87 @@
+/*
+ * Unit-tests for TLS wrappers
+ *
+ * Copyright (C) 2013 Red Hat Inc.
+ *
+ * Authors:
+ *  Paolo Bonzini <pbonzini@redhat.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include <glib.h>
+#include <errno.h>
+#include <string.h>
+
+#include "qemu-common.h"
+#include "qemu/atomic.h"
+#include "qemu/thread.h"
+#include "qemu/tls.h"
+
+DECLARE_TLS(volatile long long, cnt);
+DEFINE_TLS(volatile long long, cnt);
+
+#define NUM_THREADS 10
+
+int stop;
+
+static void *test_thread(void *arg)
+{
+    volatile long long *p_cnt = tls_alloc_cnt();
+    volatile long long **p_ret = arg;
+    long long exp = 0;
+
+    g_assert(tls_get_cnt() == p_cnt);
+    *p_ret = p_cnt;
+    g_assert(*p_cnt == 0);
+    while (atomic_mb_read(&stop) == 0) {
+        exp++;
+        (*p_cnt)++;
+        g_assert(*tls_get_cnt() == exp);
+    }
+
+    return NULL;
+}
+
+static void test_tls(void)
+{
+    volatile long long *addr[NUM_THREADS];
+    QemuThread t[NUM_THREADS];
+    int i;
+
+    for (i = 0; i < NUM_THREADS; i++) {
+        qemu_thread_create(&t[i], test_thread, &addr[i], QEMU_THREAD_JOINABLE);
+    }
+    g_usleep(1000000);
+    atomic_mb_set(&stop, 1);
+    for (i = 0; i < NUM_THREADS; i++) {
+        qemu_thread_join(&t[i]);
+    }
+    for (i = 1; i < NUM_THREADS; i++) {
+        g_assert(addr[i] != addr[i - 1]);
+    }
+}
+
+int main(int argc, char **argv)
+{
+    g_test_init(&argc, &argv, NULL);
+
+    g_test_add_func("/tls", test_tls);
+    return g_test_run();
+}
diff --git a/util/qemu-thread-win32.c b/util/qemu-thread-win32.c
index 517878d..f75e404 100644
--- a/util/qemu-thread-win32.c
+++ b/util/qemu-thread-win32.c
@@ -16,6 +16,23 @@
 #include <assert.h>
 #include <limits.h>
 
+/* TLS support.  Some versions of mingw32 provide it, others do not.  */
+
+#ifndef CONFIG_MINGW32_TLS_RUNTIME
+int __attribute__((section(".tls$AAA"))) _tls_start = 0;
+int __attribute__((section(".tls$ZZZ"))) _tls_end = 0;
+int _tls_index = 0;
+
+const IMAGE_TLS_DIRECTORY _tls_used __attribute__((used, section(".rdata$T"))) = {
+ (ULONG)(ULONG_PTR) &_tls_start, /* start of tls data */
+ (ULONG)(ULONG_PTR) &_tls_end,   /* end of tls data */
+ (ULONG)(ULONG_PTR) &_tls_index, /* address of tls_index */
+ (ULONG) 0,                      /* pointer to callbacks */
+ (ULONG) 0,                      /* size of tls zero fill */
+ (ULONG) 0                       /* characteristics */
+};
+#endif
+
 static void error_exit(int err, const char *msg)
 {
     char *pstr;
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 07/30] qemu-thread: add QemuEvent
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (5 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 06/30] qemu-thread: add TLS wrappers Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 08/30] rcu: add rcu library Paolo Bonzini
                   ` (22 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

This emulates Win32 manual-reset events using futexes or conditional
variables.  Typical ways to use them are with multi-producer,
single-consumer data structures, to test for a complex condition whose
elements come from different threads:

    for (;;) {
        qemu_event_reset(ev);
        ... test complex condition ...
        if (condition is true) {
            break;
        }
        qemu_event_wait(ev);
    }

Or more efficiently (but with some duplication):

    ... evaluate condition ...
    while (!condition) {
        qemu_event_reset(ev);
        ... evaluate condition ...
        if (!condition) {
            qemu_event_wait(ev);
            ... evaluate condition ...
        }
    }

QemuEvent provides a very fast userspace path in the common case when
no other thread is waiting, or the event is not changing state.  It
is used to report RCU quiescent states to the thread calling
synchronize_rcu (the latter being the single consumer), and to report
call_rcu invocations to the thread that receives them.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 include/qemu/thread-posix.h |   8 +++
 include/qemu/thread-win32.h |   4 ++
 include/qemu/thread.h       |   7 +++
 util/qemu-thread-posix.c    | 116 ++++++++++++++++++++++++++++++++++++++++++++
 util/qemu-thread-win32.c    |  26 ++++++++++
 5 files changed, 161 insertions(+)

diff --git a/include/qemu/thread-posix.h b/include/qemu/thread-posix.h
index 0f30dcc..916b2a7 100644
--- a/include/qemu/thread-posix.h
+++ b/include/qemu/thread-posix.h
@@ -21,6 +21,14 @@ struct QemuSemaphore {
 #endif
 };
 
+struct QemuEvent {
+#ifndef __linux__
+    pthread_mutex_t lock;
+    pthread_cond_t cond;
+#endif
+    unsigned value;
+};
+
 struct QemuThread {
     pthread_t thread;
 };
diff --git a/include/qemu/thread-win32.h b/include/qemu/thread-win32.h
index 13adb95..3d58081 100644
--- a/include/qemu/thread-win32.h
+++ b/include/qemu/thread-win32.h
@@ -17,6 +17,10 @@ struct QemuSemaphore {
     HANDLE sema;
 };
 
+struct QemuEvent {
+    HANDLE event;
+};
+
 typedef struct QemuThreadData QemuThreadData;
 struct QemuThread {
     QemuThreadData *data;
diff --git a/include/qemu/thread.h b/include/qemu/thread.h
index c02404b..3e32c65 100644
--- a/include/qemu/thread.h
+++ b/include/qemu/thread.h
@@ -7,6 +7,7 @@
 typedef struct QemuMutex QemuMutex;
 typedef struct QemuCond QemuCond;
 typedef struct QemuSemaphore QemuSemaphore;
+typedef struct QemuEvent QemuEvent;
 typedef struct QemuThread QemuThread;
 
 #ifdef _WIN32
@@ -45,6 +46,12 @@ void qemu_sem_wait(QemuSemaphore *sem);
 int qemu_sem_timedwait(QemuSemaphore *sem, int ms);
 void qemu_sem_destroy(QemuSemaphore *sem);
 
+void qemu_event_init(QemuEvent *ev, bool init);
+void qemu_event_set(QemuEvent *ev);
+void qemu_event_reset(QemuEvent *ev);
+void qemu_event_wait(QemuEvent *ev);
+void qemu_event_destroy(QemuEvent *ev);
+
 void qemu_thread_create(QemuThread *thread,
                         void *(*start_routine)(void *),
                         void *arg, int mode);
diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
index 4489abf..8178f9b 100644
--- a/util/qemu-thread-posix.c
+++ b/util/qemu-thread-posix.c
@@ -20,7 +20,12 @@
 #include <limits.h>
 #include <unistd.h>
 #include <sys/time.h>
+#ifdef __linux__
+#include <sys/syscall.h>
+#include <linux/futex.h>
+#endif
 #include "qemu/thread.h"
+#include "qemu/atomic.h"
 
 static void error_exit(int err, const char *msg)
 {
@@ -268,6 +273,117 @@ void qemu_sem_wait(QemuSemaphore *sem)
 #endif
 }
 
+#ifdef __linux__
+#define futex(...)              syscall(__NR_futex, __VA_ARGS__)
+
+static inline void futex_wake(QemuEvent *ev, int n)
+{
+    futex(ev, FUTEX_WAKE, n, NULL, NULL, 0);
+}
+
+static inline void futex_wait(QemuEvent *ev, unsigned val)
+{
+    futex(ev, FUTEX_WAIT, (int) val, NULL, NULL, 0);
+}
+#else
+static inline void futex_wake(QemuEvent *ev, int n)
+{
+    if (n == 1) {
+        pthread_cond_signal(&ev->cond);
+    } else {
+        pthread_cond_broadcast(&ev->cond);
+    }
+}
+
+static inline void futex_wait(QemuEvent *ev, unsigned val)
+{
+    pthread_mutex_lock(&ev->lock);
+    if (ev->value == val) {
+        pthread_cond_wait(&ev->cond, &ev->lock);
+    }
+    pthread_mutex_unlock(&ev->lock);
+}
+#endif
+
+/* Valid transitions:
+ * - free->set, when setting the event
+ * - busy->set, when setting the event, followed by futex_wake
+ * - set->free, when resetting the event
+ * - free->busy, when waiting
+ *
+ * set->busy does not happen (it can be observed from the outside but
+ * it really is set->free->busy).
+ *
+ * busy->free provably cannot happen; to enforce it, the set->free transition
+ * is done with an OR, which becomes a no-op if the event has concurrently
+ * transitioned to free or busy.
+ */
+
+#define EV_SET         0
+#define EV_FREE        1
+#define EV_BUSY       -1
+
+void qemu_event_init(QemuEvent *ev, bool init)
+{
+#ifndef __linux__
+    pthread_mutex_init(&ev->lock, NULL);
+    pthread_cond_init(&ev->cond, NULL);
+#endif
+
+    ev->value = (init ? EV_SET : EV_FREE);
+}
+
+void qemu_event_destroy(QemuEvent *ev)
+{
+#ifndef __linux__
+    pthread_mutex_destroy(&ev->lock);
+    pthread_cond_destroy(&ev->cond);
+#endif
+}
+
+void qemu_event_set(QemuEvent *ev)
+{
+    if (atomic_mb_read(&ev->value) != EV_SET) {
+        if (atomic_xchg(&ev->value, EV_SET) == EV_BUSY) {
+            /* There were waiters, wake them up.  */
+            futex_wake(ev, INT_MAX);
+        }
+    }
+}
+
+void qemu_event_reset(QemuEvent *ev)
+{
+    if (atomic_mb_read(&ev->value) == EV_SET) {
+        /*
+         * If there was a concurrent reset (or even reset+wait),
+         * do nothing.  Otherwise change EV_SET->EV_FREE.
+         */
+        atomic_or(&ev->value, EV_FREE);
+    }
+}
+
+void qemu_event_wait(QemuEvent *ev)
+{
+    unsigned value;
+
+    value = atomic_mb_read(&ev->value);
+    if (value != EV_SET) {
+        if (value == EV_FREE) {
+            /*
+             * Leave the event reset and tell qemu_event_set that there
+             * are waiters.  No need to retry, because there cannot be
+             * a concurent busy->free transition.  After the CAS, the
+             * event will be either set or busy.
+             */
+            if (atomic_cmpxchg(&ev->value, EV_FREE, EV_BUSY) == EV_SET) {
+                return;
+            }
+        }
+        futex_wait(ev, EV_BUSY);
+    }
+}
+
+
 void qemu_thread_create(QemuThread *thread,
                        void *(*start_routine)(void*),
                        void *arg, int mode)
diff --git a/util/qemu-thread-win32.c b/util/qemu-thread-win32.c
index f75e404..de49f1e 100644
--- a/util/qemu-thread-win32.c
+++ b/util/qemu-thread-win32.c
@@ -244,6 +244,32 @@ void qemu_sem_wait(QemuSemaphore *sem)
     }
 }
 
+void qemu_event_init(QemuEvent *ev, bool init)
+{
+    /* Manual reset.  */
+    ev->event = CreateEvent(NULL, TRUE, init, NULL);
+}
+
+void qemu_event_destroy(QemuEvent *ev)
+{
+    CloseHandle(ev->event);
+}
+
+void qemu_event_set(QemuEvent *ev)
+{
+    SetEvent(ev->event);
+}
+
+void qemu_event_reset(QemuEvent *ev)
+{
+    ResetEvent(ev->event);
+}
+
+void qemu_event_wait(QemuEvent *ev)
+{
+    WaitForSingleObject(ev->event, INFINITE);
+}
+
 struct QemuThreadData {
     /* Passed to win32_start_routine.  */
     void             *(*start_routine)(void *);
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 08/30] rcu: add rcu library
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (6 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 07/30] qemu-thread: add QemuEvent Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-07-01  9:47   ` Jan Kiszka
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 09/30] qemu-thread: register threads with RCU Paolo Bonzini
                   ` (21 subsequent siblings)
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

This includes a (mangled) copy of the urcu-qsbr code from liburcu.
The main changes are: 1) removing dependencies on many other header files
in liburcu; 2) removing for simplicity the tentative busy waiting in
synchronize_rcu, which has limited performance effects; 3) replacing
futexes in synchronize_rcu with QemuEvents for Win32 portability.
The API is the same as liburcu, so it should be possible in the future
to require liburcu on POSIX systems for example and use our copy only
on Windows.

Among the various versions available I chose urcu-qsbr, which has the
fastest rcu_read_{lock,unlock} but requires the program to manually
annotate quiescent points or intervals.  QEMU threads usually have easily
identified quiescent periods, so this should not be a problem.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/rcu.txt               | 303 +++++++++++++++++++++++++++++++++++++++++++++
 hw/9pfs/virtio-9p-synth.c  |   1 +
 include/qemu/queue.h       |  13 ++
 include/qemu/rcu-pointer.h | 110 ++++++++++++++++
 include/qemu/rcu.h         | 141 +++++++++++++++++++++
 include/qemu/thread.h      |   3 -
 libcacard/Makefile         |   3 +-
 util/Makefile.objs         |   1 +
 util/rcu.c                 | 195 +++++++++++++++++++++++++++++
 9 files changed, 766 insertions(+), 4 deletions(-)
 create mode 100644 docs/rcu.txt
 create mode 100644 include/qemu/rcu-pointer.h
 create mode 100644 include/qemu/rcu.h
 create mode 100644 util/rcu.c

diff --git a/docs/rcu.txt b/docs/rcu.txt
new file mode 100644
index 0000000..4869ec7
--- /dev/null
+++ b/docs/rcu.txt
@@ -0,0 +1,303 @@
+Using RCU (Read-Copy-Update) for synchronization
+================================================
+
+Read-copy update (RCU) is a synchronization mechanism that is used to
+protect read-mostly data structures.  RCU is very efficient and scalable
+on the read side (it is wait-free), and thus can make the read paths
+extremely fast.
+
+RCU supports concurrency between a single writer and multiple readers,
+thus it is not used alone.  Typically, the write-side will use a lock to
+serialize multiple updates, but other approaches are possible (e.g.,
+restricting updates to a single task).  In QEMU, when a lock is used,
+this will often be the "iothread mutex", also known as the "big QEMU
+lock" (BQL).  Also, restricting updates to a single task is done in
+QEMU using the "bottom half" API.
+
+RCU is fundamentally a "wait-to-finish" mechanism.  The read side marks
+sections of code with "critical sections", and the update side will wait
+for the execution of all *currently running* critical sections before
+proceeding, or before asynchronously executing a callback.
+
+The key point here is that only the currently running critical sections
+are waited for; critical sections that are started _after_ the beginning
+of the wait do not extend the wait, despite running concurrently with
+the updater.  This is the reason why RCU is more scalable than,
+for example, reader-writer locks.  It is so much more scalable that
+the system will have a single instance of the RCU mechanism; a single
+mechanism can be used for an arbitrary number of "things", without
+having to worry about things such as contention or deadlocks.
+
+How is this possible?  The basic idea is to split updates in two phases,
+"removal" and "reclamation".  During removal, we ensure that subsequent
+readers will not be able to get a reference to the old data.  After
+removal has completed, a critical section will not be able to access
+the old data.  Therefore, critical sections that begin after removal
+do not matter; as soon as all previous critical sections have finished,
+there cannot be any readers who hold references to the data structure,
+which may not be safely reclaimed (e.g., freed or unref'ed).
+
+Here is a picutre:
+
+        thread 1                  thread 2                  thread 3
+    -------------------    ------------------------    -------------------
+    enter RCU crit.sec.
+           |                finish removal phase
+           |                begin wait
+           |                      |                    enter RCU crit.sec.
+    exit RCU crit.sec             |                           |
+                            complete wait                     |
+                            begin reclamation phase           |
+                                                       exit RCU crit.sec.
+
+
+Note how thread 3 is still executing its critical section when thread 2
+starts reclaiming data.  This is possible, because the old version of the
+data structure was not accessible at the time thread 3 began executing
+that critical section.
+
+
+RCU API
+=======
+
+The core RCU API is small:
+
+     void rcu_read_lock(void);
+
+        Used by a reader to inform the reclaimer that the reader is
+        entering an RCU read-side critical section.
+
+     void rcu_read_unlock(void);
+
+        Used by a reader to inform the reclaimer that the reader is
+        exiting an RCU read-side critical section.  Note that RCU
+        read-side critical sections may be nested and/or overlapping.
+
+     void synchronize_rcu(void);
+
+        Blocks until all pre-existing RCU read-side critical sections
+        on all threads have completed.  This marks the end of the removal
+        phase and the beginning of reclamation phase.
+
+        Note that it would be valid for another update to come while
+        synchronize_rcu is running.  Because of this, it is better that
+        the updater releases any locks it may hold before calling
+        synchronize_rcu.
+
+     typeof(*p) rcu_dereference(p);
+     typeof(p) rcu_assign_pointer(p, typeof(p) v);
+
+        These macros are similar to atomic_mb_read() and atomic_mb_set()
+        respectively.  However, they make some assumptions on the code
+        that calls them, which allows a more optimized implementation.
+
+        rcu_assign_pointer assumes that the update side is not going
+        to read from the data structure after "publishing" the new
+        values; that is, it assumes that all assignments happen at
+        the very end of the removal phase.
+
+        rcu_dereference assumes that whenever a single RCU critical
+        section reads multiple shared data, these reads are either
+        data-dependent or need no ordering.  This is almost always the
+        case when using RCU.  If this were not the case, you can use
+        atomic_mb_read() or smp_rmb().
+
+        If you are going to be fetching multiple fields from the
+        RCU-protected structure, repeated rcu_dereference() calls
+        would look ugly and incur unnecessary overhead on Alpha CPUs.
+        You can then do this:
+
+        p = &rcu_dereference(head);
+        foo = head->foo;
+        bar = head->bar;
+
+
+RCU QUIESCENT STATES
+====================
+
+An efficient implementation of rcu_read_lock() and rcu_read_unlock()
+relies on the availability of fast thread-local storage.  Unfortunately,
+this is not possible on all the systems supported by QEMU (in particular
+on many POSIX systems other than Linux and Solaris).
+
+For this reason, QEMU's RCU implementation resorts to manual annotation
+of "quiescent states", i.e. points where no RCU read-side critical
+section can be active.  All threads that participate in the RCU mechanism
+need to annotate such points.
+
+Marking quiescent states is done with the following three APIs:
+
+     void rcu_quiescent_state(void);
+
+        Marks a point in the execution of the current thread where no
+        RCU read-side critical section can be active.
+
+     void rcu_thread_offline(void);
+
+        Marks the beginning of an "extended quiescent state" for the
+        current thread, i.e. an interval of time during which no
+        RCU read-side critical section can be active.
+
+     void rcu_thread_online(void);
+
+        Marks the end of an extended quiescent state for the current
+        thread.
+
+
+Furthermore, threads that participate in the RCU mechanism must communicate
+this fact using the following APIs:
+
+     void rcu_register_thread(void);
+
+        Mark a thread as taking part in the RCU mechanism.  Such a thread
+        will have to report quiescent points regularly, either manually
+        or through the QemuCond/QemuSemaphore/QemuEvent APIs.
+
+     void rcu_unregister_thread(void);
+
+        Mark a thread as not taking part anymore in the RCU mechanism.
+        It is not a problem if such a thread reports quiescent points,
+        either manually or by using the QemuCond/QemuSemaphore/QemuEvent
+        APIs.
+
+Note that these APIs are relatively heavyweight, and should _not_ be
+nested.
+
+
+DIFFERENCES WITH LINUX
+======================
+
+- Waiting on a mutex is possible, though discouraged, within an RCU critical
+  section.  This is because spinlocks are rarely (if ever) used in userspace
+  programming; not allowing this would prevent upgrading an RCU read-side
+  critical section to become an updater.
+
+- rcu_dereference takes a _pointer_ to the variable being accessed.
+  Wrong usage will be detected by the compiler.
+
+- Quiescent points must be marked explicitly in the thread.
+
+
+RCU PATTERNS
+============
+
+Many patterns using read-writer locks translate directly to RCU, with
+the advantages of higher scalability and deadlock immunity.
+
+In general, RCU can be used whenever it is possible to create a new
+"version" of a data structure every time the updater runs.  This may
+sound like a very strict restriction, however:
+
+- the updater does not mean "everything that writes to a data structure",
+  but rather "everything that involves a reclamation step".  See the
+  array example below
+
+- in some cases, creating a new version of a data structure may actually
+  be very cheap.  For example, modifying the "next" pointer of a singly
+  linked list is effectively creating a new version of the list.
+
+Here are some frequently-used RCU idioms that are worth noting.
+
+
+RCU list processing
+-------------------
+
+TBD (not yet used in QEMU)
+
+
+RCU reference counting
+----------------------
+
+Because grace periods are not allowed to complete while there is an RCU
+read-side critical section in progress, the RCU read-side primitives
+may be used as a restricted reference-counting mechanism.  For example,
+consider the following code fragment:
+
+    rcu_read_lock();
+    p = rcu_dereference(&foo);
+    /* do something with p. */
+    rcu_read_unlock();
+
+The RCU read-side critical section ensures that the value of "p" remains
+valid until after the rcu_read_unlock().  In some sense, it is acquiring
+a reference to p that is later released when the critical section ends.
+The write side looks simply like this (with appropriate locking):
+
+    qemu_mutex_lock(&foo_mutex);
+    old = foo;
+    rcu_assign_pointer(foo, new);
+    qemu_mutex_unlock(&foo_mutex);
+    synchronize_rcu();
+    free(old);
+
+Note that the same idiom would be possible with reader/writer
+locks:
+
+    read_lock(&foo_rwlock);         write_mutex_lock(&foo_rwlock);
+    p = foo;                        p = foo;
+    /* do something with p. */      foo = new;
+    read_unlock(&foo_rwlock);       free(p);
+                                    write_mutex_unlock(&foo_rwlock);
+                                    free(p);
+
+
+RCU resizable arrays
+--------------------
+
+Resizable arrays can be used with RCU.  The expensive RCU synchronization
+only needs to take place when the array is resized.  The two items to
+take care of are:
+
+- ensuring that the old version of the array is available between removal
+  and reclamation;
+
+- avoiding mismatches in the read side between the array data and the
+  array size.
+
+The first problem is avoided simply by not using realloc.  Instead,
+each resize will allocate a new array and copy the old data into it.
+The second problem would arise if the size and the data pointers were
+two members of a larger struct:
+
+    struct mystuff {
+        ...
+        int data_size;
+        int data_alloc;
+        T   *data;
+        ...
+    };
+
+Instead, we store the size of the array with the array itself:
+
+    struct arr {
+        int size;
+        int alloc;
+        T   data[];
+    };
+    struct arr *global_array;
+
+    read side:
+        rcu_read_lock();
+        struct arr *array = rcu_dereference(&global_array);
+        x = i < array->size ? array->data[i] : -1;
+        rcu_read_unlock();
+        return x;
+
+    write side (running under a lock):
+        if (global_array->size == global_array->alloc) {
+            /* Creating a new version.  */
+            new_array = g_malloc(sizeof(struct arr) +
+                                 global_array->alloc * 2 * sizeof(T));
+            new_array->size = global_array->size;
+            new_array->alloc = global_array->alloc * 2;
+            memcpy(new_array->data, global_array->data,
+                   global_array->alloc * sizeof(T));
+
+            /* Removal phase.  */
+            old_array = global_array;
+            rcu_assign_pointer(new_array->data, new_array);
+            synchronize_rcu();
+
+            /* Reclamation phase.  */
+            free(old_array);
+        }
diff --git a/hw/9pfs/virtio-9p-synth.c b/hw/9pfs/virtio-9p-synth.c
index 840e4eb..fdfea21 100644
--- a/hw/9pfs/virtio-9p-synth.c
+++ b/hw/9pfs/virtio-9p-synth.c
@@ -17,6 +17,7 @@
 #include "virtio-9p-xattr.h"
 #include "fsdev/qemu-fsdev.h"
 #include "virtio-9p-synth.h"
+#include "qemu/rcu.h"
 
 #include <sys/stat.h>
 
diff --git a/include/qemu/queue.h b/include/qemu/queue.h
index d433b90..847ddd1 100644
--- a/include/qemu/queue.h
+++ b/include/qemu/queue.h
@@ -104,6 +104,19 @@ struct {                                                                \
         (head)->lh_first = NULL;                                        \
 } while (/*CONSTCOND*/0)
 
+#define QLIST_SWAP(dstlist, srclist, field) do {                        \
+        void *tmplist;                                                  \
+        tmplist = (srclist)->lh_first;                                  \
+        (srclist)->lh_first = (dstlist)->lh_first;                      \
+        if ((srclist)->lh_first != NULL) {                              \
+            (srclist)->lh_first->field.le_prev = &(srclist)->lh_first;  \
+        }                                                               \
+        (dstlist)->lh_first = tmplist;                                  \
+        if ((dstlist)->lh_first != NULL) {                              \
+            (dstlist)->lh_first->field.le_prev = &(dstlist)->lh_first;  \
+        }                                                               \
+} while (/*CONSTCOND*/0)
+
 #define QLIST_INSERT_AFTER(listelm, elm, field) do {                    \
         if (((elm)->field.le_next = (listelm)->field.le_next) != NULL)  \
                 (listelm)->field.le_next->field.le_prev =               \
diff --git a/include/qemu/rcu-pointer.h b/include/qemu/rcu-pointer.h
new file mode 100644
index 0000000..0e6417c
--- /dev/null
+++ b/include/qemu/rcu-pointer.h
@@ -0,0 +1,110 @@
+#ifndef _URCU_POINTER_STATIC_H
+#define _URCU_POINTER_STATIC_H
+
+/*
+ * urcu-pointer-static.h
+ *
+ * Userspace RCU header. Operations on pointers.
+ *
+ * TO BE INCLUDED ONLY IN LGPL-COMPATIBLE CODE. See urcu-pointer.h for
+ * linking dynamically with the userspace rcu library.
+ *
+ * Copyright (c) 2009 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
+ * Copyright (c) 2009 Paul E. McKenney, IBM Corporation.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ *
+ * IBM's contributions to this file may be relicensed under LGPLv2 or later.
+ */
+
+#include "compiler.h"
+#include "qemu/atomic.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * rcu_dereference - reads (copy) a RCU-protected pointer to a local variable
+ * into a RCU read-side critical section. The pointer can later be safely
+ * dereferenced within the critical section.
+ *
+ * This ensures that the pointer copy is invariant thorough the whole critical
+ * section.
+ *
+ * Inserts memory barriers on architectures that require them (currently only
+ * Alpha) and documents which pointers are protected by RCU.
+ *
+ * The compiler memory barrier in atomic_read() ensures that value-speculative
+ * optimizations (e.g. VSS: Value Speculation Scheduling) does not perform the
+ * data read before the pointer read by speculating the value of the pointer.
+ * Correct ordering is ensured because the pointer is read as a volatile access.
+ * This acts as a global side-effect operation, which forbids reordering of
+ * dependent memory operations. Note that such concern about dependency-breaking
+ * optimizations will eventually be taken care of by the "memory_order_consume"
+ * addition to forthcoming C++ standard.
+ *
+ * Should match rcu_assign_pointer() or rcu_xchg_pointer().
+ */
+
+#define rcu_dereference(p)                      \
+        ({                                      \
+            typeof(p) _p1 = (p);                \
+            smp_read_barrier_depends();         \
+            *(_p1);                             \
+        })
+
+/**
+ * rcu_cmpxchg_pointer - same as rcu_set_pointer, but tests if the pointer
+ * is as expected by "old". If succeeds, returns the previous pointer to the
+ * data structure, which can be safely freed after waiting for a quiescent state
+ * using synchronize_rcu(). If fails (unexpected value), returns old (which
+ * should not be freed !).
+ */
+
+#define rcu_cmpxchg_pointer(p, old, _new)       \
+        ({                                      \
+            typeof(*p) _pold = (old);           \
+            typeof(*p) _pnew = (_new);          \
+            atomic_cmpxchg(p, _pold, _pnew);    \
+        })
+
+#define rcu_set_pointer(p, v)                   \
+        ({                                      \
+             typeof(*p) _pv = (v);              \
+             smp_wmb();                         \
+             atomic_set(p, _pv);                \
+        })
+
+/**
+ * rcu_assign_pointer - assign (publicize) a pointer to a new data structure
+ * meant to be read by RCU read-side critical sections. Returns the assigned
+ * value.
+ *
+ * Documents which pointers will be dereferenced by RCU read-side critical
+ * sections and adds the required memory barriers on architectures requiring
+ * them. It also makes sure the compiler does not reorder code initializing the
+ * data structure before its publication.
+ *
+ * Should match rcu_dereference().
+ */
+
+#define rcu_assign_pointer(p, v)    rcu_set_pointer(&(p), v)
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _URCU_POINTER_STATIC_H */
diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h
new file mode 100644
index 0000000..96841e3
--- /dev/null
+++ b/include/qemu/rcu.h
@@ -0,0 +1,141 @@
+#ifndef _URCU_QSBR_H
+#define _URCU_QSBR_H
+
+/*
+ * urcu-qsbr.h
+ *
+ * Userspace RCU QSBR header.
+ *
+ * LGPL-compatible code should include this header with :
+ *
+ * #define _LGPL_SOURCE
+ * #include <urcu.h>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ *
+ * IBM's contributions to this file may be relicensed under LGPLv2 or later.
+ */
+
+#include <stdlib.h>
+#include <assert.h>
+#include <limits.h>
+#include <unistd.h>
+#include <stdint.h>
+#include <stdbool.h>
+#include <glib.h>
+
+#include "qemu/compiler.h"
+#include "qemu/rcu-pointer.h"
+#include "qemu/thread.h"
+#include "qemu/tls.h"
+#include "qemu/queue.h"
+#include "qemu/atomic.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*
+ * Important !
+ *
+ * Each thread containing read-side critical sections must be registered
+ * with rcu_register_thread() before calling rcu_read_lock().
+ * rcu_unregister_thread() should be called before the thread exits.
+ */
+
+#ifdef DEBUG_RCU
+#define rcu_assert(args...)    assert(args)
+#else
+#define rcu_assert(args...)
+#endif
+
+#define RCU_GP_ONLINE     (1UL << 0)
+#define RCU_GP_CTR        (1UL << 1)
+
+/*
+ * Global quiescent period counter with low-order bits unused.
+ * Using a int rather than a char to eliminate false register dependencies
+ * causing stalls on some architectures.
+ */
+extern unsigned long rcu_gp_ctr;
+
+extern QemuEvent rcu_gp_event;
+
+struct rcu_reader_data {
+    /* Data used by both reader and synchronize_rcu() */
+    unsigned long ctr;
+    bool waiting;
+
+    /* Data used for registry, protected by rcu_gp_lock */
+    QLIST_ENTRY(rcu_reader_data) node;
+};
+
+DECLARE_TLS(struct rcu_reader_data, rcu_reader);
+
+static inline void rcu_read_lock(void)
+{
+    rcu_assert(tls_get_rcu_reader()->ctr);
+}
+
+static inline void rcu_read_unlock(void)
+{
+    /* Ensure that the previous reads complete before starting those
+     * in another critical section.
+     */
+    smp_rmb();
+}
+
+static inline void rcu_quiescent_state(void)
+{
+    struct rcu_reader_data *p_rcu_reader = tls_get_rcu_reader();
+
+    /* Reuses smp_rmb() in the last rcu_read_unlock().  */
+    unsigned ctr = atomic_read(&rcu_gp_ctr);
+    atomic_xchg(&p_rcu_reader->ctr, ctr);
+    if (atomic_read(&p_rcu_reader->waiting)) {
+        atomic_set(&p_rcu_reader->waiting, false);
+        qemu_event_set(&rcu_gp_event);
+    }
+}
+
+static inline void rcu_thread_offline(void)
+{
+    struct rcu_reader_data *p_rcu_reader = tls_get_rcu_reader();
+
+    atomic_xchg(&p_rcu_reader->ctr, 0);
+    if (atomic_read(&p_rcu_reader->waiting)) {
+        atomic_set(&p_rcu_reader->waiting, false);
+        qemu_event_set(&rcu_gp_event);
+    }
+}
+
+static inline void rcu_thread_online(void)
+{
+    rcu_quiescent_state();
+}
+
+extern void synchronize_rcu(void);
+
+/*
+ * Reader thread registration.
+ */
+extern void rcu_register_thread(void);
+extern void rcu_unregister_thread(void);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _URCU_QSBR_H */
diff --git a/include/qemu/thread.h b/include/qemu/thread.h
index 3e32c65..5d64a20 100644
--- a/include/qemu/thread.h
+++ b/include/qemu/thread.h
@@ -25,9 +25,6 @@ void qemu_mutex_lock(QemuMutex *mutex);
 int qemu_mutex_trylock(QemuMutex *mutex);
 void qemu_mutex_unlock(QemuMutex *mutex);
 
-#define rcu_read_lock() do { } while (0)
-#define rcu_read_unlock() do { } while (0)
-
 void qemu_cond_init(QemuCond *cond);
 void qemu_cond_destroy(QemuCond *cond);
 
diff --git a/libcacard/Makefile b/libcacard/Makefile
index 47827a0..f7a3b07 100644
--- a/libcacard/Makefile
+++ b/libcacard/Makefile
@@ -4,7 +4,8 @@ TOOLS += vscclient$(EXESUF)
 
 # objects linked into a shared library, built with libtool with -fPIC if required
 libcacard-obj-y = $(stub-obj-y) $(libcacard-y)
-libcacard-obj-y += util/osdep.o util/cutils.o util/qemu-timer-common.o util/error.o
+libcacard-obj-y += util/osdep.o util/cutils.o util/qemu-timer-common.o
+libcacard-obj-y += util/rcu.o util/error.o
 libcacard-obj-$(CONFIG_WIN32) += util/oslib-win32.o util/qemu-thread-win32.o
 libcacard-obj-$(CONFIG_POSIX) += util/oslib-posix.o util/qemu-thread-posix.o
 libcacard-obj-y += $(filter trace/%, $(util-obj-y))
diff --git a/util/Makefile.objs b/util/Makefile.objs
index dc72ab0..c34e5ee 100644
--- a/util/Makefile.objs
+++ b/util/Makefile.objs
@@ -11,3 +11,4 @@ util-obj-y += iov.o aes.o qemu-config.o qemu-sockets.o uri.o notify.o
 util-obj-y += qemu-option.o qemu-progress.o
 util-obj-y += hexdump.o
 util-obj-y += crc32c.o
+util-obj-y += rcu.o
diff --git a/util/rcu.c b/util/rcu.c
new file mode 100644
index 0000000..e1498db
--- /dev/null
+++ b/util/rcu.c
@@ -0,0 +1,195 @@
+/*
+ * urcu-qsbr.c
+ *
+ * Userspace RCU QSBR library
+ *
+ * Copyright (c) 2009 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
+ * Copyright (c) 2009 Paul E. McKenney, IBM Corporation.
+ * Copyright 2013 Red Hat, Inc.
+ *
+ * Ported to QEMU by Paolo Bonzini  <pbonzini@redhat.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ *
+ * IBM's contributions to this file may be relicensed under LGPLv2 or later.
+ */
+
+#include <stdio.h>
+#include <assert.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <errno.h>
+#include "qemu/rcu.h"
+#include "qemu/atomic.h"
+
+/*
+ * Global grace period counter.  Bit 0 is one if the thread is online.
+ * Bits 1 and above are defined in synchronize_rcu/update_counter_and_wait.
+ */
+#define RCU_GP_ONLINE           (1UL << 0)
+#define RCU_GP_CTR              (1UL << 1)
+
+unsigned long rcu_gp_ctr = RCU_GP_ONLINE;
+
+QemuEvent rcu_gp_event;
+static QemuMutex rcu_gp_lock;
+
+/*
+ * Check whether a quiescent state was crossed between the beginning of
+ * update_counter_and_wait and now.
+ */
+static inline int rcu_gp_ongoing(unsigned long *ctr)
+{
+    unsigned long v;
+
+    /* See update_counter_and_wait for the discussion of memory barriers.  */
+    v = atomic_read(ctr);
+    return v && (v != rcu_gp_ctr);
+}
+
+/* Written to only by each individual reader. Read by both the reader and the
+ * writers.
+ */
+DEFINE_TLS(struct rcu_reader_data, rcu_reader);
+
+/* Protected by rcu_gp_lock.  */
+typedef QLIST_HEAD(, rcu_reader_data) ThreadList;
+static ThreadList registry = QLIST_HEAD_INITIALIZER(registry);
+
+/* Wait for previous parity/grace period to be empty of readers.  */
+static void wait_for_readers(void)
+{
+    ThreadList qsreaders = QLIST_HEAD_INITIALIZER(qsreaders);
+    struct rcu_reader_data *index, *tmp;
+
+    for (;;) {
+        /* We want to be notified of changes made to rcu_gp_ongoing
+         * while we walk the list.
+         */
+        qemu_event_reset(&rcu_gp_event);
+
+        /* Instead of using atomic_mb_set for index->waiting, and
+         * atomic_mb_read for index->ctr, memory barriers are placed
+         * manually since writes to different threads are independent.
+         * atomic_mb_set has a smp_wmb before...
+         */
+        smp_wmb();
+        QLIST_FOREACH(index, &registry, node) {
+            atomic_set(&index->waiting, true);
+        }
+
+        /* ... and a smp_mb after.  */
+        smp_mb();
+
+        QLIST_FOREACH_SAFE(index, &registry, node, tmp) {
+            if (!rcu_gp_ongoing(&index->ctr)) {
+                QLIST_REMOVE(index, node);
+                QLIST_INSERT_HEAD(&qsreaders, index, node);
+
+                /* No need for mb_set here, worst of all we
+                 * get some extra futex wakeups.
+                 */
+                atomic_set(&index->waiting, false);
+            }
+        }
+
+        /* atomic_mb_read has smp_rmb after.  */
+        smp_rmb();
+
+        if (QLIST_EMPTY(&registry)) {
+            break;
+        }
+
+        /* Wait for one thread to report a quiescent state and
+         * try again.
+         */
+        qemu_event_wait(&rcu_gp_event);
+    }
+
+    /* put back the reader list in the registry */
+    QLIST_SWAP(&registry, &qsreaders, node);
+}
+
+void synchronize_rcu(void)
+{
+    unsigned long was_online;
+
+    was_online = tls_get_rcu_reader()->ctr;
+
+    /* Mark the writer thread offline to make sure we don't wait for
+     * our own quiescent state. This allows using synchronize_rcu()
+     * in threads registered as readers.
+     */
+    if (was_online) {
+        rcu_thread_offline();
+    }
+
+    qemu_mutex_lock(&rcu_gp_lock);
+
+    if (!QLIST_EMPTY(&registry)) {
+        /* In either case, the atomic_mb_set below blocks stores that free
+         * old RCU-protected pointers.
+         */
+        if (sizeof(rcu_gp_ctr) < 8) {
+            /* For architectures with 32-bit longs, a two-subphases algorithm
+             * ensures we do not encounter overflow bugs.
+             *
+             * Switch parity: 0 -> 1, 1 -> 0.
+             */
+            atomic_mb_set(&rcu_gp_ctr, rcu_gp_ctr ^ RCU_GP_CTR);
+            wait_for_readers();
+            atomic_mb_set(&rcu_gp_ctr, rcu_gp_ctr ^ RCU_GP_CTR);
+        } else {
+            /* Increment current grace period.  */
+            atomic_mb_set(&rcu_gp_ctr, rcu_gp_ctr + RCU_GP_CTR);
+        }
+
+        wait_for_readers();
+    }
+
+    qemu_mutex_unlock(&rcu_gp_lock);
+
+    if (was_online) {
+        rcu_thread_online();
+    }
+}
+
+void rcu_register_thread(void)
+{
+    if (!tls_get_rcu_reader()) {
+        tls_alloc_rcu_reader();
+    }
+
+    assert(tls_get_rcu_reader()->ctr == 0);
+    qemu_mutex_lock(&rcu_gp_lock);
+    QLIST_INSERT_HEAD(&registry, tls_get_rcu_reader(), node);
+    qemu_mutex_unlock(&rcu_gp_lock);
+    rcu_quiescent_state();
+}
+
+void rcu_unregister_thread(void)
+{
+    rcu_thread_offline();
+    qemu_mutex_lock(&rcu_gp_lock);
+    QLIST_REMOVE(tls_get_rcu_reader(), node);
+    qemu_mutex_unlock(&rcu_gp_lock);
+}
+
+static void __attribute__((__constructor__)) rcu_init(void)
+{
+    qemu_mutex_init(&rcu_gp_lock);
+    qemu_event_init(&rcu_gp_event, true);
+    rcu_register_thread();
+}
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 09/30] qemu-thread: register threads with RCU
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (7 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 08/30] rcu: add rcu library Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 10/30] rcu: add call_rcu Paolo Bonzini
                   ` (20 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/rcu.txt             | 13 +++++++------
 util/qemu-thread-posix.c | 28 +++++++++++++++++++++++++++-
 util/qemu-thread-win32.c |  2 ++
 3 files changed, 36 insertions(+), 7 deletions(-)

diff --git a/docs/rcu.txt b/docs/rcu.txt
index 4869ec7..118a28a 100644
--- a/docs/rcu.txt
+++ b/docs/rcu.txt
@@ -122,8 +122,8 @@ on many POSIX systems other than Linux and Solaris).
 
 For this reason, QEMU's RCU implementation resorts to manual annotation
 of "quiescent states", i.e. points where no RCU read-side critical
-section can be active.  All threads that participate in the RCU mechanism
-need to annotate such points.
+section can be active.  All threads created with qemu_thread_create
+participate in the RCU mechanism and need to annotate such points.
 
 Marking quiescent states is done with the following three APIs:
 
@@ -144,8 +144,8 @@ Marking quiescent states is done with the following three APIs:
         thread.
 
 
-Furthermore, threads that participate in the RCU mechanism must communicate
-this fact using the following APIs:
+The following APIs can be used to use RCU in a thread that is not
+created with qemu_thread_create():
 
      void rcu_register_thread(void);
 
@@ -160,8 +160,9 @@ this fact using the following APIs:
         either manually or by using the QemuCond/QemuSemaphore/QemuEvent
         APIs.
 
-Note that these APIs are relatively heavyweight, and should _not_ be
-nested.
+Note that these APIs are relatively heavyweight, should _not_ be
+nested, and should not be called in threads that are created with
+qemu_thread_create().
 
 
 DIFFERENCES WITH LINUX
diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
index 8178f9b..2df3382 100644
--- a/util/qemu-thread-posix.c
+++ b/util/qemu-thread-posix.c
@@ -26,6 +26,7 @@
 #endif
 #include "qemu/thread.h"
 #include "qemu/atomic.h"
+#include "qemu/rcu.h"
 
 static void error_exit(int err, const char *msg)
 {
@@ -384,6 +385,26 @@ void qemu_event_wait(QemuEvent *ev)
 }
 
 
+typedef struct QemuThreadData {
+    /* Passed to win32_start_routine.  */
+    void             *(*start_routine)(void *);
+    void             *arg;
+} QemuThreadData;
+
+static void *thread_start_routine(void *arg)
+{
+    QemuThreadData *data = (QemuThreadData *) arg;
+    void *(*start_routine)(void *) = data->start_routine;
+    void *thread_arg = data->arg;
+    void *ret;
+
+    rcu_register_thread();
+    g_free(data);
+    ret = start_routine(thread_arg);
+    rcu_unregister_thread();
+    return ret;
+}
+
 void qemu_thread_create(QemuThread *thread,
                        void *(*start_routine)(void*),
                        void *arg, int mode)
@@ -391,6 +412,11 @@ void qemu_thread_create(QemuThread *thread,
     sigset_t set, oldset;
     int err;
     pthread_attr_t attr;
+    QemuThreadData *data;
+
+    data = g_malloc(sizeof(*data));
+    data->start_routine = start_routine;
+    data->arg = arg;
 
     err = pthread_attr_init(&attr);
     if (err) {
@@ -406,7 +432,7 @@ void qemu_thread_create(QemuThread *thread,
     /* Leave signal handling to the iothread.  */
     sigfillset(&set);
     pthread_sigmask(SIG_SETMASK, &set, &oldset);
-    err = pthread_create(&thread->thread, &attr, start_routine, arg);
+    err = pthread_create(&thread->thread, &attr, thread_start_routine, data);
     if (err)
         error_exit(err, __func__);
 
diff --git a/util/qemu-thread-win32.c b/util/qemu-thread-win32.c
index de49f1e..18978be 100644
--- a/util/qemu-thread-win32.c
+++ b/util/qemu-thread-win32.c
@@ -295,6 +295,7 @@ static unsigned __stdcall win32_start_routine(void *arg)
         data = NULL;
     }
     qemu_thread_data = data;
+    rcu_register_thread();
     qemu_thread_exit(start_routine(thread_arg));
     abort();
 }
@@ -310,6 +311,7 @@ void qemu_thread_exit(void *arg)
         data->exited = true;
         LeaveCriticalSection(&data->cs);
     }
+    rcu_unregister_thread();
     _endthreadex(0);
 }
 
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 10/30] rcu: add call_rcu
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (8 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 09/30] qemu-thread: register threads with RCU Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 11/30] rcu: add rcutorture Paolo Bonzini
                   ` (19 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/rcu.txt       | 108 +++++++++++++++++++++++++++++++++++++++++++++--
 include/qemu/rcu.h |  22 ++++++++++
 util/rcu.c         | 120 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 246 insertions(+), 4 deletions(-)

diff --git a/docs/rcu.txt b/docs/rcu.txt
index 118a28a..ca1c099 100644
--- a/docs/rcu.txt
+++ b/docs/rcu.txt
@@ -82,7 +82,50 @@ The core RCU API is small:
         Note that it would be valid for another update to come while
         synchronize_rcu is running.  Because of this, it is better that
         the updater releases any locks it may hold before calling
-        synchronize_rcu.
+        synchronize_rcu.  If this is not possible (for example, because
+        the updater is protected by the BQL), you can use call_rcu.
+
+     void call_rcu1(struct rcu_head * head,
+                    void (*func)(struct rcu_head *head));
+
+        This function invokes func(head) after all pre-existing RCU
+        read-side critical sections on all threads have completed.  This
+        marks the end of the removal phase, with func taking care
+        asynchronously of the reclamation phase.
+
+        The foo struct needs to have an rcu_head structure added,
+        perhaps as follows:
+
+            struct foo {
+                struct rcu_head rcu;
+                int a;
+                char b;
+                long c;
+            };
+
+        so that the reclaimer function can fetch the struct foo address
+        and free it:
+
+            call_rcu1(foo_reclaim, &foo.rcu);
+
+            void foo_reclaim(struct rcu_head *rp)
+            {
+                struct foo *fp = container_of(rp, struct foo, rcu);
+                g_free(fp);
+            }
+
+        For the common case where the rcu_head member is the first of the
+        struct, you can use the following macro.
+
+     void call_rcu(T *p,
+                   void (*func)(T *p),
+                   field-name);
+
+        call_rcu1 is typically used through this macro, in the common case
+        where the "struct rcu_head" is the first field in the struct.  In
+        the above case, one could have written simply:
+
+            call_rcu(foo_reclaim, g_free, rcu);
 
      typeof(*p) rcu_dereference(p);
      typeof(p) rcu_assign_pointer(p, typeof(p) v);
@@ -176,6 +219,11 @@ DIFFERENCES WITH LINUX
 - rcu_dereference takes a _pointer_ to the variable being accessed.
   Wrong usage will be detected by the compiler.
 
+- call_rcu is a macro that has an extra argument (the name of the first
+  field in the struct, which must be a struct rcu_head), and expects the
+  type of the callback's argument to be the type of the first argument.
+  call_rcu1 is the same as Linux's call_rcu.
+
 - Quiescent points must be marked explicitly in the thread.
 
 
@@ -231,7 +279,47 @@ The write side looks simply like this (with appropriate locking):
     synchronize_rcu();
     free(old);
 
-Note that the same idiom would be possible with reader/writer
+If the processing cannot be done purely within the critical section, it
+is possible to combine this idiom with a "real" reference count:
+
+    rcu_read_lock();
+    p = rcu_dereference(&foo);
+    foo_ref(p);
+    rcu_read_unlock();
+    /* do something with p. */
+    foo_unref(p);
+
+The write side can be like this:
+
+    qemu_mutex_lock(&foo_mutex);
+    old = foo;
+    rcu_assign_pointer(foo, new);
+    qemu_mutex_unlock(&foo_mutex);
+    synchronize_rcu();
+    foo_unref(old);
+
+or with call_rcu:
+
+    qemu_mutex_lock(&foo_mutex);
+    old = foo;
+    rcu_assign_pointer(foo, new);
+    qemu_mutex_unlock(&foo_mutex);
+    call_rcu(foo_unref, old, rcu);
+
+In both cases, the write side only performs removal.  Reclamation
+happens when the last reference to a "foo" object is dropped.
+Using synchronize_rcu() is undesirably expensive, because the
+last reference may be dropped on the read side.  Hence you can
+use call_rcu() instead:
+
+     foo_unref(struct foo *p) {
+        if (atomic_dec(&p->refcount) == 0) {
+            call_rcu(foo_destroy, p, rcu);
+        }
+    }
+
+
+Note that the same idioms would be possible with reader/writer
 locks:
 
     read_lock(&foo_rwlock);         write_mutex_lock(&foo_rwlock);
@@ -241,13 +329,25 @@ locks:
                                     write_mutex_unlock(&foo_rwlock);
                                     free(p);
 
+    ------------------------------------------------------------------
+
+    read_lock(&foo_rwlock);         write_mutex_lock(&foo_rwlock);
+    p = foo;                        old = foo;
+    foo_ref(p);                     foo = new;
+    read_unlock(&foo_rwlock);       write_mutex_unlock(&foo_rwlock);
+    /* do something with p. */      foo_unref(old);
+    foo_unref(p);
+
+foo_unref could use a mechanism such as bottom halves to move deallocation
+out of hot paths.
+
 
 RCU resizable arrays
 --------------------
 
 Resizable arrays can be used with RCU.  The expensive RCU synchronization
-only needs to take place when the array is resized.  The two items to
-take care of are:
+(or call_rcu) only needs to take place when the array is resized.
+The two items to take care of are:
 
 - ensuring that the old version of the array is available between removal
   and reclamation;
diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h
index 96841e3..1ed70bb 100644
--- a/include/qemu/rcu.h
+++ b/include/qemu/rcu.h
@@ -134,6 +134,28 @@ extern void synchronize_rcu(void);
 extern void rcu_register_thread(void);
 extern void rcu_unregister_thread(void);
 
+struct rcu_head;
+typedef void RCUCBFunc(struct rcu_head *head);
+
+struct rcu_head {
+    struct rcu_head *next;
+    RCUCBFunc *func;
+};
+
+extern void call_rcu1(struct rcu_head *head, RCUCBFunc *func);
+
+/* The operands of the minus operator must have the same type,
+ * which must be the one that we specify in the cast.
+ */
+#define call_rcu(head, func, field)                                      \
+    call_rcu1(({                                                         \
+         char __attribute__((unused))                                    \
+            offset_must_be_zero[-offsetof(typeof(*(head)), field)],      \
+            func_type_invalid = (func) - (void (*)(typeof(head)))(func); \
+         &(head)->field;                                                 \
+      }),                                                                \
+      (RCUCBFunc *)(func))
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/util/rcu.c b/util/rcu.c
index e1498db..f1c5736 100644
--- a/util/rcu.c
+++ b/util/rcu.c
@@ -26,6 +26,7 @@
  * IBM's contributions to this file may be relicensed under LGPLv2 or later.
  */
 
+#include "qemu-common.h"
 #include <stdio.h>
 #include <assert.h>
 #include <stdlib.h>
@@ -33,6 +34,7 @@
 #include <errno.h>
 #include "qemu/rcu.h"
 #include "qemu/atomic.h"
+#include "qemu/thread.h"
 
 /*
  * Global grace period counter.  Bit 0 is one if the thread is online.
@@ -166,6 +168,119 @@ void synchronize_rcu(void)
     }
 }
 
+
+#define RCU_CALL_MIN_SIZE        30
+
+/* Multi-producer, single-consumer queue based on urcu/static/wfqueue.h
+ * from liburcu.  Note that head is only used by the consumer.
+ */
+static struct rcu_head dummy;
+static struct rcu_head *head = &dummy, **tail = &dummy.next;
+static int rcu_call_count;
+static QemuEvent rcu_call_ready_event;
+
+static void enqueue(struct rcu_head *node)
+{
+    struct rcu_head **old_tail;
+
+    node->next = NULL;
+    old_tail = atomic_xchg(&tail, &node->next);
+    atomic_mb_set(old_tail, node);
+}
+
+static struct rcu_head *try_dequeue(void)
+{
+    struct rcu_head *node, *next;
+
+retry:
+    /* Test for an empty list, which we do not expect.  Note that for
+     * the consumer head and tail are always consistent.  The head
+     * is consistent because only the consumer reads/writes it.
+     * The tail, because it is the first step in the enqueuing.
+     * It is only the next pointers that might be inconsistent.
+     */
+    if (head == &dummy && atomic_mb_read(&tail) == &dummy.next) {
+        abort();
+    }
+
+    /* If the head node has NULL in its next pointer, the value is
+     * wrong and we need to wait until its enqueuer finishes the update.
+     */
+    node = head;
+    next = atomic_mb_read(&head->next);
+    if (!next) {
+        return NULL;
+    }
+
+    /* Since we are the sole consumer, and we excluded the empty case
+     * above, the queue will always have at least two nodes: the
+     * dummy node, and the one being removed.  So we do not need to update
+     * the tail pointer.
+     */
+    head = next;
+
+    /* If we dequeued the dummy node, add it back at the end and retry.  */
+    if (node == &dummy) {
+        enqueue(node);
+        goto retry;
+    }
+
+    return node;
+}
+
+static void *call_rcu_thread(void *opaque)
+{
+    struct rcu_head *node;
+
+    /* This thread is just a writer.  */
+    rcu_thread_offline();
+
+    for (;;) {
+        int tries = 0;
+        int n = atomic_read(&rcu_call_count);
+
+        /* Heuristically wait for a decent number of callbacks to pile up.
+         * Fetch rcu_call_count now, we only must process elements that were
+         * added before synchronize_rcu() starts.
+         */
+        while (n < RCU_CALL_MIN_SIZE && ++tries <= 5) {
+            g_usleep(100000);
+            qemu_event_reset(&rcu_call_ready_event);
+            n = atomic_read(&rcu_call_count);
+            if (n < RCU_CALL_MIN_SIZE) {
+                qemu_event_wait(&rcu_call_ready_event);
+                n = atomic_read(&rcu_call_count);
+            }
+        }
+
+        atomic_sub(&rcu_call_count, n);
+        synchronize_rcu();
+        while (n > 0) {
+            node = try_dequeue();
+            while (!node) {
+                qemu_event_reset(&rcu_call_ready_event);
+                node = try_dequeue();
+                if (!node) {
+                    qemu_event_wait(&rcu_call_ready_event);
+                    node = try_dequeue();
+                }
+            }
+
+            n--;
+            node->func(node);
+        }
+    }
+    abort();
+}
+
+void call_rcu1(struct rcu_head *node, void (*func)(struct rcu_head *node))
+{
+    node->func = func;
+    enqueue(node);
+    atomic_inc(&rcu_call_count);
+    qemu_event_set(&rcu_call_ready_event);
+}
+
 void rcu_register_thread(void)
 {
     if (!tls_get_rcu_reader()) {
@@ -189,7 +304,12 @@ void rcu_unregister_thread(void)
 
 static void __attribute__((__constructor__)) rcu_init(void)
 {
+    QemuThread thread;
+
     qemu_mutex_init(&rcu_gp_lock);
     qemu_event_init(&rcu_gp_event, true);
+
+    qemu_event_init(&rcu_call_ready_event, false);
+    qemu_thread_create(&thread, call_rcu_thread, NULL, QEMU_THREAD_DETACHED);
     rcu_register_thread();
 }
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 11/30] rcu: add rcutorture
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (9 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 10/30] rcu: add call_rcu Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 12/30] rcu: allow nested calls to rcu_thread_offline/rcu_thread_online Paolo Bonzini
                   ` (18 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile     |   5 +-
 tests/rcutorture.c | 439 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 443 insertions(+), 1 deletion(-)
 create mode 100644 tests/rcutorture.c

diff --git a/tests/Makefile b/tests/Makefile
index 0400b39..6a81b94 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -50,6 +50,8 @@ gcov-files-test-tls-y =
 check-unit-y += tests/test-int128$(EXESUF)
 # all code tested by test-int128 is inside int128.h
 gcov-files-test-int128-y =
+check-unit-y += tests/rcutorture$(EXESUF)
+gcov-files-rcutorture-y = util/rcu.c
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -82,7 +84,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
 	tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
 	tests/test-qmp-commands.o tests/test-visitor-serialization.o \
 	tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
-	tests/test-tls.o
+	tests/test-tls.o tests/rcutorture.o
 
 test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o
 
@@ -107,6 +109,7 @@ tests/test-xbzrle$(EXESUF): tests/test-xbzrle.o xbzrle.o page_cache.o libqemuuti
 tests/test-cutils$(EXESUF): tests/test-cutils.o util/cutils.o
 tests/test-int128$(EXESUF): tests/test-int128.o
 tests/test-tls$(EXESUF): tests/test-tls.o libqemuutil.a
+tests/rcutorture$(EXESUF): tests/rcutorture.o libqemuutil.a
 
 tests/test-qapi-types.c tests/test-qapi-types.h :\
 $(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py
diff --git a/tests/rcutorture.c b/tests/rcutorture.c
new file mode 100644
index 0000000..02ab9ed
--- /dev/null
+++ b/tests/rcutorture.c
@@ -0,0 +1,439 @@
+/*
+ * rcutorture.c: simple user-level performance/stress test of RCU.
+ *
+ * Usage:
+ *     ./rcu <nreaders> rperf [ <seconds> ]
+ *         Run a read-side performance test with the specified
+ *         number of readers for <seconds> seconds.
+ *     ./rcu <nupdaters> uperf [ <seconds> ]
+ *         Run an update-side performance test with the specified
+ *         number of updaters and specified duration.
+ *     ./rcu <nreaders> perf [ <seconds> ]
+ *         Run a combined read/update performance test with the specified
+ *         number of readers and one updater and specified duration.
+ *
+ * The above tests produce output as follows:
+ *
+ * n_reads: 46008000  n_updates: 146026  nreaders: 2  nupdaters: 1 duration: 1
+ * ns/read: 43.4707  ns/update: 6848.1
+ *
+ * The first line lists the total number of RCU reads and updates executed
+ * during the test, the number of reader threads, the number of updater
+ * threads, and the duration of the test in seconds.  The second line
+ * lists the average duration of each type of operation in nanoseconds,
+ * or "nan" if the corresponding type of operation was not performed.
+ *
+ *     ./rcu <nreaders> stress [ <seconds> ]
+ *         Run a stress test with the specified number of readers and
+ *         one updater.
+ *
+ * This test produces output as follows:
+ *
+ * n_reads: 114633217  n_updates: 3903415  n_mberror: 0
+ * rcu_stress_count: 114618391 14826 0 0 0 0 0 0 0 0 0
+ *
+ * The first line lists the number of RCU read and update operations
+ * executed, followed by the number of memory-ordering violations
+ * (which will be zero in a correct RCU implementation).  The second
+ * line lists the number of readers observing progressively more stale
+ * data.  A correct RCU implementation will have all but the first two
+ * numbers non-zero.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (c) 2008 Paul E. McKenney, IBM Corporation.
+ */
+
+/*
+ * Test variables.
+ */
+
+#include <glib.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include "qemu/atomic.h"
+#include "qemu/rcu.h"
+#include "qemu/compiler.h"
+#include "qemu/thread.h"
+
+long long n_reads = 0LL;
+long n_updates = 0L;
+int nthreadsrunning;
+
+char argsbuf[64];
+
+#define GOFLAG_INIT 0
+#define GOFLAG_RUN  1
+#define GOFLAG_STOP 2
+
+static volatile int goflag = GOFLAG_INIT;
+
+#define RCU_READ_RUN 1000
+
+#define NR_THREADS 100
+static QemuThread threads[NR_THREADS];
+static struct rcu_reader_data *data[NR_THREADS];
+static int n_threads;
+
+static void create_thread(void *(*func)(void *))
+{
+    if (n_threads >= NR_THREADS) {
+        fprintf(stderr, "Thread limit of %d exceeded!\n", NR_THREADS);
+        exit(-1);
+    }
+    qemu_thread_create(&threads[n_threads], func, &data[n_threads],
+                       QEMU_THREAD_JOINABLE);
+    n_threads++;
+}
+
+static void wait_all_threads(void)
+{
+    int i;
+
+    for (i = 0; i < n_threads; i++) {
+        qemu_thread_join(&threads[i]);
+    }
+    n_threads = 0;
+}
+
+/*
+ * Performance test.
+ */
+
+static void *rcu_read_perf_test(void *arg)
+{
+    int i;
+    long long n_reads_local = 0;
+
+    *(struct rcu_reader_data **)arg = tls_get_rcu_reader();
+    atomic_inc(&nthreadsrunning);
+    rcu_thread_offline();
+    while (goflag == GOFLAG_INIT) {
+        g_usleep(1000);
+    }
+    rcu_thread_online();
+    while (goflag == GOFLAG_RUN) {
+        for (i = 0; i < RCU_READ_RUN; i++) {
+            rcu_read_lock();
+            rcu_read_unlock();
+        }
+        n_reads_local += RCU_READ_RUN;
+        rcu_quiescent_state();
+    }
+    atomic_add(&n_reads, n_reads_local);
+
+    return NULL;
+}
+
+static void *rcu_update_perf_test(void *arg)
+{
+    long long n_updates_local = 0;
+
+    *(struct rcu_reader_data **)arg = tls_get_rcu_reader();
+    atomic_inc(&nthreadsrunning);
+    while (goflag == GOFLAG_INIT) {
+        g_usleep(1000);
+    }
+    while (goflag == GOFLAG_RUN) {
+        synchronize_rcu();
+        n_updates_local++;
+    }
+    atomic_add(&n_updates, n_updates_local);
+    return NULL;
+}
+
+static void perftestinit(void)
+{
+    nthreadsrunning = 0;
+}
+
+static void perftestrun(int nthreads, int duration, int nreaders, int nupdaters)
+{
+    while (atomic_read(&nthreadsrunning) < nthreads) {
+        g_usleep(1000);
+    }
+    goflag = GOFLAG_RUN;
+    sleep(duration);
+    goflag = GOFLAG_STOP;
+    wait_all_threads();
+    printf("n_reads: %lld  n_updates: %ld  nreaders: %d  nupdaters: %d duration: %d\n",
+           n_reads, n_updates, nreaders, nupdaters, duration);
+    printf("ns/read: %g  ns/update: %g\n",
+           ((duration * 1000*1000*1000.*(double)nreaders) /
+        (double)n_reads),
+           ((duration * 1000*1000*1000.*(double)nupdaters) /
+        (double)n_updates));
+    exit(0);
+}
+
+static void perftest(int nreaders, int duration)
+{
+    int i;
+
+    perftestinit();
+    for (i = 0; i < nreaders; i++) {
+        create_thread(rcu_read_perf_test);
+    }
+    create_thread(rcu_update_perf_test);
+    perftestrun(i + 1, duration, nreaders, 1);
+}
+
+static void rperftest(int nreaders, int duration)
+{
+    int i;
+
+    perftestinit();
+    for (i = 0; i < nreaders; i++) {
+        create_thread(rcu_read_perf_test);
+    }
+    perftestrun(i, duration, nreaders, 0);
+}
+
+static void uperftest(int nupdaters, int duration)
+{
+    int i;
+
+    perftestinit();
+    for (i = 0; i < nupdaters; i++) {
+        create_thread(rcu_update_perf_test);
+    }
+    perftestrun(i, duration, 0, nupdaters);
+}
+
+/*
+ * Stress test.
+ */
+
+#define RCU_STRESS_PIPE_LEN 10
+
+struct rcu_stress {
+    int pipe_count;
+    int mbtest;
+};
+
+struct rcu_stress rcu_stress_array[RCU_STRESS_PIPE_LEN] = { { 0 } };
+struct rcu_stress *rcu_stress_current;
+int rcu_stress_idx;
+
+int n_mberror;
+long long rcu_stress_count[RCU_STRESS_PIPE_LEN + 1];
+
+
+static void *rcu_read_stress_test(void *arg)
+{
+    int i;
+    int itercnt = 0;
+    struct rcu_stress *p;
+    int pc;
+    long long n_reads_local = 0;
+    volatile int garbage;
+
+    *(struct rcu_reader_data **)arg = tls_get_rcu_reader();
+    rcu_thread_offline();
+    while (goflag == GOFLAG_INIT) {
+        g_usleep(1000);
+    }
+    rcu_thread_online();
+    while (goflag == GOFLAG_RUN) {
+        rcu_read_lock();
+        p = rcu_dereference(&rcu_stress_current);
+        if (p->mbtest == 0) {
+            n_mberror++;
+        }
+        rcu_read_lock();
+        for (i = 0; i < 100; i++)
+            garbage++;
+        rcu_read_unlock();
+        pc = p->pipe_count;
+        rcu_read_unlock();
+        if ((pc > RCU_STRESS_PIPE_LEN) || (pc < 0)) {
+            pc = RCU_STRESS_PIPE_LEN;
+        }
+        atomic_inc(&rcu_stress_count[pc]);
+        n_reads_local++;
+        rcu_quiescent_state();
+        if ((++itercnt % 0x1000) == 0) {
+            synchronize_rcu();
+        }
+    }
+    atomic_add(&n_reads, n_reads_local);
+
+    return NULL;
+}
+
+static void *rcu_update_stress_test(void *arg)
+{
+    int i;
+    struct rcu_stress *p;
+
+    *(struct rcu_reader_data **)arg = tls_get_rcu_reader();
+    while (goflag == GOFLAG_INIT) {
+        g_usleep(1000);
+    }
+    while (goflag == GOFLAG_RUN) {
+        i = rcu_stress_idx + 1;
+        if (i >= RCU_STRESS_PIPE_LEN) {
+            i = 0;
+        }
+        p = &rcu_stress_array[i];
+        p->mbtest = 0;
+        smp_mb();
+        p->pipe_count = 0;
+        p->mbtest = 1;
+        rcu_assign_pointer(rcu_stress_current, p);
+        rcu_stress_idx = i;
+        for (i = 0; i < RCU_STRESS_PIPE_LEN; i++)
+            if (i != rcu_stress_idx) {
+                rcu_stress_array[i].pipe_count++;
+            }
+        synchronize_rcu();
+        n_updates++;
+    }
+    return NULL;
+}
+
+static void *rcu_fake_update_stress_test(void *arg)
+{
+    *(struct rcu_reader_data **)arg = tls_get_rcu_reader();
+    while (goflag == GOFLAG_INIT) {
+        g_usleep(1000);
+    }
+    while (goflag == GOFLAG_RUN) {
+        synchronize_rcu();
+        g_usleep(1000);
+    }
+    return NULL;
+}
+
+static void stresstest(int nreaders, int duration)
+{
+    int i;
+
+    rcu_stress_current = &rcu_stress_array[0];
+    rcu_stress_current->pipe_count = 0;
+    rcu_stress_current->mbtest = 1;
+    for (i = 0; i < nreaders; i++) {
+        create_thread(rcu_read_stress_test);
+    }
+    create_thread(rcu_update_stress_test);
+    for (i = 0; i < 5; i++) {
+        create_thread(rcu_fake_update_stress_test);
+    }
+    goflag = GOFLAG_RUN;
+    sleep(duration);
+    goflag = GOFLAG_STOP;
+    wait_all_threads();
+    printf("n_reads: %lld  n_updates: %ld  n_mberror: %d\n",
+           n_reads, n_updates, n_mberror);
+    printf("rcu_stress_count:");
+    for (i = 0; i <= RCU_STRESS_PIPE_LEN; i++) {
+        printf(" %lld", rcu_stress_count[i]);
+    }
+    printf("\n");
+    exit(0);
+}
+
+/* GTest interface */
+
+static void gtest_stress(int nreaders, int duration)
+{
+    int i;
+
+    rcu_stress_current = &rcu_stress_array[0];
+    rcu_stress_current->pipe_count = 0;
+    rcu_stress_current->mbtest = 1;
+    for (i = 0; i < nreaders; i++) {
+        create_thread(rcu_read_stress_test);
+    }
+    create_thread(rcu_update_stress_test);
+    for (i = 0; i < 5; i++) {
+        create_thread(rcu_fake_update_stress_test);
+    }
+    goflag = GOFLAG_RUN;
+    sleep(duration);
+    goflag = GOFLAG_STOP;
+    wait_all_threads();
+    g_assert_cmpint(n_mberror, ==, 0);
+    for (i = 2; i <= RCU_STRESS_PIPE_LEN; i++) {
+        g_assert_cmpint(rcu_stress_count[i], ==, 0);
+    }
+}
+
+static void gtest_stress_1_1(void)
+{
+    gtest_stress(1, 1);
+}
+
+static void gtest_stress_10_1(void)
+{
+    gtest_stress(10, 1);
+}
+
+static void gtest_stress_1_5(void)
+{
+    gtest_stress(1, 5);
+}
+
+static void gtest_stress_10_5(void)
+{
+    gtest_stress(10, 5);
+}
+
+/*
+ * Mainprogram.
+ */
+
+static void usage(int argc, char *argv[])
+{
+    fprintf(stderr, "Usage: %s [nreaders [ perf | stress ] ]\n", argv[0]);
+    exit(-1);
+}
+
+int main(int argc, char *argv[])
+{
+    int nreaders = 1;
+    int duration = 1;
+
+    /* This thread is not part of the test.  */
+    rcu_thread_offline();
+
+    if (argc >= 2 && argv[1][0] == '-') {
+        g_test_init(&argc, &argv, NULL);
+        g_test_add_func("/rcutorture/short/1reader", gtest_stress_1_1);
+        g_test_add_func("/rcutorture/short/10readers", gtest_stress_10_1);
+        g_test_add_func("/rcutorture/long/1reader", gtest_stress_1_5);
+        g_test_add_func("/rcutorture/long/10readers", gtest_stress_10_5);
+        return g_test_run();
+    }
+
+    if (argc >= 2) {
+        nreaders = strtoul(argv[1], NULL, 0);
+    }
+    if (argc > 3) {
+        duration = strtoul(argv[3], NULL, 0);
+    }
+    if (argc < 3 || strcmp(argv[2], "stress") == 0) {
+        stresstest(nreaders, duration);
+    } else if (strcmp(argv[2], "rperf") == 0) {
+        rperftest(nreaders, duration);
+    } else if (strcmp(argv[2], "uperf") == 0) {
+        uperftest(nreaders, duration);
+    } else if (strcmp(argv[2], "perf") == 0) {
+        perftest(nreaders, duration);
+    }
+    usage(argc, argv);
+    return 0;
+}
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 12/30] rcu: allow nested calls to rcu_thread_offline/rcu_thread_online
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (10 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 11/30] rcu: add rcutorture Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 13/30] qemu-thread: report RCU quiescent states Paolo Bonzini
                   ` (17 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/rcu.txt       |  5 +++++
 include/qemu/rcu.h | 21 +++++++++++++++++++--
 2 files changed, 24 insertions(+), 2 deletions(-)

diff --git a/docs/rcu.txt b/docs/rcu.txt
index ca1c099..a3510b9 100644
--- a/docs/rcu.txt
+++ b/docs/rcu.txt
@@ -187,6 +187,11 @@ Marking quiescent states is done with the following three APIs:
         thread.
 
 
+rcu_thread_offline() and rcu_thread_online() can be nested.  The end of
+the extended quiescent state will coincide with the outermost call to
+rcu_thread_online().
+
+
 The following APIs can be used to use RCU in a thread that is not
 created with qemu_thread_create():
 
diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h
index 1ed70bb..eff6d1e 100644
--- a/include/qemu/rcu.h
+++ b/include/qemu/rcu.h
@@ -78,6 +78,9 @@ struct rcu_reader_data {
     unsigned long ctr;
     bool waiting;
 
+    /* Data used by reader only */
+    unsigned offline;
+
     /* Data used for registry, protected by rcu_gp_lock */
     QLIST_ENTRY(rcu_reader_data) node;
 };
@@ -100,9 +103,14 @@ static inline void rcu_read_unlock(void)
 static inline void rcu_quiescent_state(void)
 {
     struct rcu_reader_data *p_rcu_reader = tls_get_rcu_reader();
+    unsigned ctr;
+
+    if (p_rcu_reader->offline > 0) {
+        return;
+    }
 
     /* Reuses smp_rmb() in the last rcu_read_unlock().  */
-    unsigned ctr = atomic_read(&rcu_gp_ctr);
+    ctr = atomic_read(&rcu_gp_ctr);
     atomic_xchg(&p_rcu_reader->ctr, ctr);
     if (atomic_read(&p_rcu_reader->waiting)) {
         atomic_set(&p_rcu_reader->waiting, false);
@@ -114,6 +122,10 @@ static inline void rcu_thread_offline(void)
 {
     struct rcu_reader_data *p_rcu_reader = tls_get_rcu_reader();
 
+    if (p_rcu_reader->offline++ > 0) {
+        return;
+    }
+
     atomic_xchg(&p_rcu_reader->ctr, 0);
     if (atomic_read(&p_rcu_reader->waiting)) {
         atomic_set(&p_rcu_reader->waiting, false);
@@ -123,7 +135,12 @@ static inline void rcu_thread_offline(void)
 
 static inline void rcu_thread_online(void)
 {
-    rcu_quiescent_state();
+    struct rcu_reader_data *p_rcu_reader = tls_get_rcu_reader();
+
+    assert(p_rcu_reader->offline != 0);
+    if (--p_rcu_reader->offline == 0) {
+        rcu_quiescent_state();
+    }
 }
 
 extern void synchronize_rcu(void);
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 13/30] qemu-thread: report RCU quiescent states
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (11 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 12/30] rcu: allow nested calls to rcu_thread_offline/rcu_thread_online Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 14/30] event loop: " Paolo Bonzini
                   ` (16 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Most threads will use mutexes and other sleeping synchronization primitives
(condition variables, semaphores, events) periodically.  For these threads,
the synchronization primitives are natural places to report a quiescent
state (possibly an extended one).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/rcu.txt             | 33 ++++++++++++++++++++++++++++++++-
 util/qemu-thread-posix.c | 30 ++++++++++++++++++++++++++----
 util/qemu-thread-win32.c | 16 +++++++++++++++-
 util/rcu.c               |  3 ---
 4 files changed, 73 insertions(+), 9 deletions(-)

diff --git a/docs/rcu.txt b/docs/rcu.txt
index a3510b9..6c4a852 100644
--- a/docs/rcu.txt
+++ b/docs/rcu.txt
@@ -168,6 +168,35 @@ of "quiescent states", i.e. points where no RCU read-side critical
 section can be active.  All threads created with qemu_thread_create
 participate in the RCU mechanism and need to annotate such points.
 
+Luckily, in most cases no manual annotation is needed, because waiting
+on condition variables (qemu_cond_wait), semaphores (qemu_sem_wait,
+qemu_sem_timedwait) or events (qemu_event_wait) implicitly marks the thread
+as quiescent for the whole duration of the wait.  (There is an exception
+for semaphore waits with a zero timeout).
+
+Manual annotation is still needed in the following cases:
+
+- threads that spend their sleeping time in the kernel, for example
+  in a call to select(), poll(), sigwait() or WaitForMultipleObjects().
+  The QEMU I/O thread is an example of this case.  When running under
+  KVM, VCPUs are also in a quiescent state while running the guest.
+
+- threads that perform a lot of I/O.  In QEMU, the workers used for
+  aio=thread are an example of this case (see aio_worker in block/raw-*).
+
+- threads that run continuously until they exit.  The migration thread
+  is an example of this case.
+
+Regarding the second case, note that the workers run in the QEMU thread
+pool.  The thread pool uses semaphores for synchronization, hence it does
+report quiescent states periodically.  However, in some cases (e.g. NFS
+mounted with the "hard" option) the workers can take an arbitrarily long
+amount of time.  When this happens, synchronize_rcu() will not exit and
+call_rcu() callbacks will be delayed arbitrarily.  It is therefore a
+good idea to mark I/O system calls as quiescence points in the worker
+functions.
+
+
 Marking quiescent states is done with the following three APIs:
 
      void rcu_quiescent_state(void);
@@ -229,7 +258,9 @@ DIFFERENCES WITH LINUX
   type of the callback's argument to be the type of the first argument.
   call_rcu1 is the same as Linux's call_rcu.
 
-- Quiescent points must be marked explicitly in the thread.
+- Quiescent points must be marked explicitly unless the thread uses
+  condvars/semaphores/events for synchronization.  Note that mutexes
+  do not report quiescent points (see the first item above).
 
 
 RCU PATTERNS
diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
index 2df3382..21190be 100644
--- a/util/qemu-thread-posix.c
+++ b/util/qemu-thread-posix.c
@@ -119,7 +119,9 @@ void qemu_cond_wait(QemuCond *cond, QemuMutex *mutex)
 {
     int err;
 
+    rcu_thread_offline();
     err = pthread_cond_wait(&cond->cond, &mutex->lock);
+    rcu_thread_online();
     if (err)
         error_exit(err, __func__);
 }
@@ -212,6 +214,10 @@ int qemu_sem_timedwait(QemuSemaphore *sem, int ms)
     int rc;
     struct timespec ts;
 
+    if (ms) {
+        rcu_thread_offline();
+    }
+
 #if defined(__APPLE__) || defined(__NetBSD__)
     compute_abs_deadline(&ts, ms);
     pthread_mutex_lock(&sem->lock);
@@ -227,7 +233,10 @@ int qemu_sem_timedwait(QemuSemaphore *sem, int ms)
         }
     }
     pthread_mutex_unlock(&sem->lock);
-    return (rc == ETIMEDOUT ? -1 : 0);
+    if (rc == ETIMEDOUT) {
+        rc == -1;
+    }
+
 #else
     if (ms <= 0) {
         /* This is cheaper than sem_timedwait.  */
@@ -235,7 +244,7 @@ int qemu_sem_timedwait(QemuSemaphore *sem, int ms)
             rc = sem_trywait(&sem->sem);
         } while (rc == -1 && errno == EINTR);
         if (rc == -1 && errno == EAGAIN) {
-            return -1;
+            goto out;
         }
     } else {
         compute_abs_deadline(&ts, ms);
@@ -243,18 +252,25 @@ int qemu_sem_timedwait(QemuSemaphore *sem, int ms)
             rc = sem_timedwait(&sem->sem, &ts);
         } while (rc == -1 && errno == EINTR);
         if (rc == -1 && errno == ETIMEDOUT) {
-            return -1;
+            goto out;
         }
     }
     if (rc < 0) {
         error_exit(errno, __func__);
     }
-    return 0;
 #endif
+
+out:
+    if (ms) {
+        rcu_thread_online();
+    }
+    return rc;
 }
 
 void qemu_sem_wait(QemuSemaphore *sem)
 {
+    rcu_thread_offline();
+
 #if defined(__APPLE__) || defined(__NetBSD__)
     pthread_mutex_lock(&sem->lock);
     --sem->count;
@@ -272,6 +288,8 @@ void qemu_sem_wait(QemuSemaphore *sem)
         error_exit(errno, __func__);
     }
 #endif
+
+    rcu_thread_online();
 }
 
 #ifdef __linux__
@@ -380,7 +398,11 @@ void qemu_event_wait(QemuEvent *ev)
                 return;
             }
         }
+        rcu_thread_offline();
         futex_wait(ev, EV_BUSY);
+        rcu_thread_online();
+    } else {
+        rcu_quiescent_state();
     }
 }
 
diff --git a/util/qemu-thread-win32.c b/util/qemu-thread-win32.c
index 18978be..9c14cf1 100644
--- a/util/qemu-thread-win32.c
+++ b/util/qemu-thread-win32.c
@@ -12,6 +12,7 @@
  */
 #include "qemu-common.h"
 #include "qemu/thread.h"
+#include "qemu/rcu.h"
 #include <process.h>
 #include <assert.h>
 #include <limits.h>
@@ -187,7 +188,9 @@ void qemu_cond_wait(QemuCond *cond, QemuMutex *mutex)
      * leaving mutex unlocked before we wait on semaphore.
      */
     qemu_mutex_unlock(mutex);
+    rcu_thread_offline();
     WaitForSingleObject(cond->sema, INFINITE);
+    rcu_thread_online();
 
     /* Now waiters must rendez-vous with the signaling thread and
      * let it continue.  For cond_broadcast this has heavy contention
@@ -227,7 +230,16 @@ void qemu_sem_post(QemuSemaphore *sem)
 
 int qemu_sem_timedwait(QemuSemaphore *sem, int ms)
 {
-    int rc = WaitForSingleObject(sem->sema, ms);
+    int rc;
+
+    if (ms) {
+        rcu_thread_offline();
+    }
+    rc = WaitForSingleObject(sem->sema, ms);
+    if (ms) {
+        rcu_thread_online();
+    }
+
     if (rc == WAIT_OBJECT_0) {
         return 0;
     }
@@ -267,7 +279,9 @@ void qemu_event_reset(QemuEvent *ev)
 
 void qemu_event_wait(QemuEvent *ev)
 {
+    rcu_thread_offline();
     WaitForSingleObject(ev->event, INFINITE);
+    rcu_thread_online();
 }
 
 struct QemuThreadData {
diff --git a/util/rcu.c b/util/rcu.c
index f1c5736..654f3bb 100644
--- a/util/rcu.c
+++ b/util/rcu.c
@@ -232,9 +232,6 @@ static void *call_rcu_thread(void *opaque)
 {
     struct rcu_head *node;
 
-    /* This thread is just a writer.  */
-    rcu_thread_offline();
-
     for (;;) {
         int tries = 0;
         int n = atomic_read(&rcu_call_count);
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 14/30] event loop: report RCU quiescent states
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (12 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 13/30] qemu-thread: report RCU quiescent states Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 15/30] cpus: " Paolo Bonzini
                   ` (15 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Threads that run event loops also have places that can sleep for an extended
time.  Place an extended quiescent state there.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 aio-posix.c |  8 ++++++++
 aio-win32.c | 11 ++++++++++-
 main-loop.c |  6 ++++++
 3 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/aio-posix.c b/aio-posix.c
index b68eccd..a57b276 100644
--- a/aio-posix.c
+++ b/aio-posix.c
@@ -17,6 +17,7 @@
 #include "block/block.h"
 #include "qemu/queue.h"
 #include "qemu/sockets.h"
+#include "qemu/rcu.h"
 
 struct AioHandler
 {
@@ -175,6 +176,7 @@ bool aio_poll(AioContext *ctx, bool blocking)
     int ret;
     bool busy, progress;
 
+    rcu_quiescent_state();
     progress = false;
 
     /*
@@ -232,9 +234,15 @@ bool aio_poll(AioContext *ctx, bool blocking)
     }
 
     /* wait until next event */
+    if (blocking) {
+        rcu_thread_offline();
+    }
     ret = g_poll((GPollFD *)ctx->pollfds->data,
                  ctx->pollfds->len,
                  blocking ? -1 : 0);
+    if (blocking) {
+        rcu_thread_online();
+    }
 
     /* if we have any readable fds, dispatch event */
     if (ret > 0) {
diff --git a/aio-win32.c b/aio-win32.c
index 38723bf..9652afb 100644
--- a/aio-win32.c
+++ b/aio-win32.c
@@ -19,6 +19,7 @@
 #include "block/block.h"
 #include "qemu/queue.h"
 #include "qemu/sockets.h"
+#include "qemu/rcu.h"
 
 struct AioHandler {
     EventNotifier *e;
@@ -99,6 +100,7 @@ bool aio_poll(AioContext *ctx, bool blocking)
     bool busy, progress;
     int count;
 
+    rcu_quiescent_state();
     progress = false;
 
     /*
@@ -175,7 +177,14 @@ bool aio_poll(AioContext *ctx, bool blocking)
     /* wait until next event */
     while (count > 0) {
         int timeout = blocking ? INFINITE : 0;
-        int ret = WaitForMultipleObjects(count, events, FALSE, timeout);
+
+        if (timeout) {
+            rcu_thread_offline();
+        }
+        ret = WaitForMultipleObjects(count, events, FALSE, timeout);
+        if (timeout) {
+            rcu_thread_online();
+        }
 
         /* if we have any signaled events, dispatch event */
         if ((DWORD) (ret - WAIT_OBJECT_0) >= count) {
diff --git a/main-loop.c b/main-loop.c
index a44fff6..166357f 100644
--- a/main-loop.c
+++ b/main-loop.c
@@ -28,6 +28,7 @@
 #include "slirp/libslirp.h"
 #include "qemu/main-loop.h"
 #include "block/aio.h"
+#include "qemu/rcu.h"
 
 #ifndef _WIN32
 
@@ -220,13 +221,16 @@ static int os_host_main_loop_wait(uint32_t timeout)
     if (timeout > 0) {
         spin_counter = 0;
         qemu_mutex_unlock_iothread();
+        rcu_thread_offline();
     } else {
         spin_counter++;
+        rcu_quiescent_state();
     }
 
     ret = g_poll((GPollFD *)gpollfds->data, gpollfds->len, timeout);
 
     if (timeout > 0) {
+        rcu_thread_online();
         qemu_mutex_lock_iothread();
     }
 
@@ -424,7 +428,9 @@ static int os_host_main_loop_wait(uint32_t timeout)
     }
 
     qemu_mutex_unlock_iothread();
+    rcu_thread_offline();
     g_poll_ret = g_poll(poll_fds, n_poll_fds + w->num, poll_timeout);
+    rcu_thread_online();
     qemu_mutex_lock_iothread();
     if (g_poll_ret > 0) {
         for (i = 0; i < w->num; i++) {
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 15/30] cpus: report RCU quiescent states
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (13 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 14/30] event loop: " Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 16/30] block: " Paolo Bonzini
                   ` (14 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

CPU threads have extended quiescent states while relinquishing control
to the accelerator (except TCG).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 cpus.c    | 3 +++
 kvm-all.c | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/cpus.c b/cpus.c
index c8bc8ad..4b5cb8d 100644
--- a/cpus.c
+++ b/cpus.c
@@ -37,6 +37,7 @@
 #include "sysemu/qtest.h"
 #include "qemu/main-loop.h"
 #include "qemu/bitmap.h"
+#include "qemu/rcu.h"
 
 #ifndef _WIN32
 #include "qemu/compatfd.h"
@@ -793,6 +794,7 @@ static void *qemu_dummy_cpu_thread_fn(void *arg)
     while (1) {
         cpu_single_env = NULL;
         qemu_mutex_unlock_iothread();
+        rcu_thread_offline();
         do {
             int sig;
             r = sigwait(&waitset, &sig);
@@ -801,6 +803,7 @@ static void *qemu_dummy_cpu_thread_fn(void *arg)
             perror("sigwait");
             exit(1);
         }
+        rcu_thread_online();
         qemu_mutex_lock_iothread();
         cpu_single_env = env;
         qemu_wait_io_event_common(cpu);
diff --git a/kvm-all.c b/kvm-all.c
index 91aa7ff..547abe3 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -33,6 +33,7 @@
 #include "exec/memory.h"
 #include "exec/address-spaces.h"
 #include "qemu/event_notifier.h"
+#include "qemu/rcu.h"
 #include "trace.h"
 
 /* This check must be after config-host.h is included */
@@ -1644,7 +1645,9 @@ int kvm_cpu_exec(CPUArchState *env)
         }
         qemu_mutex_unlock_iothread();
 
+        rcu_thread_offline();
         run_ret = kvm_vcpu_ioctl(cpu, KVM_RUN, 0);
+        rcu_thread_online();
 
         qemu_mutex_lock_iothread();
         kvm_arch_post_run(cpu, run);
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 16/30] block: report RCU quiescent states
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (14 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 15/30] cpus: " Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 17/30] migration: " Paolo Bonzini
                   ` (13 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

The aio workers may spend a long time executing I/O operations;
mark that time as an extended quiescent state.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 block/raw-posix.c | 3 +++
 block/raw-win32.c | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/block/raw-posix.c b/block/raw-posix.c
index c0ccf27..637fe8a 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -26,6 +26,7 @@
 #include "qemu/log.h"
 #include "block/block_int.h"
 #include "qemu/module.h"
+#include "qemu/rcu.h"
 #include "trace.h"
 #include "block/thread-pool.h"
 #include "qemu/iov.h"
@@ -735,6 +736,7 @@ static int aio_worker(void *arg)
     RawPosixAIOData *aiocb = arg;
     ssize_t ret = 0;
 
+    rcu_thread_offline();
     switch (aiocb->aio_type & QEMU_AIO_TYPE_MASK) {
     case QEMU_AIO_READ:
         ret = handle_aiocb_rw(aiocb);
@@ -774,6 +776,7 @@ static int aio_worker(void *arg)
     }
 
     g_slice_free(RawPosixAIOData, aiocb);
+    rcu_thread_online();
     return ret;
 }
 
diff --git a/block/raw-win32.c b/block/raw-win32.c
index 7c03b6d..fc573b7 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -25,6 +25,7 @@
 #include "qemu/timer.h"
 #include "block/block_int.h"
 #include "qemu/module.h"
+#include "qemu/rcu.h"
 #include "raw-aio.h"
 #include "trace.h"
 #include "block/thread-pool.h"
@@ -99,6 +100,7 @@ static int aio_worker(void *arg)
     ssize_t ret = 0;
     size_t count;
 
+    rcu_thread_offline();
     switch (aiocb->aio_type & QEMU_AIO_TYPE_MASK) {
     case QEMU_AIO_READ:
         count = handle_aiocb_rw(aiocb);
@@ -136,6 +138,7 @@ static int aio_worker(void *arg)
     }
 
     g_slice_free(RawWin32AIOData, aiocb);
+    rcu_thread_online();
     return ret;
 }
 
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 17/30] migration: report RCU quiescent states
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (15 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 16/30] block: " Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 18/30] memory: protect current_map by RCU Paolo Bonzini
                   ` (12 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

The migration thread polls s->state periodically, it does not
use a mutex or condition variable, so it has to report quiescent
states manually.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 migration.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/migration.c b/migration.c
index 83f5691..28010dc 100644
--- a/migration.c
+++ b/migration.c
@@ -22,6 +22,7 @@
 #include "qemu/sockets.h"
 #include "migration/block.h"
 #include "qemu/thread.h"
+#include "qemu/rcu.h"
 #include "qmp-commands.h"
 #include "trace.h"
 
@@ -508,6 +509,7 @@ static void *migration_thread(void *opaque)
         int64_t current_time;
         uint64_t pending_size;
 
+        rcu_quiescent_state();
         if (!qemu_file_rate_limit(s->file)) {
             DPRINTF("iterate\n");
             pending_size = qemu_savevm_state_pending(s->file, max_size);
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 18/30] memory: protect current_map by RCU
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (16 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 17/30] migration: " Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 19/30] memory: avoid ref/unref in memory_region_find Paolo Bonzini
                   ` (11 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 include/exec/memory.h |  5 +++++
 memory.c              | 50 +++++++++++++++++++++-----------------------------
 2 files changed, 26 insertions(+), 29 deletions(-)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 9c33bba..aa7a922 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -24,6 +24,7 @@
 #include "qemu/queue.h"
 #include "qemu/int128.h"
 #include "qemu/notify.h"
+#include "qemu/rcu.h"
 
 #define MAX_PHYS_ADDR_SPACE_BITS 62
 #define MAX_PHYS_ADDR            (((hwaddr)1 << MAX_PHYS_ADDR_SPACE_BITS) - 1)
@@ -168,9 +169,13 @@ struct MemoryRegion {
  */
 struct AddressSpace {
     /* All fields are private. */
+    struct rcu_head rcu;
     char *name;
     MemoryRegion *root;
+
+    /* Accessed via RCU.  */
     struct FlatView *current_map;
+
     int ioeventfd_nb;
     struct MemoryRegionIoeventfd *ioeventfds;
     struct AddressSpaceDispatch *dispatch;
diff --git a/memory.c b/memory.c
index bb92e17..7a4fe37 100644
--- a/memory.c
+++ b/memory.c
@@ -29,26 +29,12 @@ static unsigned memory_region_transaction_depth;
 static bool memory_region_update_pending;
 static bool global_dirty_log = false;
 
-/* flat_view_mutex is taken around reading as->current_map; the critical
- * section is extremely short, so I'm using a single mutex for every AS.
- * We could also RCU for the read-side.
- *
- * The BQL is taken around transaction commits, hence both locks are taken
- * while writing to as->current_map (with the BQL taken outside).
- */
-static QemuMutex flat_view_mutex;
-
 static QTAILQ_HEAD(memory_listeners, MemoryListener) memory_listeners
     = QTAILQ_HEAD_INITIALIZER(memory_listeners);
 
 static QTAILQ_HEAD(, AddressSpace) address_spaces
     = QTAILQ_HEAD_INITIALIZER(address_spaces);
 
-static void memory_init(void)
-{
-    qemu_mutex_init(&flat_view_mutex);
-}
-
 typedef struct AddrRange AddrRange;
 
 /*
@@ -239,6 +225,7 @@ struct FlatRange {
  * order.
  */
 struct FlatView {
+    struct rcu_head rcu;
     unsigned ref;
     FlatRange *ranges;
     unsigned nr;
@@ -610,10 +597,10 @@ static FlatView *address_space_get_flatview(AddressSpace *as)
 {
     FlatView *view;
 
-    qemu_mutex_lock(&flat_view_mutex);
-    view = as->current_map;
+    rcu_read_lock();
+    view = rcu_dereference(&as->current_map);
     flatview_ref(view);
-    qemu_mutex_unlock(&flat_view_mutex);
+    rcu_read_unlock();
     return view;
 }
 
@@ -722,10 +709,9 @@ static void address_space_update_topology(AddressSpace *as)
     address_space_update_topology_pass(as, old_view, new_view, false);
     address_space_update_topology_pass(as, old_view, new_view, true);
 
-    qemu_mutex_lock(&flat_view_mutex);
-    flatview_unref(as->current_map);
-    as->current_map = new_view;
-    qemu_mutex_unlock(&flat_view_mutex);
+    /* Writes are protected by the BQL.  */
+    rcu_assign_pointer(as->current_map, new_view);
+    call_rcu(old_view, flatview_unref, rcu);
 
     /* Note that all the old MemoryRegions are still alive up to this
      * point.  This relieves most MemoryListeners from the need to
@@ -1687,10 +1673,6 @@ void memory_listener_unregister(MemoryListener *listener)
 
 void address_space_init(AddressSpace *as, MemoryRegion *root, const char *name)
 {
-    if (QTAILQ_EMPTY(&address_spaces)) {
-        memory_init();
-    }
-
     memory_region_transaction_begin();
     as->root = root;
     as->current_map = g_new(FlatView, 1);
@@ -1704,6 +1686,14 @@ void address_space_init(AddressSpace *as, MemoryRegion *root, const char *name)
     memory_region_transaction_commit();
 }
 
+static void do_address_space_destroy(AddressSpace *as)
+{
+    address_space_destroy_dispatch(as);
+    flatview_unref(as->current_map);
+    g_free(as->name);
+    g_free(as->ioeventfds);
+}
+
 void address_space_destroy(AddressSpace *as)
 {
     /* Flush out anything from MemoryListeners listening in on this */
@@ -1711,10 +1701,12 @@ void address_space_destroy(AddressSpace *as)
     as->root = NULL;
     memory_region_transaction_commit();
     QTAILQ_REMOVE(&address_spaces, as, address_spaces_link);
-    address_space_destroy_dispatch(as);
-    flatview_unref(as->current_map);
-    g_free(as->name);
-    g_free(as->ioeventfds);
+
+    /* At this point, as->dispatch and as->current_map are dummy
+     * entries that the guest should never use.  Wait for the old
+     * values to expire before freeing the data.
+     */
+    call_rcu(as, do_address_space_destroy, rcu);
 }
 
 bool io_mem_read(MemoryRegion *mr, hwaddr addr, uint64_t *pval, unsigned size)
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 19/30] memory: avoid ref/unref in memory_region_find
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (17 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 18/30] memory: protect current_map by RCU Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 20/30] exec: change well-known physical sections to macros Paolo Bonzini
                   ` (10 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Do the entire lookup under RCU.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 memory.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/memory.c b/memory.c
index 7a4fe37..4fc23a2 100644
--- a/memory.c
+++ b/memory.c
@@ -1559,7 +1559,8 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
     as = memory_region_to_address_space(root);
     range = addrrange_make(int128_make64(addr), int128_make64(size));
 
-    view = address_space_get_flatview(as);
+    rcu_read_lock();
+    view = rcu_dereference(&as->current_map);
     fr = flatview_lookup(view, range);
     if (!fr) {
         return ret;
@@ -1580,7 +1581,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
     ret.readonly = fr->readonly;
     memory_region_ref(ret.mr);
 
-    flatview_unref(view);
+    rcu_read_unlock();
     return ret;
 }
 
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 20/30] exec: change well-known physical sections to macros
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (18 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 19/30] memory: avoid ref/unref in memory_region_find Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 21/30] exec: separate current memory map from the one being built Paolo Bonzini
                   ` (9 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel; +Cc: Liu Ping Fan, Liu Ping Fan

From: Liu Ping Fan <qemulist@gmail.com>

Sections like phys_section_unassigned always have fixed address
in phys_sections.  Declared as macro, so we can use them
when having more than one phys_sections array.

Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
Signed-off-by: Liu Ping Fan <qemulist@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/exec.c b/exec.c
index a788981..7ad513c 100644
--- a/exec.c
+++ b/exec.c
@@ -113,10 +113,10 @@ typedef struct subpage_t {
 
 static MemoryRegionSection *phys_sections;
 static unsigned phys_sections_nb, phys_sections_nb_alloc;
-static uint16_t phys_section_unassigned;
-static uint16_t phys_section_notdirty;
-static uint16_t phys_section_rom;
-static uint16_t phys_section_watch;
+#define PHYS_SECTION_UNASSIGNED 0
+#define PHYS_SECTION_NOTDIRTY 1
+#define PHYS_SECTION_ROM 2
+#define PHYS_SECTION_WATCH 3
 
 /* Simple allocator for PhysPageEntry nodes */
 static PhysPageEntry (*phys_map_nodes)[L2_SIZE];
@@ -174,7 +174,7 @@ static void phys_page_set_level(PhysPageEntry *lp, hwaddr *index,
         if (level == 0) {
             for (i = 0; i < L2_SIZE; i++) {
                 p[i].is_leaf = 1;
-                p[i].ptr = phys_section_unassigned;
+                p[i].ptr = PHYS_SECTION_UNASSIGNED;
             }
         }
     } else {
@@ -213,7 +213,7 @@ static MemoryRegionSection *phys_page_find(AddressSpaceDispatch *d, hwaddr index
 
     for (i = P_L2_LEVELS - 1; i >= 0 && !lp.is_leaf; i--) {
         if (lp.ptr == PHYS_MAP_NODE_NIL) {
-            return &phys_sections[phys_section_unassigned];
+            return &phys_sections[PHYS_SECTION_UNASSIGNED];
         }
         p = phys_map_nodes[lp.ptr];
         lp = p[(index >> (i * L2_BITS)) & (L2_SIZE - 1)];
@@ -731,9 +731,9 @@ hwaddr memory_region_section_get_iotlb(CPUArchState *env,
         iotlb = (memory_region_get_ram_addr(section->mr) & TARGET_PAGE_MASK)
             + xlat;
         if (!section->readonly) {
-            iotlb |= phys_section_notdirty;
+            iotlb |= PHYS_SECTION_NOTDIRTY;
         } else {
-            iotlb |= phys_section_rom;
+            iotlb |= PHYS_SECTION_ROM;
         }
     } else {
         iotlb = section - phys_sections;
@@ -746,7 +746,7 @@ hwaddr memory_region_section_get_iotlb(CPUArchState *env,
         if (vaddr == (wp->vaddr & TARGET_PAGE_MASK)) {
             /* Avoid trapping reads of pages with a write breakpoint. */
             if ((prot & PAGE_WRITE) || (wp->flags & BP_MEM_READ)) {
-                iotlb = phys_section_watch + paddr;
+                iotlb = PHYS_SECTION_WATCH + paddr;
                 *address |= TLB_MMIO;
                 break;
             }
@@ -1670,7 +1670,7 @@ static subpage_t *subpage_init(AddressSpace *as, hwaddr base)
     printf("%s: %p base " TARGET_FMT_plx " len %08x %d\n", __func__,
            mmio, base, TARGET_PAGE_SIZE, subpage_memory);
 #endif
-    subpage_register(mmio, 0, TARGET_PAGE_SIZE-1, phys_section_unassigned);
+    subpage_register(mmio, 0, TARGET_PAGE_SIZE-1, PHYS_SECTION_UNASSIGNED);
 
     return mmio;
 }
@@ -1712,11 +1712,17 @@ static void mem_begin(MemoryListener *listener)
 
 static void core_begin(MemoryListener *listener)
 {
+    uint16_t n;
+
     phys_sections_clear();
-    phys_section_unassigned = dummy_section(&io_mem_unassigned);
-    phys_section_notdirty = dummy_section(&io_mem_notdirty);
-    phys_section_rom = dummy_section(&io_mem_rom);
-    phys_section_watch = dummy_section(&io_mem_watch);
+    n = dummy_section(&io_mem_unassigned);
+    assert(n == PHYS_SECTION_UNASSIGNED);
+    n = dummy_section(&io_mem_notdirty);
+    assert(n == PHYS_SECTION_NOTDIRTY);
+    n = dummy_section(&io_mem_rom);
+    assert(n == PHYS_SECTION_ROM);
+    n = dummy_section(&io_mem_watch);
+    assert(n == PHYS_SECTION_WATCH);
 }
 
 static void tcg_commit(MemoryListener *listener)
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 21/30] exec: separate current memory map from the one being built
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (19 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 20/30] exec: change well-known physical sections to macros Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-07-02 14:41   ` Jan Kiszka
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 22/30] memory: move MemoryListener declaration earlier Paolo Bonzini
                   ` (8 subsequent siblings)
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel; +Cc: Liu Ping Fan

Currently, phys_node_map and phys_sections are shared by all
of the AddressSpaceDispatch.  When updating mem topology, all
AddressSpaceDispatch will rebuild dispatch tables sequentially
on them.  In order to prepare for RCU access, leave the old
memory map alive while the next one is being accessed.

When rebuilding, the new dispatch tables will build and lookup
next_map; after all dispatch tables are rebuilt, we can switch
to next_* and free the previous table.

Based on a patch from Liu Ping Fan.

Signed-off-by: Liu Ping Fan <qemulist@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c | 103 ++++++++++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 63 insertions(+), 40 deletions(-)

diff --git a/exec.c b/exec.c
index 7ad513c..f138e56 100644
--- a/exec.c
+++ b/exec.c
@@ -111,16 +111,24 @@ typedef struct subpage_t {
     uint16_t sub_section[TARGET_PAGE_SIZE];
 } subpage_t;
 
-static MemoryRegionSection *phys_sections;
-static unsigned phys_sections_nb, phys_sections_nb_alloc;
 #define PHYS_SECTION_UNASSIGNED 0
 #define PHYS_SECTION_NOTDIRTY 1
 #define PHYS_SECTION_ROM 2
 #define PHYS_SECTION_WATCH 3
 
-/* Simple allocator for PhysPageEntry nodes */
-static PhysPageEntry (*phys_map_nodes)[L2_SIZE];
-static unsigned phys_map_nodes_nb, phys_map_nodes_nb_alloc;
+typedef PhysPageEntry Node[L2_SIZE];
+
+typedef struct PhysPageMap {
+    unsigned sections_nb;
+    unsigned sections_nb_alloc;
+    unsigned nodes_nb;
+    unsigned nodes_nb_alloc;
+    Node *nodes;
+    MemoryRegionSection *sections;
+} PhysPageMap;
+
+static PhysPageMap cur_map;
+static PhysPageMap next_map;
 
 #define PHYS_MAP_NODE_NIL (((uint16_t)~0) >> 1)
 
@@ -135,13 +143,13 @@ static MemoryRegion io_mem_watch;
 
 static void phys_map_node_reserve(unsigned nodes)
 {
-    if (phys_map_nodes_nb + nodes > phys_map_nodes_nb_alloc) {
-        typedef PhysPageEntry Node[L2_SIZE];
-        phys_map_nodes_nb_alloc = MAX(phys_map_nodes_nb_alloc * 2, 16);
-        phys_map_nodes_nb_alloc = MAX(phys_map_nodes_nb_alloc,
-                                      phys_map_nodes_nb + nodes);
-        phys_map_nodes = g_renew(Node, phys_map_nodes,
-                                 phys_map_nodes_nb_alloc);
+    if (next_map.nodes_nb + nodes > next_map.nodes_nb_alloc) {
+        next_map.nodes_nb_alloc = MAX(next_map.nodes_nb_alloc * 2,
+                                            16);
+        next_map.nodes_nb_alloc = MAX(next_map.nodes_nb_alloc,
+                                      next_map.nodes_nb + nodes);
+        next_map.nodes = g_renew(Node, next_map.nodes,
+                                 next_map.nodes_nb_alloc);
     }
 }
 
@@ -150,12 +158,12 @@ static uint16_t phys_map_node_alloc(void)
     unsigned i;
     uint16_t ret;
 
-    ret = phys_map_nodes_nb++;
+    ret = next_map.nodes_nb++;
     assert(ret != PHYS_MAP_NODE_NIL);
-    assert(ret != phys_map_nodes_nb_alloc);
+    assert(ret != next_map.nodes_nb_alloc);
     for (i = 0; i < L2_SIZE; ++i) {
-        phys_map_nodes[ret][i].is_leaf = 0;
-        phys_map_nodes[ret][i].ptr = PHYS_MAP_NODE_NIL;
+        next_map.nodes[ret][i].is_leaf = 0;
+        next_map.nodes[ret][i].ptr = PHYS_MAP_NODE_NIL;
     }
     return ret;
 }
@@ -170,7 +178,7 @@ static void phys_page_set_level(PhysPageEntry *lp, hwaddr *index,
 
     if (!lp->is_leaf && lp->ptr == PHYS_MAP_NODE_NIL) {
         lp->ptr = phys_map_node_alloc();
-        p = phys_map_nodes[lp->ptr];
+        p = next_map.nodes[lp->ptr];
         if (level == 0) {
             for (i = 0; i < L2_SIZE; i++) {
                 p[i].is_leaf = 1;
@@ -178,7 +186,7 @@ static void phys_page_set_level(PhysPageEntry *lp, hwaddr *index,
             }
         }
     } else {
-        p = phys_map_nodes[lp->ptr];
+        p = next_map.nodes[lp->ptr];
     }
     lp = &p[(*index >> (level * L2_BITS)) & (L2_SIZE - 1)];
 
@@ -205,20 +213,20 @@ static void phys_page_set(AddressSpaceDispatch *d,
     phys_page_set_level(&d->phys_map, &index, &nb, leaf, P_L2_LEVELS - 1);
 }
 
-static MemoryRegionSection *phys_page_find(AddressSpaceDispatch *d, hwaddr index)
+static MemoryRegionSection *phys_page_find(PhysPageEntry lp, hwaddr index,
+                                           Node *nodes, MemoryRegionSection *sections)
 {
-    PhysPageEntry lp = d->phys_map;
     PhysPageEntry *p;
     int i;
 
     for (i = P_L2_LEVELS - 1; i >= 0 && !lp.is_leaf; i--) {
         if (lp.ptr == PHYS_MAP_NODE_NIL) {
-            return &phys_sections[PHYS_SECTION_UNASSIGNED];
+            return &sections[PHYS_SECTION_UNASSIGNED];
         }
-        p = phys_map_nodes[lp.ptr];
+        p = nodes[lp.ptr];
         lp = p[(index >> (i * L2_BITS)) & (L2_SIZE - 1)];
     }
-    return &phys_sections[lp.ptr];
+    return &sections[lp.ptr];
 }
 
 bool memory_region_is_unassigned(MemoryRegion *mr)
@@ -234,10 +242,11 @@ static MemoryRegionSection *address_space_lookup_region(AddressSpace *as,
     MemoryRegionSection *section;
     subpage_t *subpage;
 
-    section = phys_page_find(as->dispatch, addr >> TARGET_PAGE_BITS);
+    section = phys_page_find(as->dispatch->phys_map, addr >> TARGET_PAGE_BITS,
+                             cur_map.nodes, cur_map.sections);
     if (resolve_subpage && section->mr->subpage) {
         subpage = container_of(section->mr, subpage_t, iomem);
-        section = &phys_sections[subpage->sub_section[SUBPAGE_IDX(addr)]];
+        section = &cur_map.sections[subpage->sub_section[SUBPAGE_IDX(addr)]];
     }
     return section;
 }
@@ -736,7 +745,7 @@ hwaddr memory_region_section_get_iotlb(CPUArchState *env,
             iotlb |= PHYS_SECTION_ROM;
         }
     } else {
-        iotlb = section - phys_sections;
+        iotlb = section - cur_map.sections;
         iotlb += xlat;
     }
 
@@ -769,16 +778,17 @@ static uint16_t phys_section_add(MemoryRegionSection *section)
      * pointer to produce the iotlb entries.  Thus it should
      * never overflow into the page-aligned value.
      */
-    assert(phys_sections_nb < TARGET_PAGE_SIZE);
+    assert(next_map.sections_nb < TARGET_PAGE_SIZE);
 
-    if (phys_sections_nb == phys_sections_nb_alloc) {
-        phys_sections_nb_alloc = MAX(phys_sections_nb_alloc * 2, 16);
-        phys_sections = g_renew(MemoryRegionSection, phys_sections,
-                                phys_sections_nb_alloc);
+    if (next_map.sections_nb == next_map.sections_nb_alloc) {
+        next_map.sections_nb_alloc = MAX(next_map.sections_nb_alloc * 2,
+                                         16);
+        next_map.sections = g_renew(MemoryRegionSection, next_map.sections,
+                                    next_map.sections_nb_alloc);
     }
-    phys_sections[phys_sections_nb] = *section;
+    next_map.sections[next_map.sections_nb] = *section;
     memory_region_ref(section->mr);
-    return phys_sections_nb++;
+    return next_map.sections_nb++;
 }
 
 static void phys_section_destroy(MemoryRegion *mr)
@@ -792,13 +802,14 @@ static void phys_section_destroy(MemoryRegion *mr)
     }
 }
 
-static void phys_sections_clear(void)
+static void phys_sections_clear(PhysPageMap *map)
 {
-    while (phys_sections_nb > 0) {
-        MemoryRegionSection *section = &phys_sections[--phys_sections_nb];
+    while (map->sections_nb > 0) {
+        MemoryRegionSection *section = &map->sections[--map->sections_nb];
         phys_section_destroy(section->mr);
     }
-    phys_map_nodes_nb = 0;
+    g_free(map->sections);
+    g_free(map->nodes);
 }
 
 static void register_subpage(AddressSpaceDispatch *d, MemoryRegionSection *section)
@@ -806,7 +817,8 @@ static void register_subpage(AddressSpaceDispatch *d, MemoryRegionSection *secti
     subpage_t *subpage;
     hwaddr base = section->offset_within_address_space
         & TARGET_PAGE_MASK;
-    MemoryRegionSection *existing = phys_page_find(d, base >> TARGET_PAGE_BITS);
+    MemoryRegionSection *existing = phys_page_find(d->phys_map, base >> TARGET_PAGE_BITS,
+                                                   next_map.nodes, next_map.sections);
     MemoryRegionSection subsection = {
         .offset_within_address_space = base,
         .size = int128_make64(TARGET_PAGE_SIZE),
@@ -1689,7 +1701,7 @@ static uint16_t dummy_section(MemoryRegion *mr)
 
 MemoryRegion *iotlb_to_region(hwaddr index)
 {
-    return phys_sections[index & ~TARGET_PAGE_MASK].mr;
+    return cur_map.sections[index & ~TARGET_PAGE_MASK].mr;
 }
 
 static void io_mem_init(void)
@@ -1714,7 +1726,7 @@ static void core_begin(MemoryListener *listener)
 {
     uint16_t n;
 
-    phys_sections_clear();
+    memset(&next_map, 0, sizeof(next_map));
     n = dummy_section(&io_mem_unassigned);
     assert(n == PHYS_SECTION_UNASSIGNED);
     n = dummy_section(&io_mem_notdirty);
@@ -1725,6 +1737,16 @@ static void core_begin(MemoryListener *listener)
     assert(n == PHYS_SECTION_WATCH);
 }
 
+/* This listener's commit run after the other AddressSpaceDispatch listeners'.
+ * All AddressSpaceDispatch instances have switched to the next map.
+ */
+static void core_commit(MemoryListener *listener)
+{
+    PhysPageMap info = cur_map;
+    cur_map = next_map;
+    phys_sections_clear(&info);
+}
+
 static void tcg_commit(MemoryListener *listener)
 {
     CPUArchState *env;
@@ -1749,6 +1771,7 @@ static void core_log_global_stop(MemoryListener *listener)
 
 static MemoryListener core_memory_listener = {
     .begin = core_begin,
+    .commit = core_commit,
     .log_global_start = core_log_global_start,
     .log_global_stop = core_log_global_stop,
     .priority = 1,
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 22/30] memory: move MemoryListener declaration earlier
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (20 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 21/30] exec: separate current memory map from the one being built Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-07-02 14:41   ` Jan Kiszka
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 23/30] exec: move listener from AddressSpaceDispatch to AddressSpace Paolo Bonzini
                   ` (7 subsequent siblings)
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 include/exec/memory.h | 66 +++++++++++++++++++++++++--------------------------
 1 file changed, 33 insertions(+), 33 deletions(-)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index aa7a922..913ac45 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -164,6 +164,39 @@ struct MemoryRegion {
     NotifierList iommu_notify;
 };
 
+typedef struct MemoryListener MemoryListener;
+
+/**
+ * MemoryListener: callbacks structure for updates to the physical memory map
+ *
+ * Allows a component to adjust to changes in the guest-visible memory map.
+ * Use with memory_listener_register() and memory_listener_unregister().
+ */
+struct MemoryListener {
+    void (*begin)(MemoryListener *listener);
+    void (*commit)(MemoryListener *listener);
+    void (*region_add)(MemoryListener *listener, MemoryRegionSection *section);
+    void (*region_del)(MemoryListener *listener, MemoryRegionSection *section);
+    void (*region_nop)(MemoryListener *listener, MemoryRegionSection *section);
+    void (*log_start)(MemoryListener *listener, MemoryRegionSection *section);
+    void (*log_stop)(MemoryListener *listener, MemoryRegionSection *section);
+    void (*log_sync)(MemoryListener *listener, MemoryRegionSection *section);
+    void (*log_global_start)(MemoryListener *listener);
+    void (*log_global_stop)(MemoryListener *listener);
+    void (*eventfd_add)(MemoryListener *listener, MemoryRegionSection *section,
+                        bool match_data, uint64_t data, EventNotifier *e);
+    void (*eventfd_del)(MemoryListener *listener, MemoryRegionSection *section,
+                        bool match_data, uint64_t data, EventNotifier *e);
+    void (*coalesced_mmio_add)(MemoryListener *listener, MemoryRegionSection *section,
+                               hwaddr addr, hwaddr len);
+    void (*coalesced_mmio_del)(MemoryListener *listener, MemoryRegionSection *section,
+                               hwaddr addr, hwaddr len);
+    /* Lower = earlier (during add), later (during del) */
+    unsigned priority;
+    AddressSpace *address_space_filter;
+    QTAILQ_ENTRY(MemoryListener) link;
+};
+
 /**
  * AddressSpace: describes a mapping of addresses to #MemoryRegion objects
  */
@@ -202,39 +235,6 @@ struct MemoryRegionSection {
     bool readonly;
 };
 
-typedef struct MemoryListener MemoryListener;
-
-/**
- * MemoryListener: callbacks structure for updates to the physical memory map
- *
- * Allows a component to adjust to changes in the guest-visible memory map.
- * Use with memory_listener_register() and memory_listener_unregister().
- */
-struct MemoryListener {
-    void (*begin)(MemoryListener *listener);
-    void (*commit)(MemoryListener *listener);
-    void (*region_add)(MemoryListener *listener, MemoryRegionSection *section);
-    void (*region_del)(MemoryListener *listener, MemoryRegionSection *section);
-    void (*region_nop)(MemoryListener *listener, MemoryRegionSection *section);
-    void (*log_start)(MemoryListener *listener, MemoryRegionSection *section);
-    void (*log_stop)(MemoryListener *listener, MemoryRegionSection *section);
-    void (*log_sync)(MemoryListener *listener, MemoryRegionSection *section);
-    void (*log_global_start)(MemoryListener *listener);
-    void (*log_global_stop)(MemoryListener *listener);
-    void (*eventfd_add)(MemoryListener *listener, MemoryRegionSection *section,
-                        bool match_data, uint64_t data, EventNotifier *e);
-    void (*eventfd_del)(MemoryListener *listener, MemoryRegionSection *section,
-                        bool match_data, uint64_t data, EventNotifier *e);
-    void (*coalesced_mmio_add)(MemoryListener *listener, MemoryRegionSection *section,
-                               hwaddr addr, hwaddr len);
-    void (*coalesced_mmio_del)(MemoryListener *listener, MemoryRegionSection *section,
-                               hwaddr addr, hwaddr len);
-    /* Lower = earlier (during add), later (during del) */
-    unsigned priority;
-    AddressSpace *address_space_filter;
-    QTAILQ_ENTRY(MemoryListener) link;
-};
-
 /**
  * memory_region_init: Initialize a memory region
  *
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 23/30] exec: move listener from AddressSpaceDispatch to AddressSpace
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (21 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 22/30] memory: move MemoryListener declaration earlier Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-07-02 14:41   ` Jan Kiszka
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 24/30] exec: separate current radix tree from the one being built Paolo Bonzini
                   ` (6 subsequent siblings)
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

This will help having two copies of AddressSpaceDispatch during the
recreation of the radix tree (one being built, and one that is complete
and accessed under RCU).  We do not want to have to unregister and
re-register the listener.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c                | 17 +++++++++--------
 include/exec/memory.h |  2 ++
 2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/exec.c b/exec.c
index f138e56..dffdf23 100644
--- a/exec.c
+++ b/exec.c
@@ -99,7 +99,6 @@ struct AddressSpaceDispatch {
      * The bottom level has pointers to MemoryRegionSections.
      */
     PhysPageEntry phys_map;
-    MemoryListener listener;
     AddressSpace *as;
 };
 
@@ -855,7 +854,8 @@ static void register_multipage(AddressSpaceDispatch *d,
 
 static void mem_add(MemoryListener *listener, MemoryRegionSection *section)
 {
-    AddressSpaceDispatch *d = container_of(listener, AddressSpaceDispatch, listener);
+    AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
+    AddressSpaceDispatch *d = as->dispatch;
     MemoryRegionSection now = *section, remain = *section;
     Int128 page_size = int128_make64(TARGET_PAGE_SIZE);
 
@@ -1717,7 +1717,8 @@ static void io_mem_init(void)
 
 static void mem_begin(MemoryListener *listener)
 {
-    AddressSpaceDispatch *d = container_of(listener, AddressSpaceDispatch, listener);
+    AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
+    AddressSpaceDispatch *d = as->dispatch;
 
     d->phys_map.ptr = PHYS_MAP_NODE_NIL;
 }
@@ -1786,22 +1787,22 @@ void address_space_init_dispatch(AddressSpace *as)
     AddressSpaceDispatch *d = g_new(AddressSpaceDispatch, 1);
 
     d->phys_map  = (PhysPageEntry) { .ptr = PHYS_MAP_NODE_NIL, .is_leaf = 0 };
-    d->listener = (MemoryListener) {
+    d->as = as;
+    as->dispatch = d;
+    as->dispatch_listener = (MemoryListener) {
         .begin = mem_begin,
         .region_add = mem_add,
         .region_nop = mem_add,
         .priority = 0,
     };
-    d->as = as;
-    as->dispatch = d;
-    memory_listener_register(&d->listener, as);
+    memory_listener_register(&as->dispatch_listener, as);
 }
 
 void address_space_destroy_dispatch(AddressSpace *as)
 {
     AddressSpaceDispatch *d = as->dispatch;
 
-    memory_listener_unregister(&d->listener);
+    memory_listener_unregister(&as->dispatch_listener);
     g_free(d);
     as->dispatch = NULL;
 }
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 913ac45..1cd1f50 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -212,6 +212,8 @@ struct AddressSpace {
     int ioeventfd_nb;
     struct MemoryRegionIoeventfd *ioeventfds;
     struct AddressSpaceDispatch *dispatch;
+    MemoryListener dispatch_listener;
+
     QTAILQ_ENTRY(AddressSpace) address_spaces_link;
 };
 
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 24/30] exec: separate current radix tree from the one being built
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (22 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 23/30] exec: move listener from AddressSpaceDispatch to AddressSpace Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-07-02 14:41   ` Jan Kiszka
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 25/30] exec: put memory map in AddressSpaceDispatch Paolo Bonzini
                   ` (5 subsequent siblings)
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

This same treatment previously done to phys_node_map and phys_sections
is now applied to the dispatch field of AddressSpace.  Topology updates
use as->next_dispatch while accesses use as->dispatch.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c                | 23 ++++++++++++++++-------
 include/exec/memory.h |  1 +
 2 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/exec.c b/exec.c
index dffdf23..0d852ee 100644
--- a/exec.c
+++ b/exec.c
@@ -855,7 +855,7 @@ static void register_multipage(AddressSpaceDispatch *d,
 static void mem_add(MemoryListener *listener, MemoryRegionSection *section)
 {
     AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
-    AddressSpaceDispatch *d = as->dispatch;
+    AddressSpaceDispatch *d = as->next_dispatch;
     MemoryRegionSection now = *section, remain = *section;
     Int128 page_size = int128_make64(TARGET_PAGE_SIZE);
 
@@ -1718,9 +1718,21 @@ static void io_mem_init(void)
 static void mem_begin(MemoryListener *listener)
 {
     AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
+    AddressSpaceDispatch *d = g_new(AddressSpaceDispatch, 1);
+
+    d->phys_map  = (PhysPageEntry) { .ptr = PHYS_MAP_NODE_NIL, .is_leaf = 0 };
+    d->as = as;
+    as->next_dispatch = d;
+}
+
+static void mem_commit(MemoryListener *listener)
+{
+    AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
     AddressSpaceDispatch *d = as->dispatch;
 
-    d->phys_map.ptr = PHYS_MAP_NODE_NIL;
+    /* cur_map will soon be switched to next_map, too.  */
+    as->dispatch = as->next_dispatch;
+    g_free(d);
 }
 
 static void core_begin(MemoryListener *listener)
@@ -1784,13 +1796,10 @@ static MemoryListener tcg_memory_listener = {
 
 void address_space_init_dispatch(AddressSpace *as)
 {
-    AddressSpaceDispatch *d = g_new(AddressSpaceDispatch, 1);
-
-    d->phys_map  = (PhysPageEntry) { .ptr = PHYS_MAP_NODE_NIL, .is_leaf = 0 };
-    d->as = as;
-    as->dispatch = d;
+    as->dispatch = NULL;
     as->dispatch_listener = (MemoryListener) {
         .begin = mem_begin,
+        .commit = mem_commit,
         .region_add = mem_add,
         .region_nop = mem_add,
         .priority = 0,
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 1cd1f50..b21a460 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -212,6 +212,7 @@ struct AddressSpace {
     int ioeventfd_nb;
     struct MemoryRegionIoeventfd *ioeventfds;
     struct AddressSpaceDispatch *dispatch;
+    struct AddressSpaceDispatch *next_dispatch;
     MemoryListener dispatch_listener;
 
     QTAILQ_ENTRY(AddressSpace) address_spaces_link;
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 25/30] exec: put memory map in AddressSpaceDispatch
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (23 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 24/30] exec: separate current radix tree from the one being built Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-07-02 14:42   ` Jan Kiszka
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 26/30] exec: remove cur_map Paolo Bonzini
                   ` (4 subsequent siblings)
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

This lets us get a consistent (phys_map, nodes, sections) using
RCU.  After this patch, cur_map is not used anymore except for freeing
it at the end of the topology update.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c | 28 +++++++++++++++++-----------
 1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/exec.c b/exec.c
index 0d852ee..581f0c4 100644
--- a/exec.c
+++ b/exec.c
@@ -94,11 +94,15 @@ struct PhysPageEntry {
     uint16_t ptr : 15;
 };
 
+typedef PhysPageEntry Node[L2_SIZE];
+
 struct AddressSpaceDispatch {
     /* This is a multi-level map on the physical address space.
      * The bottom level has pointers to MemoryRegionSections.
      */
     PhysPageEntry phys_map;
+    Node *nodes;
+    MemoryRegionSection *sections;
     AddressSpace *as;
 };
 
@@ -115,8 +119,6 @@ typedef struct subpage_t {
 #define PHYS_SECTION_ROM 2
 #define PHYS_SECTION_WATCH 3
 
-typedef PhysPageEntry Node[L2_SIZE];
-
 typedef struct PhysPageMap {
     unsigned sections_nb;
     unsigned sections_nb_alloc;
@@ -238,14 +240,15 @@ static MemoryRegionSection *address_space_lookup_region(AddressSpace *as,
                                                         hwaddr addr,
                                                         bool resolve_subpage)
 {
+    AddressSpaceDispatch *d = as->dispatch;
     MemoryRegionSection *section;
     subpage_t *subpage;
 
-    section = phys_page_find(as->dispatch->phys_map, addr >> TARGET_PAGE_BITS,
-                             cur_map.nodes, cur_map.sections);
+    section = phys_page_find(d->phys_map, addr >> TARGET_PAGE_BITS,
+                             d->nodes, d->sections);
     if (resolve_subpage && section->mr->subpage) {
         subpage = container_of(section->mr, subpage_t, iomem);
-        section = &cur_map.sections[subpage->sub_section[SUBPAGE_IDX(addr)]];
+        section = &d->sections[subpage->sub_section[SUBPAGE_IDX(addr)]];
     }
     return section;
 }
@@ -744,7 +747,7 @@ hwaddr memory_region_section_get_iotlb(CPUArchState *env,
             iotlb |= PHYS_SECTION_ROM;
         }
     } else {
-        iotlb = section - cur_map.sections;
+        iotlb = section - address_space_memory.dispatch->sections;
         iotlb += xlat;
     }
 
@@ -1701,7 +1704,7 @@ static uint16_t dummy_section(MemoryRegion *mr)
 
 MemoryRegion *iotlb_to_region(hwaddr index)
 {
-    return cur_map.sections[index & ~TARGET_PAGE_MASK].mr;
+    return address_space_memory.dispatch->sections[index & ~TARGET_PAGE_MASK].mr;
 }
 
 static void io_mem_init(void)
@@ -1728,11 +1731,14 @@ static void mem_begin(MemoryListener *listener)
 static void mem_commit(MemoryListener *listener)
 {
     AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
-    AddressSpaceDispatch *d = as->dispatch;
+    AddressSpaceDispatch *cur = as->dispatch;
+    AddressSpaceDispatch *next = as->next_dispatch;
 
-    /* cur_map will soon be switched to next_map, too.  */
-    as->dispatch = as->next_dispatch;
-    g_free(d);
+    next->nodes = next_map.nodes;
+    next->sections = next_map.sections;
+
+    as->dispatch = next;
+    g_free(cur);
 }
 
 static void core_begin(MemoryListener *listener)
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 26/30] exec: remove cur_map
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (24 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 25/30] exec: put memory map in AddressSpaceDispatch Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 27/30] exec: change some APIs to take AddressSpaceDispatch Paolo Bonzini
                   ` (3 subsequent siblings)
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

cur_map is not used anymore; instead, each AddressSpaceDispatch
has its own nodes/sections pair.  The priorities of the
MemoryListeners, and in the future RCU, guarantee that the
nodes/sections are not freed while they are still in use.

(In fact, next_map itself is not needed except to free the data on the
next update).

To avoid incorrect use, replace cur_map with a temporary copy that
is only valid while the topology is being updated.  If you use it,
the name prev_map makes it clear that you're doing something weird.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/exec.c b/exec.c
index 581f0c4..7f87e16 100644
--- a/exec.c
+++ b/exec.c
@@ -128,7 +128,7 @@ typedef struct PhysPageMap {
     MemoryRegionSection *sections;
 } PhysPageMap;
 
-static PhysPageMap cur_map;
+static PhysPageMap *prev_map;
 static PhysPageMap next_map;
 
 #define PHYS_MAP_NODE_NIL (((uint16_t)~0) >> 1)
@@ -804,7 +804,7 @@ static void phys_section_destroy(MemoryRegion *mr)
     }
 }
 
-static void phys_sections_clear(PhysPageMap *map)
+static void phys_sections_free(PhysPageMap *map)
 {
     while (map->sections_nb > 0) {
         MemoryRegionSection *section = &map->sections[--map->sections_nb];
@@ -812,6 +812,7 @@ static void phys_sections_clear(PhysPageMap *map)
     }
     g_free(map->sections);
     g_free(map->nodes);
+    g_free(map);
 }
 
 static void register_subpage(AddressSpaceDispatch *d, MemoryRegionSection *section)
@@ -1745,6 +1746,9 @@ static void core_begin(MemoryListener *listener)
 {
     uint16_t n;
 
+    prev_map = g_new(PhysPageMap, 1);
+    *prev_map = next_map;
+
     memset(&next_map, 0, sizeof(next_map));
     n = dummy_section(&io_mem_unassigned);
     assert(n == PHYS_SECTION_UNASSIGNED);
@@ -1761,9 +1765,7 @@ static void core_begin(MemoryListener *listener)
  */
 static void core_commit(MemoryListener *listener)
 {
-    PhysPageMap info = cur_map;
-    cur_map = next_map;
-    phys_sections_clear(&info);
+    phys_sections_free(prev_map);
 }
 
 static void tcg_commit(MemoryListener *listener)
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 27/30] exec: change some APIs to take AddressSpaceDispatch
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (25 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 26/30] exec: remove cur_map Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-07-02 14:47   ` Jan Kiszka
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 28/30] exec: change iotlb " Paolo Bonzini
                   ` (2 subsequent siblings)
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/exec.c b/exec.c
index 7f87e16..528c4d7 100644
--- a/exec.c
+++ b/exec.c
@@ -236,11 +236,10 @@ bool memory_region_is_unassigned(MemoryRegion *mr)
         && mr != &io_mem_watch;
 }
 
-static MemoryRegionSection *address_space_lookup_region(AddressSpace *as,
+static MemoryRegionSection *address_space_lookup_region(AddressSpaceDispatch *d,
                                                         hwaddr addr,
                                                         bool resolve_subpage)
 {
-    AddressSpaceDispatch *d = as->dispatch;
     MemoryRegionSection *section;
     subpage_t *subpage;
 
@@ -254,13 +253,13 @@ static MemoryRegionSection *address_space_lookup_region(AddressSpace *as,
 }
 
 static MemoryRegionSection *
-address_space_translate_internal(AddressSpace *as, hwaddr addr, hwaddr *xlat,
+address_space_translate_internal(AddressSpaceDispatch *d, hwaddr addr, hwaddr *xlat,
                                  hwaddr *plen, bool resolve_subpage)
 {
     MemoryRegionSection *section;
     Int128 diff;
 
-    section = address_space_lookup_region(as, addr, resolve_subpage);
+    section = address_space_lookup_region(d, addr, resolve_subpage);
     /* Compute offset within MemoryRegionSection */
     addr -= section->offset_within_address_space;
 
@@ -282,7 +281,7 @@ MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
     hwaddr len = *plen;
 
     for (;;) {
-        section = address_space_translate_internal(as, addr, &addr, plen, true);
+        section = address_space_translate_internal(as->dispatch, addr, &addr, plen, true);
         mr = section->mr;
 
         if (!mr->iommu_ops) {
@@ -311,7 +310,7 @@ address_space_translate_for_iotlb(AddressSpace *as, hwaddr addr, hwaddr *xlat,
                                   hwaddr *plen)
 {
     MemoryRegionSection *section;
-    section = address_space_translate_internal(as, addr, xlat, plen, false);
+    section = address_space_translate_internal(as->dispatch, addr, xlat, plen, false);
 
     assert(!section->mr->iommu_ops);
     return section;
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 28/30] exec: change iotlb APIs to take AddressSpaceDispatch
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (26 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 27/30] exec: change some APIs to take AddressSpaceDispatch Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-07-02 10:00   ` Jan Kiszka
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 29/30] exec: add a reference to the region returned by address_space_translate Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 30/30] exec: put address space dispatch under RCU critical section Paolo Bonzini
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

This makes it possible to start following RCU rules, which require
not dereferencing as->dispatch more than once.  It is not covering
the whole of TCG, since the TLB data structures are not RCU-friendly,
but it is enough for exec.c.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 cputlb.c              | 7 ++++---
 exec.c                | 9 +++++----
 include/exec/cputlb.h | 9 ++++++---
 3 files changed, 15 insertions(+), 10 deletions(-)

diff --git a/cputlb.c b/cputlb.c
index 51381ae..82875b1 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -253,6 +253,7 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
                   hwaddr paddr, int prot,
                   int mmu_idx, target_ulong size)
 {
+    AddressSpaceDispatch *d;
     MemoryRegionSection *section;
     unsigned int index;
     target_ulong address;
@@ -267,8 +268,8 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
     }
 
     sz = size;
-    section = address_space_translate_for_iotlb(&address_space_memory, paddr,
-                                                &xlat, &sz);
+    d = address_space_memory.dispatch;
+    section = address_space_translate_for_iotlb(d, paddr, &xlat, &sz);
     assert(sz >= TARGET_PAGE_SIZE);
 
 #if defined(DEBUG_TLB)
@@ -288,7 +289,7 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
     }
 
     code_address = address;
-    iotlb = memory_region_section_get_iotlb(env, section, vaddr, paddr, xlat,
+    iotlb = memory_region_section_get_iotlb(d, env, section, vaddr, paddr, xlat,
                                             prot, &address);
 
     index = (vaddr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1);
diff --git a/exec.c b/exec.c
index 528c4d7..3e1a576 100644
--- a/exec.c
+++ b/exec.c
@@ -306,11 +306,11 @@ MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
 }
 
 MemoryRegionSection *
-address_space_translate_for_iotlb(AddressSpace *as, hwaddr addr, hwaddr *xlat,
+address_space_translate_for_iotlb(AddressSpaceDispatch *d, hwaddr addr, hwaddr *xlat,
                                   hwaddr *plen)
 {
     MemoryRegionSection *section;
-    section = address_space_translate_internal(as->dispatch, addr, xlat, plen, false);
+    section = address_space_translate_internal(d, addr, xlat, plen, false);
 
     assert(!section->mr->iommu_ops);
     return section;
@@ -726,7 +726,8 @@ static int cpu_physical_memory_set_dirty_tracking(int enable)
     return ret;
 }
 
-hwaddr memory_region_section_get_iotlb(CPUArchState *env,
+hwaddr memory_region_section_get_iotlb(AddressSpaceDispatch *d,
+                                       CPUArchState *env,
                                        MemoryRegionSection *section,
                                        target_ulong vaddr,
                                        hwaddr paddr, hwaddr xlat,
@@ -746,7 +747,7 @@ hwaddr memory_region_section_get_iotlb(CPUArchState *env,
             iotlb |= PHYS_SECTION_ROM;
         }
     } else {
-        iotlb = section - address_space_memory.dispatch->sections;
+        iotlb = section - d->sections;
         iotlb += xlat;
     }
 
diff --git a/include/exec/cputlb.h b/include/exec/cputlb.h
index e21cb60..968b6a4 100644
--- a/include/exec/cputlb.h
+++ b/include/exec/cputlb.h
@@ -31,12 +31,15 @@ void tlb_set_dirty(CPUArchState *env, target_ulong vaddr);
 extern int tlb_flush_count;
 
 /* exec.c */
+typedef struct AddressSpaceDispatch AddressSpaceDispatch;
+
 void tb_flush_jmp_cache(CPUArchState *env, target_ulong addr);
 
 MemoryRegionSection *
-address_space_translate_for_iotlb(AddressSpace *as, hwaddr addr, hwaddr *xlat,
-                                  hwaddr *plen);
-hwaddr memory_region_section_get_iotlb(CPUArchState *env,
+address_space_translate_for_iotlb(AddressSpaceDispatch *d, hwaddr addr,
+                                  hwaddr *xlat, hwaddr *plen);
+hwaddr memory_region_section_get_iotlb(AddressSpaceDispatch *d,
+                                       CPUArchState *env,
                                        MemoryRegionSection *section,
                                        target_ulong vaddr,
                                        hwaddr paddr, hwaddr xlat,
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 29/30] exec: add a reference to the region returned by address_space_translate
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (27 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 28/30] exec: change iotlb " Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 30/30] exec: put address space dispatch under RCU critical section Paolo Bonzini
  29 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

Once address_space_translate will only be protected by RCU, the returned
MemoryRegion might disappear as soon as the RCU read-side critical section
ends.  Avoid this by adding a reference to the region, and dropping it
in the caller of address_space_translate.

This generalizes what was done for address_space_map to other callers of
address_space_translate.  It is necessary to call address_space_translate
out of the BQL.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c                | 19 +++++++++++++++----
 include/exec/memory.h |  3 ++-
 2 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/exec.c b/exec.c
index 3e1a576..e564014 100644
--- a/exec.c
+++ b/exec.c
@@ -302,6 +302,7 @@ MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
 
     *plen = len;
     *xlat = addr;
+    memory_region_ref(mr);
     return mr;
 }
 
@@ -1990,6 +1991,7 @@ bool address_space_rw(AddressSpace *as, hwaddr addr, uint8_t *buf,
                 memcpy(buf, ptr, l);
             }
         }
+        memory_region_unref(mr);
         len -= l;
         buf += l;
         addr += l;
@@ -2146,7 +2148,7 @@ void *address_space_map(AddressSpace *as,
         bounce.addr = addr;
         bounce.len = l;
 
-        memory_region_ref(mr);
+        /* Keep the reference to mr until address_space_unmap.  */
         bounce.mr = mr;
         if (!is_write) {
             address_space_read(as, addr, bounce.buffer, l);
@@ -2169,12 +2171,13 @@ void *address_space_map(AddressSpace *as,
 
         l = len;
         this_mr = address_space_translate(as, addr, &xlat, &l, is_write);
+        memory_region_unref(this_mr);
         if (this_mr != mr || xlat != base + done) {
             break;
         }
     }
 
-    memory_region_ref(mr);
+    /* Keep the reference to mr until address_space_unmap.  */
     *plen = done;
     return qemu_ram_ptr_length(raddr + base, plen);
 }
@@ -2272,6 +2275,7 @@ static inline uint32_t ldl_phys_internal(hwaddr addr,
             break;
         }
     }
+    memory_region_unref(mr);
     return val;
 }
 
@@ -2331,6 +2335,7 @@ static inline uint64_t ldq_phys_internal(hwaddr addr,
             break;
         }
     }
+    memory_region_unref(mr);
     return val;
 }
 
@@ -2398,6 +2403,7 @@ static inline uint32_t lduw_phys_internal(hwaddr addr,
             break;
         }
     }
+    memory_region_unref(mr);
     return val;
 }
 
@@ -2445,6 +2451,7 @@ void stl_phys_notdirty(hwaddr addr, uint32_t val)
             }
         }
     }
+    memory_region_unref(mr);
 }
 
 /* warning: addr must be aligned */
@@ -2486,6 +2493,7 @@ static inline void stl_phys_internal(hwaddr addr, uint32_t val,
         }
         invalidate_and_set_dirty(addr1, 4);
     }
+    memory_region_unref(mr);
 }
 
 void stl_phys(hwaddr addr, uint32_t val)
@@ -2549,6 +2557,7 @@ static inline void stw_phys_internal(hwaddr addr, uint32_t val,
         }
         invalidate_and_set_dirty(addr1, 2);
     }
+    memory_region_unref(mr);
 }
 
 void stw_phys(hwaddr addr, uint32_t val)
@@ -2638,11 +2647,13 @@ bool cpu_physical_memory_is_io(hwaddr phys_addr)
 {
     MemoryRegion*mr;
     hwaddr l = 1;
+    bool res;
 
     mr = address_space_translate(&address_space_memory,
                                  phys_addr, &phys_addr, &l, false);
 
-    return !(memory_region_is_ram(mr) ||
-             memory_region_is_romd(mr));
+    res = !(memory_region_is_ram(mr) || memory_region_is_romd(mr));
+    memory_region_unref(mr);
+    return res;
 }
 #endif
diff --git a/include/exec/memory.h b/include/exec/memory.h
index b21a460..66226b1 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -986,7 +986,8 @@ bool address_space_write(AddressSpace *as, hwaddr addr,
 bool address_space_read(AddressSpace *as, hwaddr addr, uint8_t *buf, int len);
 
 /* address_space_translate: translate an address range into an address space
- * into a MemoryRegion and an address range into that section
+ * into a MemoryRegion and an address range into that section.  Add a reference
+ * to that region.
  *
  * @as: #AddressSpace to be accessed
  * @addr: address within that address space
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* [Qemu-devel] [PATCH 30/30] exec: put address space dispatch under RCU critical section
  2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
                   ` (28 preceding siblings ...)
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 29/30] exec: add a reference to the region returned by address_space_translate Paolo Bonzini
@ 2013-06-28 18:26 ` Paolo Bonzini
  2013-06-28 19:38   ` Jan Kiszka
  29 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-06-28 18:26 UTC (permalink / raw)
  To: qemu-devel

With this change, address space dispatch can be moved outside the
BQL.  The actual I/O would still have to happen within the lock.

The next step would be to introduce a function that can only
be called from outside the BQL, address_space_rw_unlocked.  The
function would do something like

    mr = address_space_translate(...)
    if (!mr->unlocked) {
        locked = true;
        qemu_mutex_lock_iothread();
    }
    ...dispatch...
    if (locked) {
        qemu_mutex_unlock_iothread();
    }

(Note that subpages are already ready to be changed to unlocked
access).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 cputlb.c             |  4 +++-
 exec.c               | 30 ++++++++++++++++++++++++++----
 hw/ppc/spapr_iommu.c | 10 +++++++++-
 3 files changed, 38 insertions(+), 6 deletions(-)

diff --git a/cputlb.c b/cputlb.c
index 82875b1..97bfe70 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -267,8 +267,9 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
         tlb_add_large_page(env, vaddr, size);
     }
 
+    rcu_read_lock();
     sz = size;
-    d = address_space_memory.dispatch;
+    d = rcu_dereference(&address_space_memory.dispatch);
     section = address_space_translate_for_iotlb(d, paddr, &xlat, &sz);
     assert(sz >= TARGET_PAGE_SIZE);
 
@@ -321,6 +322,7 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
     } else {
         te->addr_write = -1;
     }
+    rcu_read_unlock();
 }
 
 /* NOTE: this function can trigger an exception */
diff --git a/exec.c b/exec.c
index e564014..8c6f925 100644
--- a/exec.c
+++ b/exec.c
@@ -97,6 +97,8 @@ struct PhysPageEntry {
 typedef PhysPageEntry Node[L2_SIZE];
 
 struct AddressSpaceDispatch {
+    struct rcu_head rcu;
+
     /* This is a multi-level map on the physical address space.
      * The bottom level has pointers to MemoryRegionSections.
      */
@@ -120,6 +122,8 @@ typedef struct subpage_t {
 #define PHYS_SECTION_WATCH 3
 
 typedef struct PhysPageMap {
+    struct rcu_head rcu;
+
     unsigned sections_nb;
     unsigned sections_nb_alloc;
     unsigned nodes_nb;
@@ -236,6 +240,7 @@ bool memory_region_is_unassigned(MemoryRegion *mr)
         && mr != &io_mem_watch;
 }
 
+/* Called from RCU critical section */
 static MemoryRegionSection *address_space_lookup_region(AddressSpaceDispatch *d,
                                                         hwaddr addr,
                                                         bool resolve_subpage)
@@ -252,6 +257,7 @@ static MemoryRegionSection *address_space_lookup_region(AddressSpaceDispatch *d,
     return section;
 }
 
+/* Called from RCU critical section */
 static MemoryRegionSection *
 address_space_translate_internal(AddressSpaceDispatch *d, hwaddr addr, hwaddr *xlat,
                                  hwaddr *plen, bool resolve_subpage)
@@ -280,8 +286,10 @@ MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
     MemoryRegion *mr;
     hwaddr len = *plen;
 
+    rcu_read_lock();
     for (;;) {
-        section = address_space_translate_internal(as->dispatch, addr, &addr, plen, true);
+        AddressSpaceDispatch *d = rcu_dereference(&as->dispatch);
+        section = address_space_translate_internal(d, addr, &addr, plen, true);
         mr = section->mr;
 
         if (!mr->iommu_ops) {
@@ -303,9 +311,11 @@ MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
     *plen = len;
     *xlat = addr;
     memory_region_ref(mr);
+    rcu_read_unlock();
     return mr;
 }
 
+/* Called from RCU critical section */
 MemoryRegionSection *
 address_space_translate_for_iotlb(AddressSpaceDispatch *d, hwaddr addr, hwaddr *xlat,
                                   hwaddr *plen)
@@ -727,6 +737,7 @@ static int cpu_physical_memory_set_dirty_tracking(int enable)
     return ret;
 }
 
+/* Called from RCU critical section */
 hwaddr memory_region_section_get_iotlb(AddressSpaceDispatch *d,
                                        CPUArchState *env,
                                        MemoryRegionSection *section,
@@ -1706,6 +1717,11 @@ static uint16_t dummy_section(MemoryRegion *mr)
 
 MemoryRegion *iotlb_to_region(hwaddr index)
 {
+    /* This assumes that address_space_memory.dispatch->sections
+     * does not change between address_space_translate_for_iotlb and
+     * here.  This is currently true because TCG runs within the
+     * BQL.
+     */
     return address_space_memory.dispatch->sections[index & ~TARGET_PAGE_MASK].mr;
 }
 
@@ -1762,11 +1778,12 @@ static void core_begin(MemoryListener *listener)
 }
 
 /* This listener's commit run after the other AddressSpaceDispatch listeners'.
- * All AddressSpaceDispatch instances have switched to the next map.
+ * All AddressSpaceDispatch instances have switched to the next map, but
+ * there may be concurrent readers.
  */
 static void core_commit(MemoryListener *listener)
 {
-    phys_sections_free(prev_map);
+    call_rcu(prev_map, phys_sections_free, rcu);
 }
 
 static void tcg_commit(MemoryListener *listener)
@@ -1816,13 +1833,18 @@ void address_space_init_dispatch(AddressSpace *as)
     memory_listener_register(&as->dispatch_listener, as);
 }
 
+static void address_space_dispatch_free(AddressSpaceDispatch *d)
+{
+    g_free(d);
+}
+
 void address_space_destroy_dispatch(AddressSpace *as)
 {
     AddressSpaceDispatch *d = as->dispatch;
 
     memory_listener_unregister(&as->dispatch_listener);
-    g_free(d);
     as->dispatch = NULL;
+    call_rcu(d, address_space_dispatch_free, rcu);
 }
 
 static void memory_map_init(void)
diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
index 89b33a5..e16b94b 100644
--- a/hw/ppc/spapr_iommu.c
+++ b/hw/ppc/spapr_iommu.c
@@ -37,6 +37,7 @@ enum sPAPRTCEAccess {
 };
 
 struct sPAPRTCETable {
+    struct rcu_head rcu;
     uint32_t liobn;
     uint32_t window_size;
     sPAPRTCE *table;
@@ -68,6 +69,8 @@ static sPAPRTCETable *spapr_tce_find_by_liobn(uint32_t liobn)
     return NULL;
 }
 
+/* Called within RCU critical section */
+
 static IOMMUTLBEntry spapr_tce_translate_iommu(MemoryRegion *iommu, hwaddr addr)
 {
     sPAPRTCETable *tcet = container_of(iommu, sPAPRTCETable, iommu);
@@ -159,7 +162,7 @@ sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn, size_t wi
     return tcet;
 }
 
-void spapr_tce_free(sPAPRTCETable *tcet)
+static void spapr_tce_do_free(sPAPRTCETable *tcet)
 {
     QLIST_REMOVE(tcet, list);
 
@@ -172,6 +175,11 @@ void spapr_tce_free(sPAPRTCETable *tcet)
     g_free(tcet);
 }
 
+void spapr_tce_free(sPAPRTCETable *tcet)
+{
+    call_rcu(tcet, spapr_tce_do_free, rcu);
+}
+
 MemoryRegion *spapr_tce_get_iommu(sPAPRTCETable *tcet)
 {
     return &tcet->iommu;
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 30/30] exec: put address space dispatch under RCU critical section
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 30/30] exec: put address space dispatch under RCU critical section Paolo Bonzini
@ 2013-06-28 19:38   ` Jan Kiszka
  2013-07-01 11:48     ` Paolo Bonzini
  0 siblings, 1 reply; 68+ messages in thread
From: Jan Kiszka @ 2013-06-28 19:38 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

[-- Attachment #1: Type: text/plain, Size: 4226 bytes --]

On 2013-06-28 20:26, Paolo Bonzini wrote:
> With this change, address space dispatch can be moved outside the
> BQL.  The actual I/O would still have to happen within the lock.
> 
> The next step would be to introduce a function that can only
> be called from outside the BQL, address_space_rw_unlocked.  The
> function would do something like
> 
>     mr = address_space_translate(...)
>     if (!mr->unlocked) {
>         locked = true;
>         qemu_mutex_lock_iothread();
>     }
>     ...dispatch...
>     if (locked) {
>         qemu_mutex_unlock_iothread();
>     }
> 
> (Note that subpages are already ready to be changed to unlocked
> access).
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  cputlb.c             |  4 +++-
>  exec.c               | 30 ++++++++++++++++++++++++++----
>  hw/ppc/spapr_iommu.c | 10 +++++++++-
>  3 files changed, 38 insertions(+), 6 deletions(-)
> 
> diff --git a/cputlb.c b/cputlb.c
> index 82875b1..97bfe70 100644
> --- a/cputlb.c
> +++ b/cputlb.c
> @@ -267,8 +267,9 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
>          tlb_add_large_page(env, vaddr, size);
>      }
>  
> +    rcu_read_lock();
>      sz = size;
> -    d = address_space_memory.dispatch;
> +    d = rcu_dereference(&address_space_memory.dispatch);
>      section = address_space_translate_for_iotlb(d, paddr, &xlat, &sz);
>      assert(sz >= TARGET_PAGE_SIZE);
>  
> @@ -321,6 +322,7 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
>      } else {
>          te->addr_write = -1;
>      }
> +    rcu_read_unlock();
>  }
>  
>  /* NOTE: this function can trigger an exception */
> diff --git a/exec.c b/exec.c
> index e564014..8c6f925 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -97,6 +97,8 @@ struct PhysPageEntry {
>  typedef PhysPageEntry Node[L2_SIZE];
>  
>  struct AddressSpaceDispatch {
> +    struct rcu_head rcu;
> +
>      /* This is a multi-level map on the physical address space.
>       * The bottom level has pointers to MemoryRegionSections.
>       */
> @@ -120,6 +122,8 @@ typedef struct subpage_t {
>  #define PHYS_SECTION_WATCH 3
>  
>  typedef struct PhysPageMap {
> +    struct rcu_head rcu;
> +
>      unsigned sections_nb;
>      unsigned sections_nb_alloc;
>      unsigned nodes_nb;
> @@ -236,6 +240,7 @@ bool memory_region_is_unassigned(MemoryRegion *mr)
>          && mr != &io_mem_watch;
>  }
>  
> +/* Called from RCU critical section */
>  static MemoryRegionSection *address_space_lookup_region(AddressSpaceDispatch *d,
>                                                          hwaddr addr,
>                                                          bool resolve_subpage)
> @@ -252,6 +257,7 @@ static MemoryRegionSection *address_space_lookup_region(AddressSpaceDispatch *d,
>      return section;
>  }
>  
> +/* Called from RCU critical section */
>  static MemoryRegionSection *
>  address_space_translate_internal(AddressSpaceDispatch *d, hwaddr addr, hwaddr *xlat,
>                                   hwaddr *plen, bool resolve_subpage)
> @@ -280,8 +286,10 @@ MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
>      MemoryRegion *mr;
>      hwaddr len = *plen;
>  
> +    rcu_read_lock();
>      for (;;) {
> -        section = address_space_translate_internal(as->dispatch, addr, &addr, plen, true);
> +        AddressSpaceDispatch *d = rcu_dereference(&as->dispatch);
> +        section = address_space_translate_internal(d, addr, &addr, plen, true);
>          mr = section->mr;
>  
>          if (!mr->iommu_ops) {
> @@ -303,9 +311,11 @@ MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
>      *plen = len;
>      *xlat = addr;
>      memory_region_ref(mr);
> +    rcu_read_unlock();
>      return mr;
>  }

At this point, do we still have unowned memory regions? If so, we must
not return them here if the caller is not holding the BQL (which is
supposed to protect their registration/modification/deregistration).

I played with a version today that returns NULL in this case. Then the
caller of address_space_translate can take the BQL and retry the
translation.

Jan



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 01/30] memory: access FlatView from a local variable
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 01/30] memory: access FlatView from a local variable Paolo Bonzini
@ 2013-06-28 20:01   ` Anthony Liguori
  0 siblings, 0 replies; 68+ messages in thread
From: Anthony Liguori @ 2013-06-28 20:01 UTC (permalink / raw)
  To: Paolo Bonzini, qemu-devel

Paolo Bonzini <pbonzini@redhat.com> writes:

> We will soon require accesses to as->current_map to be placed under
> a lock (with reference counting so as to keep the critical section
> small).  To simplify this change, always fetch as->current_map into
> a local variable and access it through that variable.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>

Regards,

Anthony Liguori

> ---
>  memory.c | 31 +++++++++++++++++++++----------
>  1 file changed, 21 insertions(+), 10 deletions(-)
>
> diff --git a/memory.c b/memory.c
> index 688c817..1f44cd1 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -578,13 +578,15 @@ static void address_space_add_del_ioeventfds(AddressSpace *as,
>  
>  static void address_space_update_ioeventfds(AddressSpace *as)
>  {
> +    FlatView *view;
>      FlatRange *fr;
>      unsigned ioeventfd_nb = 0;
>      MemoryRegionIoeventfd *ioeventfds = NULL;
>      AddrRange tmp;
>      unsigned i;
>  
> -    FOR_EACH_FLAT_RANGE(fr, as->current_map) {
> +    view = as->current_map;
> +    FOR_EACH_FLAT_RANGE(fr, view) {
>          for (i = 0; i < fr->mr->ioeventfd_nb; ++i) {
>              tmp = addrrange_shift(fr->mr->ioeventfds[i].addr,
>                                    int128_sub(fr->addr.start,
> @@ -1142,7 +1144,8 @@ void memory_region_sync_dirty_bitmap(MemoryRegion *mr)
>      FlatRange *fr;
>  
>      QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) {
> -        FOR_EACH_FLAT_RANGE(fr, as->current_map) {
> +        FlatView *view = as->current_map;
> +        FOR_EACH_FLAT_RANGE(fr, view) {
>              if (fr->mr == mr) {
>                  MEMORY_LISTENER_UPDATE_REGION(fr, as, Forward, log_sync);
>              }
> @@ -1192,12 +1195,14 @@ void *memory_region_get_ram_ptr(MemoryRegion *mr)
>  
>  static void memory_region_update_coalesced_range_as(MemoryRegion *mr, AddressSpace *as)
>  {
> +    FlatView *view;
>      FlatRange *fr;
>      CoalescedMemoryRange *cmr;
>      AddrRange tmp;
>      MemoryRegionSection section;
>  
> -    FOR_EACH_FLAT_RANGE(fr, as->current_map) {
> +    view = as->current_map;
> +    FOR_EACH_FLAT_RANGE(fr, view) {
>          if (fr->mr == mr) {
>              section = (MemoryRegionSection) {
>                  .address_space = as,
> @@ -1488,9 +1493,9 @@ static int cmp_flatrange_addr(const void *addr_, const void *fr_)
>      return 0;
>  }
>  
> -static FlatRange *address_space_lookup(AddressSpace *as, AddrRange addr)
> +static FlatRange *flatview_lookup(FlatView *view, AddrRange addr)
>  {
> -    return bsearch(&addr, as->current_map->ranges, as->current_map->nr,
> +    return bsearch(&addr, view->ranges, view->nr,
>                     sizeof(FlatRange), cmp_flatrange_addr);
>  }
>  
> @@ -1501,6 +1506,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
>      MemoryRegion *root;
>      AddressSpace *as;
>      AddrRange range;
> +    FlatView *view;
>      FlatRange *fr;
>  
>      addr += mr->addr;
> @@ -1511,13 +1517,14 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
>  
>      as = memory_region_to_address_space(root);
>      range = addrrange_make(int128_make64(addr), int128_make64(size));
> -    fr = address_space_lookup(as, range);
> +
> +    view = as->current_map;
> +    fr = flatview_lookup(view, range);
>      if (!fr) {
>          return ret;
>      }
>  
> -    while (fr > as->current_map->ranges
> -           && addrrange_intersects(fr[-1].addr, range)) {
> +    while (fr > view->ranges && addrrange_intersects(fr[-1].addr, range)) {
>          --fr;
>      }
>  
> @@ -1537,9 +1544,11 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
>  
>  void address_space_sync_dirty_bitmap(AddressSpace *as)
>  {
> +    FlatView *view;
>      FlatRange *fr;
>  
> -    FOR_EACH_FLAT_RANGE(fr, as->current_map) {
> +    view = as->current_map;
> +    FOR_EACH_FLAT_RANGE(fr, view) {
>          MEMORY_LISTENER_UPDATE_REGION(fr, as, Forward, log_sync);
>      }
>  }
> @@ -1559,6 +1568,7 @@ void memory_global_dirty_log_stop(void)
>  static void listener_add_address_space(MemoryListener *listener,
>                                         AddressSpace *as)
>  {
> +    FlatView *view;
>      FlatRange *fr;
>  
>      if (listener->address_space_filter
> @@ -1572,7 +1582,8 @@ static void listener_add_address_space(MemoryListener *listener,
>          }
>      }
>  
> -    FOR_EACH_FLAT_RANGE(fr, as->current_map) {
> +    view = as->current_map;
> +    FOR_EACH_FLAT_RANGE(fr, view) {
>          MemoryRegionSection section = {
>              .mr = fr->mr,
>              .address_space = as,
> -- 
> 1.8.1.4

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 02/30] memory: use a new FlatView pointer on every topology update
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 02/30] memory: use a new FlatView pointer on every topology update Paolo Bonzini
@ 2013-06-28 20:02   ` Anthony Liguori
  0 siblings, 0 replies; 68+ messages in thread
From: Anthony Liguori @ 2013-06-28 20:02 UTC (permalink / raw)
  To: Paolo Bonzini, qemu-devel

Paolo Bonzini <pbonzini@redhat.com> writes:

> This is the first step towards converting as->current_map to
> RCU-style updates, where the FlatView updates run concurrently
> with uses of an old FlatView.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>

Regards,

Anthony Liguori

> ---
>  memory.c | 34 ++++++++++++++++++----------------
>  1 file changed, 18 insertions(+), 16 deletions(-)
>
> diff --git a/memory.c b/memory.c
> index 1f44cd1..319894e 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -276,6 +276,7 @@ static void flatview_destroy(FlatView *view)
>          memory_region_unref(view->ranges[i].mr);
>      }
>      g_free(view->ranges);
> +    g_free(view);
>  }
>  
>  static bool can_merge(FlatRange *r1, FlatRange *r2)
> @@ -512,17 +513,18 @@ static void render_memory_region(FlatView *view,
>  }
>  
>  /* Render a memory topology into a list of disjoint absolute ranges. */
> -static FlatView generate_memory_topology(MemoryRegion *mr)
> +static FlatView *generate_memory_topology(MemoryRegion *mr)
>  {
> -    FlatView view;
> +    FlatView *view;
>  
> -    flatview_init(&view);
> +    view = g_new(FlatView, 1);
> +    flatview_init(view);
>  
>      if (mr) {
> -        render_memory_region(&view, mr, int128_zero(),
> +        render_memory_region(view, mr, int128_zero(),
>                               addrrange_make(int128_zero(), int128_2_64()), false);
>      }
> -    flatview_simplify(&view);
> +    flatview_simplify(view);
>  
>      return view;
>  }
> @@ -610,8 +612,8 @@ static void address_space_update_ioeventfds(AddressSpace *as)
>  }
>  
>  static void address_space_update_topology_pass(AddressSpace *as,
> -                                               FlatView old_view,
> -                                               FlatView new_view,
> +                                               const FlatView *old_view,
> +                                               const FlatView *new_view,
>                                                 bool adding)
>  {
>      unsigned iold, inew;
> @@ -621,14 +623,14 @@ static void address_space_update_topology_pass(AddressSpace *as,
>       * Kill ranges in the old map, and instantiate ranges in the new map.
>       */
>      iold = inew = 0;
> -    while (iold < old_view.nr || inew < new_view.nr) {
> -        if (iold < old_view.nr) {
> -            frold = &old_view.ranges[iold];
> +    while (iold < old_view->nr || inew < new_view->nr) {
> +        if (iold < old_view->nr) {
> +            frold = &old_view->ranges[iold];
>          } else {
>              frold = NULL;
>          }
> -        if (inew < new_view.nr) {
> -            frnew = &new_view.ranges[inew];
> +        if (inew < new_view->nr) {
> +            frnew = &new_view->ranges[inew];
>          } else {
>              frnew = NULL;
>          }
> @@ -674,14 +676,14 @@ static void address_space_update_topology_pass(AddressSpace *as,
>  
>  static void address_space_update_topology(AddressSpace *as)
>  {
> -    FlatView old_view = *as->current_map;
> -    FlatView new_view = generate_memory_topology(as->root);
> +    FlatView *old_view = as->current_map;
> +    FlatView *new_view = generate_memory_topology(as->root);
>  
>      address_space_update_topology_pass(as, old_view, new_view, false);
>      address_space_update_topology_pass(as, old_view, new_view, true);
>  
> -    *as->current_map = new_view;
> -    flatview_destroy(&old_view);
> +    as->current_map = new_view;
> +    flatview_destroy(old_view);
>      address_space_update_ioeventfds(as);
>  }
>  
> -- 
> 1.8.1.4

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 03/30] memory: add reference counting to FlatView
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 03/30] memory: add reference counting to FlatView Paolo Bonzini
@ 2013-06-28 20:07   ` Anthony Liguori
  0 siblings, 0 replies; 68+ messages in thread
From: Anthony Liguori @ 2013-06-28 20:07 UTC (permalink / raw)
  To: Paolo Bonzini, qemu-devel

Paolo Bonzini <pbonzini@redhat.com> writes:

> With this change, a FlatView can be used even after a concurrent
> update has replaced it.  Because we do not have RCU, we use a
> mutex to protect the small critical sections that read/write the
> as->current_map pointer.  Accesses to the FlatView can be done
> outside the mutex.
>
> If a MemoryRegion will be used after the FlatView is unref-ed (or after
> a MemoryListener callback is returned), a reference has to be added to
> that MemoryRegion.  For example, memory_region_find adds a reference to
> the MemoryRegion that it returns.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  memory.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------
>  1 file changed, 69 insertions(+), 10 deletions(-)
>
> diff --git a/memory.c b/memory.c
> index 319894e..bb92e17 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -29,12 +29,26 @@ static unsigned memory_region_transaction_depth;
>  static bool memory_region_update_pending;
>  static bool global_dirty_log = false;
>  
> +/* flat_view_mutex is taken around reading as->current_map; the critical
> + * section is extremely short, so I'm using a single mutex for every AS.
> + * We could also RCU for the read-side.
> + *
> + * The BQL is taken around transaction commits, hence both locks are taken
> + * while writing to as->current_map (with the BQL taken outside).
> + */
> +static QemuMutex flat_view_mutex;
> +
>  static QTAILQ_HEAD(memory_listeners, MemoryListener) memory_listeners
>      = QTAILQ_HEAD_INITIALIZER(memory_listeners);
>  
>  static QTAILQ_HEAD(, AddressSpace) address_spaces
>      = QTAILQ_HEAD_INITIALIZER(address_spaces);
>  
> +static void memory_init(void)
> +{
> +    qemu_mutex_init(&flat_view_mutex);
> +}
> +
>  typedef struct AddrRange AddrRange;
>  
>  /*
> @@ -225,6 +239,7 @@ struct FlatRange {
>   * order.
>   */
>  struct FlatView {
> +    unsigned ref;
>      FlatRange *ranges;
>      unsigned nr;
>      unsigned nr_allocated;
> @@ -246,6 +261,7 @@ static bool flatrange_equal(FlatRange *a, FlatRange *b)
>  
>  static void flatview_init(FlatView *view)
>  {
> +    view->ref = 1;
>      view->ranges = NULL;
>      view->nr = 0;
>      view->nr_allocated = 0;
> @@ -279,6 +295,18 @@ static void flatview_destroy(FlatView *view)
>      g_free(view);
>  }
>  
> +static void flatview_ref(FlatView *view)
> +{
> +    __sync_fetch_and_add(&view->ref, 1);
> +}
> +
> +static void flatview_unref(FlatView *view)
> +{
> +    if (__sync_fetch_and_sub(&view->ref, 1) == 1) {
> +        flatview_destroy(view);
> +    }
> +}
> +
>  static bool can_merge(FlatRange *r1, FlatRange *r2)
>  {
>      return int128_eq(addrrange_end(r1->addr), r2->addr.start)
> @@ -578,6 +606,17 @@ static void address_space_add_del_ioeventfds(AddressSpace *as,
>      }
>  }
>  
> +static FlatView *address_space_get_flatview(AddressSpace *as)
> +{
> +    FlatView *view;
> +
> +    qemu_mutex_lock(&flat_view_mutex);
> +    view = as->current_map;
> +    flatview_ref(view);
> +    qemu_mutex_unlock(&flat_view_mutex);
> +    return view;
> +}
> +
>  static void address_space_update_ioeventfds(AddressSpace *as)
>  {
>      FlatView *view;
> @@ -587,7 +626,7 @@ static void address_space_update_ioeventfds(AddressSpace *as)
>      AddrRange tmp;
>      unsigned i;
>  
> -    view = as->current_map;
> +    view = address_space_get_flatview(as);
>      FOR_EACH_FLAT_RANGE(fr, view) {
>          for (i = 0; i < fr->mr->ioeventfd_nb; ++i) {
>              tmp = addrrange_shift(fr->mr->ioeventfds[i].addr,
> @@ -609,6 +648,7 @@ static void address_space_update_ioeventfds(AddressSpace *as)
>      g_free(as->ioeventfds);
>      as->ioeventfds = ioeventfds;
>      as->ioeventfd_nb = ioeventfd_nb;
> +    flatview_unref(view);
>  }
>  
>  static void address_space_update_topology_pass(AddressSpace *as,
> @@ -676,14 +716,25 @@ static void address_space_update_topology_pass(AddressSpace *as,
>  
>  static void address_space_update_topology(AddressSpace *as)
>  {
> -    FlatView *old_view = as->current_map;
> +    FlatView *old_view = address_space_get_flatview(as);
>      FlatView *new_view = generate_memory_topology(as->root);
>  
>      address_space_update_topology_pass(as, old_view, new_view, false);
>      address_space_update_topology_pass(as, old_view, new_view, true);
>  
> +    qemu_mutex_lock(&flat_view_mutex);
> +    flatview_unref(as->current_map);
>      as->current_map = new_view;
> -    flatview_destroy(old_view);
> +    qemu_mutex_unlock(&flat_view_mutex);
> +
> +    /* Note that all the old MemoryRegions are still alive up to this
> +     * point.  This relieves most MemoryListeners from the need to
> +     * ref/unref the MemoryRegions they get---unless they use them
> +     * outside the iothread mutex, in which case precise reference
> +     * counting is necessary.
> +     */
> +    flatview_unref(old_view);
> +
>      address_space_update_ioeventfds(as);
>  }
>  
> @@ -1146,12 +1197,13 @@ void memory_region_sync_dirty_bitmap(MemoryRegion *mr)
>      FlatRange *fr;
>  
>      QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) {
> -        FlatView *view = as->current_map;
> +        FlatView *view = address_space_get_flatview(as);
>          FOR_EACH_FLAT_RANGE(fr, view) {
>              if (fr->mr == mr) {
>                  MEMORY_LISTENER_UPDATE_REGION(fr, as, Forward, log_sync);
>              }
>          }
> +        flatview_unref(view);
>      }
>  }
>  
> @@ -1203,7 +1255,7 @@ static void memory_region_update_coalesced_range_as(MemoryRegion *mr, AddressSpa
>      AddrRange tmp;
>      MemoryRegionSection section;
>  
> -    view = as->current_map;
> +    view = address_space_get_flatview(as);
>      FOR_EACH_FLAT_RANGE(fr, view) {
>          if (fr->mr == mr) {
>              section = (MemoryRegionSection) {
> @@ -1229,6 +1281,7 @@ static void memory_region_update_coalesced_range_as(MemoryRegion *mr, AddressSpa
>              }
>          }
>      }
> +    flatview_unref(view);
>  }
>  
>  static void memory_region_update_coalesced_range(MemoryRegion *mr)
> @@ -1520,7 +1573,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
>      as = memory_region_to_address_space(root);
>      range = addrrange_make(int128_make64(addr), int128_make64(size));
>  
> -    view = as->current_map;
> +    view = address_space_get_flatview(as);
>      fr = flatview_lookup(view, range);
>      if (!fr) {
>          return ret;
> @@ -1541,6 +1594,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
>      ret.readonly = fr->readonly;
>      memory_region_ref(ret.mr);
>  
> +    flatview_unref(view);
>      return ret;
>  }
>  
> @@ -1549,10 +1603,11 @@ void address_space_sync_dirty_bitmap(AddressSpace *as)
>      FlatView *view;
>      FlatRange *fr;
>  
> -    view = as->current_map;
> +    view = address_space_get_flatview(as);
>      FOR_EACH_FLAT_RANGE(fr, view) {
>          MEMORY_LISTENER_UPDATE_REGION(fr, as, Forward, log_sync);
>      }
> +    flatview_unref(view);
>  }
>  
>  void memory_global_dirty_log_start(void)
> @@ -1584,7 +1639,7 @@ static void listener_add_address_space(MemoryListener *listener,
>          }
>      }
>  
> -    view = as->current_map;
> +    view = address_space_get_flatview(as);
>      FOR_EACH_FLAT_RANGE(fr, view) {
>          MemoryRegionSection section = {
>              .mr = fr->mr,
> @@ -1598,6 +1653,7 @@ static void listener_add_address_space(MemoryListener *listener,
>              listener->region_add(listener, &section);
>          }
>      }
> +    flatview_unref(view);
>  }
>  
>  void memory_listener_register(MemoryListener *listener, AddressSpace *filter)
> @@ -1631,6 +1687,10 @@ void memory_listener_unregister(MemoryListener *listener)
>  
>  void address_space_init(AddressSpace *as, MemoryRegion *root, const char *name)
>  {
> +    if (QTAILQ_EMPTY(&address_spaces)) {
> +        memory_init();
> +    }
> +

While this is fine, I see no harm in using a type_init() constructor to
do the global initialization.

Weirdness could ensue if we ever supported removing address spaces which
isn't that crazy of an idea I think.

Regards,

Anthony Liguori

>      memory_region_transaction_begin();
>      as->root = root;
>      as->current_map = g_new(FlatView, 1);
> @@ -1652,9 +1712,8 @@ void address_space_destroy(AddressSpace *as)
>      memory_region_transaction_commit();
>      QTAILQ_REMOVE(&address_spaces, as, address_spaces_link);
>      address_space_destroy_dispatch(as);
> -    flatview_destroy(as->current_map);
> +    flatview_unref(as->current_map);
>      g_free(as->name);
> -    g_free(as->current_map);
>      g_free(as->ioeventfds);
>  }
>  
> -- 
> 1.8.1.4

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 04/30] add a header file for atomic operations Paolo Bonzini
@ 2013-06-28 20:41   ` Anthony Liguori
  2013-07-01 10:21     ` Paolo Bonzini
  2013-07-01 11:08     ` Peter Maydell
  2013-07-03  2:24   ` liu ping fan
  1 sibling, 2 replies; 68+ messages in thread
From: Anthony Liguori @ 2013-06-28 20:41 UTC (permalink / raw)
  To: Paolo Bonzini, qemu-devel

Paolo Bonzini <pbonzini@redhat.com> writes:

> We're already using them in several places, but __sync builtins are just
> too ugly to type, and do not provide seqcst load/store operations.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  docs/atomics.txt         | 345 +++++++++++++++++++++++++++++++++++++++++++++++
>  hw/display/qxl.c         |   3 +-
>  hw/virtio/vhost.c        |   9 +-
>  include/qemu/atomic.h    | 190 +++++++++++++++++++++-----
>  migration.c              |   3 +-
>  tests/test-thread-pool.c |   8 +-
>  6 files changed, 514 insertions(+), 44 deletions(-)
>  create mode 100644 docs/atomics.txt
>
> diff --git a/docs/atomics.txt b/docs/atomics.txt
> new file mode 100644
> index 0000000..e2ce04b
> --- /dev/null
> +++ b/docs/atomics.txt

Really nice write-up!

> diff --git a/hw/display/qxl.c b/hw/display/qxl.c
> index 3862d7a..f24cb4e 100644
> --- a/hw/display/qxl.c
> +++ b/hw/display/qxl.c
> @@ -23,6 +23,7 @@
>  #include "qemu-common.h"
>  #include "qemu/timer.h"
>  #include "qemu/queue.h"
> +#include "qemu/atomic.h"
>  #include "monitor/monitor.h"
>  #include "sysemu/sysemu.h"
>  #include "trace.h"
> @@ -1726,7 +1727,7 @@ static void qxl_send_events(PCIQXLDevice *d, uint32_t events)
>          trace_qxl_send_events_vm_stopped(d->id, events);
>          return;
>      }
> -    old_pending = __sync_fetch_and_or(&d->ram->int_pending, le_events);
> +    old_pending = atomic_or(&d->ram->int_pending, le_events);
>      if ((old_pending & le_events) == le_events) {
>          return;
>      }
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 96ab625..8f6ab13 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -16,6 +16,7 @@
>  #include <sys/ioctl.h>
>  #include "hw/virtio/vhost.h"
>  #include "hw/hw.h"
> +#include "qemu/atomic.h"
>  #include "qemu/range.h"
>  #include <linux/vhost.h>
>  #include "exec/address-spaces.h"
> @@ -47,11 +48,9 @@ static void vhost_dev_sync_region(struct vhost_dev *dev,
>              addr += VHOST_LOG_CHUNK;
>              continue;
>          }
> -        /* Data must be read atomically. We don't really
> -         * need the barrier semantics of __sync
> -         * builtins, but it's easier to use them than
> -         * roll our own. */
> -        log = __sync_fetch_and_and(from, 0);
> +        /* Data must be read atomically. We don't really need barrier semantics
> +         * but it's easier to use atomic_* than roll our own. */
> +        log = atomic_xchg(from, 0);
>          while ((bit = sizeof(log) > sizeof(int) ?
>                  ffsll(log) : ffs(log))) {
>              hwaddr page_addr;
> diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
> index 10becb6..04d64d0 100644
> --- a/include/qemu/atomic.h
> +++ b/include/qemu/atomic.h
> @@ -1,68 +1,194 @@
> -#ifndef __QEMU_BARRIER_H
> -#define __QEMU_BARRIER_H 1
> +/*
> + * Simple interface for atomic operations.
> + *
> + * Copyright (C) 2013 Red Hat, Inc.
> + *
> + * Author: Paolo Bonzini <pbonzini@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
>  
> -/* Compiler barrier */
> -#define barrier()   asm volatile("" ::: "memory")
> +#ifndef __QEMU_ATOMIC_H
> +#define __QEMU_ATOMIC_H 1
>  
> -#if defined(__i386__)
> +#include "qemu/compiler.h"
>  
> -#include "qemu/compiler.h"        /* QEMU_GNUC_PREREQ */
> +/* For C11 atomic ops */
>  
> -/*
> - * Because of the strongly ordered x86 storage model, wmb() and rmb() are nops
> - * on x86(well, a compiler barrier only).  Well, at least as long as
> - * qemu doesn't do accesses to write-combining memory or non-temporal
> - * load/stores from C code.
> - */
> -#define smp_wmb()   barrier()
> -#define smp_rmb()   barrier()
> +/* Compiler barrier */
> +#define barrier()   ({ asm volatile("" ::: "memory"); (void)0; })
> +
> +#ifndef __ATOMIC_RELAXED
>  
>  /*
> - * We use GCC builtin if it's available, as that can use
> - * mfence on 32 bit as well, e.g. if built with -march=pentium-m.
> - * However, on i386, there seem to be known bugs as recently as 4.3.
> - * */
> -#if QEMU_GNUC_PREREQ(4, 4)
> -#define smp_mb() __sync_synchronize()
> + * We use GCC builtin if it's available, as that can use mfence on
> + * 32-bit as well, e.g. if built with -march=pentium-m. However, on
> + * i386 the spec is buggy, and the implementation followed it until
> + * 4.3 (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36793).
> + */
> +#if defined(__i386__) || defined(__x86_64__)
> +#if !QEMU_GNUC_PREREQ(4, 4)
> +#if defined __x86_64__
> +#define smp_mb()    ({ asm volatile("mfence" ::: "memory"); (void)0; })
>  #else
> -#define smp_mb() asm volatile("lock; addl $0,0(%%esp) " ::: "memory")
> +#define smp_mb()    ({ asm volatile("lock; addl $0,0(%%esp) " ::: "memory"); (void)0; })
> +#endif
> +#endif
>  #endif
>  
> -#elif defined(__x86_64__)
>  
> +#ifdef __alpha__
> +#define smp_read_barrier_depends()   asm volatile("mb":::"memory")
> +#endif
> +
> +#if defined(__i386__) || defined(__x86_64__) || defined(__s390x__)
> +
> +/*
> + * Because of the strongly ordered storage model, wmb() and rmb() are nops
> + * here (a compiler barrier only).  QEMU doesn't do accesses to write-combining
> + * qemu memory or non-temporal load/stores from C code.

Tiny copy/paste error here: s/qemu memory/memory/g".

One thing I've been thinking about reviewing this code, what should we
be doing in virtio.c?

We have barriers but we're relying on st[u][wlb]_phys having atomic
semantics.  I think it's okay in practice but if we're taking a more
diligent approach here should we introduce atomic variants that work on
guest phys addresses?

Otherwise:

Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>

Regards,

Anthony Liguori

> + */
>  #define smp_wmb()   barrier()
>  #define smp_rmb()   barrier()
> -#define smp_mb() asm volatile("mfence" ::: "memory")
> +
> +/*
> + * __sync_lock_test_and_set() is documented to be an acquire barrier only,
> + * but it is a full barrier at the hardware level.  Add a compiler barrier
> + * to make it a full barrier also at the compiler level.
> + */
> +#define atomic_xchg(ptr, i)    (barrier(), __sync_lock_test_and_set(ptr, i))
> +
> +/*
> + * Load/store with Java volatile semantics.
> + */
> +#define atomic_mb_set(ptr, i)  ((void)atomic_xchg(ptr, i))
>  
>  #elif defined(_ARCH_PPC)
>  
>  /*
>   * We use an eieio() for wmb() on powerpc.  This assumes we don't
>   * need to order cacheable and non-cacheable stores with respect to
> - * each other
> + * each other.
> + *
> + * smp_mb has the same problem as on x86 for not-very-new GCC
> + * (http://patchwork.ozlabs.org/patch/126184/, Nov 2011).
>   */
> -#define smp_wmb()   asm volatile("eieio" ::: "memory")
> -
> +#define smp_wmb()   ({ asm volatile("eieio" ::: "memory"); (void)0; })
>  #if defined(__powerpc64__)
> -#define smp_rmb()   asm volatile("lwsync" ::: "memory")
> +#define smp_rmb()   ({ asm volatile("lwsync" ::: "memory"); (void)0; })
>  #else
> -#define smp_rmb()   asm volatile("sync" ::: "memory")
> +#define smp_rmb()   ({ asm volatile("sync" ::: "memory"); (void)0; })
>  #endif
> +#define smp_mb()    ({ asm volatile("sync" ::: "memory"); (void)0; })
>  
> -#define smp_mb()   asm volatile("sync" ::: "memory")
> +#endif /* _ARCH_PPC */
>  
> -#else
> +#endif /* C11 atomics */
>  
>  /*
>   * For (host) platforms we don't have explicit barrier definitions
>   * for, we use the gcc __sync_synchronize() primitive to generate a
>   * full barrier.  This should be safe on all platforms, though it may
> - * be overkill for wmb() and rmb().
> + * be overkill for smp_wmb() and smp_rmb().
>   */
> +#ifndef smp_mb
> +#define smp_mb()    __sync_synchronize()
> +#endif
> +
> +#ifndef smp_wmb
> +#ifdef __ATOMIC_RELEASE
> +#define smp_wmb()   __atomic_thread_fence(__ATOMIC_RELEASE)
> +#else
>  #define smp_wmb()   __sync_synchronize()
> -#define smp_mb()   __sync_synchronize()
> +#endif
> +#endif
> +
> +#ifndef smp_rmb
> +#ifdef __ATOMIC_ACQUIRE
> +#define smp_rmb()   __atomic_thread_fence(__ATOMIC_ACQUIRE)
> +#else
>  #define smp_rmb()   __sync_synchronize()
> +#endif
> +#endif
> +
> +#ifndef smp_read_barrier_depends
> +#ifdef __ATOMIC_CONSUME
> +#define smp_read_barrier_depends()   __atomic_thread_fence(__ATOMIC_CONSUME)
> +#else
> +#define smp_read_barrier_depends()   barrier()
> +#endif
> +#endif
>  
> +#ifndef atomic_read
> +#define atomic_read(ptr)       (*(__typeof__(*ptr) *volatile) (ptr))
>  #endif
>  
> +#ifndef atomic_set
> +#define atomic_set(ptr, i)     ((*(__typeof__(*ptr) *volatile) (ptr)) = (i))
> +#endif
> +
> +/* These have the same semantics as Java volatile variables.
> + * See http://gee.cs.oswego.edu/dl/jmm/cookbook.html:
> + * "1. Issue a StoreStore barrier (wmb) before each volatile store."
> + *  2. Issue a StoreLoad barrier after each volatile store.
> + *     Note that you could instead issue one before each volatile load, but
> + *     this would be slower for typical programs using volatiles in which
> + *     reads greatly outnumber writes. Alternatively, if available, you
> + *     can implement volatile store as an atomic instruction (for example
> + *     XCHG on x86) and omit the barrier. This may be more efficient if
> + *     atomic instructions are cheaper than StoreLoad barriers.
> + *  3. Issue LoadLoad and LoadStore barriers after each volatile load."
> + *
> + * If you prefer to think in terms of "pairing" of memory barriers,
> + * an atomic_mb_read pairs with an atomic_mb_set.
> + *
> + * And for the few ia64 lovers that exist, an atomic_mb_read is a ld.acq,
> + * while an atomic_mb_set is a st.rel followed by a memory barrier.
> + *
> + * These are a bit weaker than __atomic_load/store with __ATOMIC_SEQ_CST
> + * (see docs/atomics.txt), and I'm not sure that __ATOMIC_ACQ_REL is enough.
> + * Just always use the barriers manually by the rules above.
> + */
> +#ifndef atomic_mb_read
> +#define atomic_mb_read(ptr)    ({           \
> +    typeof(*ptr) _val = atomic_read(ptr);   \
> +    smp_rmb();                              \
> +    _val;                                   \
> +})
> +#endif
> +
> +#ifndef atomic_mb_set
> +#define atomic_mb_set(ptr, i)  do {         \
> +    smp_wmb();                              \
> +    atomic_set(ptr, i);                     \
> +    smp_mb();                               \
> +} while (0)
> +#endif
> +
> +#ifndef atomic_xchg
> +#ifdef __ATOMIC_SEQ_CST
> +#define atomic_xchg(ptr, i)    ({                           \
> +    typeof(*ptr) _new = (i), _old;                          \
> +    __atomic_exchange(ptr, &_new, &_old, __ATOMIC_SEQ_CST); \
> +    _old;                                                   \
> +})
> +#elif defined __clang__
> +#define atomic_xchg(ptr, i)    __sync_exchange(ptr, i)
> +#else
> +/* __sync_lock_test_and_set() is documented to be an acquire barrier only.  */
> +#define atomic_xchg(ptr, i)    (smp_mb(), __sync_lock_test_and_set(ptr, i))
> +#endif
> +#endif
> +
> +/* Provide shorter names for GCC atomic builtins.  */
> +#define atomic_inc(ptr)        __sync_fetch_and_add(ptr, 1)
> +#define atomic_dec(ptr)        __sync_fetch_and_add(ptr, -1)
> +#define atomic_add             __sync_fetch_and_add
> +#define atomic_sub             __sync_fetch_and_sub
> +#define atomic_and             __sync_fetch_and_and
> +#define atomic_or              __sync_fetch_and_or
> +#define atomic_cmpxchg         __sync_val_compare_and_swap
> +
>  #endif
> diff --git a/migration.c b/migration.c
> index 058f9e6..83f5691 100644
> --- a/migration.c
> +++ b/migration.c
> @@ -290,8 +290,7 @@ static void migrate_fd_cleanup(void *opaque)
>  
>  static void migrate_finish_set_state(MigrationState *s, int new_state)
>  {
> -    if (__sync_val_compare_and_swap(&s->state, MIG_STATE_ACTIVE,
> -                                    new_state) == new_state) {
> +    if (atomic_cmpxchg(&s->state, MIG_STATE_ACTIVE, new_state) == new_state) {
>          trace_migrate_set_state(new_state);
>      }
>  }
> diff --git a/tests/test-thread-pool.c b/tests/test-thread-pool.c
> index 22915aa..dbf2fc8 100644
> --- a/tests/test-thread-pool.c
> +++ b/tests/test-thread-pool.c
> @@ -17,15 +17,15 @@ typedef struct {
>  static int worker_cb(void *opaque)
>  {
>      WorkerTestData *data = opaque;
> -    return __sync_fetch_and_add(&data->n, 1);
> +    return atomic_inc(&data->n);
>  }
>  
>  static int long_cb(void *opaque)
>  {
>      WorkerTestData *data = opaque;
> -    __sync_fetch_and_add(&data->n, 1);
> +    atomic_inc(&data->n);
>      g_usleep(2000000);
> -    __sync_fetch_and_add(&data->n, 1);
> +    atomic_inc(&data->n);
>      return 0;
>  }
>  
> @@ -169,7 +169,7 @@ static void test_cancel(void)
>      /* Cancel the jobs that haven't been started yet.  */
>      num_canceled = 0;
>      for (i = 0; i < 100; i++) {
> -        if (__sync_val_compare_and_swap(&data[i].n, 0, 3) == 0) {
> +        if (atomic_cmpxchg(&data[i].n, 0, 3) == 0) {
>              data[i].ret = -ECANCELED;
>              bdrv_aio_cancel(data[i].aiocb);
>              active--;
> -- 
> 1.8.1.4

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h Paolo Bonzini
@ 2013-06-28 20:43   ` Anthony Liguori
  2013-06-28 23:53   ` Ed Maste
  2013-06-29 10:55   ` Peter Maydell
  2 siblings, 0 replies; 68+ messages in thread
From: Anthony Liguori @ 2013-06-28 20:43 UTC (permalink / raw)
  To: Paolo Bonzini, qemu-devel

Paolo Bonzini <pbonzini@redhat.com> writes:

> The next patch will change qemu/tls.h to support more platforms, but at
> some performance cost.  Declare cpu_single_env directly instead of using
> the tls.h abstractions.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>

Regards,

Anthony Liguori

> ---
>  exec.c                 | 10 ++++++++--
>  include/exec/cpu-all.h | 14 +++++++++++---
>  include/qemu/tls.h     | 52 --------------------------------------------------
>  3 files changed, 19 insertions(+), 57 deletions(-)
>  delete mode 100644 include/qemu/tls.h
>
> diff --git a/exec.c b/exec.c
> index d28403b..a788981 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -70,9 +70,15 @@ static MemoryRegion io_mem_unassigned;
>  #endif
>  
>  CPUArchState *first_cpu;
> +
>  /* current CPU in the current thread. It is only valid inside
> -   cpu_exec() */
> -DEFINE_TLS(CPUArchState *,cpu_single_env);
> + * cpu_exec().  See comment in include/exec/cpu-all.h.  */
> +#if defined CONFIG_KVM || (defined CONFIG_USER_ONLY && defined CONFIG_USE_NPTL)
> +__thread CPUArchState *cpu_single_env;
> +#else
> +CPUArchState *cpu_single_env;
> +#endif
> +
>  /* 0 = Do not count executed instructions.
>     1 = Precise instruction counting.
>     2 = Adaptive rate instruction counting.  */
> diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
> index e9c3717..2202ba3 100644
> --- a/include/exec/cpu-all.h
> +++ b/include/exec/cpu-all.h
> @@ -20,7 +20,6 @@
>  #define CPU_ALL_H
>  
>  #include "qemu-common.h"
> -#include "qemu/tls.h"
>  #include "exec/cpu-common.h"
>  #include "qemu/thread.h"
>  
> @@ -368,8 +367,17 @@ void cpu_dump_statistics(CPUArchState *env, FILE *f, fprintf_function cpu_fprint
>  void QEMU_NORETURN cpu_abort(CPUArchState *env, const char *fmt, ...)
>      GCC_FMT_ATTR(2, 3);
>  extern CPUArchState *first_cpu;
> -DECLARE_TLS(CPUArchState *,cpu_single_env);
> -#define cpu_single_env tls_var(cpu_single_env)
> +
> +/* This is thread-local depending on __linux__ because:
> + *  - the only -user mode supporting multiple VCPU threads is linux-user
> + *  - TCG system mode is single-threaded regarding VCPUs
> + *  - KVM system mode is multi-threaded but limited to Linux
> + */
> +#if defined CONFIG_KVM || (defined CONFIG_USER_ONLY && defined CONFIG_USE_NPTL)
> +extern __thread CPUArchState *cpu_single_env;
> +#else
> +extern CPUArchState *cpu_single_env;
> +#endif
>  
>  /* Flags for use in ENV->INTERRUPT_PENDING.
>  
> diff --git a/include/qemu/tls.h b/include/qemu/tls.h
> deleted file mode 100644
> index b92ea9d..0000000
> --- a/include/qemu/tls.h
> +++ /dev/null
> @@ -1,52 +0,0 @@
> -/*
> - * Abstraction layer for defining and using TLS variables
> - *
> - * Copyright (c) 2011 Red Hat, Inc
> - * Copyright (c) 2011 Linaro Limited
> - *
> - * Authors:
> - *  Paolo Bonzini <pbonzini@redhat.com>
> - *  Peter Maydell <peter.maydell@linaro.org>
> - *
> - * This program is free software; you can redistribute it and/or
> - * modify it under the terms of the GNU General Public License as
> - * published by the Free Software Foundation; either version 2 of
> - * the License, or (at your option) any later version.
> - *
> - * This program is distributed in the hope that it will be useful,
> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> - * GNU General Public License for more details.
> - *
> - * You should have received a copy of the GNU General Public License along
> - * with this program; if not, see <http://www.gnu.org/licenses/>.
> - */
> -
> -#ifndef QEMU_TLS_H
> -#define QEMU_TLS_H
> -
> -/* Per-thread variables. Note that we only have implementations
> - * which are really thread-local on Linux; the dummy implementations
> - * define plain global variables.
> - *
> - * This means that for the moment use should be restricted to
> - * per-VCPU variables, which are OK because:
> - *  - the only -user mode supporting multiple VCPU threads is linux-user
> - *  - TCG system mode is single-threaded regarding VCPUs
> - *  - KVM system mode is multi-threaded but limited to Linux
> - *
> - * TODO: proper implementations via Win32 .tls sections and
> - * POSIX pthread_getspecific.
> - */
> -#ifdef __linux__
> -#define DECLARE_TLS(type, x) extern DEFINE_TLS(type, x)
> -#define DEFINE_TLS(type, x)  __thread __typeof__(type) tls__##x
> -#define tls_var(x)           tls__##x
> -#else
> -/* Dummy implementations which define plain global variables */
> -#define DECLARE_TLS(type, x) extern DEFINE_TLS(type, x)
> -#define DEFINE_TLS(type, x)  __typeof__(type) tls__##x
> -#define tls_var(x)           tls__##x
> -#endif
> -
> -#endif
> -- 
> 1.8.1.4

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h Paolo Bonzini
  2013-06-28 20:43   ` Anthony Liguori
@ 2013-06-28 23:53   ` Ed Maste
  2013-07-01 10:16     ` Paolo Bonzini
  2013-06-29 10:55   ` Peter Maydell
  2 siblings, 1 reply; 68+ messages in thread
From: Ed Maste @ 2013-06-28 23:53 UTC (permalink / raw)
  To: qemu-devel

On 28 June 2013 14:26, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> +/* This is thread-local depending on __linux__ because:

Is the comment perhaps unchanged from an earlier revision that used a
different test?  It seems odd to me to reference __linux__ here.

> + *  - the only -user mode supporting multiple VCPU threads is linux-user
> + *  - TCG system mode is single-threaded regarding VCPUs
> + *  - KVM system mode is multi-threaded but limited to Linux
> + */
> +#if defined CONFIG_KVM || (defined CONFIG_USER_ONLY && defined CONFIG_USE_NPTL)

Also, in discussion on the FreeBSD bsd-user patch set the suggestion
was made that we do away with a flag, and just have thread support
always enabled.  Would you suggest this test then become KVM ||
(USER_ONLY && (USE_NPTL || __FreeBSD__))?

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h Paolo Bonzini
  2013-06-28 20:43   ` Anthony Liguori
  2013-06-28 23:53   ` Ed Maste
@ 2013-06-29 10:55   ` Peter Maydell
  2013-07-01 10:45     ` Paolo Bonzini
  2 siblings, 1 reply; 68+ messages in thread
From: Peter Maydell @ 2013-06-29 10:55 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 28 June 2013 19:26, Paolo Bonzini <pbonzini@redhat.com> wrote:
> The next patch will change qemu/tls.h to support more platforms, but at
> some performance cost.  Declare cpu_single_env directly instead of using
> the tls.h abstractions.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  exec.c                 | 10 ++++++++--
>  include/exec/cpu-all.h | 14 +++++++++++---
>  include/qemu/tls.h     | 52 --------------------------------------------------
>  3 files changed, 19 insertions(+), 57 deletions(-)
>  delete mode 100644 include/qemu/tls.h
>
> diff --git a/exec.c b/exec.c
> index d28403b..a788981 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -70,9 +70,15 @@ static MemoryRegion io_mem_unassigned;
>  #endif
>
>  CPUArchState *first_cpu;
> +
>  /* current CPU in the current thread. It is only valid inside
> -   cpu_exec() */
> -DEFINE_TLS(CPUArchState *,cpu_single_env);
> + * cpu_exec().  See comment in include/exec/cpu-all.h.  */
> +#if defined CONFIG_KVM || (defined CONFIG_USER_ONLY && defined CONFIG_USE_NPTL)
> +__thread CPUArchState *cpu_single_env;
> +#else
> +CPUArchState *cpu_single_env;
> +#endif

I don't like having the semantics of this variable differ
depending on whether CONFIG_KVM was defined. In particular
this means that the variable is per-thread if you're running
TCG on a QEMU that was configured with KVM support, but
not per-thread if you're running TCG on a QEMU that was
configured without per-thread support. That's just bizarre
and a recipe for confusion and for bugs creeping in in the
less-well-tested config combinations.

We should just be consistent and always make this be
per-thread.

thanks
-- PMM

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 08/30] rcu: add rcu library
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 08/30] rcu: add rcu library Paolo Bonzini
@ 2013-07-01  9:47   ` Jan Kiszka
  0 siblings, 0 replies; 68+ messages in thread
From: Jan Kiszka @ 2013-07-01  9:47 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 2013-06-28 20:26, Paolo Bonzini wrote:
> This includes a (mangled) copy of the urcu-qsbr code from liburcu.
> The main changes are: 1) removing dependencies on many other header files
> in liburcu; 2) removing for simplicity the tentative busy waiting in
> synchronize_rcu, which has limited performance effects; 3) replacing
> futexes in synchronize_rcu with QemuEvents for Win32 portability.
> The API is the same as liburcu, so it should be possible in the future
> to require liburcu on POSIX systems for example and use our copy only
> on Windows.
> 
> Among the various versions available I chose urcu-qsbr, which has the
> fastest rcu_read_{lock,unlock} but requires the program to manually
> annotate quiescent points or intervals.  QEMU threads usually have easily
> identified quiescent periods, so this should not be a problem.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  docs/rcu.txt               | 303 +++++++++++++++++++++++++++++++++++++++++++++
>  hw/9pfs/virtio-9p-synth.c  |   1 +
>  include/qemu/queue.h       |  13 ++
>  include/qemu/rcu-pointer.h | 110 ++++++++++++++++
>  include/qemu/rcu.h         | 141 +++++++++++++++++++++
>  include/qemu/thread.h      |   3 -
>  libcacard/Makefile         |   3 +-
>  util/Makefile.objs         |   1 +
>  util/rcu.c                 | 195 +++++++++++++++++++++++++++++
>  9 files changed, 766 insertions(+), 4 deletions(-)
>  create mode 100644 docs/rcu.txt
>  create mode 100644 include/qemu/rcu-pointer.h
>  create mode 100644 include/qemu/rcu.h
>  create mode 100644 util/rcu.c
> 
> diff --git a/docs/rcu.txt b/docs/rcu.txt
> new file mode 100644
> index 0000000..4869ec7
> --- /dev/null
> +++ b/docs/rcu.txt
> @@ -0,0 +1,303 @@
> +Using RCU (Read-Copy-Update) for synchronization
> +================================================
> +
> +Read-copy update (RCU) is a synchronization mechanism that is used to
> +protect read-mostly data structures.  RCU is very efficient and scalable
> +on the read side (it is wait-free), and thus can make the read paths
> +extremely fast.
> +
> +RCU supports concurrency between a single writer and multiple readers,
> +thus it is not used alone.  Typically, the write-side will use a lock to
> +serialize multiple updates, but other approaches are possible (e.g.,
> +restricting updates to a single task).  In QEMU, when a lock is used,
> +this will often be the "iothread mutex", also known as the "big QEMU
> +lock" (BQL).  Also, restricting updates to a single task is done in
> +QEMU using the "bottom half" API.
> +
> +RCU is fundamentally a "wait-to-finish" mechanism.  The read side marks
> +sections of code with "critical sections", and the update side will wait
> +for the execution of all *currently running* critical sections before
> +proceeding, or before asynchronously executing a callback.
> +
> +The key point here is that only the currently running critical sections
> +are waited for; critical sections that are started _after_ the beginning
> +of the wait do not extend the wait, despite running concurrently with
> +the updater.  This is the reason why RCU is more scalable than,
> +for example, reader-writer locks.  It is so much more scalable that
> +the system will have a single instance of the RCU mechanism; a single
> +mechanism can be used for an arbitrary number of "things", without
> +having to worry about things such as contention or deadlocks.
> +
> +How is this possible?  The basic idea is to split updates in two phases,
> +"removal" and "reclamation".  During removal, we ensure that subsequent
> +readers will not be able to get a reference to the old data.  After
> +removal has completed, a critical section will not be able to access
> +the old data.  Therefore, critical sections that begin after removal
> +do not matter; as soon as all previous critical sections have finished,
> +there cannot be any readers who hold references to the data structure,
> +which may not be safely reclaimed (e.g., freed or unref'ed).
> +
> +Here is a picutre:
> +
> +        thread 1                  thread 2                  thread 3
> +    -------------------    ------------------------    -------------------
> +    enter RCU crit.sec.
> +           |                finish removal phase
> +           |                begin wait
> +           |                      |                    enter RCU crit.sec.
> +    exit RCU crit.sec             |                           |
> +                            complete wait                     |
> +                            begin reclamation phase           |
> +                                                       exit RCU crit.sec.
> +
> +
> +Note how thread 3 is still executing its critical section when thread 2
> +starts reclaiming data.  This is possible, because the old version of the
> +data structure was not accessible at the time thread 3 began executing
> +that critical section.
> +
> +
> +RCU API
> +=======
> +
> +The core RCU API is small:
> +
> +     void rcu_read_lock(void);
> +
> +        Used by a reader to inform the reclaimer that the reader is
> +        entering an RCU read-side critical section.
> +
> +     void rcu_read_unlock(void);
> +
> +        Used by a reader to inform the reclaimer that the reader is
> +        exiting an RCU read-side critical section.  Note that RCU
> +        read-side critical sections may be nested and/or overlapping.
> +
> +     void synchronize_rcu(void);
> +
> +        Blocks until all pre-existing RCU read-side critical sections
> +        on all threads have completed.  This marks the end of the removal
> +        phase and the beginning of reclamation phase.
> +
> +        Note that it would be valid for another update to come while
> +        synchronize_rcu is running.  Because of this, it is better that
> +        the updater releases any locks it may hold before calling
> +        synchronize_rcu.
> +
> +     typeof(*p) rcu_dereference(p);
> +     typeof(p) rcu_assign_pointer(p, typeof(p) v);
> +
> +        These macros are similar to atomic_mb_read() and atomic_mb_set()
> +        respectively.  However, they make some assumptions on the code
> +        that calls them, which allows a more optimized implementation.
> +
> +        rcu_assign_pointer assumes that the update side is not going
> +        to read from the data structure after "publishing" the new
> +        values; that is, it assumes that all assignments happen at
> +        the very end of the removal phase.
> +
> +        rcu_dereference assumes that whenever a single RCU critical
> +        section reads multiple shared data, these reads are either
> +        data-dependent or need no ordering.  This is almost always the
> +        case when using RCU.  If this were not the case, you can use
> +        atomic_mb_read() or smp_rmb().
> +
> +        If you are going to be fetching multiple fields from the
> +        RCU-protected structure, repeated rcu_dereference() calls
> +        would look ugly and incur unnecessary overhead on Alpha CPUs.
> +        You can then do this:
> +
> +        p = &rcu_dereference(head);
> +        foo = head->foo;
> +        bar = head->bar;
> +
> +
> +RCU QUIESCENT STATES
> +====================
> +
> +An efficient implementation of rcu_read_lock() and rcu_read_unlock()
> +relies on the availability of fast thread-local storage.  Unfortunately,
> +this is not possible on all the systems supported by QEMU (in particular
> +on many POSIX systems other than Linux and Solaris).
> +
> +For this reason, QEMU's RCU implementation resorts to manual annotation
> +of "quiescent states", i.e. points where no RCU read-side critical
> +section can be active.  All threads that participate in the RCU mechanism
> +need to annotate such points.

"... can be active" also implies that

rcu_read_lock();
rcu_thread_offline();

is actually an illegal ordering, right? Can we add some optional debug
code that performs runtime checking of this requirements? That should
also catch (sooner or later) leaking RCU locks.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-06-28 23:53   ` Ed Maste
@ 2013-07-01 10:16     ` Paolo Bonzini
  0 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-07-01 10:16 UTC (permalink / raw)
  To: Ed Maste; +Cc: qemu-devel

Il 29/06/2013 01:53, Ed Maste ha scritto:
> On 28 June 2013 14:26, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>> +/* This is thread-local depending on __linux__ because:
> 
> Is the comment perhaps unchanged from an earlier revision that used a
> different test?  It seems odd to me to reference __linux__ here.
> 
>> + *  - the only -user mode supporting multiple VCPU threads is linux-user
>> + *  - TCG system mode is single-threaded regarding VCPUs
>> + *  - KVM system mode is multi-threaded but limited to Linux
>> + */
>> +#if defined CONFIG_KVM || (defined CONFIG_USER_ONLY && defined CONFIG_USE_NPTL)
> 
> Also, in discussion on the FreeBSD bsd-user patch set the suggestion
> was made that we do away with a flag, and just have thread support
> always enabled.  Would you suggest this test then become KVM ||
> (USER_ONLY && (USE_NPTL || __FreeBSD__))?

I would suggest that you have something like CONFIG_USER_THREADS that
can be used by both linux-user and bsd-user.

Paolo

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-06-28 20:41   ` Anthony Liguori
@ 2013-07-01 10:21     ` Paolo Bonzini
  2013-07-01 13:00       ` Anthony Liguori
  2013-07-01 11:08     ` Peter Maydell
  1 sibling, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-07-01 10:21 UTC (permalink / raw)
  To: Anthony Liguori; +Cc: qemu-devel

Il 28/06/2013 22:41, Anthony Liguori ha scritto:
> Tiny copy/paste error here: s/qemu memory/memory/g".
> 
> One thing I've been thinking about reviewing this code, what should we
> be doing in virtio.c?
> 
> We have barriers but we're relying on st[u][wlb]_phys having atomic
> semantics.  I think it's okay in practice but if we're taking a more
> diligent approach here should we introduce atomic variants that work on
> guest phys addresses?

I think it's part of the semantics of stu?[wlb]_phys that they (a) are
not CSE'd by the compiler (b) are atomic for aligned addresses.  I
cannot find the commit exactly, but I think mst added specific code for
that.

Paolo

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-06-29 10:55   ` Peter Maydell
@ 2013-07-01 10:45     ` Paolo Bonzini
  2013-07-01 11:05       ` Peter Maydell
  0 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-07-01 10:45 UTC (permalink / raw)
  To: Peter Maydell; +Cc: qemu-devel

Il 29/06/2013 12:55, Peter Maydell ha scritto:
> On 28 June 2013 19:26, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> The next patch will change qemu/tls.h to support more platforms, but at
>> some performance cost.  Declare cpu_single_env directly instead of using
>> the tls.h abstractions.
>>
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> ---
>>  exec.c                 | 10 ++++++++--
>>  include/exec/cpu-all.h | 14 +++++++++++---
>>  include/qemu/tls.h     | 52 --------------------------------------------------
>>  3 files changed, 19 insertions(+), 57 deletions(-)
>>  delete mode 100644 include/qemu/tls.h
>>
>> diff --git a/exec.c b/exec.c
>> index d28403b..a788981 100644
>> --- a/exec.c
>> +++ b/exec.c
>> @@ -70,9 +70,15 @@ static MemoryRegion io_mem_unassigned;
>>  #endif
>>
>>  CPUArchState *first_cpu;
>> +
>>  /* current CPU in the current thread. It is only valid inside
>> -   cpu_exec() */
>> -DEFINE_TLS(CPUArchState *,cpu_single_env);
>> + * cpu_exec().  See comment in include/exec/cpu-all.h.  */
>> +#if defined CONFIG_KVM || (defined CONFIG_USER_ONLY && defined CONFIG_USE_NPTL)
>> +__thread CPUArchState *cpu_single_env;
>> +#else
>> +CPUArchState *cpu_single_env;
>> +#endif
> 
> I don't like having the semantics of this variable differ
> depending on whether CONFIG_KVM was defined. In particular
> this means that the variable is per-thread if you're running
> TCG on a QEMU that was configured with KVM support, but
> not per-thread if you're running TCG on a QEMU that was
> configured without per-thread support. That's just bizarre
> and a recipe for confusion and for bugs creeping in in the
> less-well-tested config combinations.
> 
> We should just be consistent and always make this be
> per-thread.

If it's okay to make cpu_single_env accesses more expensive by a factor
of 4 on TLS-deficient hosts (at least OpenBSD; do Darwin and NetBSD
support thread-local storage?), I'm all for it.  I (and I guess Stefan
too) do not want to introduce performance regressions in these patches.
 Making it simpler is something that one would do after having tested at
least one of OpenBSD/Darwin/whatever.

This patch does not make things worse than before.  If anything, it's
better because *more* targets have non-TLS semantics: namely non-KVM
targets on Linux become non-TLS.

Paolo

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-07-01 10:45     ` Paolo Bonzini
@ 2013-07-01 11:05       ` Peter Maydell
  2013-07-01 16:21         ` Paolo Bonzini
  0 siblings, 1 reply; 68+ messages in thread
From: Peter Maydell @ 2013-07-01 11:05 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 1 July 2013 11:45, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 29/06/2013 12:55, Peter Maydell ha scritto:
>> We should just be consistent and always make this be
>> per-thread.
>
> If it's okay to make cpu_single_env accesses more expensive by a factor
> of 4 on TLS-deficient hosts (at least OpenBSD; do Darwin and NetBSD
> support thread-local storage?), I'm all for it.  I (and I guess Stefan
> too) do not want to introduce performance regressions in these patches.
>  Making it simpler is something that one would do after having tested at
> least one of OpenBSD/Darwin/whatever.

MacOSX 10.8 supports __thread if you build with clang (not if you
build with gcc), though I can't speak to its performance since it
compiles to a function call to get the variable's address.

> This patch does not make things worse than before.  If anything, it's
> better because *more* targets have non-TLS semantics: namely non-KVM
> targets on Linux become non-TLS.

That is making things worse! Non-TLS is the untested minority
case, we want to be taking things out of it, not pushing
configs that were previously TLS into it.

-- PMM

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-06-28 20:41   ` Anthony Liguori
  2013-07-01 10:21     ` Paolo Bonzini
@ 2013-07-01 11:08     ` Peter Maydell
  1 sibling, 0 replies; 68+ messages in thread
From: Peter Maydell @ 2013-07-01 11:08 UTC (permalink / raw)
  To: Anthony Liguori; +Cc: Paolo Bonzini, qemu-devel

On 28 June 2013 21:41, Anthony Liguori <anthony@codemonkey.ws> wrote:
> One thing I've been thinking about reviewing this code, what should we
> be doing in virtio.c?
>
> We have barriers but we're relying on st[u][wlb]_phys having atomic
> semantics.  I think it's okay in practice but if we're taking a more
> diligent approach here should we introduce atomic variants that work on
> guest phys addresses?

Those accesses are weird anyway, because they implicitly depend
on going through a path which is cache-coherent with the CPU
that was used to run the guest VCPU. So they're not strictly
speaking writing "guest physical memory" necessarily...

-- PMM

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 30/30] exec: put address space dispatch under RCU critical section
  2013-06-28 19:38   ` Jan Kiszka
@ 2013-07-01 11:48     ` Paolo Bonzini
  0 siblings, 0 replies; 68+ messages in thread
From: Paolo Bonzini @ 2013-07-01 11:48 UTC (permalink / raw)
  To: Jan Kiszka; +Cc: qemu-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Il 28/06/2013 21:38, Jan Kiszka ha scritto:
> At this point, do we still have unowned memory regions?

Yes, for stuff that is registered by the boards or by non-qdevified
devices.  In either case, they cannot disappear so it's fine that you
call address_space_translate outside the BQL---even if they lack an owner.

Paolo

> If so, we must not return them here if the caller is not holding 
> the BQL (which is supposed to protect their 
> registration/modification/deregistration).
> 
> I played with a version today that returns NULL in this case. Then 
> the caller of address_space_translate can take the BQL and retry 
> the translation.
> 
> Jan
> 
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJR0WyRAAoJEBvWZb6bTYbyJioP/jj39ehtAAjJA0I3heu6oCwW
fGezYd20NnFQg7kWzNceJM52RT7pIqX/z1a1YRYWHz+izk6ZyfJ36EM0OOa4QSJK
yz03mpcSh52XS+kDPzlMYs6OpKRcIPCcR3WU8CP3/ic1lU/74VlXT5jIme0omIk7
3IWBviwrsV69Fz++2kKOTmmPjT63gudKOuZbCrvHnIkwatg3U6wQVDkOn/rFnVCC
S+LjVJpdBDO9m4y4kwGtSsboepO+Yhv31Q2HihLFOFUG9qwBkeqASIQl/kYxjIGl
J17gS37waujKLBNdUEYvzTf9km/XUppc8Bdp1IjPQo6bLJeB6UFFTTkmjdH2qmLn
Qy6eO7LrS1ag7NCaJqY1tZQFyp+fkRf5TTAlf6to7N/4pmb3jFIOueLohQGDeL8b
m6yJmmViaAGRdX1dcpzu7kYlwHpQkAccK/Bg6LnvP9iF3DC/vulVupiDl6BIwuDF
qYzOuQyIqQdp2m2Xf8x5+VJgwLSjHV8I7PxKwFD3q6pIhqRrmKuwc9TgWHQwVBWA
04X/PclkMXfAo82Pcuncztibk+Ft9X7U2svSy2OLsZC9c1qY+BUwpNPKRIBbOdM4
2mZsJCMqDvLZwf6GHKAt4NqY0dyRycHENIqPYcfO+CK2FGsodf4/xQ27/g/5l1Zu
+oflF+G6n8yKtEz0Qz2q
=S4/P
-----END PGP SIGNATURE-----

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-07-01 10:21     ` Paolo Bonzini
@ 2013-07-01 13:00       ` Anthony Liguori
  2013-07-01 13:04         ` Paolo Bonzini
  0 siblings, 1 reply; 68+ messages in thread
From: Anthony Liguori @ 2013-07-01 13:00 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

Paolo Bonzini <pbonzini@redhat.com> writes:

> Il 28/06/2013 22:41, Anthony Liguori ha scritto:
>> Tiny copy/paste error here: s/qemu memory/memory/g".
>> 
>> One thing I've been thinking about reviewing this code, what should we
>> be doing in virtio.c?
>> 
>> We have barriers but we're relying on st[u][wlb]_phys having atomic
>> semantics.  I think it's okay in practice but if we're taking a more
>> diligent approach here should we introduce atomic variants that work on
>> guest phys addresses?
>
> I think it's part of the semantics of stu?[wlb]_phys that they (a) are
> not CSE'd by the compiler (b) are atomic for aligned addresses.  I
> cannot find the commit exactly, but I think mst added specific code for
> that.

Right, I'm not questioning whether these functions have strong enough
semantics in their implementation, but asking what their contract should
be.

Either we should document that these functions have atomic semantics or
we should introduce another variant that guarantee atomic access.

I think the later makes more sense since the majority of users probably
don't need atomic semantics.

Regards,

Anthony Liguori

>
> Paolo

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-07-01 13:00       ` Anthony Liguori
@ 2013-07-01 13:04         ` Paolo Bonzini
  2013-07-01 13:20           ` Anthony Liguori
  0 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-07-01 13:04 UTC (permalink / raw)
  To: Anthony Liguori; +Cc: qemu-devel

Il 01/07/2013 15:00, Anthony Liguori ha scritto:
>> I
>> > cannot find the commit exactly, but I think mst added specific code for
>> > that.
> Right, I'm not questioning whether these functions have strong enough
> semantics in their implementation, but asking what their contract should
> be.
> 
> Either we should document that these functions have atomic semantics or
> we should introduce another variant that guarantee atomic access.
> 
> I think the later makes more sense since the majority of users probably
> don't need atomic semantics.

I think many of these loads and stores do, actually; perhaps most.  It
also matches what hardware does.

Paolo

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-07-01 13:04         ` Paolo Bonzini
@ 2013-07-01 13:20           ` Anthony Liguori
  2013-07-04  5:24             ` liu ping fan
  0 siblings, 1 reply; 68+ messages in thread
From: Anthony Liguori @ 2013-07-01 13:20 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

Paolo Bonzini <pbonzini@redhat.com> writes:

> Il 01/07/2013 15:00, Anthony Liguori ha scritto:
>>> I
>>> > cannot find the commit exactly, but I think mst added specific code for
>>> > that.
>> Right, I'm not questioning whether these functions have strong enough
>> semantics in their implementation, but asking what their contract should
>> be.
>> 
>> Either we should document that these functions have atomic semantics or
>> we should introduce another variant that guarantee atomic access.
>> 
>> I think the later makes more sense since the majority of users probably
>> don't need atomic semantics.
>
> I think many of these loads and stores do, actually; perhaps most.  It
> also matches what hardware does.

Hrm, I'm not sure if that's true.  PCI has an explicit LOCK# bit to
enable exclusive access so my assumption would be that it doesn't by
default.

But either way, we should document the semantics.

Regards,

Anthony Liguori

>
> Paolo

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-07-01 11:05       ` Peter Maydell
@ 2013-07-01 16:21         ` Paolo Bonzini
  2013-07-01 16:26           ` Peter Maydell
  0 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-07-01 16:21 UTC (permalink / raw)
  To: Peter Maydell; +Cc: qemu-devel

Il 01/07/2013 13:05, Peter Maydell ha scritto:
>> > This patch does not make things worse than before.  If anything, it's
>> > better because *more* targets have non-TLS semantics: namely non-KVM
>> > targets on Linux become non-TLS.
> That is making things worse! Non-TLS is the untested minority
> case, we want to be taking things out of it, not pushing
> configs that were previously TLS into it.

I think we should strive for one of these two:

(1) all targets are TLS;

(2) all targets are non-TLS if this is possible.

Either maximizes the homogeneity across platforms.

Paolo

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-07-01 16:21         ` Paolo Bonzini
@ 2013-07-01 16:26           ` Peter Maydell
  2013-07-01 20:52             ` Paolo Bonzini
  0 siblings, 1 reply; 68+ messages in thread
From: Peter Maydell @ 2013-07-01 16:26 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 1 July 2013 17:21, Paolo Bonzini <pbonzini@redhat.com> wrote:
> I think we should strive for one of these two:
>
> (1) all targets are TLS;
>
> (2) all targets are non-TLS if this is possible.
>
> Either maximizes the homogeneity across platforms.

Since the two largest cases are both "cpu_single_env must be TLS"
(ie (a) system emulation built with KVM support and (b) linux-user),
the set of targets which can be non-TLS is really really small,
and I think (1) makes much more sense.

(I'm assuming you don't want to try to support cpu_single_env
being both per-thread and not-per-thread in a single binary
depending on whether the user passes -enable-kvm or not.)

-- PMM

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-07-01 16:26           ` Peter Maydell
@ 2013-07-01 20:52             ` Paolo Bonzini
  2013-07-01 21:34               ` Peter Maydell
  0 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-07-01 20:52 UTC (permalink / raw)
  To: Peter Maydell; +Cc: qemu-devel

Il 01/07/2013 18:26, Peter Maydell ha scritto:
> > I think we should strive for one of these two:
> >
> > (1) all targets are TLS;
> >
> > (2) all targets are non-TLS if this is possible.
> >
> > Either maximizes the homogeneity across platforms.
> 
> Since the two largest cases are both "cpu_single_env must be TLS"
> (ie (a) system emulation built with KVM support and (b) linux-user),
> the set of targets which can be non-TLS is really really small,
> and I think (1) makes much more sense.

Not many linux-user targets support threads (including not i386).

> (I'm assuming you don't want to try to support cpu_single_env
> being both per-thread and not-per-thread in a single binary
> depending on whether the user passes -enable-kvm or not.)

No, of course not.

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-07-01 20:52             ` Paolo Bonzini
@ 2013-07-01 21:34               ` Peter Maydell
  2013-07-02 13:40                 ` Andreas Färber
  0 siblings, 1 reply; 68+ messages in thread
From: Peter Maydell @ 2013-07-01 21:34 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 1 July 2013 21:52, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 01/07/2013 18:26, Peter Maydell ha scritto:
>> Since the two largest cases are both "cpu_single_env must be TLS"
>> (ie (a) system emulation built with KVM support and (b) linux-user),
>> the set of targets which can be non-TLS is really really small,
>> and I think (1) makes much more sense.
>
> Not many linux-user targets support threads (including not i386).

i386 guest is a comparatively rare case for linux-user (because
most people have an i386 box they can run them on). Also I'm
hoping to get most of the linux-user guests up to the point where
we can just have CONFIG_NPTL be true for all of them -- I have
several patches on-list which are trying to head in that direction.

thanks
-- PMM

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 28/30] exec: change iotlb APIs to take AddressSpaceDispatch
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 28/30] exec: change iotlb " Paolo Bonzini
@ 2013-07-02 10:00   ` Jan Kiszka
  0 siblings, 0 replies; 68+ messages in thread
From: Jan Kiszka @ 2013-07-02 10:00 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 2013-06-28 20:26, Paolo Bonzini wrote:
> This makes it possible to start following RCU rules, which require
> not dereferencing as->dispatch more than once.  It is not covering
> the whole of TCG, since the TLB data structures are not RCU-friendly,
> but it is enough for exec.c.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  cputlb.c              | 7 ++++---
>  exec.c                | 9 +++++----
>  include/exec/cputlb.h | 9 ++++++---
>  3 files changed, 15 insertions(+), 10 deletions(-)
> 
> diff --git a/cputlb.c b/cputlb.c
> index 51381ae..82875b1 100644
> --- a/cputlb.c
> +++ b/cputlb.c
> @@ -253,6 +253,7 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
>                    hwaddr paddr, int prot,
>                    int mmu_idx, target_ulong size)
>  {
> +    AddressSpaceDispatch *d;
>      MemoryRegionSection *section;
>      unsigned int index;
>      target_ulong address;
> @@ -267,8 +268,8 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
>      }
>  
>      sz = size;
> -    section = address_space_translate_for_iotlb(&address_space_memory, paddr,
> -                                                &xlat, &sz);
> +    d = address_space_memory.dispatch;
> +    section = address_space_translate_for_iotlb(d, paddr, &xlat, &sz);
>      assert(sz >= TARGET_PAGE_SIZE);
>  
>  #if defined(DEBUG_TLB)
> @@ -288,7 +289,7 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
>      }
>  
>      code_address = address;
> -    iotlb = memory_region_section_get_iotlb(env, section, vaddr, paddr, xlat,
> +    iotlb = memory_region_section_get_iotlb(d, env, section, vaddr, paddr, xlat,
>                                              prot, &address);
>  
>      index = (vaddr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1);
> diff --git a/exec.c b/exec.c
> index 528c4d7..3e1a576 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -306,11 +306,11 @@ MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
>  }
>  
>  MemoryRegionSection *
> -address_space_translate_for_iotlb(AddressSpace *as, hwaddr addr, hwaddr *xlat,
> +address_space_translate_for_iotlb(AddressSpaceDispatch *d, hwaddr addr, hwaddr *xlat,
>                                    hwaddr *plen)
>  {
>      MemoryRegionSection *section;
> -    section = address_space_translate_internal(as->dispatch, addr, xlat, plen, false);
> +    section = address_space_translate_internal(d, addr, xlat, plen, false);
>  
>      assert(!section->mr->iommu_ops);
>      return section;
> @@ -726,7 +726,8 @@ static int cpu_physical_memory_set_dirty_tracking(int enable)
>      return ret;
>  }
>  
> -hwaddr memory_region_section_get_iotlb(CPUArchState *env,
> +hwaddr memory_region_section_get_iotlb(AddressSpaceDispatch *d,
> +                                       CPUArchState *env,
>                                         MemoryRegionSection *section,
>                                         target_ulong vaddr,
>                                         hwaddr paddr, hwaddr xlat,
> @@ -746,7 +747,7 @@ hwaddr memory_region_section_get_iotlb(CPUArchState *env,
>              iotlb |= PHYS_SECTION_ROM;
>          }
>      } else {
> -        iotlb = section - address_space_memory.dispatch->sections;
> +        iotlb = section - d->sections;
>          iotlb += xlat;
>      }
>  
> diff --git a/include/exec/cputlb.h b/include/exec/cputlb.h
> index e21cb60..968b6a4 100644
> --- a/include/exec/cputlb.h
> +++ b/include/exec/cputlb.h
> @@ -31,12 +31,15 @@ void tlb_set_dirty(CPUArchState *env, target_ulong vaddr);
>  extern int tlb_flush_count;
>  
>  /* exec.c */
> +typedef struct AddressSpaceDispatch AddressSpaceDispatch;
> +
>  void tb_flush_jmp_cache(CPUArchState *env, target_ulong addr);
>  
>  MemoryRegionSection *
> -address_space_translate_for_iotlb(AddressSpace *as, hwaddr addr, hwaddr *xlat,
> -                                  hwaddr *plen);
> -hwaddr memory_region_section_get_iotlb(CPUArchState *env,
> +address_space_translate_for_iotlb(AddressSpaceDispatch *d, hwaddr addr,
> +                                  hwaddr *xlat, hwaddr *plen);
> +hwaddr memory_region_section_get_iotlb(AddressSpaceDispatch *d,
> +                                       CPUArchState *env,
>                                         MemoryRegionSection *section,
>                                         target_ulong vaddr,
>                                         hwaddr paddr, hwaddr xlat,
> 

Not sure if this version was also affected, but the current one in your
git requires this to build:

diff --git a/include/exec/cputlb.h b/include/exec/cputlb.h
index 968b6a4..bd7ff2f 100644
--- a/include/exec/cputlb.h
+++ b/include/exec/cputlb.h
@@ -19,6 +19,8 @@
 #ifndef CPUTLB_H
 #define CPUTLB_H
 
+#include <exec/memory-internal.h>
+
 #if !defined(CONFIG_USER_ONLY)
 /* cputlb.c */
 void tlb_protect_code(ram_addr_t ram_addr);
@@ -31,8 +33,6 @@ void tlb_set_dirty(CPUArchState *env, target_ulong vaddr);
 extern int tlb_flush_count;
 
 /* exec.c */
-typedef struct AddressSpaceDispatch AddressSpaceDispatch;
-
 void tb_flush_jmp_cache(CPUArchState *env, target_ulong addr);
 
 MemoryRegionSection *


Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply related	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-07-01 21:34               ` Peter Maydell
@ 2013-07-02 13:40                 ` Andreas Färber
  2013-07-02 14:06                   ` Alexander Graf
  0 siblings, 1 reply; 68+ messages in thread
From: Andreas Färber @ 2013-07-02 13:40 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Paolo Bonzini, qemu-devel, Alexander Graf

Am 01.07.2013 23:34, schrieb Peter Maydell:
> On 1 July 2013 21:52, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> Il 01/07/2013 18:26, Peter Maydell ha scritto:
>>> Since the two largest cases are both "cpu_single_env must be TLS"
>>> (ie (a) system emulation built with KVM support and (b) linux-user),
>>> the set of targets which can be non-TLS is really really small,
>>> and I think (1) makes much more sense.
>>
>> Not many linux-user targets support threads (including not i386).
> 
> i386 guest is a comparatively rare case for linux-user (because
> most people have an i386 box they can run them on). Also I'm
> hoping to get most of the linux-user guests up to the point where
> we can just have CONFIG_NPTL be true for all of them -- I have
> several patches on-list which are trying to head in that direction.

Alex had posted a patch which implements NPTL for i386, tested with
WINE, I believe.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h
  2013-07-02 13:40                 ` Andreas Färber
@ 2013-07-02 14:06                   ` Alexander Graf
  0 siblings, 0 replies; 68+ messages in thread
From: Alexander Graf @ 2013-07-02 14:06 UTC (permalink / raw)
  To: Andreas Färber; +Cc: Peter Maydell, qemu-devel, Paolo Bonzini

On 07/02/2013 03:40 PM, Andreas Färber wrote:
> Am 01.07.2013 23:34, schrieb Peter Maydell:
>> On 1 July 2013 21:52, Paolo Bonzini<pbonzini@redhat.com>  wrote:
>>> Il 01/07/2013 18:26, Peter Maydell ha scritto:
>>>> Since the two largest cases are both "cpu_single_env must be TLS"
>>>> (ie (a) system emulation built with KVM support and (b) linux-user),
>>>> the set of targets which can be non-TLS is really really small,
>>>> and I think (1) makes much more sense.
>>> Not many linux-user targets support threads (including not i386).
>> i386 guest is a comparatively rare case for linux-user (because
>> most people have an i386 box they can run them on). Also I'm
>> hoping to get most of the linux-user guests up to the point where
>> we can just have CONFIG_NPTL be true for all of them -- I have
>> several patches on-list which are trying to head in that direction.
> Alex had posted a patch which implements NPTL for i386, tested with
> WINE, I believe.

Not sure I posted it, but I do have it ready. I'll rebase it once the 
NPTL refactorings are in. They look very sane to me.


Alex

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 21/30] exec: separate current memory map from the one being built
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 21/30] exec: separate current memory map from the one being built Paolo Bonzini
@ 2013-07-02 14:41   ` Jan Kiszka
  0 siblings, 0 replies; 68+ messages in thread
From: Jan Kiszka @ 2013-07-02 14:41 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel, Liu Ping Fan

On 2013-06-28 20:26, Paolo Bonzini wrote:
> Currently, phys_node_map and phys_sections are shared by all
> of the AddressSpaceDispatch.  When updating mem topology, all
> AddressSpaceDispatch will rebuild dispatch tables sequentially
> on them.  In order to prepare for RCU access, leave the old
> memory map alive while the next one is being accessed.
> 
> When rebuilding, the new dispatch tables will build and lookup
> next_map; after all dispatch tables are rebuilt, we can switch
> to next_* and free the previous table.
> 
> Based on a patch from Liu Ping Fan.
> 
> Signed-off-by: Liu Ping Fan <qemulist@gmail.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  exec.c | 103 ++++++++++++++++++++++++++++++++++++++++-------------------------
>  1 file changed, 63 insertions(+), 40 deletions(-)
> 
> diff --git a/exec.c b/exec.c
> index 7ad513c..f138e56 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -111,16 +111,24 @@ typedef struct subpage_t {
>      uint16_t sub_section[TARGET_PAGE_SIZE];
>  } subpage_t;
>  
> -static MemoryRegionSection *phys_sections;
> -static unsigned phys_sections_nb, phys_sections_nb_alloc;
>  #define PHYS_SECTION_UNASSIGNED 0
>  #define PHYS_SECTION_NOTDIRTY 1
>  #define PHYS_SECTION_ROM 2
>  #define PHYS_SECTION_WATCH 3
>  
> -/* Simple allocator for PhysPageEntry nodes */
> -static PhysPageEntry (*phys_map_nodes)[L2_SIZE];
> -static unsigned phys_map_nodes_nb, phys_map_nodes_nb_alloc;
> +typedef PhysPageEntry Node[L2_SIZE];
> +
> +typedef struct PhysPageMap {
> +    unsigned sections_nb;
> +    unsigned sections_nb_alloc;
> +    unsigned nodes_nb;
> +    unsigned nodes_nb_alloc;
> +    Node *nodes;
> +    MemoryRegionSection *sections;
> +} PhysPageMap;
> +
> +static PhysPageMap cur_map;
> +static PhysPageMap next_map;
>  
>  #define PHYS_MAP_NODE_NIL (((uint16_t)~0) >> 1)
>  
> @@ -135,13 +143,13 @@ static MemoryRegion io_mem_watch;
>  
>  static void phys_map_node_reserve(unsigned nodes)
>  {
> -    if (phys_map_nodes_nb + nodes > phys_map_nodes_nb_alloc) {
> -        typedef PhysPageEntry Node[L2_SIZE];
> -        phys_map_nodes_nb_alloc = MAX(phys_map_nodes_nb_alloc * 2, 16);
> -        phys_map_nodes_nb_alloc = MAX(phys_map_nodes_nb_alloc,
> -                                      phys_map_nodes_nb + nodes);
> -        phys_map_nodes = g_renew(Node, phys_map_nodes,
> -                                 phys_map_nodes_nb_alloc);
> +    if (next_map.nodes_nb + nodes > next_map.nodes_nb_alloc) {
> +        next_map.nodes_nb_alloc = MAX(next_map.nodes_nb_alloc * 2,
> +                                            16);
> +        next_map.nodes_nb_alloc = MAX(next_map.nodes_nb_alloc,
> +                                      next_map.nodes_nb + nodes);
> +        next_map.nodes = g_renew(Node, next_map.nodes,
> +                                 next_map.nodes_nb_alloc);
>      }
>  }
>  
> @@ -150,12 +158,12 @@ static uint16_t phys_map_node_alloc(void)
>      unsigned i;
>      uint16_t ret;
>  
> -    ret = phys_map_nodes_nb++;
> +    ret = next_map.nodes_nb++;
>      assert(ret != PHYS_MAP_NODE_NIL);
> -    assert(ret != phys_map_nodes_nb_alloc);
> +    assert(ret != next_map.nodes_nb_alloc);
>      for (i = 0; i < L2_SIZE; ++i) {
> -        phys_map_nodes[ret][i].is_leaf = 0;
> -        phys_map_nodes[ret][i].ptr = PHYS_MAP_NODE_NIL;
> +        next_map.nodes[ret][i].is_leaf = 0;
> +        next_map.nodes[ret][i].ptr = PHYS_MAP_NODE_NIL;
>      }
>      return ret;
>  }
> @@ -170,7 +178,7 @@ static void phys_page_set_level(PhysPageEntry *lp, hwaddr *index,
>  
>      if (!lp->is_leaf && lp->ptr == PHYS_MAP_NODE_NIL) {
>          lp->ptr = phys_map_node_alloc();
> -        p = phys_map_nodes[lp->ptr];
> +        p = next_map.nodes[lp->ptr];
>          if (level == 0) {
>              for (i = 0; i < L2_SIZE; i++) {
>                  p[i].is_leaf = 1;
> @@ -178,7 +186,7 @@ static void phys_page_set_level(PhysPageEntry *lp, hwaddr *index,
>              }
>          }
>      } else {
> -        p = phys_map_nodes[lp->ptr];
> +        p = next_map.nodes[lp->ptr];
>      }
>      lp = &p[(*index >> (level * L2_BITS)) & (L2_SIZE - 1)];
>  
> @@ -205,20 +213,20 @@ static void phys_page_set(AddressSpaceDispatch *d,
>      phys_page_set_level(&d->phys_map, &index, &nb, leaf, P_L2_LEVELS - 1);
>  }
>  
> -static MemoryRegionSection *phys_page_find(AddressSpaceDispatch *d, hwaddr index)
> +static MemoryRegionSection *phys_page_find(PhysPageEntry lp, hwaddr index,
> +                                           Node *nodes, MemoryRegionSection *sections)

As both nodes and sections should belong to the same PhysPageMap,
passing a reference to the latter would be a marginally cleaner.

>  {
> -    PhysPageEntry lp = d->phys_map;
>      PhysPageEntry *p;
>      int i;
>  
>      for (i = P_L2_LEVELS - 1; i >= 0 && !lp.is_leaf; i--) {
>          if (lp.ptr == PHYS_MAP_NODE_NIL) {
> -            return &phys_sections[PHYS_SECTION_UNASSIGNED];
> +            return &sections[PHYS_SECTION_UNASSIGNED];
>          }
> -        p = phys_map_nodes[lp.ptr];
> +        p = nodes[lp.ptr];
>          lp = p[(index >> (i * L2_BITS)) & (L2_SIZE - 1)];
>      }
> -    return &phys_sections[lp.ptr];
> +    return &sections[lp.ptr];
>  }
>  
>  bool memory_region_is_unassigned(MemoryRegion *mr)
> @@ -234,10 +242,11 @@ static MemoryRegionSection *address_space_lookup_region(AddressSpace *as,
>      MemoryRegionSection *section;
>      subpage_t *subpage;
>  
> -    section = phys_page_find(as->dispatch, addr >> TARGET_PAGE_BITS);
> +    section = phys_page_find(as->dispatch->phys_map, addr >> TARGET_PAGE_BITS,
> +                             cur_map.nodes, cur_map.sections);
>      if (resolve_subpage && section->mr->subpage) {
>          subpage = container_of(section->mr, subpage_t, iomem);
> -        section = &phys_sections[subpage->sub_section[SUBPAGE_IDX(addr)]];
> +        section = &cur_map.sections[subpage->sub_section[SUBPAGE_IDX(addr)]];
>      }
>      return section;
>  }
> @@ -736,7 +745,7 @@ hwaddr memory_region_section_get_iotlb(CPUArchState *env,
>              iotlb |= PHYS_SECTION_ROM;
>          }
>      } else {
> -        iotlb = section - phys_sections;
> +        iotlb = section - cur_map.sections;
>          iotlb += xlat;
>      }
>  
> @@ -769,16 +778,17 @@ static uint16_t phys_section_add(MemoryRegionSection *section)
>       * pointer to produce the iotlb entries.  Thus it should
>       * never overflow into the page-aligned value.
>       */
> -    assert(phys_sections_nb < TARGET_PAGE_SIZE);
> +    assert(next_map.sections_nb < TARGET_PAGE_SIZE);
>  
> -    if (phys_sections_nb == phys_sections_nb_alloc) {
> -        phys_sections_nb_alloc = MAX(phys_sections_nb_alloc * 2, 16);
> -        phys_sections = g_renew(MemoryRegionSection, phys_sections,
> -                                phys_sections_nb_alloc);
> +    if (next_map.sections_nb == next_map.sections_nb_alloc) {
> +        next_map.sections_nb_alloc = MAX(next_map.sections_nb_alloc * 2,
> +                                         16);
> +        next_map.sections = g_renew(MemoryRegionSection, next_map.sections,
> +                                    next_map.sections_nb_alloc);
>      }
> -    phys_sections[phys_sections_nb] = *section;
> +    next_map.sections[next_map.sections_nb] = *section;
>      memory_region_ref(section->mr);
> -    return phys_sections_nb++;
> +    return next_map.sections_nb++;
>  }
>  
>  static void phys_section_destroy(MemoryRegion *mr)
> @@ -792,13 +802,14 @@ static void phys_section_destroy(MemoryRegion *mr)
>      }
>  }
>  
> -static void phys_sections_clear(void)
> +static void phys_sections_clear(PhysPageMap *map)
>  {
> -    while (phys_sections_nb > 0) {
> -        MemoryRegionSection *section = &phys_sections[--phys_sections_nb];
> +    while (map->sections_nb > 0) {
> +        MemoryRegionSection *section = &map->sections[--map->sections_nb];
>          phys_section_destroy(section->mr);
>      }
> -    phys_map_nodes_nb = 0;
> +    g_free(map->sections);
> +    g_free(map->nodes);
>  }
>  
>  static void register_subpage(AddressSpaceDispatch *d, MemoryRegionSection *section)
> @@ -806,7 +817,8 @@ static void register_subpage(AddressSpaceDispatch *d, MemoryRegionSection *secti
>      subpage_t *subpage;
>      hwaddr base = section->offset_within_address_space
>          & TARGET_PAGE_MASK;
> -    MemoryRegionSection *existing = phys_page_find(d, base >> TARGET_PAGE_BITS);
> +    MemoryRegionSection *existing = phys_page_find(d->phys_map, base >> TARGET_PAGE_BITS,
> +                                                   next_map.nodes, next_map.sections);
>      MemoryRegionSection subsection = {
>          .offset_within_address_space = base,
>          .size = int128_make64(TARGET_PAGE_SIZE),
> @@ -1689,7 +1701,7 @@ static uint16_t dummy_section(MemoryRegion *mr)
>  
>  MemoryRegion *iotlb_to_region(hwaddr index)
>  {
> -    return phys_sections[index & ~TARGET_PAGE_MASK].mr;
> +    return cur_map.sections[index & ~TARGET_PAGE_MASK].mr;
>  }
>  
>  static void io_mem_init(void)
> @@ -1714,7 +1726,7 @@ static void core_begin(MemoryListener *listener)
>  {
>      uint16_t n;
>  
> -    phys_sections_clear();
> +    memset(&next_map, 0, sizeof(next_map));
>      n = dummy_section(&io_mem_unassigned);
>      assert(n == PHYS_SECTION_UNASSIGNED);
>      n = dummy_section(&io_mem_notdirty);
> @@ -1725,6 +1737,16 @@ static void core_begin(MemoryListener *listener)
>      assert(n == PHYS_SECTION_WATCH);
>  }
>  
> +/* This listener's commit run after the other AddressSpaceDispatch listeners'.
> + * All AddressSpaceDispatch instances have switched to the next map.
> + */
> +static void core_commit(MemoryListener *listener)
> +{
> +    PhysPageMap info = cur_map;

"info"? Why not "last_map"? Oh, it disappears later on anyway.

> +    cur_map = next_map;
> +    phys_sections_clear(&info);
> +}
> +
>  static void tcg_commit(MemoryListener *listener)
>  {
>      CPUArchState *env;
> @@ -1749,6 +1771,7 @@ static void core_log_global_stop(MemoryListener *listener)
>  
>  static MemoryListener core_memory_listener = {
>      .begin = core_begin,
> +    .commit = core_commit,
>      .log_global_start = core_log_global_start,
>      .log_global_stop = core_log_global_stop,
>      .priority = 1,
> 

Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com>

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 22/30] memory: move MemoryListener declaration earlier
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 22/30] memory: move MemoryListener declaration earlier Paolo Bonzini
@ 2013-07-02 14:41   ` Jan Kiszka
  0 siblings, 0 replies; 68+ messages in thread
From: Jan Kiszka @ 2013-07-02 14:41 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 2013-06-28 20:26, Paolo Bonzini wrote:
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  include/exec/memory.h | 66 +++++++++++++++++++++++++--------------------------
>  1 file changed, 33 insertions(+), 33 deletions(-)
> 
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index aa7a922..913ac45 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -164,6 +164,39 @@ struct MemoryRegion {
>      NotifierList iommu_notify;
>  };
>  
> +typedef struct MemoryListener MemoryListener;
> +
> +/**
> + * MemoryListener: callbacks structure for updates to the physical memory map
> + *
> + * Allows a component to adjust to changes in the guest-visible memory map.
> + * Use with memory_listener_register() and memory_listener_unregister().
> + */
> +struct MemoryListener {
> +    void (*begin)(MemoryListener *listener);
> +    void (*commit)(MemoryListener *listener);
> +    void (*region_add)(MemoryListener *listener, MemoryRegionSection *section);
> +    void (*region_del)(MemoryListener *listener, MemoryRegionSection *section);
> +    void (*region_nop)(MemoryListener *listener, MemoryRegionSection *section);
> +    void (*log_start)(MemoryListener *listener, MemoryRegionSection *section);
> +    void (*log_stop)(MemoryListener *listener, MemoryRegionSection *section);
> +    void (*log_sync)(MemoryListener *listener, MemoryRegionSection *section);
> +    void (*log_global_start)(MemoryListener *listener);
> +    void (*log_global_stop)(MemoryListener *listener);
> +    void (*eventfd_add)(MemoryListener *listener, MemoryRegionSection *section,
> +                        bool match_data, uint64_t data, EventNotifier *e);
> +    void (*eventfd_del)(MemoryListener *listener, MemoryRegionSection *section,
> +                        bool match_data, uint64_t data, EventNotifier *e);
> +    void (*coalesced_mmio_add)(MemoryListener *listener, MemoryRegionSection *section,
> +                               hwaddr addr, hwaddr len);
> +    void (*coalesced_mmio_del)(MemoryListener *listener, MemoryRegionSection *section,
> +                               hwaddr addr, hwaddr len);
> +    /* Lower = earlier (during add), later (during del) */
> +    unsigned priority;
> +    AddressSpace *address_space_filter;
> +    QTAILQ_ENTRY(MemoryListener) link;
> +};
> +
>  /**
>   * AddressSpace: describes a mapping of addresses to #MemoryRegion objects
>   */
> @@ -202,39 +235,6 @@ struct MemoryRegionSection {
>      bool readonly;
>  };
>  
> -typedef struct MemoryListener MemoryListener;
> -
> -/**
> - * MemoryListener: callbacks structure for updates to the physical memory map
> - *
> - * Allows a component to adjust to changes in the guest-visible memory map.
> - * Use with memory_listener_register() and memory_listener_unregister().
> - */
> -struct MemoryListener {
> -    void (*begin)(MemoryListener *listener);
> -    void (*commit)(MemoryListener *listener);
> -    void (*region_add)(MemoryListener *listener, MemoryRegionSection *section);
> -    void (*region_del)(MemoryListener *listener, MemoryRegionSection *section);
> -    void (*region_nop)(MemoryListener *listener, MemoryRegionSection *section);
> -    void (*log_start)(MemoryListener *listener, MemoryRegionSection *section);
> -    void (*log_stop)(MemoryListener *listener, MemoryRegionSection *section);
> -    void (*log_sync)(MemoryListener *listener, MemoryRegionSection *section);
> -    void (*log_global_start)(MemoryListener *listener);
> -    void (*log_global_stop)(MemoryListener *listener);
> -    void (*eventfd_add)(MemoryListener *listener, MemoryRegionSection *section,
> -                        bool match_data, uint64_t data, EventNotifier *e);
> -    void (*eventfd_del)(MemoryListener *listener, MemoryRegionSection *section,
> -                        bool match_data, uint64_t data, EventNotifier *e);
> -    void (*coalesced_mmio_add)(MemoryListener *listener, MemoryRegionSection *section,
> -                               hwaddr addr, hwaddr len);
> -    void (*coalesced_mmio_del)(MemoryListener *listener, MemoryRegionSection *section,
> -                               hwaddr addr, hwaddr len);
> -    /* Lower = earlier (during add), later (during del) */
> -    unsigned priority;
> -    AddressSpace *address_space_filter;
> -    QTAILQ_ENTRY(MemoryListener) link;
> -};
> -
>  /**
>   * memory_region_init: Initialize a memory region
>   *
> 

Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com>

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 23/30] exec: move listener from AddressSpaceDispatch to AddressSpace
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 23/30] exec: move listener from AddressSpaceDispatch to AddressSpace Paolo Bonzini
@ 2013-07-02 14:41   ` Jan Kiszka
  0 siblings, 0 replies; 68+ messages in thread
From: Jan Kiszka @ 2013-07-02 14:41 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 2013-06-28 20:26, Paolo Bonzini wrote:
> This will help having two copies of AddressSpaceDispatch during the
> recreation of the radix tree (one being built, and one that is complete
> and accessed under RCU).  We do not want to have to unregister and
> re-register the listener.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  exec.c                | 17 +++++++++--------
>  include/exec/memory.h |  2 ++
>  2 files changed, 11 insertions(+), 8 deletions(-)
> 
> diff --git a/exec.c b/exec.c
> index f138e56..dffdf23 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -99,7 +99,6 @@ struct AddressSpaceDispatch {
>       * The bottom level has pointers to MemoryRegionSections.
>       */
>      PhysPageEntry phys_map;
> -    MemoryListener listener;
>      AddressSpace *as;
>  };
>  
> @@ -855,7 +854,8 @@ static void register_multipage(AddressSpaceDispatch *d,
>  
>  static void mem_add(MemoryListener *listener, MemoryRegionSection *section)
>  {
> -    AddressSpaceDispatch *d = container_of(listener, AddressSpaceDispatch, listener);
> +    AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
> +    AddressSpaceDispatch *d = as->dispatch;
>      MemoryRegionSection now = *section, remain = *section;
>      Int128 page_size = int128_make64(TARGET_PAGE_SIZE);
>  
> @@ -1717,7 +1717,8 @@ static void io_mem_init(void)
>  
>  static void mem_begin(MemoryListener *listener)
>  {
> -    AddressSpaceDispatch *d = container_of(listener, AddressSpaceDispatch, listener);
> +    AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
> +    AddressSpaceDispatch *d = as->dispatch;
>  
>      d->phys_map.ptr = PHYS_MAP_NODE_NIL;
>  }
> @@ -1786,22 +1787,22 @@ void address_space_init_dispatch(AddressSpace *as)
>      AddressSpaceDispatch *d = g_new(AddressSpaceDispatch, 1);
>  
>      d->phys_map  = (PhysPageEntry) { .ptr = PHYS_MAP_NODE_NIL, .is_leaf = 0 };
> -    d->listener = (MemoryListener) {
> +    d->as = as;
> +    as->dispatch = d;
> +    as->dispatch_listener = (MemoryListener) {
>          .begin = mem_begin,
>          .region_add = mem_add,
>          .region_nop = mem_add,
>          .priority = 0,
>      };
> -    d->as = as;
> -    as->dispatch = d;
> -    memory_listener_register(&d->listener, as);
> +    memory_listener_register(&as->dispatch_listener, as);
>  }
>  
>  void address_space_destroy_dispatch(AddressSpace *as)
>  {
>      AddressSpaceDispatch *d = as->dispatch;

You could save this line, but only if you need to touch the patch for
some other reason.

>  
> -    memory_listener_unregister(&d->listener);
> +    memory_listener_unregister(&as->dispatch_listener);
>      g_free(d);
>      as->dispatch = NULL;
>  }
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index 913ac45..1cd1f50 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -212,6 +212,8 @@ struct AddressSpace {
>      int ioeventfd_nb;
>      struct MemoryRegionIoeventfd *ioeventfds;
>      struct AddressSpaceDispatch *dispatch;
> +    MemoryListener dispatch_listener;
> +
>      QTAILQ_ENTRY(AddressSpace) address_spaces_link;
>  };
>  
> 

Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com>

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 24/30] exec: separate current radix tree from the one being built
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 24/30] exec: separate current radix tree from the one being built Paolo Bonzini
@ 2013-07-02 14:41   ` Jan Kiszka
  0 siblings, 0 replies; 68+ messages in thread
From: Jan Kiszka @ 2013-07-02 14:41 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 2013-06-28 20:26, Paolo Bonzini wrote:
> This same treatment previously done to phys_node_map and phys_sections
> is now applied to the dispatch field of AddressSpace.  Topology updates
> use as->next_dispatch while accesses use as->dispatch.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  exec.c                | 23 ++++++++++++++++-------
>  include/exec/memory.h |  1 +
>  2 files changed, 17 insertions(+), 7 deletions(-)
> 
> diff --git a/exec.c b/exec.c
> index dffdf23..0d852ee 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -855,7 +855,7 @@ static void register_multipage(AddressSpaceDispatch *d,
>  static void mem_add(MemoryListener *listener, MemoryRegionSection *section)
>  {
>      AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
> -    AddressSpaceDispatch *d = as->dispatch;
> +    AddressSpaceDispatch *d = as->next_dispatch;
>      MemoryRegionSection now = *section, remain = *section;
>      Int128 page_size = int128_make64(TARGET_PAGE_SIZE);
>  
> @@ -1718,9 +1718,21 @@ static void io_mem_init(void)
>  static void mem_begin(MemoryListener *listener)
>  {
>      AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
> +    AddressSpaceDispatch *d = g_new(AddressSpaceDispatch, 1);
> +
> +    d->phys_map  = (PhysPageEntry) { .ptr = PHYS_MAP_NODE_NIL, .is_leaf = 0 };
> +    d->as = as;
> +    as->next_dispatch = d;
> +}
> +
> +static void mem_commit(MemoryListener *listener)
> +{
> +    AddressSpace *as = container_of(listener, AddressSpace, dispatch_listener);
>      AddressSpaceDispatch *d = as->dispatch;
>  
> -    d->phys_map.ptr = PHYS_MAP_NODE_NIL;
> +    /* cur_map will soon be switched to next_map, too.  */
> +    as->dispatch = as->next_dispatch;
> +    g_free(d);
>  }
>  
>  static void core_begin(MemoryListener *listener)
> @@ -1784,13 +1796,10 @@ static MemoryListener tcg_memory_listener = {
>  
>  void address_space_init_dispatch(AddressSpace *as)
>  {
> -    AddressSpaceDispatch *d = g_new(AddressSpaceDispatch, 1);
> -
> -    d->phys_map  = (PhysPageEntry) { .ptr = PHYS_MAP_NODE_NIL, .is_leaf = 0 };
> -    d->as = as;
> -    as->dispatch = d;
> +    as->dispatch = NULL;
>      as->dispatch_listener = (MemoryListener) {
>          .begin = mem_begin,
> +        .commit = mem_commit,
>          .region_add = mem_add,
>          .region_nop = mem_add,
>          .priority = 0,
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index 1cd1f50..b21a460 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -212,6 +212,7 @@ struct AddressSpace {
>      int ioeventfd_nb;
>      struct MemoryRegionIoeventfd *ioeventfds;
>      struct AddressSpaceDispatch *dispatch;
> +    struct AddressSpaceDispatch *next_dispatch;
>      MemoryListener dispatch_listener;
>  
>      QTAILQ_ENTRY(AddressSpace) address_spaces_link;
> 

Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com>

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 25/30] exec: put memory map in AddressSpaceDispatch
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 25/30] exec: put memory map in AddressSpaceDispatch Paolo Bonzini
@ 2013-07-02 14:42   ` Jan Kiszka
  2013-07-02 15:08     ` Paolo Bonzini
  0 siblings, 1 reply; 68+ messages in thread
From: Jan Kiszka @ 2013-07-02 14:42 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 2013-06-28 20:26, Paolo Bonzini wrote:
> This lets us get a consistent (phys_map, nodes, sections) using
> RCU.  After this patch, cur_map is not used anymore except for freeing
> it at the end of the topology update.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  exec.c | 28 +++++++++++++++++-----------
>  1 file changed, 17 insertions(+), 11 deletions(-)
> 
> diff --git a/exec.c b/exec.c
> index 0d852ee..581f0c4 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -94,11 +94,15 @@ struct PhysPageEntry {
>      uint16_t ptr : 15;
>  };
>  
> +typedef PhysPageEntry Node[L2_SIZE];
> +
>  struct AddressSpaceDispatch {
>      /* This is a multi-level map on the physical address space.
>       * The bottom level has pointers to MemoryRegionSections.
>       */
>      PhysPageEntry phys_map;
> +    Node *nodes;
> +    MemoryRegionSection *sections;

Why not sticking the whole current PhysPageMap into here? Wouldn't that
also allow to overcome prev_map completely (next patch)?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 27/30] exec: change some APIs to take AddressSpaceDispatch
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 27/30] exec: change some APIs to take AddressSpaceDispatch Paolo Bonzini
@ 2013-07-02 14:47   ` Jan Kiszka
  0 siblings, 0 replies; 68+ messages in thread
From: Jan Kiszka @ 2013-07-02 14:47 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 2013-06-28 20:26, Paolo Bonzini wrote:
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  exec.c | 11 +++++------
>  1 file changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/exec.c b/exec.c
> index 7f87e16..528c4d7 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -236,11 +236,10 @@ bool memory_region_is_unassigned(MemoryRegion *mr)
>          && mr != &io_mem_watch;
>  }
>  
> -static MemoryRegionSection *address_space_lookup_region(AddressSpace *as,
> +static MemoryRegionSection *address_space_lookup_region(AddressSpaceDispatch *d,
>                                                          hwaddr addr,
>                                                          bool resolve_subpage)
>  {
> -    AddressSpaceDispatch *d = as->dispatch;
>      MemoryRegionSection *section;
>      subpage_t *subpage;
>  
> @@ -254,13 +253,13 @@ static MemoryRegionSection *address_space_lookup_region(AddressSpace *as,
>  }
>  
>  static MemoryRegionSection *
> -address_space_translate_internal(AddressSpace *as, hwaddr addr, hwaddr *xlat,
> +address_space_translate_internal(AddressSpaceDispatch *d, hwaddr addr, hwaddr *xlat,
>                                   hwaddr *plen, bool resolve_subpage)
>  {
>      MemoryRegionSection *section;
>      Int128 diff;
>  
> -    section = address_space_lookup_region(as, addr, resolve_subpage);
> +    section = address_space_lookup_region(d, addr, resolve_subpage);
>      /* Compute offset within MemoryRegionSection */
>      addr -= section->offset_within_address_space;
>  
> @@ -282,7 +281,7 @@ MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
>      hwaddr len = *plen;
>  
>      for (;;) {
> -        section = address_space_translate_internal(as, addr, &addr, plen, true);
> +        section = address_space_translate_internal(as->dispatch, addr, &addr, plen, true);
>          mr = section->mr;
>  
>          if (!mr->iommu_ops) {
> @@ -311,7 +310,7 @@ address_space_translate_for_iotlb(AddressSpace *as, hwaddr addr, hwaddr *xlat,
>                                    hwaddr *plen)
>  {
>      MemoryRegionSection *section;
> -    section = address_space_translate_internal(as, addr, xlat, plen, false);
> +    section = address_space_translate_internal(as->dispatch, addr, xlat, plen, false);
>  
>      assert(!section->mr->iommu_ops);
>      return section;
> 

Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com>

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 25/30] exec: put memory map in AddressSpaceDispatch
  2013-07-02 14:42   ` Jan Kiszka
@ 2013-07-02 15:08     ` Paolo Bonzini
  2013-07-02 15:48       ` Jan Kiszka
  0 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-07-02 15:08 UTC (permalink / raw)
  To: Jan Kiszka; +Cc: qemu-devel

Il 02/07/2013 16:42, Jan Kiszka ha scritto:
>> > +typedef PhysPageEntry Node[L2_SIZE];
>> > +
>> >  struct AddressSpaceDispatch {
>> >      /* This is a multi-level map on the physical address space.
>> >       * The bottom level has pointers to MemoryRegionSections.
>> >       */
>> >      PhysPageEntry phys_map;
>> > +    Node *nodes;
>> > +    MemoryRegionSection *sections;
> Why not sticking the whole current PhysPageMap into here? Wouldn't that
> also allow to overcome prev_map completely (next patch)?

(BTW, this is also why patch 21 has two separate arguments to
phys_page_find).

Sticking a pointer would add one useless pointer chase.  Storing it by
value makes it unclear who owns the PhysPageMap and doesn't say that
AddressSpaceDispatch cannot extend the vectors.  In the end, this seemed
like a good compromise.

Paolo

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 25/30] exec: put memory map in AddressSpaceDispatch
  2013-07-02 15:08     ` Paolo Bonzini
@ 2013-07-02 15:48       ` Jan Kiszka
  0 siblings, 0 replies; 68+ messages in thread
From: Jan Kiszka @ 2013-07-02 15:48 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On 2013-07-02 17:08, Paolo Bonzini wrote:
> Il 02/07/2013 16:42, Jan Kiszka ha scritto:
>>>> +typedef PhysPageEntry Node[L2_SIZE];
>>>> +
>>>>  struct AddressSpaceDispatch {
>>>>      /* This is a multi-level map on the physical address space.
>>>>       * The bottom level has pointers to MemoryRegionSections.
>>>>       */
>>>>      PhysPageEntry phys_map;
>>>> +    Node *nodes;
>>>> +    MemoryRegionSection *sections;
>> Why not sticking the whole current PhysPageMap into here? Wouldn't that
>> also allow to overcome prev_map completely (next patch)?
> 
> (BTW, this is also why patch 21 has two separate arguments to
> phys_page_find).
> 
> Sticking a pointer would add one useless pointer chase.  Storing it by
> value makes it unclear who owns the PhysPageMap and doesn't say that
> AddressSpaceDispatch cannot extend the vectors.  In the end, this seemed
> like a good compromise.

Well, PhysPageMap is still globally owned. And as long as this is not
broken up, you are likely right. So let's wait with this for the next
refactoring round.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-06-28 18:26 ` [Qemu-devel] [PATCH 04/30] add a header file for atomic operations Paolo Bonzini
  2013-06-28 20:41   ` Anthony Liguori
@ 2013-07-03  2:24   ` liu ping fan
  2013-07-03  5:59     ` Paolo Bonzini
  1 sibling, 1 reply; 68+ messages in thread
From: liu ping fan @ 2013-07-03  2:24 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On Sat, Jun 29, 2013 at 2:26 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> We're already using them in several places, but __sync builtins are just
> too ugly to type, and do not provide seqcst load/store operations.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  docs/atomics.txt         | 345 +++++++++++++++++++++++++++++++++++++++++++++++
>  hw/display/qxl.c         |   3 +-
>  hw/virtio/vhost.c        |   9 +-
>  include/qemu/atomic.h    | 190 +++++++++++++++++++++-----
>  migration.c              |   3 +-
>  tests/test-thread-pool.c |   8 +-
>  6 files changed, 514 insertions(+), 44 deletions(-)
>  create mode 100644 docs/atomics.txt
>
> diff --git a/docs/atomics.txt b/docs/atomics.txt
> new file mode 100644
> index 0000000..e2ce04b
> --- /dev/null
> +++ b/docs/atomics.txt
> @@ -0,0 +1,345 @@
> +CPUs perform independent memory operations effectively in random order.
> +but this can be a problem for CPU-CPU interaction (including interactions
> +between QEMU and the guest).  Multi-threaded programs use various tools
> +to instruct the compiler and the CPU to restrict the order to something
> +that is consistent with the expectations of the programmer.
> +
> +The most basic tool is locking.  Mutexes, condition variables and
> +semaphores are used in QEMU, and should be the default approach to
> +synchronization.  Anything else is considerably harder, but it's
> +also justified more often than one would like.  The two tools that
> +are provided by qemu/atomic.h are memory barriers and atomic operations.
> +
> +Macros defined by qemu/atomic.h fall in three camps:
> +
> +- compiler barriers: barrier();
> +
> +- weak atomic access and manual memory barriers: atomic_read(),
> +  atomic_set(), smp_rmb(), smp_wmb(), smp_mb(), smp_read_barrier_depends();
> +
> +- sequentially consistent atomic access: everything else.
> +
> +
> +COMPILER MEMORY BARRIER
> +=======================
> +
> +barrier() prevents the compiler from moving the memory accesses either
> +side of it to the other side.  The compiler barrier has no direct effect
> +on the CPU, which may then reorder things however it wishes.
> +
> +barrier() is mostly used within qemu/atomic.h itself.  On some
> +architectures, CPU guarantees are strong enough that blocking compiler
> +optimizations already ensures the correct order of execution.  In this
> +case, qemu/atomic.h will reduce stronger memory barriers to simple
> +compiler barriers.
> +
> +Still, barrier() can be useful when writing code that can be interrupted
> +by signal handlers.
> +
> +
> +SEQUENTIALLY CONSISTENT ATOMIC ACCESS
> +=====================================
> +
> +Most of the operations in the qemu/atomic.h header ensure *sequential
> +consistency*, where "the result of any execution is the same as if the
> +operations of all the processors were executed in some sequential order,
> +and the operations of each individual processor appear in this sequence
> +in the order specified by its program".
> +
> +qemu/atomic.h provides the following set of atomic read-modify-write
> +operations:
> +
> +    typeof(*ptr) atomic_inc(ptr)
> +    typeof(*ptr) atomic_dec(ptr)
> +    typeof(*ptr) atomic_add(ptr, val)
> +    typeof(*ptr) atomic_sub(ptr, val)
> +    typeof(*ptr) atomic_and(ptr, val)
> +    typeof(*ptr) atomic_or(ptr, val)
> +    typeof(*ptr) atomic_xchg(ptr, val
> +    typeof(*ptr) atomic_cmpxchg(ptr, old, new)
> +
> +all of which return the old value of *ptr.  These operations are
> +polymorphic; they operate on any type that is as wide as an int.
> +
> +Sequentially consistent loads and stores can be done using:
> +
> +    atomic_add(ptr, 0) for loads
> +    atomic_xchg(ptr, val) for stores
> +
> +However, they are quite expensive on some platforms, notably POWER and
> +ARM.  Therefore, qemu/atomic.h provides two primitives with slightly
> +weaker constraints:
> +
> +    typeof(*ptr) atomic_mb_read(ptr)
> +    void         atomic_mb_set(ptr, val)
> +
> +The semantics of these primitives map to Java volatile variables,
> +and are strongly related to memory barriers as used in the Linux
> +kernel (see below).
> +
> +As long as you use atomic_mb_read and atomic_mb_set, accesses cannot
> +be reordered with each other, and it is also not possible to reorder
> +"normal" accesses around them.
> +
> +However, and this is the important difference between
> +atomic_mb_read/atomic_mb_set and sequential consistency, it is important
> +for both threads to access the same volatile variable.  It is not the
> +case that everything visible to thread A when it writes volatile field f
> +becomes visible to thread B after it reads volatile field g. The store
> +and load have to "match" (i.e., be performed on the same volatile
> +field) to achieve the right semantics.
> +
> +
> +These operations operate on any type that is as wide as an int or smaller.
> +
> +
> +WEAK ATOMIC ACCESS AND MANUAL MEMORY BARRIERS
> +=============================================
> +
> +Compared to sequentially consistent atomic access, programming with
> +weaker consistency models can be considerably more complicated.
> +In general, if the algorithm you are writing includes both writes
> +and reads on the same side, it is generally simpler to use sequentially
> +consistent primitives.
> +
> +When using this model, variables are accessed with atomic_read() and
> +atomic_set(), and restrictions to the ordering of accesses is enforced
> +using the smp_rmb(), smp_wmb(), smp_mb() and smp_read_barrier_depends()
> +memory barriers.
> +
> +atomic_read() and atomic_set() prevents the compiler from using
> +optimizations that might otherwise optimize accesses out of existence
> +on the one hand, or that might create unsolicited accesses on the other.
> +In general this should not have any effect, because the same compiler
> +barriers are already implied by memory barriers.  However, it is useful
> +to do so, because it tells readers which variables are shared with
> +other threads, and which are local to the current thread or protected
> +by other, more mundane means.
> +
> +Memory barriers control the order of references to shared memory.
> +They come in four kinds:
> +
> +- smp_rmb() guarantees that all the LOAD operations specified before
> +  the barrier will appear to happen before all the LOAD operations
> +  specified after the barrier with respect to the other components of
> +  the system.
> +
> +  In other words, smp_rmb() puts a partial ordering on loads, but is not
> +  required to have any effect on stores.
> +
> +- smp_wmb() guarantees that all the STORE operations specified before
> +  the barrier will appear to happen before all the STORE operations
> +  specified after the barrier with respect to the other components of
> +  the system.
> +
> +  In other words, smp_wmb() puts a partial ordering on stores, but is not
> +  required to have any effect on loads.
> +
> +- smp_mb() guarantees that all the LOAD and STORE operations specified
> +  before the barrier will appear to happen before all the LOAD and
> +  STORE operations specified after the barrier with respect to the other
> +  components of the system.
> +
> +  smp_mb() puts a partial ordering on both loads and stores.  It is
> +  stronger than both a read and a write memory barrier; it implies both
> +  smp_rmb() and smp_wmb(), but it also prevents STOREs coming before the
> +  barrier from overtaking LOADs coming after the barrier and vice versa.
> +
> +- smp_read_barrier_depends() is a weaker kind of read barrier.  On
> +  most processors, whenever two loads are performed such that the
> +  second depends on the result of the first (e.g., the first load
> +  retrieves the address to which the second load will be directed),
> +  the processor will guarantee that the first LOAD will appear to happen
> +  before the second with respect to the other components of the system.
> +  However, this is not always true---for example, it was not true on
> +  Alpha processors.  Whenever this kind of access happens to shared
> +  memory (that is not protected by a lock), a read barrier is needed,
> +  and smp_read_barrier_depends() can be used instead of smp_rmb().
> +
> +  Note that the first load really has to have a _data_ dependency and not
> +  a control dependency.  If the address for the second load is dependent
> +  on the first load, but the dependency is through a conditional rather
> +  than actually loading the address itself, then it's a _control_
> +  dependency and a full read barrier or better is required.
> +
> +
> +This is the set of barriers that is required *between* two atomic_read()
> +and atomic_set() operations to achieve sequential consistency:
> +
> +                    |               2nd operation             |
> +                    |-----------------------------------------|
> +     1st operation  | (after last) | atomic_read | atomic_set |
> +     ---------------+--------------+-------------+------------|
> +     (before first) |              | none        | smp_wmb()  |
> +     ---------------+--------------+-------------+------------|
> +     atomic_read    | smp_rmb()    | smp_rmb()*  | **         |
> +     ---------------+--------------+-------------+------------|
> +     atomic_set     | none         | smp_mb()*** | smp_wmb()  |
> +     ---------------+--------------+-------------+------------|
> +
> +       * Or smp_read_barrier_depends().
> +
> +      ** This requires a load-store barrier.  How to achieve this varies
> +         depending on the machine, but in practice smp_rmb()+smp_wmb()
> +         should have the desired effect.  For example, on PowerPC the
> +         lwsync instruction is a combined load-load, load-store and
> +         store-store barrier.
> +
> +     *** This requires a store-load barrier.  On most machines, the only
> +         way to achieve this is a full barrier.
> +
> +
> +You can see that the two possible definitions of atomic_mb_read()
> +and atomic_mb_set() are the following:
> +
> +    1) atomic_mb_read(p)   = atomic_read(p); smp_rmb()
> +       atomic_mb_set(p, v) = smp_wmb(); atomic_set(p, v); smp_mb()
> +
> +    2) atomic_mb_read(p)   = smp_mb() atomic_read(p); smp_rmb()
> +       atomic_mb_set(p, v) = smp_wmb(); atomic_set(p, v);
> +
> +Usually the former is used, because smp_mb() is expensive and a program
> +normally has more reads than writes.  Therefore it makes more sense to
> +make atomic_mb_set() the more expensive operation.
> +
> +There are two common cases in which atomic_mb_read and atomic_mb_set
> +generate too many memory barriers, and thus it can be useful to manually
> +place barriers instead:
> +
> +- when a data structure has one thread that is always a writer
> +  and one thread that is always a reader, manual placement of
> +  memory barriers makes the write side faster.  Furthermore,
> +  correctness is easy to check for in this case using the "pairing"
> +  trick that is explained below:
> +
> +     thread 1                                thread 1
> +     -------------------------               ------------------------
> +     (other writes)
> +                                             smp_wmb()
> +     atomic_mb_set(&a, x)                    atomic_set(&a, x)
> +                                             smp_wmb()
> +     atomic_mb_set(&b, y)                    atomic_set(&b, y)
> +
> +                                       =>
> +     thread 2                                thread 2
> +     -------------------------               ------------------------
> +     y = atomic_mb_read(&b)                  y = atomic_read(&b)
> +                                             smp_rmb()
> +     x = atomic_mb_read(&a)                  x = atomic_read(&a)
> +                                             smp_rmb()
> +
> +- sometimes, a thread is accessing many variables that are otherwise
> +  unrelated to each other (for example because, apart from the current
> +  thread, exactly one other thread will read or write each of these
> +  variables).  In this case, it is possible to "hoist" the implicit
> +  barriers provided by atomic_mb_read() and atomic_mb_set() outside
> +  a loop.  For example, the above definition atomic_mb_read() gives
> +  the following transformation:
> +
> +     n = 0;                                  n = 0;
> +     for (i = 0; i < 10; i++)          =>    for (i = 0; i < 10; i++)
> +       n += atomic_mb_read(&a[i]);             n += atomic_read(&a[i]);
> +                                             smp_rmb();
> +
> +  Similarly, atomic_mb_set() can be transformed as follows:
> +  smp_mb():
> +
> +                                             smp_wmb();
> +     for (i = 0; i < 10; i++)          =>    for (i = 0; i < 10; i++)
> +       atomic_mb_set(&a[i], false);            atomic_set(&a[i], false);
> +                                             smp_mb();
> +
> +
> +The two tricks can be combined.  In this case, splitting a loop in
> +two lets you hoist the barriers out of the loops _and_ eliminate the
> +expensive smp_mb():
> +
> +                                             smp_wmb();
> +     for (i = 0; i < 10; i++) {        =>    for (i = 0; i < 10; i++)
> +       atomic_mb_set(&a[i], false);            atomic_set(&a[i], false);
> +       atomic_mb_set(&b[i], false);          smb_wmb();
> +     }                                       for (i = 0; i < 10; i++)
> +                                               atomic_set(&a[i], false);
> +                                             smp_mb();
> +
> +  The other thread can still use atomic_mb_read()/atomic_mb_set()
> +
> +
> +Memory barrier pairing
> +----------------------
> +
> +A useful rule of thumb is that memory barriers should always, or almost
> +always, be paired with another barrier.  In the case of QEMU, however,
> +note that the other barrier may actually be in a driver that runs in
> +the guest!
> +
> +For the purposes of pairing, smp_read_barrier_depends() and smp_rmb()
> +both count as read barriers.  A read barriers shall pair with a write
> +barrier or a full barrier; a write barrier shall pair with a read
> +barrier or a full barrier.  A full barrier can pair with anything.
> +For example:
> +
> +        thread 1             thread 2
> +        ===============      ===============
> +        a = 1;
> +        smp_wmb();
> +        b = 2;               x = b;
> +                             smp_rmb();
> +                             y = a;
> +
> +Note that the "writing" thread are accessing the variables in the
> +opposite order as the "reading" thread.  This is expected: stores
> +before the write barrier will normally match the loads after the
> +read barrier, and vice versa.  The same is true for more than 2
> +access and for data dependency barriers:
> +
> +        thread 1             thread 2
> +        ===============      ===============
> +        b[2] = 1;
> +        smp_wmb();
> +        x->i = 2;
> +        smp_wmb();
> +        a = x;               x = a;
> +                             smp_read_barrier_depends();
> +                             y = x->i;
> +                             smp_read_barrier_depends();
> +                             z = b[y];
> +
> +smp_wmb() also pairs with atomic_mb_read(), and smp_rmb() also pairs
> +with atomic_mb_set().
> +
> +
> +COMPARISON WITH LINUX KERNEL MEMORY BARRIERS
> +============================================
> +
> +Here is a list of differences between Linux kernel atomic operations
> +and memory barriers, and the equivalents in QEMU:
> +
> +- atomic operations in Linux are always on a 32-bit int type and

32-bit?  int should be the integer type that the target processor is
most efficient working with.

Regards,
Pingfan
> +  use a boxed atomic_t type; atomic operations in QEMU are polymorphic
> +  and use normal C types.
> +
> +- atomic_read and atomic_set in Linux give no guarantee at all;
> +  atomic_read and atomic_set in QEMU include a compiler barrier
> +  (similar to the ACCESS_ONCE macro in Linux).
> +
> +- most atomic read-modify-write operations in Linux return void;
> +  in QEMU, all of them return the old value of the variable.
> +
> +- different atomic read-modify-write operations in Linux imply
> +  a different set of memory barriers; in QEMU, all of them enforce
> +  sequential consistency, which means they imply full memory barriers
> +  before and after the operation.
> +
> +- Linux does not have an equivalent of atomic_mb_read() and
> +  atomic_mb_set().  In particular, note that set_mb() is a little
> +  weaker than atomic_mb_set().
> +
> +
> +SOURCES
> +=======
> +
> +* Documentation/memory-barriers.txt from the Linux kernel
> +
> +* "The JSR-133 Cookbook for Compiler Writers", available at
> +  http://g.oswego.edu/dl/jmm/cookbook.html
> diff --git a/hw/display/qxl.c b/hw/display/qxl.c
> index 3862d7a..f24cb4e 100644
> --- a/hw/display/qxl.c
> +++ b/hw/display/qxl.c
> @@ -23,6 +23,7 @@
>  #include "qemu-common.h"
>  #include "qemu/timer.h"
>  #include "qemu/queue.h"
> +#include "qemu/atomic.h"
>  #include "monitor/monitor.h"
>  #include "sysemu/sysemu.h"
>  #include "trace.h"
> @@ -1726,7 +1727,7 @@ static void qxl_send_events(PCIQXLDevice *d, uint32_t events)
>          trace_qxl_send_events_vm_stopped(d->id, events);
>          return;
>      }
> -    old_pending = __sync_fetch_and_or(&d->ram->int_pending, le_events);
> +    old_pending = atomic_or(&d->ram->int_pending, le_events);
>      if ((old_pending & le_events) == le_events) {
>          return;
>      }
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 96ab625..8f6ab13 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -16,6 +16,7 @@
>  #include <sys/ioctl.h>
>  #include "hw/virtio/vhost.h"
>  #include "hw/hw.h"
> +#include "qemu/atomic.h"
>  #include "qemu/range.h"
>  #include <linux/vhost.h>
>  #include "exec/address-spaces.h"
> @@ -47,11 +48,9 @@ static void vhost_dev_sync_region(struct vhost_dev *dev,
>              addr += VHOST_LOG_CHUNK;
>              continue;
>          }
> -        /* Data must be read atomically. We don't really
> -         * need the barrier semantics of __sync
> -         * builtins, but it's easier to use them than
> -         * roll our own. */
> -        log = __sync_fetch_and_and(from, 0);
> +        /* Data must be read atomically. We don't really need barrier semantics
> +         * but it's easier to use atomic_* than roll our own. */
> +        log = atomic_xchg(from, 0);
>          while ((bit = sizeof(log) > sizeof(int) ?
>                  ffsll(log) : ffs(log))) {
>              hwaddr page_addr;
> diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
> index 10becb6..04d64d0 100644
> --- a/include/qemu/atomic.h
> +++ b/include/qemu/atomic.h
> @@ -1,68 +1,194 @@
> -#ifndef __QEMU_BARRIER_H
> -#define __QEMU_BARRIER_H 1
> +/*
> + * Simple interface for atomic operations.
> + *
> + * Copyright (C) 2013 Red Hat, Inc.
> + *
> + * Author: Paolo Bonzini <pbonzini@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
>
> -/* Compiler barrier */
> -#define barrier()   asm volatile("" ::: "memory")
> +#ifndef __QEMU_ATOMIC_H
> +#define __QEMU_ATOMIC_H 1
>
> -#if defined(__i386__)
> +#include "qemu/compiler.h"
>
> -#include "qemu/compiler.h"        /* QEMU_GNUC_PREREQ */
> +/* For C11 atomic ops */
>
> -/*
> - * Because of the strongly ordered x86 storage model, wmb() and rmb() are nops
> - * on x86(well, a compiler barrier only).  Well, at least as long as
> - * qemu doesn't do accesses to write-combining memory or non-temporal
> - * load/stores from C code.
> - */
> -#define smp_wmb()   barrier()
> -#define smp_rmb()   barrier()
> +/* Compiler barrier */
> +#define barrier()   ({ asm volatile("" ::: "memory"); (void)0; })
> +
> +#ifndef __ATOMIC_RELAXED
>
>  /*
> - * We use GCC builtin if it's available, as that can use
> - * mfence on 32 bit as well, e.g. if built with -march=pentium-m.
> - * However, on i386, there seem to be known bugs as recently as 4.3.
> - * */
> -#if QEMU_GNUC_PREREQ(4, 4)
> -#define smp_mb() __sync_synchronize()
> + * We use GCC builtin if it's available, as that can use mfence on
> + * 32-bit as well, e.g. if built with -march=pentium-m. However, on
> + * i386 the spec is buggy, and the implementation followed it until
> + * 4.3 (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36793).
> + */
> +#if defined(__i386__) || defined(__x86_64__)
> +#if !QEMU_GNUC_PREREQ(4, 4)
> +#if defined __x86_64__
> +#define smp_mb()    ({ asm volatile("mfence" ::: "memory"); (void)0; })
>  #else
> -#define smp_mb() asm volatile("lock; addl $0,0(%%esp) " ::: "memory")
> +#define smp_mb()    ({ asm volatile("lock; addl $0,0(%%esp) " ::: "memory"); (void)0; })
> +#endif
> +#endif
>  #endif
>
> -#elif defined(__x86_64__)
>
> +#ifdef __alpha__
> +#define smp_read_barrier_depends()   asm volatile("mb":::"memory")
> +#endif
> +
> +#if defined(__i386__) || defined(__x86_64__) || defined(__s390x__)
> +
> +/*
> + * Because of the strongly ordered storage model, wmb() and rmb() are nops
> + * here (a compiler barrier only).  QEMU doesn't do accesses to write-combining
> + * qemu memory or non-temporal load/stores from C code.
> + */
>  #define smp_wmb()   barrier()
>  #define smp_rmb()   barrier()
> -#define smp_mb() asm volatile("mfence" ::: "memory")
> +
> +/*
> + * __sync_lock_test_and_set() is documented to be an acquire barrier only,
> + * but it is a full barrier at the hardware level.  Add a compiler barrier
> + * to make it a full barrier also at the compiler level.
> + */
> +#define atomic_xchg(ptr, i)    (barrier(), __sync_lock_test_and_set(ptr, i))
> +
> +/*
> + * Load/store with Java volatile semantics.
> + */
> +#define atomic_mb_set(ptr, i)  ((void)atomic_xchg(ptr, i))
>
>  #elif defined(_ARCH_PPC)
>
>  /*
>   * We use an eieio() for wmb() on powerpc.  This assumes we don't
>   * need to order cacheable and non-cacheable stores with respect to
> - * each other
> + * each other.
> + *
> + * smp_mb has the same problem as on x86 for not-very-new GCC
> + * (http://patchwork.ozlabs.org/patch/126184/, Nov 2011).
>   */
> -#define smp_wmb()   asm volatile("eieio" ::: "memory")
> -
> +#define smp_wmb()   ({ asm volatile("eieio" ::: "memory"); (void)0; })
>  #if defined(__powerpc64__)
> -#define smp_rmb()   asm volatile("lwsync" ::: "memory")
> +#define smp_rmb()   ({ asm volatile("lwsync" ::: "memory"); (void)0; })
>  #else
> -#define smp_rmb()   asm volatile("sync" ::: "memory")
> +#define smp_rmb()   ({ asm volatile("sync" ::: "memory"); (void)0; })
>  #endif
> +#define smp_mb()    ({ asm volatile("sync" ::: "memory"); (void)0; })
>
> -#define smp_mb()   asm volatile("sync" ::: "memory")
> +#endif /* _ARCH_PPC */
>
> -#else
> +#endif /* C11 atomics */
>
>  /*
>   * For (host) platforms we don't have explicit barrier definitions
>   * for, we use the gcc __sync_synchronize() primitive to generate a
>   * full barrier.  This should be safe on all platforms, though it may
> - * be overkill for wmb() and rmb().
> + * be overkill for smp_wmb() and smp_rmb().
>   */
> +#ifndef smp_mb
> +#define smp_mb()    __sync_synchronize()
> +#endif
> +
> +#ifndef smp_wmb
> +#ifdef __ATOMIC_RELEASE
> +#define smp_wmb()   __atomic_thread_fence(__ATOMIC_RELEASE)
> +#else
>  #define smp_wmb()   __sync_synchronize()
> -#define smp_mb()   __sync_synchronize()
> +#endif
> +#endif
> +
> +#ifndef smp_rmb
> +#ifdef __ATOMIC_ACQUIRE
> +#define smp_rmb()   __atomic_thread_fence(__ATOMIC_ACQUIRE)
> +#else
>  #define smp_rmb()   __sync_synchronize()
> +#endif
> +#endif
> +
> +#ifndef smp_read_barrier_depends
> +#ifdef __ATOMIC_CONSUME
> +#define smp_read_barrier_depends()   __atomic_thread_fence(__ATOMIC_CONSUME)
> +#else
> +#define smp_read_barrier_depends()   barrier()
> +#endif
> +#endif
>
> +#ifndef atomic_read
> +#define atomic_read(ptr)       (*(__typeof__(*ptr) *volatile) (ptr))
>  #endif
>
> +#ifndef atomic_set
> +#define atomic_set(ptr, i)     ((*(__typeof__(*ptr) *volatile) (ptr)) = (i))
> +#endif
> +
> +/* These have the same semantics as Java volatile variables.
> + * See http://gee.cs.oswego.edu/dl/jmm/cookbook.html:
> + * "1. Issue a StoreStore barrier (wmb) before each volatile store."
> + *  2. Issue a StoreLoad barrier after each volatile store.
> + *     Note that you could instead issue one before each volatile load, but
> + *     this would be slower for typical programs using volatiles in which
> + *     reads greatly outnumber writes. Alternatively, if available, you
> + *     can implement volatile store as an atomic instruction (for example
> + *     XCHG on x86) and omit the barrier. This may be more efficient if
> + *     atomic instructions are cheaper than StoreLoad barriers.
> + *  3. Issue LoadLoad and LoadStore barriers after each volatile load."
> + *
> + * If you prefer to think in terms of "pairing" of memory barriers,
> + * an atomic_mb_read pairs with an atomic_mb_set.
> + *
> + * And for the few ia64 lovers that exist, an atomic_mb_read is a ld.acq,
> + * while an atomic_mb_set is a st.rel followed by a memory barrier.
> + *
> + * These are a bit weaker than __atomic_load/store with __ATOMIC_SEQ_CST
> + * (see docs/atomics.txt), and I'm not sure that __ATOMIC_ACQ_REL is enough.
> + * Just always use the barriers manually by the rules above.
> + */
> +#ifndef atomic_mb_read
> +#define atomic_mb_read(ptr)    ({           \
> +    typeof(*ptr) _val = atomic_read(ptr);   \
> +    smp_rmb();                              \
> +    _val;                                   \
> +})
> +#endif
> +
> +#ifndef atomic_mb_set
> +#define atomic_mb_set(ptr, i)  do {         \
> +    smp_wmb();                              \
> +    atomic_set(ptr, i);                     \
> +    smp_mb();                               \
> +} while (0)
> +#endif
> +
> +#ifndef atomic_xchg
> +#ifdef __ATOMIC_SEQ_CST
> +#define atomic_xchg(ptr, i)    ({                           \
> +    typeof(*ptr) _new = (i), _old;                          \
> +    __atomic_exchange(ptr, &_new, &_old, __ATOMIC_SEQ_CST); \
> +    _old;                                                   \
> +})
> +#elif defined __clang__
> +#define atomic_xchg(ptr, i)    __sync_exchange(ptr, i)
> +#else
> +/* __sync_lock_test_and_set() is documented to be an acquire barrier only.  */
> +#define atomic_xchg(ptr, i)    (smp_mb(), __sync_lock_test_and_set(ptr, i))
> +#endif
> +#endif
> +
> +/* Provide shorter names for GCC atomic builtins.  */
> +#define atomic_inc(ptr)        __sync_fetch_and_add(ptr, 1)
> +#define atomic_dec(ptr)        __sync_fetch_and_add(ptr, -1)
> +#define atomic_add             __sync_fetch_and_add
> +#define atomic_sub             __sync_fetch_and_sub
> +#define atomic_and             __sync_fetch_and_and
> +#define atomic_or              __sync_fetch_and_or
> +#define atomic_cmpxchg         __sync_val_compare_and_swap
> +
>  #endif
> diff --git a/migration.c b/migration.c
> index 058f9e6..83f5691 100644
> --- a/migration.c
> +++ b/migration.c
> @@ -290,8 +290,7 @@ static void migrate_fd_cleanup(void *opaque)
>
>  static void migrate_finish_set_state(MigrationState *s, int new_state)
>  {
> -    if (__sync_val_compare_and_swap(&s->state, MIG_STATE_ACTIVE,
> -                                    new_state) == new_state) {
> +    if (atomic_cmpxchg(&s->state, MIG_STATE_ACTIVE, new_state) == new_state) {
>          trace_migrate_set_state(new_state);
>      }
>  }
> diff --git a/tests/test-thread-pool.c b/tests/test-thread-pool.c
> index 22915aa..dbf2fc8 100644
> --- a/tests/test-thread-pool.c
> +++ b/tests/test-thread-pool.c
> @@ -17,15 +17,15 @@ typedef struct {
>  static int worker_cb(void *opaque)
>  {
>      WorkerTestData *data = opaque;
> -    return __sync_fetch_and_add(&data->n, 1);
> +    return atomic_inc(&data->n);
>  }
>
>  static int long_cb(void *opaque)
>  {
>      WorkerTestData *data = opaque;
> -    __sync_fetch_and_add(&data->n, 1);
> +    atomic_inc(&data->n);
>      g_usleep(2000000);
> -    __sync_fetch_and_add(&data->n, 1);
> +    atomic_inc(&data->n);
>      return 0;
>  }
>
> @@ -169,7 +169,7 @@ static void test_cancel(void)
>      /* Cancel the jobs that haven't been started yet.  */
>      num_canceled = 0;
>      for (i = 0; i < 100; i++) {
> -        if (__sync_val_compare_and_swap(&data[i].n, 0, 3) == 0) {
> +        if (atomic_cmpxchg(&data[i].n, 0, 3) == 0) {
>              data[i].ret = -ECANCELED;
>              bdrv_aio_cancel(data[i].aiocb);
>              active--;
> --
> 1.8.1.4
>
>
>

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-07-03  2:24   ` liu ping fan
@ 2013-07-03  5:59     ` Paolo Bonzini
  2013-07-03  7:07       ` liu ping fan
  0 siblings, 1 reply; 68+ messages in thread
From: Paolo Bonzini @ 2013-07-03  5:59 UTC (permalink / raw)
  To: liu ping fan; +Cc: qemu-devel

Il 03/07/2013 04:24, liu ping fan ha scritto:
>> > +- atomic operations in Linux are always on a 32-bit int type and
> 32-bit?  int should be the integer type that the target processor is
> most efficient working with.

Has Linux ever been ported to a machine where sizeof(int) != 4?  (Honest
question.  I don't know).

Paolo

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-07-03  5:59     ` Paolo Bonzini
@ 2013-07-03  7:07       ` liu ping fan
  0 siblings, 0 replies; 68+ messages in thread
From: liu ping fan @ 2013-07-03  7:07 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: qemu-devel

On Wed, Jul 3, 2013 at 1:59 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 03/07/2013 04:24, liu ping fan ha scritto:
>>> > +- atomic operations in Linux are always on a 32-bit int type and
>> 32-bit?  int should be the integer type that the target processor is
>> most efficient working with.
>
> Has Linux ever been ported to a machine where sizeof(int) != 4?  (Honest
> question.  I don't know).
>
Do not know either:)  But it depends on compiler, as for gcc, it would
be 32-bits.

Regards,
Pingfan
> Paolo

^ permalink raw reply	[flat|nested] 68+ messages in thread

* Re: [Qemu-devel] [PATCH 04/30] add a header file for atomic operations
  2013-07-01 13:20           ` Anthony Liguori
@ 2013-07-04  5:24             ` liu ping fan
  0 siblings, 0 replies; 68+ messages in thread
From: liu ping fan @ 2013-07-04  5:24 UTC (permalink / raw)
  To: Anthony Liguori; +Cc: Paolo Bonzini, qemu-devel

On Mon, Jul 1, 2013 at 9:20 PM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> Paolo Bonzini <pbonzini@redhat.com> writes:
>
>> Il 01/07/2013 15:00, Anthony Liguori ha scritto:
>>>> I
>>>> > cannot find the commit exactly, but I think mst added specific code for
>>>> > that.
>>> Right, I'm not questioning whether these functions have strong enough
>>> semantics in their implementation, but asking what their contract should
>>> be.
>>>
>>> Either we should document that these functions have atomic semantics or
>>> we should introduce another variant that guarantee atomic access.
>>>
>>> I think the later makes more sense since the majority of users probably
>>> don't need atomic semantics.
>>
>> I think many of these loads and stores do, actually; perhaps most.  It
>> also matches what hardware does.
>
> Hrm, I'm not sure if that's true.  PCI has an explicit LOCK# bit to
> enable exclusive access so my assumption would be that it doesn't by
> default.
>
Is it a huge but another topic -- implement atomic on device register
other than using?  For virtio,  using fence around but outside the
st[u][wlb]_phys when needed?

Regards,
Pingfan
> But either way, we should document the semantics.
>
> Regards,
>
> Anthony Liguori
>
>>
>> Paolo
>

^ permalink raw reply	[flat|nested] 68+ messages in thread

end of thread, other threads:[~2013-07-04  5:24 UTC | newest]

Thread overview: 68+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-06-28 18:26 [Qemu-devel] [PATCH 00/30] Memory API changes for 1.6: RCU-protected address space dispatch Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 01/30] memory: access FlatView from a local variable Paolo Bonzini
2013-06-28 20:01   ` Anthony Liguori
2013-06-28 18:26 ` [Qemu-devel] [PATCH 02/30] memory: use a new FlatView pointer on every topology update Paolo Bonzini
2013-06-28 20:02   ` Anthony Liguori
2013-06-28 18:26 ` [Qemu-devel] [PATCH 03/30] memory: add reference counting to FlatView Paolo Bonzini
2013-06-28 20:07   ` Anthony Liguori
2013-06-28 18:26 ` [Qemu-devel] [PATCH 04/30] add a header file for atomic operations Paolo Bonzini
2013-06-28 20:41   ` Anthony Liguori
2013-07-01 10:21     ` Paolo Bonzini
2013-07-01 13:00       ` Anthony Liguori
2013-07-01 13:04         ` Paolo Bonzini
2013-07-01 13:20           ` Anthony Liguori
2013-07-04  5:24             ` liu ping fan
2013-07-01 11:08     ` Peter Maydell
2013-07-03  2:24   ` liu ping fan
2013-07-03  5:59     ` Paolo Bonzini
2013-07-03  7:07       ` liu ping fan
2013-06-28 18:26 ` [Qemu-devel] [PATCH 05/30] exec: do not use qemu/tls.h Paolo Bonzini
2013-06-28 20:43   ` Anthony Liguori
2013-06-28 23:53   ` Ed Maste
2013-07-01 10:16     ` Paolo Bonzini
2013-06-29 10:55   ` Peter Maydell
2013-07-01 10:45     ` Paolo Bonzini
2013-07-01 11:05       ` Peter Maydell
2013-07-01 16:21         ` Paolo Bonzini
2013-07-01 16:26           ` Peter Maydell
2013-07-01 20:52             ` Paolo Bonzini
2013-07-01 21:34               ` Peter Maydell
2013-07-02 13:40                 ` Andreas Färber
2013-07-02 14:06                   ` Alexander Graf
2013-06-28 18:26 ` [Qemu-devel] [PATCH 06/30] qemu-thread: add TLS wrappers Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 07/30] qemu-thread: add QemuEvent Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 08/30] rcu: add rcu library Paolo Bonzini
2013-07-01  9:47   ` Jan Kiszka
2013-06-28 18:26 ` [Qemu-devel] [PATCH 09/30] qemu-thread: register threads with RCU Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 10/30] rcu: add call_rcu Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 11/30] rcu: add rcutorture Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 12/30] rcu: allow nested calls to rcu_thread_offline/rcu_thread_online Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 13/30] qemu-thread: report RCU quiescent states Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 14/30] event loop: " Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 15/30] cpus: " Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 16/30] block: " Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 17/30] migration: " Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 18/30] memory: protect current_map by RCU Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 19/30] memory: avoid ref/unref in memory_region_find Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 20/30] exec: change well-known physical sections to macros Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 21/30] exec: separate current memory map from the one being built Paolo Bonzini
2013-07-02 14:41   ` Jan Kiszka
2013-06-28 18:26 ` [Qemu-devel] [PATCH 22/30] memory: move MemoryListener declaration earlier Paolo Bonzini
2013-07-02 14:41   ` Jan Kiszka
2013-06-28 18:26 ` [Qemu-devel] [PATCH 23/30] exec: move listener from AddressSpaceDispatch to AddressSpace Paolo Bonzini
2013-07-02 14:41   ` Jan Kiszka
2013-06-28 18:26 ` [Qemu-devel] [PATCH 24/30] exec: separate current radix tree from the one being built Paolo Bonzini
2013-07-02 14:41   ` Jan Kiszka
2013-06-28 18:26 ` [Qemu-devel] [PATCH 25/30] exec: put memory map in AddressSpaceDispatch Paolo Bonzini
2013-07-02 14:42   ` Jan Kiszka
2013-07-02 15:08     ` Paolo Bonzini
2013-07-02 15:48       ` Jan Kiszka
2013-06-28 18:26 ` [Qemu-devel] [PATCH 26/30] exec: remove cur_map Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 27/30] exec: change some APIs to take AddressSpaceDispatch Paolo Bonzini
2013-07-02 14:47   ` Jan Kiszka
2013-06-28 18:26 ` [Qemu-devel] [PATCH 28/30] exec: change iotlb " Paolo Bonzini
2013-07-02 10:00   ` Jan Kiszka
2013-06-28 18:26 ` [Qemu-devel] [PATCH 29/30] exec: add a reference to the region returned by address_space_translate Paolo Bonzini
2013-06-28 18:26 ` [Qemu-devel] [PATCH 30/30] exec: put address space dispatch under RCU critical section Paolo Bonzini
2013-06-28 19:38   ` Jan Kiszka
2013-07-01 11:48     ` Paolo Bonzini

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.