* [PATCH 0/3] fuzz: Add a sparse-memory device to accelerate fuzzing
@ 2021-03-11 5:36 Alexander Bulekov
2021-03-11 5:36 ` [PATCH 1/3] memory: add a sparse memory device Alexander Bulekov
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Alexander Bulekov @ 2021-03-11 5:36 UTC (permalink / raw)
To: qemu-devel
Cc: Alexander Bulekov, f4bug, lidong.chen, darren.kenny, bsd,
stefanha, pbonzini
Hello,
The generic-fuzzer often provides virtual-devices with bogus DMA
addresses (e.g. 0x4141414141414141). The probability that these fuzzed
addresses actually land within RAM is quite small. The fuzzer eventually
finds valid addresses, however, this takes some time, and this problem is
compounded when the device accesses multiple DMA regions. This series
adds a "sparse" memory device, and configures it for the generic-fuzzer.
This allows us to simulate 16 EB ram (only a tiny portion actually
populated). Thus, almost any randomly generated 64-bit address will land
in memory that the fuzzer can populate with data.
Patch 1 adds the sparse-mem device
Patch 2 adds tests for the device
Patch 3 configures the generic-fuzzer to use the sparse-mem device
-Alex
Alexander Bulekov (3):
memory: add a sparse memory device
memory: add tests for the sparse-mem device
fuzz: configure a sparse-mem device, by default
MAINTAINERS | 2 +
hw/mem/meson.build | 1 +
hw/mem/sparse-mem.c | 154 ++++++++++++++++++++++++++++++++
tests/qtest/fuzz/generic_fuzz.c | 2 +-
tests/qtest/meson.build | 3 +-
tests/qtest/sparse-mem-test.c | 88 ++++++++++++++++++
6 files changed, 248 insertions(+), 2 deletions(-)
create mode 100644 hw/mem/sparse-mem.c
create mode 100644 tests/qtest/sparse-mem-test.c
--
2.28.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/3] memory: add a sparse memory device
2021-03-11 5:36 [PATCH 0/3] fuzz: Add a sparse-memory device to accelerate fuzzing Alexander Bulekov
@ 2021-03-11 5:36 ` Alexander Bulekov
2021-03-11 14:25 ` Philippe Mathieu-Daudé
2021-03-11 5:36 ` [PATCH 2/3] memory: add tests for the sparse-mem device Alexander Bulekov
2021-03-11 5:36 ` [PATCH 3/3] fuzz: configure a sparse-mem device, by default Alexander Bulekov
2 siblings, 1 reply; 6+ messages in thread
From: Alexander Bulekov @ 2021-03-11 5:36 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Alexander Bulekov, Michael S. Tsirkin, f4bug,
lidong.chen, darren.kenny, bsd, stefanha, Igor Mammedov,
pbonzini
For testing, it can be useful to simulate an enormous amount of memory
(e.g. 2^64 RAM). This adds an MMIO device that acts as sparse memory.
When something writes a nonzero value to a sparse-mem address, we
allocate a block of memory. This block is kept around, until all of the
bytes within the block are zero-ed. The device has a very low priority
(so it can be mapped beneath actual RAM, and virtual device MMIO
regions).
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
---
MAINTAINERS | 1 +
hw/mem/meson.build | 1 +
hw/mem/sparse-mem.c | 154 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 156 insertions(+)
create mode 100644 hw/mem/sparse-mem.c
diff --git a/MAINTAINERS b/MAINTAINERS
index f22d83c178..9e3d8b1401 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2618,6 +2618,7 @@ R: Thomas Huth <thuth@redhat.com>
S: Maintained
F: tests/qtest/fuzz/
F: scripts/oss-fuzz/
+F: hw/mem/sparse-mem.c
F: docs/devel/fuzzing.rst
Register API
diff --git a/hw/mem/meson.build b/hw/mem/meson.build
index 0d22f2b572..732f459e0a 100644
--- a/hw/mem/meson.build
+++ b/hw/mem/meson.build
@@ -1,5 +1,6 @@
mem_ss = ss.source_set()
mem_ss.add(files('memory-device.c'))
+mem_ss.add(files('sparse-mem.c'))
mem_ss.add(when: 'CONFIG_DIMM', if_true: files('pc-dimm.c'))
mem_ss.add(when: 'CONFIG_NPCM7XX', if_true: files('npcm7xx_mc.c'))
mem_ss.add(when: 'CONFIG_NVDIMM', if_true: files('nvdimm.c'))
diff --git a/hw/mem/sparse-mem.c b/hw/mem/sparse-mem.c
new file mode 100644
index 0000000000..ffda6f76b4
--- /dev/null
+++ b/hw/mem/sparse-mem.c
@@ -0,0 +1,154 @@
+/*
+ * A sparse memory device
+ *
+ * Copyright Red Hat Inc., 2021
+ *
+ * Authors:
+ * Alexander Bulekov <alxndr@bu.edu>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+
+#include "exec/address-spaces.h"
+#include "hw/qdev-properties.h"
+
+#define TYPE_SPARSE_MEM "sparse-mem"
+#define SPARSE_MEM(obj) OBJECT_CHECK(SparseMemState, (obj), TYPE_SPARSE_MEM)
+
+#define SPARSE_BLOCK_SIZE 0x1000
+
+typedef struct SparseMemState {
+ DeviceState parent_obj;
+ MemoryRegion mmio;
+ uint64_t baseaddr;
+ uint64_t length;
+ uint64_t usage;
+ uint64_t maxsize;
+ GHashTable *mapped;
+} SparseMemState;
+
+typedef struct sparse_mem_block {
+ uint16_t nonzeros;
+ uint8_t data[SPARSE_BLOCK_SIZE];
+} sparse_mem_block;
+
+static uint64_t sparse_mem_read(void *opaque, hwaddr addr, unsigned int size)
+{
+ SparseMemState *s = opaque;
+ uint64_t ret = 0;
+ size_t pfn = addr / SPARSE_BLOCK_SIZE;
+ size_t offset = addr % SPARSE_BLOCK_SIZE;
+ sparse_mem_block *block;
+
+ block = g_hash_table_lookup(s->mapped, (void *)pfn);
+ if (block) {
+ assert(offset + size <= sizeof(block->data));
+ memcpy(&ret, block->data + offset, size);
+ }
+ return ret;
+}
+
+static void sparse_mem_write(void *opaque, hwaddr addr, uint64_t v,
+ unsigned int size)
+{
+ SparseMemState *s = opaque;
+ size_t pfn = addr / SPARSE_BLOCK_SIZE;
+ size_t offset = addr % SPARSE_BLOCK_SIZE;
+ int nonzeros = 0;
+ sparse_mem_block *block;
+
+ if (!g_hash_table_lookup(s->mapped, (void *)pfn) &&
+ s->usage + SPARSE_BLOCK_SIZE < s->maxsize && v) {
+ g_hash_table_insert(s->mapped, (void *)pfn,
+ g_new0(sparse_mem_block, 1));
+ s->usage += sizeof(block->data);
+ }
+ block = g_hash_table_lookup(s->mapped, (void *)pfn);
+ if (!block) {
+ return;
+ }
+
+ assert(offset + size <= sizeof(block->data));
+
+ /*
+ * Track the number of nonzeros, so we can adjust the block's nonzero count
+ * after writing the value v
+ */
+ for (int i = 0; i < size; i++) {
+ nonzeros -= (block->data[offset + i] != 0);
+ }
+
+ memcpy(block->data + offset, &v, size);
+
+ for (int i = 0; i < size; i++) {
+ nonzeros += (block->data[offset + i] != 0);
+ }
+
+ /* Update the number of nonzeros in the block, free it, if it's empty */
+ assert(block->nonzeros + nonzeros < sizeof(block->data));
+ assert((int)block->nonzeros + nonzeros >= 0);
+ block->nonzeros += nonzeros;
+
+ if (block->nonzeros == 0) {
+ g_free(block);
+ g_hash_table_remove(s->mapped, (void *)pfn);
+ s->usage -= sizeof(block->data);
+ }
+}
+
+static const MemoryRegionOps sparse_mem_ops = {
+ .read = sparse_mem_read,
+ .write = sparse_mem_write,
+ .endianness = DEVICE_LITTLE_ENDIAN,
+ .valid = {
+ .min_access_size = 1,
+ .max_access_size = 8,
+ .unaligned = false,
+ },
+};
+
+static Property sparse_mem_properties[] = {
+ /* The base address of the memory */
+ DEFINE_PROP_UINT64("baseaddr", SparseMemState, baseaddr, 0x0),
+ /* The length of the sparse memory region */
+ DEFINE_PROP_UINT64("length", SparseMemState, length, UINT64_MAX),
+ /* Max amount of actual memory that can be used to back the sparse memory */
+ DEFINE_PROP_UINT64("maxsize", SparseMemState, maxsize, 0x100000),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static void sparse_mem_realize(DeviceState *dev, Error **errp)
+{
+ SparseMemState *s = SPARSE_MEM(dev);
+
+ assert(s->baseaddr + s->length > s->baseaddr);
+
+ s->mapped = g_hash_table_new(NULL, NULL);
+ memory_region_init_io(&(s->mmio), OBJECT(s), &sparse_mem_ops, s,
+ "sparse-mem", s->length);
+ memory_region_add_subregion_overlap(get_system_memory(), s->baseaddr,
+ &(s->mmio), -100);
+}
+
+static void sparse_mem_class_init(ObjectClass *klass, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(klass);
+
+ device_class_set_props(dc, sparse_mem_properties);
+
+ dc->desc = "Sparse Memory Device";
+ dc->realize = sparse_mem_realize;
+}
+
+static const TypeInfo sparse_mem_types[] = {
+ {
+ .name = TYPE_SPARSE_MEM,
+ .parent = TYPE_DEVICE,
+ .instance_size = sizeof(SparseMemState),
+ .class_init = sparse_mem_class_init,
+ },
+};
+DEFINE_TYPES(sparse_mem_types);
--
2.28.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/3] memory: add tests for the sparse-mem device
2021-03-11 5:36 [PATCH 0/3] fuzz: Add a sparse-memory device to accelerate fuzzing Alexander Bulekov
2021-03-11 5:36 ` [PATCH 1/3] memory: add a sparse memory device Alexander Bulekov
@ 2021-03-11 5:36 ` Alexander Bulekov
2021-03-11 5:36 ` [PATCH 3/3] fuzz: configure a sparse-mem device, by default Alexander Bulekov
2 siblings, 0 replies; 6+ messages in thread
From: Alexander Bulekov @ 2021-03-11 5:36 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Thomas Huth, Alexander Bulekov, f4bug,
lidong.chen, darren.kenny, bsd, stefanha, pbonzini
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
---
MAINTAINERS | 1 +
tests/qtest/meson.build | 3 +-
tests/qtest/sparse-mem-test.c | 88 +++++++++++++++++++++++++++++++++++
3 files changed, 91 insertions(+), 1 deletion(-)
create mode 100644 tests/qtest/sparse-mem-test.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 9e3d8b1401..88b35c0d23 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2619,6 +2619,7 @@ S: Maintained
F: tests/qtest/fuzz/
F: scripts/oss-fuzz/
F: hw/mem/sparse-mem.c
+F: tests/qtest/sparse-mem-test.c
F: docs/devel/fuzzing.rst
Register API
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index 58efc46144..21e3b55050 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -78,7 +78,8 @@ qtests_i386 = \
'vmgenid-test',
'migration-test',
'test-x86-cpuid-compat',
- 'numa-test']
+ 'numa-test',
+ 'sparse-mem-test']
dbus_daemon = find_program('dbus-daemon', required: false)
if dbus_daemon.found() and config_host.has_key('GDBUS_CODEGEN')
diff --git a/tests/qtest/sparse-mem-test.c b/tests/qtest/sparse-mem-test.c
new file mode 100644
index 0000000000..cb73976889
--- /dev/null
+++ b/tests/qtest/sparse-mem-test.c
@@ -0,0 +1,88 @@
+/*
+ * QTest testcases for the sparse memory device
+ *
+ * Copyright Red Hat Inc., 2021
+ *
+ * Authors:
+ * Alexander Bulekov <alxndr@bu.edu>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+
+#include "libqos/libqtest.h"
+
+static void test_sparse_memwrite(void)
+{
+ QTestState *s;
+ uint8_t *buf;
+ const int bufsize = 0x10000;
+
+ s = qtest_init("-device sparse-mem");
+
+ buf = malloc(bufsize);
+
+ for (int i = 0; i < bufsize; i++) {
+ buf[i] = (uint8_t)i;
+ }
+ qtest_memwrite(s, 0x100000000, buf, bufsize);
+ memset(buf, 0, bufsize);
+ qtest_memread(s, 0x100000000, buf, bufsize);
+
+ for (int i = 0; i < bufsize; i++) {
+ assert(buf[i] == (uint8_t)i);
+ }
+
+ free(buf);
+ qtest_quit(s);
+}
+
+static void test_sparse_int_writes(void)
+{
+ QTestState *s;
+ const int num_writes = 0x1000;
+
+ s = qtest_init("-device sparse-mem");
+
+ size_t addr = 0x10000000;
+ for (uint64_t i = 0; i < num_writes; i++) {
+ qtest_writeq(s, addr, i);
+ addr += sizeof(uint64_t);
+ }
+
+ addr = 0x10000000;
+ for (uint64_t i = 0; i < num_writes; i++) {
+ assert(qtest_readq(s, addr) == i);
+ addr += sizeof(uint64_t);
+ }
+
+ addr = 0x10000002;
+ for (uint64_t i = 0; i < num_writes; i++) {
+ qtest_writeq(s, addr, i);
+ addr += sizeof(uint64_t);
+ }
+
+ addr = 0x10000002;
+ for (uint64_t i = 0; i < num_writes; i++) {
+ assert(qtest_readq(s, addr) == i);
+ addr += sizeof(uint64_t);
+ }
+
+ qtest_quit(s);
+}
+
+int main(int argc, char **argv)
+{
+ const char *arch = qtest_get_arch();
+
+ g_test_init(&argc, &argv, NULL);
+
+ if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) {
+ qtest_add_func("sparse-mem/memwrite", test_sparse_memwrite);
+ qtest_add_func("sparse-mem/ints", test_sparse_int_writes);
+ }
+
+ return g_test_run();
+}
--
2.28.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 3/3] fuzz: configure a sparse-mem device, by default
2021-03-11 5:36 [PATCH 0/3] fuzz: Add a sparse-memory device to accelerate fuzzing Alexander Bulekov
2021-03-11 5:36 ` [PATCH 1/3] memory: add a sparse memory device Alexander Bulekov
2021-03-11 5:36 ` [PATCH 2/3] memory: add tests for the sparse-mem device Alexander Bulekov
@ 2021-03-11 5:36 ` Alexander Bulekov
2 siblings, 0 replies; 6+ messages in thread
From: Alexander Bulekov @ 2021-03-11 5:36 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Thomas Huth, Alexander Bulekov, f4bug,
lidong.chen, darren.kenny, bsd, stefanha, pbonzini
The generic-fuzzer often provides randomized DMA addresses to
virtual-devices. For a 64-bit address-space, the chance of these
randomized addresses coinciding with RAM regions, is fairly small. Even
though the fuzzer's instrumentation eventually finds valid addresses,
this can take some-time, and slows-down fuzzing progress (especially,
when multiple DMA buffers are involved). To work around this, create
"fake" sparse-memory that spans all of the 64-bit address-space.
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
---
tests/qtest/fuzz/generic_fuzz.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c
index 387ae2020a..08a6c5ac5e 100644
--- a/tests/qtest/fuzz/generic_fuzz.c
+++ b/tests/qtest/fuzz/generic_fuzz.c
@@ -940,7 +940,7 @@ static GString *generic_fuzz_cmdline(FuzzTarget *t)
if (!getenv("QEMU_FUZZ_ARGS")) {
usage();
}
- g_string_append_printf(cmd_line, " -display none \
+ g_string_append_printf(cmd_line, " -display none -device sparse-mem \
-machine accel=qtest, \
-m 512M %s ", getenv("QEMU_FUZZ_ARGS"));
return cmd_line;
--
2.28.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 1/3] memory: add a sparse memory device
2021-03-11 5:36 ` [PATCH 1/3] memory: add a sparse memory device Alexander Bulekov
@ 2021-03-11 14:25 ` Philippe Mathieu-Daudé
2021-03-12 3:08 ` Alexander Bulekov
0 siblings, 1 reply; 6+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-03-11 14:25 UTC (permalink / raw)
To: Alexander Bulekov, qemu-devel, Thomas Huth
Cc: Michael S. Tsirkin, lidong.chen, darren.kenny, bsd, stefanha,
pbonzini, Igor Mammedov
On 3/11/21 6:36 AM, Alexander Bulekov wrote:
> For testing, it can be useful to simulate an enormous amount of memory
> (e.g. 2^64 RAM). This adds an MMIO device that acts as sparse memory.
> When something writes a nonzero value to a sparse-mem address, we
> allocate a block of memory. This block is kept around, until all of the
> bytes within the block are zero-ed. The device has a very low priority
> (so it can be mapped beneath actual RAM, and virtual device MMIO
> regions).
I'm not convinced we need this, but still added some comments while
reviewing.
>
> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
> ---
> MAINTAINERS | 1 +
> hw/mem/meson.build | 1 +
> hw/mem/sparse-mem.c | 154 ++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 156 insertions(+)
> create mode 100644 hw/mem/sparse-mem.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index f22d83c178..9e3d8b1401 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2618,6 +2618,7 @@ R: Thomas Huth <thuth@redhat.com>
> S: Maintained
> F: tests/qtest/fuzz/
> F: scripts/oss-fuzz/
> +F: hw/mem/sparse-mem.c
> F: docs/devel/fuzzing.rst
>
> Register API
> diff --git a/hw/mem/meson.build b/hw/mem/meson.build
> index 0d22f2b572..732f459e0a 100644
> --- a/hw/mem/meson.build
> +++ b/hw/mem/meson.build
> @@ -1,5 +1,6 @@
> mem_ss = ss.source_set()
> mem_ss.add(files('memory-device.c'))
> +mem_ss.add(files('sparse-mem.c'))
> mem_ss.add(when: 'CONFIG_DIMM', if_true: files('pc-dimm.c'))
> mem_ss.add(when: 'CONFIG_NPCM7XX', if_true: files('npcm7xx_mc.c'))
> mem_ss.add(when: 'CONFIG_NVDIMM', if_true: files('nvdimm.c'))
> diff --git a/hw/mem/sparse-mem.c b/hw/mem/sparse-mem.c
> new file mode 100644
> index 0000000000..ffda6f76b4
> --- /dev/null
> +++ b/hw/mem/sparse-mem.c
> @@ -0,0 +1,154 @@
> +/*
> + * A sparse memory device
> + *
> + * Copyright Red Hat Inc., 2021
> + *
> + * Authors:
> + * Alexander Bulekov <alxndr@bu.edu>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +
> +#include "qemu/osdep.h"
> +
> +#include "exec/address-spaces.h"
> +#include "hw/qdev-properties.h"
> +
> +#define TYPE_SPARSE_MEM "sparse-mem"
> +#define SPARSE_MEM(obj) OBJECT_CHECK(SparseMemState, (obj), TYPE_SPARSE_MEM)
> +
> +#define SPARSE_BLOCK_SIZE 0x1000
> +
> +typedef struct SparseMemState {
> + DeviceState parent_obj;
> + MemoryRegion mmio;
> + uint64_t baseaddr;
> + uint64_t length;
> + uint64_t usage;
usage -> size_used?
> + uint64_t maxsize;
> + GHashTable *mapped;
> +} SparseMemState;
> +
> +typedef struct sparse_mem_block {
> + uint16_t nonzeros;
> + uint8_t data[SPARSE_BLOCK_SIZE];
> +} sparse_mem_block;
> +
> +static const MemoryRegionOps sparse_mem_ops = {
> + .read = sparse_mem_read,
> + .write = sparse_mem_write,
> + .endianness = DEVICE_LITTLE_ENDIAN,
> + .valid = {
> + .min_access_size = 1,
> + .max_access_size = 8,
> + .unaligned = false,
Why restrict unaligned accesses?
> + },
> +};
> +
> +static Property sparse_mem_properties[] = {
> + /* The base address of the memory */
> + DEFINE_PROP_UINT64("baseaddr", SparseMemState, baseaddr, 0x0),
> + /* The length of the sparse memory region */
> + DEFINE_PROP_UINT64("length", SparseMemState, length, UINT64_MAX),
> + /* Max amount of actual memory that can be used to back the sparse memory */
> + DEFINE_PROP_UINT64("maxsize", SparseMemState, maxsize, 0x100000),
0x100000 -> 1 * MiB
> + DEFINE_PROP_END_OF_LIST(),
> +};
> +
> +static void sparse_mem_realize(DeviceState *dev, Error **errp)
> +{
> + SparseMemState *s = SPARSE_MEM(dev);
Anyhow, we should restrict this device to QTest accelerator, right?
Maybe:
if (!qtest_enabled()) {
error_setg(errp, "sparse_mem device requires QTest");
return;
}
> +
> + assert(s->baseaddr + s->length > s->baseaddr);
Don't you need more than 64-bit to do this check?
> +
> + s->mapped = g_hash_table_new(NULL, NULL);
> + memory_region_init_io(&(s->mmio), OBJECT(s), &sparse_mem_ops, s,
> + "sparse-mem", s->length);
> + memory_region_add_subregion_overlap(get_system_memory(), s->baseaddr,
> + &(s->mmio), -100);
mr_add() to sysmem from a non-sysbus device is odd... Maybe it is
acceptable, I don't know enough.
> +}
> +
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/3] memory: add a sparse memory device
2021-03-11 14:25 ` Philippe Mathieu-Daudé
@ 2021-03-12 3:08 ` Alexander Bulekov
0 siblings, 0 replies; 6+ messages in thread
From: Alexander Bulekov @ 2021-03-12 3:08 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Thomas Huth, Michael S. Tsirkin, qemu-devel, lidong.chen,
darren.kenny, bsd, stefanha, pbonzini, Igor Mammedov
On 210311 1525, Philippe Mathieu-Daudé wrote:
> On 3/11/21 6:36 AM, Alexander Bulekov wrote:
> > For testing, it can be useful to simulate an enormous amount of memory
> > (e.g. 2^64 RAM). This adds an MMIO device that acts as sparse memory.
> > When something writes a nonzero value to a sparse-mem address, we
> > allocate a block of memory. This block is kept around, until all of the
> > bytes within the block are zero-ed. The device has a very low priority
> > (so it can be mapped beneath actual RAM, and virtual device MMIO
> > regions).
>
> I'm not convinced we need this, but still added some comments while
> reviewing.
>
> >
> > Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
> > ---
> > MAINTAINERS | 1 +
> > hw/mem/meson.build | 1 +
> > hw/mem/sparse-mem.c | 154 ++++++++++++++++++++++++++++++++++++++++++++
> > 3 files changed, 156 insertions(+)
> > create mode 100644 hw/mem/sparse-mem.c
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index f22d83c178..9e3d8b1401 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -2618,6 +2618,7 @@ R: Thomas Huth <thuth@redhat.com>
> > S: Maintained
> > F: tests/qtest/fuzz/
> > F: scripts/oss-fuzz/
> > +F: hw/mem/sparse-mem.c
> > F: docs/devel/fuzzing.rst
> >
> > Register API
> > diff --git a/hw/mem/meson.build b/hw/mem/meson.build
> > index 0d22f2b572..732f459e0a 100644
> > --- a/hw/mem/meson.build
> > +++ b/hw/mem/meson.build
> > @@ -1,5 +1,6 @@
> > mem_ss = ss.source_set()
> > mem_ss.add(files('memory-device.c'))
> > +mem_ss.add(files('sparse-mem.c'))
> > mem_ss.add(when: 'CONFIG_DIMM', if_true: files('pc-dimm.c'))
> > mem_ss.add(when: 'CONFIG_NPCM7XX', if_true: files('npcm7xx_mc.c'))
> > mem_ss.add(when: 'CONFIG_NVDIMM', if_true: files('nvdimm.c'))
> > diff --git a/hw/mem/sparse-mem.c b/hw/mem/sparse-mem.c
> > new file mode 100644
> > index 0000000000..ffda6f76b4
> > --- /dev/null
> > +++ b/hw/mem/sparse-mem.c
> > @@ -0,0 +1,154 @@
> > +/*
> > + * A sparse memory device
> > + *
> > + * Copyright Red Hat Inc., 2021
> > + *
> > + * Authors:
> > + * Alexander Bulekov <alxndr@bu.edu>
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > + * See the COPYING file in the top-level directory.
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +
> > +#include "exec/address-spaces.h"
> > +#include "hw/qdev-properties.h"
> > +
> > +#define TYPE_SPARSE_MEM "sparse-mem"
> > +#define SPARSE_MEM(obj) OBJECT_CHECK(SparseMemState, (obj), TYPE_SPARSE_MEM)
> > +
> > +#define SPARSE_BLOCK_SIZE 0x1000
> > +
> > +typedef struct SparseMemState {
> > + DeviceState parent_obj;
> > + MemoryRegion mmio;
> > + uint64_t baseaddr;
> > + uint64_t length;
> > + uint64_t usage;
>
> usage -> size_used?
>
Ok - that's nicer.
> > + uint64_t maxsize;
> > + GHashTable *mapped;
> > +} SparseMemState;
> > +
> > +typedef struct sparse_mem_block {
> > + uint16_t nonzeros;
> > + uint8_t data[SPARSE_BLOCK_SIZE];
> > +} sparse_mem_block;
> > +
>
> > +static const MemoryRegionOps sparse_mem_ops = {
> > + .read = sparse_mem_read,
> > + .write = sparse_mem_write,
> > + .endianness = DEVICE_LITTLE_ENDIAN,
> > + .valid = {
> > + .min_access_size = 1,
> > + .max_access_size = 8,
> > + .unaligned = false,
>
> Why restrict unaligned accesses?
>
It is mostly a shortcut to avoid dealing with accesses that span
multiple "blocks". E.g. a read from (uint32_t*)0x1ffe would require
looking both at the 0x1000 and 0x2000 blocks.
> > + },
> > +};
> > +
> > +static Property sparse_mem_properties[] = {
> > + /* The base address of the memory */
> > + DEFINE_PROP_UINT64("baseaddr", SparseMemState, baseaddr, 0x0),
> > + /* The length of the sparse memory region */
> > + DEFINE_PROP_UINT64("length", SparseMemState, length, UINT64_MAX),
> > + /* Max amount of actual memory that can be used to back the sparse memory */
> > + DEFINE_PROP_UINT64("maxsize", SparseMemState, maxsize, 0x100000),
>
> 0x100000 -> 1 * MiB
>
Ok.
> > + DEFINE_PROP_END_OF_LIST(),
> > +};
> > +
> > +static void sparse_mem_realize(DeviceState *dev, Error **errp)
> > +{
> > + SparseMemState *s = SPARSE_MEM(dev);
>
> Anyhow, we should restrict this device to QTest accelerator, right?
>
> Maybe:
> if (!qtest_enabled()) {
> error_setg(errp, "sparse_mem device requires QTest");
> return;
> }
>
> > +
> > + assert(s->baseaddr + s->length > s->baseaddr);
>
> Don't you need more than 64-bit to do this check?
The check is to make sure that baseaddr + length doesn't overflow the
64-bit address-space.
>
> > +
> > + s->mapped = g_hash_table_new(NULL, NULL);
> > + memory_region_init_io(&(s->mmio), OBJECT(s), &sparse_mem_ops, s,
> > + "sparse-mem", s->length);
> > + memory_region_add_subregion_overlap(get_system_memory(), s->baseaddr,
> > + &(s->mmio), -100);
>
> mr_add() to sysmem from a non-sysbus device is odd... Maybe it is
> acceptable, I don't know enough.
>
I will try to find a more standard way to do this.
Thanks
-Alex
> > +}
> > +
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-03-12 3:10 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-11 5:36 [PATCH 0/3] fuzz: Add a sparse-memory device to accelerate fuzzing Alexander Bulekov
2021-03-11 5:36 ` [PATCH 1/3] memory: add a sparse memory device Alexander Bulekov
2021-03-11 14:25 ` Philippe Mathieu-Daudé
2021-03-12 3:08 ` Alexander Bulekov
2021-03-11 5:36 ` [PATCH 2/3] memory: add tests for the sparse-mem device Alexander Bulekov
2021-03-11 5:36 ` [PATCH 3/3] fuzz: configure a sparse-mem device, by default Alexander Bulekov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.