[PATCH v7 00/19] Enable build of full Xen for RISC-V

[PATCH v7 00/19] Enable build of full Xen for RISC-V

[Oleksii Kurochko]

This patch series performs all of the additions necessary to drop the
build overrides for RISCV and enable the full Xen build. Except in cases
where compatibile implementations already exist (e.g. atomic.h and
bitops.h), the newly added definitions are simple.

The patch series is based on the following patch series:
- [PATCH 0/7] xen/bitops: Reduce the mess, starting with ffs() [1]
- [PATCH] move __read_mostly to xen/cache.h  [2]
- [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE() [3]


Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.

[1] https://lore.kernel.org/xen-devel/20240313172716.2325427-1-andrew.cooper3@citrix.com/T/#t
[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/
[3] https://lore.kernel.org/xen-devel/42fc6ae8d3eb802429d29c774502ff232340dc84.1706259490.git.federico.serafini@bugseng.com/

---
Changes in V7:
 - Patch was merged to staging:
   [PATCH v6 15/20] xen/riscv: add minimal stuff to processor.h to build full Xen.
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V6:
 - Update the cover letter message: drop already merged dependecies and add
   a new one.
 - Patches were merged to staging:
   - [PATCH v5 02/23] xen/riscv: use some asm-generic headers ( even v4 was
     merged to staging branch, I just wasn't apply changes on top of the latest staging branch )
   - [PATCH v5 03/23] xen/riscv: introduce nospec.h
   - [PATCH v5 10/23] xen/riscv: introduces acrquire, release and full barriers
 - Introduce new patches:
   - xen/riscv: introduce extenstion support check by compiler
   - xen/bitops: put __ffs() and ffz() into linux compatible header
   - xen/bitops: implement fls{l}() in common logic
 - The following patches were dropped:
   - drop some patches related to bitops operations as they were introduced in another
     patch series [...]
   - introduce new version for generic __ffs(), ffz() and fls{l}().
 - Merge patch from patch series "[PATCH v9 0/7]  Introduce generic headers" to this patch
   series as only one patch left in the generic headers patch series and it is more about
   RISC-V.
 - Other changes are specific to specific patches. please look at specific patch.
---
Changes in V5:
 - Update the cover letter as one of the dependencies were merged to staging.
 - Was introduced asm-generic for atomic ops and separate patches for asm-generic bit ops
 - Moved fence.h to separate patch to deal with some patches dependecies on fence.h
 - Patches were dropped as they were merged to staging:
   * [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
   * [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
   * [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
   * [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
   * [PATCH v4 08/30] xen/riscv: introduce setup.h
   * [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
   * [PATCH v4 11/30] xen/riscv: introduce smp.h
   * [PATCH v4 15/30] xen/riscv: introduce irq.h
   * [PATCH v4 16/30] xen/riscv: introduce p2m.h
   * [PATCH v4 17/30] xen/riscv: introduce regs.h
   * [PATCH v4 18/30] xen/riscv: introduce time.h
   * [PATCH v4 19/30] xen/riscv: introduce event.h
   * [PATCH v4 22/30] xen/riscv: define an address of frame table
 - Other changes are specific to specific patches. please look at specific patch
---
Changes in V4:
 - Update the cover letter message: new patch series dependencies.
 - Some patches were merged to staging, so they were dropped in this patch series:
     [PATCH v3 09/34] xen/riscv: introduce system.h
     [PATCH v3 18/34] xen/riscv: introduce domain.h
     [PATCH v3 19/34] xen/riscv: introduce guest_access.h
 - Was sent out of this patch series:
     [PATCH v3 16/34] xen/lib: introduce generic find next bit operations
 - [PATCH v3 17/34] xen/riscv: add compilation of generic find-next-bit.c was
   droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
 - All other changes are specific to a specific patch.
---
Changes in V3:
 - Update the cover letter message
 - The following patches were dropped as they were merged to staging:
    [PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h
    [PATCH v2 04/39] xen/riscv: add public arch-riscv.h
    [PATCH v2 05/39] xen/riscv: introduce spinlock.h
    [PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S
    [PATCH v2 34/39] xen: add RISCV support for pmu.h
    [PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
 - Instead of the following patches were introduced new:
    [PATCH v2 10/39] xen/riscv: introduce asm/iommu.h
    [PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
 - remove "asm/"  for commit messages which start with "xen/riscv:"
 - code style updates.
 - add emulation of {cmp}xchg_* for 1 and 2 bytes types.
 - code style fixes.
 - add SPDX and footer for the newly added headers.
 - introduce generic find-next-bit.c.
 - some other mionor changes. ( details please find in a patch )
---
Changes in V2:
  - Drop the following patches as they are the part of [2]:
      [PATCH v1 06/57] xen/riscv: introduce paging.h
      [PATCH v1 08/57] xen/riscv: introduce asm/device.h
      [PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h
      [PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h
      [PATCH v1 13/57] xen/riscv: introduce asm/iocap.h
      [PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h
      [PATCH v1 18/57] xen/riscv: introduce asm/random.h
      [PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h
      [PATCH v1 24/57] xen/riscv: introduce asm/percpu.h
      [PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h
      [PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h
      [PATCH v1 38/57] xen/riscv: introduce asm/monitor.h
      [PATCH v1 39/57] xen/riscv: introduce asm/numa.h
      [PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
  - xen/lib.h in most of the cases were changed to xen/bug.h as
    mostly functionilty of bug.h is used.
  - align arch-riscv.h with Arm's version of it.
  - change the Author of commit with introduction of asm/atomic.h.
  - update some definition from spinlock.h.
  - code style changes.
---

Oleksii Kurochko (19):
  automation: introduce fixed randconfig for RISC-V
  xen/riscv: disable unnecessary configs
  xen/riscv: introduce extenstion support check by compiler
  xen: introduce generic non-atomic test_*bit()
  xen/bitops: implement fls{l}() in common logic
  xen/bitops: put __ffs() into linux compatible header
  xen/riscv: introduce bitops.h
  xen/riscv: introduce cmpxchg.h
  xen/riscv: introduce io.h
  xen/riscv: introduce atomic.h
  xen/riscv: introduce monitor.h
  xen/riscv: add definition of __read_mostly
  xen/riscv: add required things to current.h
  xen/riscv: add minimal stuff to page.h to build full Xen
  xen/riscv: add minimal stuff to mm.h to build full Xen
  xen/riscv: introduce vm_event_*() functions
  xen/riscv: add minimal amount of stubs to build full Xen
  xen/riscv: enable full Xen build
  xen/README: add compiler and binutils versions for RISC-V64

 README                                  |   4 +
 automation/gitlab-ci/build.yaml         |  19 +-
 docs/misc/riscv/booting.txt             |  16 +
 xen/arch/arm/arm64/livepatch.c          |   1 -
 xen/arch/arm/include/asm/arm32/bitops.h |   2 +-
 xen/arch/arm/include/asm/arm64/bitops.h |  27 +-
 xen/arch/arm/include/asm/bitops.h       |  74 +----
 xen/arch/ppc/include/asm/bitops.h       |  76 -----
 xen/arch/ppc/include/asm/page.h         |   2 +-
 xen/arch/ppc/mm-radix.c                 |   2 +-
 xen/arch/riscv/Makefile                 |  18 +-
 xen/arch/riscv/arch.mk                  |  19 +-
 xen/arch/riscv/configs/tiny64_defconfig |  11 +-
 xen/arch/riscv/early_printk.c           | 168 ----------
 xen/arch/riscv/include/asm/atomic.h     | 261 +++++++++++++++
 xen/arch/riscv/include/asm/bitops.h     | 146 +++++++++
 xen/arch/riscv/include/asm/cache.h      |   2 +
 xen/arch/riscv/include/asm/cmpxchg.h    | 227 +++++++++++++
 xen/arch/riscv/include/asm/config.h     |   2 +
 xen/arch/riscv/include/asm/current.h    |  19 ++
 xen/arch/riscv/include/asm/io.h         | 168 ++++++++++
 xen/arch/riscv/include/asm/mm.h         | 240 ++++++++++++++
 xen/arch/riscv/include/asm/monitor.h    |  26 ++
 xen/arch/riscv/include/asm/page.h       |  19 ++
 xen/arch/riscv/mm.c                     |  52 ++-
 xen/arch/riscv/setup.c                  |  10 +-
 xen/arch/riscv/stubs.c                  | 415 ++++++++++++++++++++++++
 xen/arch/riscv/traps.c                  |  25 ++
 xen/arch/riscv/vm_event.c               |  19 ++
 xen/arch/x86/include/asm/bitops.h       |  34 +-
 xen/common/bitops.c                     |  22 ++
 xen/drivers/passthrough/arm/smmu-v3.c   |   2 +
 xen/include/asm-generic/atomic-ops.h    |  97 ++++++
 xen/include/xen/bitops.h                | 154 +++++++++
 xen/include/xen/linux-compat.h          |   2 +
 xen/lib/find-next-bit.c                 |   3 +
 36 files changed, 1999 insertions(+), 385 deletions(-)
 create mode 100644 docs/misc/riscv/booting.txt
 create mode 100644 xen/arch/riscv/include/asm/atomic.h
 create mode 100644 xen/arch/riscv/include/asm/bitops.h
 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h
 create mode 100644 xen/arch/riscv/include/asm/io.h
 create mode 100644 xen/arch/riscv/include/asm/monitor.h
 create mode 100644 xen/arch/riscv/stubs.c
 create mode 100644 xen/arch/riscv/vm_event.c
 create mode 100644 xen/include/asm-generic/atomic-ops.h

-- 
2.43.0

[PATCH v7 01/19] automation: introduce fixed randconfig for RISC-V

[Oleksii Kurochko]

This patch introduces the anchor riscv-fixed-randconfig,
which includes all configurations that should be disabled for
randconfig builds.

Suggested-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
---
Changes in V7:
 - Nothing changed. Only rebase.
---
Changes in V6:
 - new patch for this patch series
 - Reviewed-by and Acked-by was added. ( in another patch series they were
   recieved. )
---
 automation/gitlab-ci/build.yaml | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 6d2cb18b88..aac29ee13a 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -512,6 +512,14 @@ alpine-3.18-gcc-debug-arm64-boot-cpupools:
       CONFIG_BOOT_TIME_CPUPOOLS=y
 
 # RISC-V 64 cross-build
+.riscv-fixed-randconfig:
+  variables: &riscv-fixed-randconfig
+    EXTRA_FIXED_RANDCONFIG: |
+      CONFIG_COVERAGE=n
+      CONFIG_EXPERT=y
+      CONFIG_GRANT_TABLE=n
+      CONFIG_MEM_ACCESS=n
+
 archlinux-current-gcc-riscv64:
   extends: .gcc-riscv64-cross-build
   variables:
@@ -532,8 +540,7 @@ archlinux-current-gcc-riscv64-randconfig:
     CONTAINER: archlinux:current-riscv64
     KBUILD_DEFCONFIG: tiny64_defconfig
     RANDCONFIG: y
-    EXTRA_FIXED_RANDCONFIG:
-      CONFIG_COVERAGE=n
+    <<: *riscv-fixed-randconfig
 
 archlinux-current-gcc-riscv64-debug-randconfig:
   extends: .gcc-riscv64-cross-build-debug
@@ -541,8 +548,7 @@ archlinux-current-gcc-riscv64-debug-randconfig:
     CONTAINER: archlinux:current-riscv64
     KBUILD_DEFCONFIG: tiny64_defconfig
     RANDCONFIG: y
-    EXTRA_FIXED_RANDCONFIG:
-      CONFIG_COVERAGE=n
+    <<: *riscv-fixed-randconfig
 
 # Power cross-build
 debian-bullseye-gcc-ppc64le:
-- 
2.43.0

[PATCH v7 02/19] xen/riscv: disable unnecessary configs

[Oleksii Kurochko]

This patch disables unnecessary configs for two cases:
1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
2. By using tiny64_defconfig for non-randconfig builds.

Only configs which lead to compilation issues were disabled.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V7:
 - Disable only configs which cause compilation issues.
 - Update the commit message.
---
Changes in V6:
 - Nothing changed. Only rebase.
---
Changes in V5:
 - Rebase and drop duplicated configs in EXTRA_FIXED_RANDCONFIG list
 - Update the commit message
---
Changes in V4:
 - Nothing changed. Only rebase
---
Changes in V3:
 - Remove EXTRA_FIXED_RANDCONFIG for non-randconfig jobs.
   For non-randconfig jobs, it is sufficient to disable configs by using the defconfig.
 - Remove double blank lines in build.yaml file before archlinux-current-gcc-riscv64-debug
---
Changes in V2:
 - update the commit message.
 - remove xen/arch/riscv/Kconfig changes.
---
 automation/gitlab-ci/build.yaml         |  5 +++++
 xen/arch/riscv/configs/tiny64_defconfig | 11 +++++------
 2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index aac29ee13a..43faeaed9c 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -519,6 +519,11 @@ alpine-3.18-gcc-debug-arm64-boot-cpupools:
       CONFIG_EXPERT=y
       CONFIG_GRANT_TABLE=n
       CONFIG_MEM_ACCESS=n
+      CONFIG_HYPFS=n
+      CONFIG_ARGO=n
+      CONFIG_PERF_COUNTERS=n
+      CONFIG_LIVEPATCH=n
+      CONFIG_XSM=n
 
 archlinux-current-gcc-riscv64:
   extends: .gcc-riscv64-cross-build
diff --git a/xen/arch/riscv/configs/tiny64_defconfig b/xen/arch/riscv/configs/tiny64_defconfig
index 09defe236b..24a807a5f9 100644
--- a/xen/arch/riscv/configs/tiny64_defconfig
+++ b/xen/arch/riscv/configs/tiny64_defconfig
@@ -1,12 +1,11 @@
-# CONFIG_SCHED_CREDIT is not set
-# CONFIG_SCHED_RTDS is not set
-# CONFIG_SCHED_NULL is not set
-# CONFIG_SCHED_ARINC653 is not set
-# CONFIG_TRACEBUFFER is not set
 # CONFIG_HYPFS is not set
 # CONFIG_GRANT_TABLE is not set
-# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
 # CONFIG_MEM_ACCESS is not set
+# CONFIG_ARGO is not set
+# CONFIG_PERF_COUNTERS is not set
+# CONFIG_COVERAGE is not set
+# CONFIG_LIVEPATCH is not set
+# CONFIG_XSM is not set
 
 CONFIG_RISCV_64=y
 CONFIG_DEBUG=y
-- 
2.43.0

[PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler

[Oleksii Kurochko]

Currently, RISC-V requires two extensions: _zbb and _zihintpause.

This patch introduces a compiler check to check if these extensions
are supported.
Additionally, it introduces the riscv/booting.txt file, which contains
information about the extensions that should be supported by the platform.

In the future, a feature will be introduced to check whether an extension
is supported at runtime.
However, this feature requires functionality for parsing device tree
source (DTS), which is not yet available.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V7:
 - add variables for each extension separately.
 - create variable for abi and compilation flags to not repeat the same in several
   places.
 - update architectures to use generic implementations
---
Changes in V6:
 - new patch for this patch series
---
 docs/misc/riscv/booting.txt | 16 ++++++++++++++++
 xen/arch/riscv/arch.mk      | 15 +++++++++++++--
 2 files changed, 29 insertions(+), 2 deletions(-)
 create mode 100644 docs/misc/riscv/booting.txt

diff --git a/docs/misc/riscv/booting.txt b/docs/misc/riscv/booting.txt
new file mode 100644
index 0000000000..cb4d79f12c
--- /dev/null
+++ b/docs/misc/riscv/booting.txt
@@ -0,0 +1,16 @@
+System requirements
+===================
+
+The following extensions are expected to be supported by a system on which
+Xen is run:
+- Zbb:
+  RISC-V doesn't have a CLZ instruction in the base ISA.
+  As a consequence, __builtin_ffs() emits a library call to ffs() on GCC,
+  or a de Bruijn sequence on Clang.
+  Zbb extension adds a CLZ instruction, after which __builtin_ffs() emits
+  a very simple sequence.
+  The similar issue occurs with other __builtin_<bitop>, so it is needed to
+  provide a generic version of bitops in RISC-V bitops.h
+- Zihintpause:
+  On a system that doesn't have this extension, cpu_relax() should be
+  implemented properly.
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index 8403f96b6f..24a7461bcc 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -3,16 +3,27 @@
 
 $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
 
-CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64
+riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32
+riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64
 
 riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
 riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
 
+riscv-generic-flags := $(riscv-abi-y) -march=$(riscv-march-y)
+
+zbb := $(call as-insn,$(CC) $(riscv-generic-flags)_zbb,"",_zbb)
+zihintpause := $(call as-insn,\
+               $(CC) $(riscv-generic-flags)_zihintpause,"pause",_zihintpause)
+
+extensions := $(zbb) $(zihintpause)
+
+extensions := $(subst $(space),,$(extensions))
+
 # Note that -mcmodel=medany is used so that Xen can be mapped
 # into the upper half _or_ the lower half of the address space.
 # -mcmodel=medlow would force Xen into the lower half.
 
-CFLAGS += -march=$(riscv-march-y) -mstrict-align -mcmodel=medany
+CFLAGS += $(riscv-generic-flags)$(extensions) -mstrict-align -mcmodel=medany
 
 # TODO: Drop override when more of the build is working
 override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
-- 
2.43.0

[PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Oleksii Kurochko]

The patch introduces the following generic functions:
* test_bit
* generic__test_and_set_bit
* generic__test_and_clear_bit
* generic__test_and_change_bit

Also, the patch introduces the following generics which are
used by the functions mentioned above:
* BITOP_BITS_PER_WORD
* BITOP_MASK
* BITOP_WORD
* BITOP_TYPE

These functions and macros can be useful for architectures
that don't have corresponding arch-specific instructions.

Because of that x86 has the following check in the macros test_bit(),
__test_and_set_bit(), __test_and_clear_bit(), __test_and_change_bit():
    if ( bitop_bad_size(addr) ) __bitop_bad_size();
It was necessary to move the check to generic code and define as 0
for other architectures as they do not require this check.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Changes in V7:
  - move everything to xen/bitops.h to follow the same approach for all generic
    bit ops.
  - put together BITOP_BITS_PER_WORD and bitops_uint_t.
  - make BITOP_MASK more generic.
  - drop #ifdef ... #endif around BITOP_MASK, BITOP_WORD as they are generic
    enough.
  - drop "_" for generic__{test_and_set_bit,...}().
  - drop " != 0" for functions which return bool.
  - add volatile during the cast for generic__{...}().
  - update the commit message.
  - update arch related code to follow the proposed generic approach.
---
 Changes in V6:
  - Nothing changed ( only rebase )
---
 Changes in V5:
   - new patch
---
 xen/arch/arm/arm64/livepatch.c    |   1 -
 xen/arch/arm/include/asm/bitops.h |  67 -------------
 xen/arch/ppc/include/asm/bitops.h |  64 -------------
 xen/arch/ppc/include/asm/page.h   |   2 +-
 xen/arch/ppc/mm-radix.c           |   2 +-
 xen/arch/x86/include/asm/bitops.h |  28 ++----
 xen/include/xen/bitops.h          | 154 ++++++++++++++++++++++++++++++
 7 files changed, 165 insertions(+), 153 deletions(-)

diff --git a/xen/arch/arm/arm64/livepatch.c b/xen/arch/arm/arm64/livepatch.c
index df2cebedde..4bc8ed9be5 100644
--- a/xen/arch/arm/arm64/livepatch.c
+++ b/xen/arch/arm/arm64/livepatch.c
@@ -10,7 +10,6 @@
 #include <xen/mm.h>
 #include <xen/vmap.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 #include <asm/insn.h>
 #include <asm/livepatch.h>
diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index 5104334e48..8e16335e76 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -22,9 +22,6 @@
 #define __set_bit(n,p)            set_bit(n,p)
 #define __clear_bit(n,p)          clear_bit(n,p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1UL << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
 #define BITS_PER_BYTE           8
 
 #define ADDR (*(volatile int *) addr)
@@ -76,70 +73,6 @@ bool test_and_change_bit_timeout(int nr, volatile void *p,
 bool clear_mask16_timeout(uint16_t mask, volatile void *p,
                           unsigned int max_try);
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old | mask;
-        return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old & ~mask;
-        return (old & mask) != 0;
-}
-
-/* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr,
-                                            volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old ^ mask;
-        return (old & mask) != 0;
-}
-
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-        const volatile unsigned int *p = (const volatile unsigned int *)addr;
-        return 1UL & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD-1)));
-}
-
 /*
  * On ARMv5 and above those functions can be implemented around
  * the clz instruction for much better code efficiency.
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index 989d341a44..a17060c7c2 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -15,9 +15,6 @@
 #define __set_bit(n, p)         set_bit(n, p)
 #define __clear_bit(n, p)       clear_bit(n, p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1U << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
 #define BITS_PER_BYTE           8
 
 /* PPC bit number conversion */
@@ -69,17 +66,6 @@ static inline void clear_bit(int nr, volatile void *addr)
     clear_bits(BITOP_MASK(nr), (volatile unsigned int *)addr + BITOP_WORD(nr));
 }
 
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-    const volatile unsigned int *p = addr;
-    return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD - 1)));
-}
-
 static inline unsigned int test_and_clear_bits(
     unsigned int mask,
     volatile unsigned int *p)
@@ -133,56 +119,6 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
         (volatile unsigned int *)addr + BITOP_WORD(nr)) != 0;
 }
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-    unsigned int mask = BITOP_MASK(nr);
-    volatile unsigned int *p = (volatile unsigned int *)addr + BITOP_WORD(nr);
-    unsigned int old = *p;
-
-    *p = old | mask;
-    return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-    unsigned int mask = BITOP_MASK(nr);
-    volatile unsigned int *p = (volatile unsigned int *)addr + BITOP_WORD(nr);
-    unsigned int old = *p;
-
-    *p = old & ~mask;
-    return (old & mask) != 0;
-}
-
-#define flsl(x) generic_flsl(x)
-#define fls(x) generic_fls(x)
-
-/* Based on linux/include/asm-generic/bitops/ffz.h */
-/*
- * ffz - find first zero in word.
- * @word: The word to search
- *
- * Undefined if no zero exists, so code should check against ~0UL first.
- */
-#define ffz(x) __ffs(~(x))
-
 /**
  * hweightN - returns the hamming weight of a N-bit word
  * @x: the word to weigh
diff --git a/xen/arch/ppc/include/asm/page.h b/xen/arch/ppc/include/asm/page.h
index 890e285051..482053b023 100644
--- a/xen/arch/ppc/include/asm/page.h
+++ b/xen/arch/ppc/include/asm/page.h
@@ -4,7 +4,7 @@
 
 #include <xen/types.h>
 
-#include <asm/bitops.h>
+#include <xen/bitops.h>
 #include <asm/byteorder.h>
 
 #define PDE_VALID     PPC_BIT(0)
diff --git a/xen/arch/ppc/mm-radix.c b/xen/arch/ppc/mm-radix.c
index daa411a6fa..3cd8c4635a 100644
--- a/xen/arch/ppc/mm-radix.c
+++ b/xen/arch/ppc/mm-radix.c
@@ -1,11 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
+#include <xen/bitops.h>
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/mm.h>
 #include <xen/types.h>
 #include <xen/lib.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 #include <asm/early_printk.h>
 #include <asm/page.h>
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index dd439b69a0..81b43da5db 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -175,7 +175,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_set_bit - Set a bit and return its old value
+ * arch__test_and_set_bit - Set a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -183,7 +183,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_set_bit(int nr, void *addr)
+static inline int arch__test_and_set_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -194,10 +194,7 @@ static inline int __test_and_set_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_set_bit(nr, addr) ({                 \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_set_bit(nr, addr);                       \
-})
+#define arch__test_and_set_bit arch__test_and_set_bit
 
 /**
  * test_and_clear_bit - Clear a bit and return its old value
@@ -224,7 +221,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_clear_bit - Clear a bit and return its old value
+ * arch__test_and_clear_bit - Clear a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -232,7 +229,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_clear_bit(int nr, void *addr)
+static inline int arch__test_and_clear_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -243,13 +240,10 @@ static inline int __test_and_clear_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_clear_bit(nr, addr) ({               \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_clear_bit(nr, addr);                     \
-})
+#define arch__test_and_clear_bit arch__test_and_clear_bit
 
 /* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr, void *addr)
+static inline int arch__test_and_change_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -260,10 +254,7 @@ static inline int __test_and_change_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_change_bit(nr, addr) ({              \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_change_bit(nr, addr);                    \
-})
+#define arch__test_and_change_bit arch__test_and_change_bit
 
 /**
  * test_and_change_bit - Change a bit and return its new value
@@ -307,8 +298,7 @@ static inline int variable_test_bit(int nr, const volatile void *addr)
     return oldbit;
 }
 
-#define test_bit(nr, addr) ({                           \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+#define arch_test_bit(nr, addr) ({                      \
     __builtin_constant_p(nr) ?                          \
         constant_test_bit(nr, addr) :                   \
         variable_test_bit(nr, addr);                    \
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index f14ad0d33a..685c7540cc 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -65,10 +65,164 @@ static inline int generic_flsl(unsigned long x)
  * scope
  */
 
+#define BITOP_BITS_PER_WORD 32
+/* typedef uint32_t bitop_uint_t; */
+#define bitop_uint_t uint32_t
+
+#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
+
+#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
+
 /* --------------------- Please tidy above here --------------------- */
 
 #include <asm/bitops.h>
 
+#ifndef bitop_bad_size
+extern void __bitop_bad_size(void);
+#define bitop_bad_size(addr) 0
+#endif
+
+/**
+ * generic__test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline __pure bool
+generic__test_and_set_bit(unsigned long nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = ((volatile bitop_uint_t *)addr) + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old | mask;
+    return (old & mask);
+}
+
+/**
+ * generic__test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline __pure bool
+generic__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = ((volatile bitop_uint_t *)addr) + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old & ~mask;
+    return (old & mask);
+}
+
+/* WARNING: non atomic and it can be reordered! */
+static always_inline __pure bool
+generic__test_and_change_bit(unsigned long nr, volatile void *addr)
+{
+    bitop_uint_t mask = BITOP_MASK(nr);
+    volatile bitop_uint_t *p = ((volatile bitop_uint_t *)addr) + BITOP_WORD(nr);
+    bitop_uint_t old = *p;
+
+    *p = old ^ mask;
+    return (old & mask);
+}
+/**
+ * generic_test_bit - Determine whether a bit is set
+ * @nr: bit number to test
+ * @addr: Address to start counting from
+ */
+static always_inline __pure int generic_test_bit(int nr, const volatile void *addr)
+{
+    const volatile bitop_uint_t *p = addr;
+    return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD - 1)));
+}
+
+static always_inline __pure bool
+__test_and_set_bit(unsigned long nr, volatile void *addr)
+{
+#ifndef arch__test_and_set_bit
+#define arch__test_and_set_bit generic__test_and_set_bit
+#endif
+
+    return arch__test_and_set_bit(nr, addr);
+}
+#define __test_and_set_bit(nr, addr) ({             \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
+    __test_and_set_bit(nr, addr);                   \
+})
+
+static always_inline __pure bool
+__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
+{
+#ifndef arch__test_and_clear_bit
+#define arch__test_and_clear_bit generic__test_and_clear_bit
+#endif
+
+    return arch__test_and_clear_bit(nr, addr);
+}
+#define __test_and_clear_bit(nr, addr) ({           \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
+    __test_and_clear_bit(nr, addr);                 \
+})
+
+static always_inline __pure bool
+__test_and_change_bit(unsigned long nr, volatile void *addr)
+{
+#ifndef arch__test_and_change_bit
+#define arch__test_and_change_bit generic__test_and_change_bit
+#endif
+
+    return arch__test_and_change_bit(nr, addr);
+}
+#define __test_and_change_bit(nr, addr) ({              \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+    __test_and_change_bit(nr, addr);                    \
+})
+
+static always_inline __pure int test_bit(int nr, const volatile void *addr)
+{
+#ifndef arch_test_bit
+#define arch_test_bit generic_test_bit
+#endif
+
+    return arch_test_bit(nr, addr);
+}
+#define test_bit(nr, addr) ({                           \
+    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+    test_bit(nr, addr);                                 \
+})
+
+static always_inline __pure int fls(unsigned int x)
+{
+    if (__builtin_constant_p(x))
+        return generic_fls(x);
+
+#ifndef arch_fls
+#define arch_fls generic_fls
+#endif
+
+    return arch_fls(x);
+}
+
+static always_inline __pure int flsl(unsigned long x)
+{
+    if (__builtin_constant_p(x))
+        return generic_flsl(x);
+
+#ifndef arch_flsl
+#define arch_flsl generic_flsl
+#endif
+
+    return arch_flsl(x);
+}
+
 /*
  * Find First Set bit.  Bits are labelled from 1.
  */
-- 
2.43.0

[PATCH v7 05/19] xen/bitops: implement fls{l}() in common logic

[Oleksii Kurochko]

Return type was left 'int' because of the following compilation error:

./include/xen/kernel.h:18:21: error: comparison of distinct pointer types lacks a cast [-Werror]
       18 |         (void) (&_x == &_y);            \
          |                     ^~
    common/page_alloc.c:1843:34: note: in expansion of macro 'min'
     1843 |         unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);

generic_fls{l} was used instead of __builtin_clz{l}(x) as if x is 0,
the result in undefined.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V7:
 - Code style fixes
---
Changes in V6:
 - new patch for the patch series.
---
 xen/arch/arm/include/asm/arm32/bitops.h |  2 +-
 xen/arch/arm/include/asm/arm64/bitops.h |  6 ++----
 xen/arch/arm/include/asm/bitops.h       |  7 ++-----
 xen/arch/x86/include/asm/bitops.h       |  6 ++++--
 xen/common/bitops.c                     | 22 ++++++++++++++++++++++
 xen/include/xen/bitops.h                |  4 ++--
 6 files changed, 33 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/include/asm/arm32/bitops.h b/xen/arch/arm/include/asm/arm32/bitops.h
index d0309d47c1..5552d4e945 100644
--- a/xen/arch/arm/include/asm/arm32/bitops.h
+++ b/xen/arch/arm/include/asm/arm32/bitops.h
@@ -1,7 +1,7 @@
 #ifndef _ARM_ARM32_BITOPS_H
 #define _ARM_ARM32_BITOPS_H
 
-#define flsl fls
+#define arch_flsl fls
 
 /*
  * Little endian assembly bitops.  nr = 0 -> byte 0 bit 0.
diff --git a/xen/arch/arm/include/asm/arm64/bitops.h b/xen/arch/arm/include/asm/arm64/bitops.h
index 0efde29068..5f5d97faa0 100644
--- a/xen/arch/arm/include/asm/arm64/bitops.h
+++ b/xen/arch/arm/include/asm/arm64/bitops.h
@@ -22,17 +22,15 @@ static /*__*/always_inline unsigned long __ffs(unsigned long word)
  */
 #define ffz(x)  __ffs(~(x))
 
-static inline int flsl(unsigned long x)
+static inline int arch_flsl(unsigned long x)
 {
         uint64_t ret;
 
-        if (__builtin_constant_p(x))
-               return generic_flsl(x);
-
         asm("clz\t%0, %1" : "=r" (ret) : "r" (x));
 
         return BITS_PER_LONG - ret;
 }
+#define arch_flsl arch_flsl
 
 /* Based on linux/include/asm-generic/bitops/find.h */
 
diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h
index 8e16335e76..860d6d4689 100644
--- a/xen/arch/arm/include/asm/bitops.h
+++ b/xen/arch/arm/include/asm/bitops.h
@@ -78,17 +78,14 @@ bool clear_mask16_timeout(uint16_t mask, volatile void *p,
  * the clz instruction for much better code efficiency.
  */
 
-static inline int fls(unsigned int x)
+static inline int arch_fls(unsigned int x)
 {
         int ret;
 
-        if (__builtin_constant_p(x))
-               return generic_fls(x);
-
         asm("clz\t%"__OP32"0, %"__OP32"1" : "=r" (ret) : "r" (x));
         return 32 - ret;
 }
-
+#define arch_fls arch_fls
 
 #define arch_ffs(x) ({ unsigned int __t = (x); fls(ISOLATE_LSB(__t)); })
 #define arch_ffsl(x) ({ unsigned long __t = (x); flsl(ISOLATE_LSB(__t)); })
diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h
index 81b43da5db..9c4ab52df7 100644
--- a/xen/arch/x86/include/asm/bitops.h
+++ b/xen/arch/x86/include/asm/bitops.h
@@ -428,7 +428,7 @@ static always_inline unsigned int arch_ffsl(unsigned long x)
  *
  * This is defined the same way as ffs.
  */
-static inline int flsl(unsigned long x)
+static always_inline int arch_flsl(unsigned long x)
 {
     long r;
 
@@ -438,8 +438,9 @@ static inline int flsl(unsigned long x)
           "1:" : "=r" (r) : "rm" (x));
     return (int)r+1;
 }
+#define arch_flsl arch_flsl
 
-static inline int fls(unsigned int x)
+static always_inline int arch_fls(unsigned int x)
 {
     int r;
 
@@ -449,6 +450,7 @@ static inline int fls(unsigned int x)
           "1:" : "=r" (r) : "rm" (x));
     return r + 1;
 }
+#define arch_fls arch_fls
 
 /**
  * hweightN - returns the hamming weight of a N-bit word
diff --git a/xen/common/bitops.c b/xen/common/bitops.c
index a8c32f6767..95bc47176b 100644
--- a/xen/common/bitops.c
+++ b/xen/common/bitops.c
@@ -62,9 +62,31 @@ static void test_ffs(void)
     CHECK(ffs64, (uint64_t)0x8000000000000000, 64);
 }
 
+static void test_fls(void)
+{
+    /* unsigned int ffs(unsigned int) */
+    CHECK(fls, 1, 1);
+    CHECK(fls, 3, 2);
+    CHECK(fls, 3U << 30, 32);
+
+    /* unsigned int flsl(unsigned long) */
+    CHECK(flsl, 1, 1);
+    CHECK(flsl, 1UL << (BITS_PER_LONG - 1), BITS_PER_LONG);
+#if BITS_PER_LONG > 32
+    CHECK(flsl, 3UL << 32, 34);
+#endif
+
+    /* unsigned int fls64(uint64_t) */
+    CHECK(fls64, 1, 1);
+    CHECK(fls64, 0x00000000C0000000ULL, 32);
+    CHECK(fls64, 0x0000000180000000ULL, 33);
+    CHECK(fls64, 0xC000000000000000ULL, 64);
+}
+
 static int __init cf_check test_bitops(void)
 {
     test_ffs();
+    test_fls();
 
     return 0;
 }
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index 685c7540cc..bc8ae53997 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -201,7 +201,7 @@ static always_inline __pure int test_bit(int nr, const volatile void *addr)
 
 static always_inline __pure int fls(unsigned int x)
 {
-    if (__builtin_constant_p(x))
+    if ( __builtin_constant_p(x) )
         return generic_fls(x);
 
 #ifndef arch_fls
@@ -213,7 +213,7 @@ static always_inline __pure int fls(unsigned int x)
 
 static always_inline __pure int flsl(unsigned long x)
 {
-    if (__builtin_constant_p(x))
+    if ( __builtin_constant_p(x) )
         return generic_flsl(x);
 
 #ifndef arch_flsl
-- 
2.43.0

[PATCH v7 06/19] xen/bitops: put __ffs() into linux compatible header

[Oleksii Kurochko]

The mentioned macros exist only because of Linux compatible purpose.

The patch defines __ffs() in terms of Xen bitops and it is safe
to define in this way ( as __ffs() - 1 ) as considering that __ffs()
was defined as __builtin_ctzl(x), which has undefined behavior when x=0,
so it is assumed that such cases are not encountered in the current code.

To not include <xen/linux-compat.h> to Xen library files __ffs() and __ffz()
were defined locally in find-next-bit.c.

Except __ffs() usage in find-next-bit.c only one usage of __ffs() leave
in smmu-v3.c. It seems that it __ffs can be changed to ffsl(x)-1 in
this file, but to keep smmu-v3.c looks close to linux it was deciced just
to define __ffs() in xen/linux-compat.h and include it in smmu-v3.c

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V7:
 - introduce ffz(),__ffs() locally in find-next-bit.c
 - drop inclusion of <xen/linux-compat.h> in find-next-bit.c.
 - update the commit message.
---
Changes in V6:
 - new patch for the patch series.
---
 xen/arch/arm/include/asm/arm64/bitops.h | 21 ---------------------
 xen/arch/ppc/include/asm/bitops.h       | 12 ------------
 xen/drivers/passthrough/arm/smmu-v3.c   |  2 ++
 xen/include/xen/linux-compat.h          |  2 ++
 xen/lib/find-next-bit.c                 |  3 +++
 5 files changed, 7 insertions(+), 33 deletions(-)

diff --git a/xen/arch/arm/include/asm/arm64/bitops.h b/xen/arch/arm/include/asm/arm64/bitops.h
index 5f5d97faa0..2deb134388 100644
--- a/xen/arch/arm/include/asm/arm64/bitops.h
+++ b/xen/arch/arm/include/asm/arm64/bitops.h
@@ -1,27 +1,6 @@
 #ifndef _ARM_ARM64_BITOPS_H
 #define _ARM_ARM64_BITOPS_H
 
-/* Based on linux/include/asm-generic/bitops/builtin-__ffs.h */
-/**
- * __ffs - find first bit in word.
- * @word: The word to search
- *
- * Undefined if no bit exists, so code should check against 0 first.
- */
-static /*__*/always_inline unsigned long __ffs(unsigned long word)
-{
-        return __builtin_ctzl(word);
-}
-
-/* Based on linux/include/asm-generic/bitops/ffz.h */
-/*
- * ffz - find first zero in word.
- * @word: The word to search
- *
- * Undefined if no zero exists, so code should check against ~0UL first.
- */
-#define ffz(x)  __ffs(~(x))
-
 static inline int arch_flsl(unsigned long x)
 {
         uint64_t ret;
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index a17060c7c2..2237b9f8f4 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -130,16 +130,4 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr)
 #define hweight16(x) __builtin_popcount((uint16_t)(x))
 #define hweight8(x)  __builtin_popcount((uint8_t)(x))
 
-/* Based on linux/include/asm-generic/bitops/builtin-__ffs.h */
-/**
- * __ffs - find first bit in word.
- * @word: The word to search
- *
- * Undefined if no bit exists, so code should check against 0 first.
- */
-static always_inline unsigned long __ffs(unsigned long word)
-{
-    return __builtin_ctzl(word);
-}
-
 #endif /* _ASM_PPC_BITOPS_H */
diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index b1c40c2c0a..6904962467 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -72,12 +72,14 @@
  */
 
 #include <xen/acpi.h>
+#include <xen/bitops.h>
 #include <xen/config.h>
 #include <xen/delay.h>
 #include <xen/errno.h>
 #include <xen/err.h>
 #include <xen/irq.h>
 #include <xen/lib.h>
+#include <xen/linux-compat.h>
 #include <xen/list.h>
 #include <xen/mm.h>
 #include <xen/rbtree.h>
diff --git a/xen/include/xen/linux-compat.h b/xen/include/xen/linux-compat.h
index 62ba71485c..10db80df57 100644
--- a/xen/include/xen/linux-compat.h
+++ b/xen/include/xen/linux-compat.h
@@ -19,4 +19,6 @@ typedef int64_t __s64;
 
 typedef paddr_t phys_addr_t;
 
+#define __ffs(x) (ffsl(x) - 1)
+
 #endif /* __XEN_LINUX_COMPAT_H__ */
diff --git a/xen/lib/find-next-bit.c b/xen/lib/find-next-bit.c
index ca6f82277e..761b027398 100644
--- a/xen/lib/find-next-bit.c
+++ b/xen/lib/find-next-bit.c
@@ -12,6 +12,9 @@
 
 #include <asm/byteorder.h>
 
+#define __ffs(x) (ffsl(x) - 1)
+#define ffz(x) __ffs(~(x))
+
 #ifndef find_next_bit
 /*
  * Find the next set bit in a memory region.
-- 
2.43.0

[PATCH v7 07/19] xen/riscv: introduce bitops.h

[Oleksii Kurochko]

Taken from Linux-6.4.0-rc1

Xen's bitops.h consists of several Linux's headers:
* linux/arch/include/asm/bitops.h:
  * The following function were removed as they aren't used in Xen:
        * test_and_set_bit_lock
        * clear_bit_unlock
        * __clear_bit_unlock
  * The following functions were renamed in the way how they are
    used by common code:
        * __test_and_set_bit
        * __test_and_clear_bit
  * The declaration and implementation of the following functios
    were updated to make Xen build happy:
        * clear_bit
        * set_bit
        * __test_and_clear_bit
        * __test_and_set_bit
  * linux/include/asm-generic/bitops/generic-non-atomic.h with the
    following changes:
     * Only functions that can be reused in Xen were left;
       others were removed.
     * it was updated the message inside #ifndef ... #endif.
     * __always_inline -> always_inline to be align with definition in
       xen/compiler.h.
     * convert identations from tabs to spaces.
     * inside generic__test_and_* use 'bitops_uint_t' instead of 'unsigned long'
        to be generic.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V7:
 - Update the commit message.
 - Drop "__" for __op_bit and __op_bit_ord as they are atomic.
 - add comment above __set_bit and __clear_bit about why they are defined as atomic.
 - align bitops_uint_t with __AMO().
 - make changes after  generic non-atomic test_*bit() were changed.
 - s/__asm__ __volatile__/asm volatile
---
Changes in V6:
 - rebase clean ups were done: drop unused asm-generic includes
---
 Changes in V5:
   - new patch
---
 xen/arch/riscv/include/asm/bitops.h | 146 ++++++++++++++++++++++++++++
 1 file changed, 146 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h

diff --git a/xen/arch/riscv/include/asm/bitops.h b/xen/arch/riscv/include/asm/bitops.h
new file mode 100644
index 0000000000..6f0212e5ac
--- /dev/null
+++ b/xen/arch/riscv/include/asm/bitops.h
@@ -0,0 +1,146 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#undef BITOP_BITS_PER_WORD
+#undef bitop_uint_t
+
+#define BITOP_BITS_PER_WORD BITS_PER_LONG
+#define bitop_uint_t unsigned long
+
+#if BITS_PER_LONG == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITS_PER_LONG == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITS_PER_LONG"
+#endif
+
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could alternatively
+ * implement with local interrupt masking.
+ */
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    unsigned long res, mask;                            \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                       \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))
+
+/**
+ * test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ */
+static inline int test_and_set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ */
+static inline int test_and_clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(and, NOT, nr, addr);
+}
+
+/**
+ * set_bit - Atomically set a bit in memory
+ * @nr: the bit to set
+ * @addr: the address to start counting from
+ *
+ * Note that @nr may be almost arbitrarily large; this function is not
+ * restricted to acting on a single-word quantity.
+ */
+static inline void set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * clear_bit - Clears a bit in memory
+ * @nr: Bit to clear
+ * @addr: Address to start counting from
+ */
+static inline void clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(and, NOT, nr, addr);
+}
+
+/**
+ * test_and_change_bit - Toggle (change) a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is atomic and cannot be reordered.
+ * It also implies a memory barrier.
+ */
+static inline int test_and_change_bit(int nr, volatile unsigned long *addr)
+{
+	return test_and_op_bit(xor, NOP, nr, addr);
+}
+
+#undef test_and_op_bit
+#undef __op_bit
+#undef NOP
+#undef NOT
+#undef __AMO
+
+#endif /* _ASM_RISCV_BITOPS_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0

[PATCH v7 08/19] xen/riscv: introduce cmpxchg.h

[Oleksii Kurochko]

The header was taken from Linux kernl 6.4.0-rc1.

Addionally, were updated:
* add emulation of {cmp}xchg for 1/2 byte types using 32-bit atomic
  access.
* replace tabs with spaces
* replace __* variale with *__
* introduce generic version of xchg_* and cmpxchg_*.
* drop {cmp}xchg{release,relaxed,acquire} as Xen doesn't use them
* drop barries and use instruction suffixices instead ( .aq, .rl, .aqrl )

Implementation of 4- and 8-byte cases were updated according to the spec:
```
              ....
Linux Construct         RVWMO AMO Mapping
    ...
atomic <op>             amo<op>.{w|d}.aqrl
Linux Construct         RVWMO LR/SC Mapping
    ...
atomic <op>             loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez loop

Table A.5: Mappings from Linux memory primitives to RISC-V primitives

```

The current implementation is the same with 8e86f0b409a4
("arm64: atomics: fix use of acquire + release for full barrier
semantics") [1].
RISC-V could combine acquire and release into the SC
instructions and it could reduce a fence instruction to gain better
performance. Here is related description from RISC-V ISA 10.2
Load-Reserved/Store-Conditional Instructions:

 - .aq:   The LR/SC sequence can be given acquire semantics by
          setting the aq bit on the LR instruction.
 - .rl:   The LR/SC sequence can be given release semantics by
          setting the rl bit on the SC instruction.
 - .aqrl: Setting the aq bit on the LR instruction, and setting
          both the aq and the rl bit on the SC instruction makes
          the LR/SC sequence sequentially consistent, meaning that
          it cannot be reordered with earlier or later memory
          operations from the same hart.

 Software should not set the rl bit on an LR instruction unless
 the aq bit is also set, nor should software set the aq bit on an
 SC instruction unless the rl bit is also set. LR.rl and SC.aq
 instructions are not guaranteed to provide any stronger ordering
 than those with both bits clear, but may result in lower
 performance.

Also, I way of transforming ".rl + full barrier" to ".aqrl" was approved
by (the author of the RVWMO spec) [2]

[1] https://patchwork.kernel.org/project/linux-arm-kernel/patch/1391516953-14541-1-git-send-email-will.deacon@arm.com/
[2] https://lore.kernel.org/linux-riscv/41e01514-74ca-84f2-f5cc-2645c444fd8e@nvidia.com/

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V7:
 - replace __*() -> _*() in cmpxchg.h
 - add () around ptr in _amoswap_generic(), emulate_xchg_1_2()
 - fix typos
 - code style fixes.
 - refactor emulate_xcgh_1_2():
   - add parentheses for new argument.
   - use instead of constant 0x4 -> sizeof(*aligned_ptr).
   - add alignment_mask to save  sizeof(*aligned_ptr) - sizeof(*(ptr));
 - s/CONFIG_32BIT/CONFIG_RISCV_32
 - drop unnecessary parentheses in xchg()
 - drop register in _generic_cmpxchg()
 - refactor and update prototype of _generic_cmpxchg():
   add named operands, return value instead of passing ret as an argument, drop %z and J
   constraints for mask operand as it can't be zero
 - refactor and code style fixes in emulate_cmpxchg_1_2():
   - add explanatory comment for emulate_cmpxchg_1_2().
   - add parentheses for old and new arguments.
   - use instead of constant 0x4 -> sizeof(*aligned_ptr).
   - add alignment_mask to save  sizeof(*aligned_ptr) - sizeof(*(ptr));
 - drop unnessary parenthesses in cmpxchg().
 - update the commit message.
 - s/__asm__ __volatile__/asm volatile
---
Changes in V6:
-  update the commit message? ( As before I don't understand this point. Can you give an example of what sort of opcode / instruction is missing?)
 - Code style fixes
 - change sizeof(*ptr) -> sizeof(*(ptr))
 - update operands names and some local variables for macros emulate_xchg_1_2() and emulate_cmpxchg_1_2()
 - drop {cmp}xchg_{relaxed,acquire,release) versions as they aren't needed for Xen
 - update __amoswap_generic() prototype and defintion: drop pre and post barries.
 - update emulate_xchg_1_2() prototype and definion: add lr_sfx, drop pre and post barries.
 - rename __xchg_generic to __xchg(), make __xchg as static inline function to be able to "#ifndef CONFIG_32BIT case 8:... " 
---
Changes in V5:
 - update the commit message.
 - drop ALIGN_DOWN().
 - update the definition of emulate_xchg_1_2(): 
   - lr.d -> lr.w, sc.d -> sc.w.
   - drop ret argument.
   - code style fixes around asm volatile.
   - update prototype.
   - use asm named operands.
   - rename local variables.
   - add comment above the macros
 - update the definition of __xchg_generic:
   - rename to __xchg()
   - transform it to static inline
   - code style fixes around switch()
   - update prototype.
 - redefine cmpxchg()
 - update emulate_cmpxchg_1_2():
   - update prototype
   - update local variables names and usage of them
   - use name asm operands.
   - add comment above the macros
 - drop pre and post, and use .aq,.rl, .aqrl suffixes.
 - drop {cmp}xchg_{relaxed, aquire, release} as they are not used by Xen.
 - drop unnessary details in comment above emulate_cmpxchg_1_2()
---
Changes in V4:
 - Code style fixes.
 - enforce in __xchg_*() has the same type for new and *ptr, also "\n"
   was removed at the end of asm instruction.
 - dependency from https://lore.kernel.org/xen-devel/cover.1706259490.git.federico.serafini@bugseng.com/
 - switch from ASSERT_UNREACHABLE to STATIC_ASSERT_UNREACHABLE().
 - drop xchg32(ptr, x) and xchg64(ptr, x) as they aren't used.
 - drop cmpxcg{32,64}_{local} as they aren't used.
 - introduce generic version of xchg_* and cmpxchg_*.
 - update the commit message.
---
Changes in V3:
 - update the commit message
 - add emulation of {cmp}xchg_... for 1 and 2 bytes types
---
Changes in V2:
 - update the comment at the top of the header.
 - change xen/lib.h to xen/bug.h.
 - sort inclusion of headers properly.
---
 xen/arch/riscv/include/asm/cmpxchg.h | 227 +++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/config.h  |   2 +
 2 files changed, 229 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h

diff --git a/xen/arch/riscv/include/asm/cmpxchg.h b/xen/arch/riscv/include/asm/cmpxchg.h
new file mode 100644
index 0000000000..9e78035dff
--- /dev/null
+++ b/xen/arch/riscv/include/asm/cmpxchg.h
@@ -0,0 +1,227 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2014 Regents of the University of California */
+
+#ifndef _ASM_RISCV_CMPXCHG_H
+#define _ASM_RISCV_CMPXCHG_H
+
+#include <xen/compiler.h>
+#include <xen/lib.h>
+
+#include <asm/fence.h>
+#include <asm/io.h>
+#include <asm/system.h>
+
+#define _amoswap_generic(ptr, new, ret, sfx) \
+    asm volatile ( \
+        " amoswap" sfx " %0, %2, %1" \
+        : "=r" (ret), "+A" (*(ptr)) \
+        : "r" (new) \
+        : "memory" );
+
+/*
+ * For LR and SC, the A extension requires that the address held in rs1 be
+ * naturally aligned to the size of the operand (i.e., eight-byte aligned
+ * for 64-bit words and four-byte aligned for 32-bit words).
+ * If the address is not naturally aligned, an address-misaligned exception
+ * or an access-fault exception will be generated.
+ *
+ * Thereby:
+ * - for 1-byte xchg access the containing word by clearing low two bits.
+ * - for 2-byte xchg access the containing word by clearing bit 1.
+ *
+ * If resulting 4-byte access is still misalgined, it will fault just as
+ * non-emulated 4-byte access would.
+ */
+#define emulate_xchg_1_2(ptr, new, lr_sfx, sc_sfx) \
+({ \
+    uint32_t *aligned_ptr; \
+    unsigned long alignment_mask = sizeof(*aligned_ptr) - sizeof(*(ptr)); \
+    unsigned int new_val_bit = \
+        ((unsigned long)(ptr) & alignment_mask) * BITS_PER_BYTE; \
+    unsigned long mask = \
+        GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_bit; \
+    unsigned int new_ = (new) << new_val_bit; \
+    unsigned int old; \
+    unsigned int scratch; \
+    \
+    aligned_ptr = (uint32_t *)((unsigned long)(ptr) & ~alignment_mask); \
+    \
+    asm volatile ( \
+        "0: lr.w" lr_sfx " %[old], %[ptr_]\n" \
+        "   andn  %[scratch], %[old], %[mask]\n" \
+        "   or   %[scratch], %[scratch], %z[new_]\n" \
+        "   sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \
+        "   bnez %[scratch], 0b\n" \
+        : [old] "=&r" (old), [scratch] "=&r" (scratch), \
+          [ptr_] "+A" (*aligned_ptr) \
+        : [new_] "rJ" (new_), [mask] "r" (mask) \
+        : "memory" ); \
+    \
+    (__typeof__(*(ptr)))((old & mask) >> new_val_bit); \
+})
+
+static always_inline unsigned long __xchg(volatile void *ptr, unsigned long new, int size)
+{
+    unsigned long ret;
+
+    switch ( size )
+    {
+    case 1:
+        ret = emulate_xchg_1_2((volatile uint8_t *)ptr, new, ".aq", ".aqrl");
+        break;
+    case 2:
+        ret = emulate_xchg_1_2((volatile uint16_t *)ptr, new, ".aq", ".aqrl");
+        break;
+    case 4:
+        _amoswap_generic((volatile uint32_t *)ptr, new, ret, ".w.aqrl");
+        break;
+#ifndef CONFIG_RISCV_32
+    case 8:
+        _amoswap_generic((volatile uint64_t *)ptr, new, ret, ".d.aqrl");
+        break;
+#endif
+    default:
+        STATIC_ASSERT_UNREACHABLE();
+    }
+
+    return ret;
+}
+
+#define xchg(ptr, x) \
+({ \
+    __typeof__(*(ptr)) n_ = (x); \
+    (__typeof__(*(ptr))) \
+        __xchg((ptr), (unsigned long)n_, sizeof(*(ptr))); \
+})
+
+#define _generic_cmpxchg(ptr, old, new, lr_sfx, sc_sfx) \
+ ({ \
+    unsigned int rc; \
+    unsigned long ret; \
+    unsigned long mask = GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0); \
+    asm volatile ( \
+        "0: lr" lr_sfx " %[ret], %[ptr_]\n" \
+        "   and  %[ret], %[ret], %[mask]\n" \
+        "   bne  %[ret], %z[old_], 1f\n" \
+        "   sc" sc_sfx " %[rc], %z[new_], %[ptr_]\n" \
+        "   bnez %[rc], 0b\n" \
+        "1:\n" \
+        : [ret] "=&r" (ret), [rc] "=&r" (rc), [ptr_] "+A" (*ptr) \
+        : [old_] "rJ" (old), [new_] "rJ" (new), [mask] "r" (mask)  \
+        : "memory" ); \
+    ret; \
+ })
+
+/*
+ * For LR and SC, the A extension requires that the address held in rs1 be
+ * naturally aligned to the size of the operand (i.e., eight-byte aligned
+ * for 64-bit words and four-byte aligned for 32-bit words).
+ * If the address is not naturally aligned, an address-misaligned exception
+ * or an access-fault exception will be generated.
+ *
+ * Thereby:
+ * - for 1-byte xchg access the containing word by clearing low two bits
+ * - for 2-byte xchg ccess the containing word by clearing first bit.
+ * 
+ * If resulting 4-byte access is still misalgined, it will fault just as
+ * non-emulated 4-byte access would.
+ *
+ * old_val was casted to unsigned long for cmpxchgptr()
+ */
+#define emulate_cmpxchg_1_2(ptr, old, new, lr_sfx, sc_sfx) \
+({ \
+    uint32_t *aligned_ptr; \
+    unsigned long alignment_mask = sizeof(*aligned_ptr) - sizeof(*(ptr)); \
+    uint8_t new_val_bit = \
+        ((unsigned long)(ptr) & alignment_mask) * BITS_PER_BYTE; \
+    unsigned long mask = \
+        GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_bit; \
+    unsigned int old_ = (old) << new_val_bit; \
+    unsigned int new_ = (new) << new_val_bit; \
+    unsigned int old_val; \
+    unsigned int scratch; \
+    \
+    aligned_ptr = (uint32_t *)((unsigned long)ptr & ~alignment_mask); \
+    \
+    asm volatile ( \
+        "0: lr.w" lr_sfx " %[scratch], %[ptr_]\n" \
+        "   and  %[old_val], %[scratch], %[mask]\n" \
+        "   bne  %[old_val], %z[old_], 1f\n" \
+        /* the following line is an equivalent to:
+         *     scratch = old_val & ~mask;
+         * And to elimanate one ( likely register ) input it was decided
+         * to use:
+         *     scratch = old_val ^ scratch
+         */ \
+        "   xor  %[scratch], %[old_val], %[scratch]\n" \
+        "   or   %[scratch], %[scratch], %z[new_]\n" \
+        "   sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \
+        "   bnez %[scratch], 0b\n" \
+        "1:\n" \
+        : [old_val] "=&r" (old_val), [scratch] "=&r" (scratch), \
+          [ptr_] "+A" (*aligned_ptr) \
+        : [old_] "rJ" (old_), [new_] "rJ" (new_), \
+          [mask] "r" (mask) \
+        : "memory" ); \
+    \
+    (__typeof__(*(ptr)))((unsigned long)old_val >> new_val_bit); \
+})
+
+/*
+ * Atomic compare and exchange.  Compare OLD with MEM, if identical,
+ * store NEW in MEM.  Return the initial value in MEM.  Success is
+ * indicated by comparing RETURN with OLD.
+ */
+static always_inline unsigned long __cmpxchg(volatile void *ptr,
+                                             unsigned long old,
+                                             unsigned long new,
+                                             int size)
+{
+    unsigned long ret;
+
+    switch ( size )
+    {
+    case 1:
+        ret = emulate_cmpxchg_1_2((volatile uint8_t *)ptr, old, new,
+                                  ".aq", ".aqrl");
+        break;
+    case 2:
+        ret = emulate_cmpxchg_1_2((volatile uint16_t *)ptr, old, new,
+                                   ".aq", ".aqrl");
+        break;
+    case 4:
+        ret = _generic_cmpxchg((volatile uint32_t *)ptr, old, new,
+                          ".w.aq", ".w.aqrl");
+        break;
+#ifndef CONFIG_32BIT
+    case 8:
+        ret = _generic_cmpxchg((volatile uint64_t *)ptr, old, new,
+                           ".d.aq", ".d.aqrl");
+        break;
+#endif
+    default:
+        STATIC_ASSERT_UNREACHABLE();
+    }
+
+    return ret;
+}
+
+#define cmpxchg(ptr, o, n) \
+({ \
+    __typeof__(*(ptr)) o_ = (o); \
+    __typeof__(*(ptr)) n_ = (n); \
+    (__typeof__(*(ptr))) \
+    __cmpxchg((ptr), (unsigned long)o_, (unsigned long)n_, \
+              sizeof(*(ptr))); \
+})
+
+#endif /* _ASM_RISCV_CMPXCHG_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/riscv/include/asm/config.h b/xen/arch/riscv/include/asm/config.h
index c5f93e6a01..50583aafdc 100644
--- a/xen/arch/riscv/include/asm/config.h
+++ b/xen/arch/riscv/include/asm/config.h
@@ -119,6 +119,8 @@
 
 #define BITS_PER_LLONG 64
 
+#define BITS_PER_BYTE 8
+
 /* xen_ulong_t is always 64 bits */
 #define BITS_PER_XEN_ULONG 64
 
-- 
2.43.0

[PATCH v7 09/19] xen/riscv: introduce io.h

[Oleksii Kurochko]

The header taken form Linux 6.4.0-rc1 and is based on
arch/riscv/include/asm/mmio.h with the following changes:
- drop forcing of endianess for read*(), write*() functions as
  no matter what CPU endianness, what endianness a particular device
  (and hence its MMIO region(s)) is using is entirely independent.
  Hence conversion, where necessary, needs to occur at a layer up.
  Another one reason to drop endianess conversion here is:
  https://patchwork.kernel.org/project/linux-riscv/patch/20190411115623.5749-3-hch@lst.de/
  One of the answers of the author of the commit:
    And we don't know if Linux will be around if that ever changes.
    The point is:
     a) the current RISC-V spec is LE only
     b) the current linux port is LE only except for this little bit
    There is no point in leaving just this bitrotting code around.  It
    just confuses developers, (very very slightly) slows down compiles
    and will bitrot.  It also won't be any significant help to a future
    developer down the road doing a hypothetical BE RISC-V Linux port.
- drop unused argument of __io_ar() macros.
- drop "#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}"
  as they are unnecessary.
- Adopt the Xen code style for this header, considering that significant changes
  are not anticipated in the future.
  In the event of any issues, adapting them to Xen style should be easily
  manageable.
- drop unnecessary  __r variables in macros read*_cpu()
- update inline assembler constraints for addr argument for
  __raw_read{b,w,l,q} and __raw_write{b,w,l,q} to tell a compiler that
 *addr will be accessed.
- add stubs for __raw_readq() and __raw_writeq() for RISCV_32

Addionally, to the header was added definions of ioremap_*().

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V7:
 - update the comment message in riscv/io.h at the top.
 - code style fixes.
 - back const in places where it should be.
---
Changes in V6:
 - drop unnecessary spaces and fix typos in the file comment.
 - s/CONFIG_64BIT/CONFIG_RISCV_32 as .d suffix for instruction doesn't exist for RV32.
 - add stubs for __raw_readq() and __raw_writeq() for RISCV_32
 - update inline assembler constraints for addr argument for __raw_read{b,w,l,q} and
   __raw_write{b,w,l,q} to tell compiler that *addr will be accessed.
 - s/u8/uint8_t
 - update the commit message
---
Changes in V5:
 - Xen code style related fixes
 - drop #define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}
 - drop cpu_to_le16()
 - remove unuused argument in _io_ar()
 - update the commit message 
 - drop unnessary __r variables in macros read*_cpu()
 - update the comments at the top of the header.
---
Changes in V4:
 - delete inner parentheses in macros.
 - s/u<N>/uint<N>.
---
Changes in V3:
 - re-sync with linux kernel
 - update the commit message
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/io.h | 168 ++++++++++++++++++++++++++++++++
 1 file changed, 168 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/io.h

diff --git a/xen/arch/riscv/include/asm/io.h b/xen/arch/riscv/include/asm/io.h
new file mode 100644
index 0000000000..8d9535e973
--- /dev/null
+++ b/xen/arch/riscv/include/asm/io.h
@@ -0,0 +1,168 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ *  The header taken form Linux 6.4.0-rc1 and is based on
+ *  arch/riscv/include/asm/mmio.h with the following changes:
+ *   - drop forcing of endianess for read*(), write*() functions as
+ *     no matter what CPU endianness, what endianness a particular device
+ *     (and hence its MMIO region(s)) is using is entirely independent.
+ *     Hence conversion, where necessary, needs to occur at a layer up.
+ *     Another one reason to drop endianess conversion is:
+ *     https://patchwork.kernel.org/project/linux-riscv/patch/20190411115623.5749-3-hch@lst.de/
+ *     One of the answers of the author of the commit:
+ *       And we don't know if Linux will be around if that ever changes.
+ *       The point is:
+ *        a) the current RISC-V spec is LE only
+ *        b) the current linux port is LE only except for this little bit
+ *       There is no point in leaving just this bitrotting code around.  It
+ *       just confuses developers, (very very slightly) slows down compiles
+ *      and will bitrot.  It also won't be any significant help to a future
+ *       developer down the road doing a hypothetical BE RISC-V Linux port.
+ *   - drop unused argument of __io_ar() macros.
+ *   - drop "#define _raw_{read,write}{b,w,l,q} _raw_{read,write}{b,w,l,q}"
+ *     as they are unnecessary.
+ *   - Adopt the Xen code style for this header, considering that significant
+ *     changes are not anticipated in the future.
+ *     In the event of any issues, adapting them to Xen style should be easily
+ *     manageable.
+ *   - drop unnecessary __r variables in macros read*_cpu()
+ *   - update inline assembler constraints for addr argument for
+ *     __raw_read{b,w,l,q} and __raw_write{b,w,l,q} to tell a compiler that
+ *     *addr will be accessed.
+ *
+ * Copyright (C) 1996-2000 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ * Copyright (C) 2014 Regents of the University of California
+ * Copyright (C) 2024 Vates
+ */
+
+#ifndef _ASM_RISCV_IO_H
+#define _ASM_RISCV_IO_H
+
+#include <asm/byteorder.h>
+
+/*
+ * The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't
+ * change the properties of memory regions.  This should be fixed by the
+ * upcoming platform spec.
+ */
+#define ioremap_nocache(addr, size) ioremap(addr, size)
+#define ioremap_wc(addr, size) ioremap(addr, size)
+#define ioremap_wt(addr, size) ioremap(addr, size)
+
+/* Generic IO read/write.  These perform native-endian accesses. */
+static inline void __raw_writeb(uint8_t val, volatile void __iomem *addr)
+{
+    asm volatile ( "sb %1, %0"
+                   : "=m" (*(volatile uint8_t __force *)addr) : "r" (val) );
+}
+
+static inline void __raw_writew(uint16_t val, volatile void __iomem *addr)
+{
+    asm volatile ( "sh %1, %0"
+                   : "=m" (*(volatile uint16_t __force *)addr) : "r" (val) );
+}
+
+static inline void __raw_writel(uint32_t val, volatile void __iomem *addr)
+{
+    asm volatile ( "sw %1, %0"
+                   : "=m" (*(volatile uint32_t __force *)addr) : "r" (val) );
+}
+
+static inline void __raw_writeq(uint64_t val, volatile void __iomem *addr)
+{
+#ifdef CONFIG_RISCV_32
+    BUILD_BUG_ON("unimplemented");
+#else
+    asm volatile ( "sd %1, %0"
+                   : "=m" (*(volatile uint64_t __force *)addr) : "r" (val) );
+#endif
+}
+
+static inline uint8_t __raw_readb(const volatile void __iomem *addr)
+{
+    uint8_t val;
+
+    asm volatile ( "lb %0, %1" : "=r" (val)
+                   : "m" (*(const volatile uint8_t __force *)addr) );
+    return val;
+}
+
+static inline uint16_t __raw_readw(const volatile void __iomem *addr)
+{
+    uint16_t val;
+
+    asm volatile ( "lh %0, %1" : "=r" (val)
+                   : "m" (*(const volatile uint16_t __force *)addr) );
+    return val;
+}
+
+static inline uint32_t __raw_readl(const volatile void __iomem *addr)
+{
+    uint32_t val;
+
+    asm volatile ( "lw %0, %1" : "=r" (val)
+                   : "m" (*(const volatile uint32_t __force *)addr) );
+    return val;
+}
+
+static inline uint64_t __raw_readq(const volatile void __iomem *addr)
+{
+    uint64_t val;
+
+#ifdef CONFIG_RISCV_32
+    BUILD_BUG_ON("unimplemented");
+#else
+    asm volatile ( "ld %0, %1" : "=r" (val)
+                   : "m" (*(const volatile uint64_t __force *)addr) );
+#endif
+
+    return val;
+}
+
+
+/*
+ * Unordered I/O memory access primitives.  These are even more relaxed than
+ * the relaxed versions, as they don't even order accesses between successive
+ * operations to the I/O regions.
+ */
+#define readb_cpu(c)        __raw_readb(c)
+#define readw_cpu(c)        __raw_readw(c)
+#define readl_cpu(c)        __raw_readl(c)
+#define readq_cpu(c)        __raw_readq(c)
+
+#define writeb_cpu(v, c)    __raw_writeb(v, c)
+#define writew_cpu(v, c)    __raw_writew(v, c)
+#define writel_cpu(v, c)    __raw_writel(v, c)
+#define writeq_cpu(v, c)    __raw_writeq(v, c)
+
+/*
+ * I/O memory access primitives. Reads are ordered relative to any
+ * following Normal memory access. Writes are ordered relative to any prior
+ * Normal memory access.  The memory barriers here are necessary as RISC-V
+ * doesn't define any ordering between the memory space and the I/O space.
+ */
+#define __io_br()   do { } while (0)
+#define __io_ar()   asm volatile ( "fence i,r" : : : "memory" );
+#define __io_bw()   asm volatile ( "fence w,o" : : : "memory" );
+#define __io_aw()   do { } while (0)
+
+#define readb(c) ({ uint8_t  v_; __io_br(); v_ = readb_cpu(c); __io_ar(); v_; })
+#define readw(c) ({ uint16_t v_; __io_br(); v_ = readw_cpu(c); __io_ar(); v_; })
+#define readl(c) ({ uint32_t v_; __io_br(); v_ = readl_cpu(c); __io_ar(); v_; })
+#define readq(c) ({ uint64_t v_; __io_br(); v_ = readq_cpu(c); __io_ar(); v_; })
+
+#define writeb(v, c)    ({ __io_bw(); writeb_cpu(v, c); __io_aw(); })
+#define writew(v, c)    ({ __io_bw(); writew_cpu(v, c); __io_aw(); })
+#define writel(v, c)    ({ __io_bw(); writel_cpu(v, c); __io_aw(); })
+#define writeq(v, c)    ({ __io_bw(); writeq_cpu(v, c); __io_aw(); })
+
+#endif /* _ASM_RISCV_IO_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0

[PATCH v7 10/19] xen/riscv: introduce atomic.h

[Oleksii Kurochko]

Initially the patch was introduced by Bobby, who takes the header from
Linux kernel.

The following changes were done on top of Bobby's changes:
 - atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated
   to use__*xchg_generic()
 - drop casts in write_atomic() as they are unnecessary
 - drop introduction of WRITE_ONCE() and READ_ONCE().
   Xen provides ACCESS_ONCE()
 - remove zero-length array access in read_atomic()
 - drop defines similar to pattern:
   #define atomic_add_return_relaxed   atomic_add_return_relaxed
 - move not RISC-V specific functions to asm-generic/atomics-ops.h
 - drop  atomic##prefix##_{cmp}xchg_{release, aquire, release}() as they
   are not used in Xen.
 - update the defintion of  atomic##prefix##_{cmp}xchg according to
   {cmp}xchg() implementation in Xen.

The current implementation is the same with 8e86f0b409a4
("arm64: atomics: fix use of acquire + release for full barrier
semantics") [1].
RISC-V could combine acquire and release into the SC
instructions and it could reduce a fence instruction to gain better
performance. Here is related description from RISC-V ISA 10.2
Load-Reserved/Store-Conditional Instructions:

 - .aq:   The LR/SC sequence can be given acquire semantics by
          setting the aq bit on the LR instruction.
 - .rl:   The LR/SC sequence can be given release semantics by
              setting the rl bit on the SC instruction.
 - .aqrl: Setting the aq bit on the LR instruction, and setting
          both the aq and the rl bit on the SC instruction makes
          the LR/SC sequence sequentially consistent, meaning that
          it cannot be reordered with earlier or later memory
          operations from the same hart.

 Software should not set the rl bit on an LR instruction unless
 the aq bit is also set, nor should software set the aq bit on an
 SC instruction unless the rl bit is also set. LR.rl and SC.aq
 instructions are not guaranteed to provide any stronger ordering
 than those with both bits clear, but may result in lower
 performance.

Also, I way of transforming ".rl + full barrier" to ".aqrl" was approved
by (the author of the RVWMO spec) [2]

[1] https://patchwork.kernel.org/project/linux-arm-kernel/patch/1391516953-14541-1-git-send-email-will.deacon@arm.com/
[2] https://lore.kernel.org/linux-riscv/41e01514-74ca-84f2-f5cc-2645c444fd8e@nvidia.com/

Signed-off-by: Bobby Eshleman <bobbyeshleman@gmail.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V7:
 - drop relaxed version of atomic ops as they are not used.
 - update the commit message
 - code style fixes
 - refactor functions write_atomic(), add_sized() to be able to use #ifdef CONFIG_RISCV_32 ... #endif
   for {write,read}q().
 - update ATOMIC_OPS to receive unary operator.
 - update the header on top of atomic-ops.h.
 - some minor movements of function inside atomic-ops.h header.
---
Changes in V6:
 - drop atomic##prefix##_{cmp}xchg_{release, aquire, relaxed} as they aren't used
   by Xen
 - code style fixes.
 - %s/__asm__ __volatile__/asm volatile
 - add explanational comments.
 - move inclusion of "#include <asm-generic/atomic-ops.h>" further down in atomic.h
   header.
---
Changes in V5:
 - fence.h changes were moved to separate patch as patches related to io.h and cmpxchg.h,
   which are dependecies for this patch, also needed changes in fence.h
 - remove accessing of zero-length array
 - drops cast in write_atomic()
 - drop introduction of WRITE_ONCE() and READ_ONCE().
 - drop defines similar to pattern #define atomic_add_return_relaxed   atomic_add_return_relaxed
 - Xen code style fixes
 - move not RISC-V specific functions to asm-generic/atomics-ops.h
---
Changes in V4:
 - do changes related to the updates of [PATCH v3 13/34] xen/riscv: introduce cmpxchg.h
 - drop casts in read_atomic_size(), write_atomic(), add_sized()
 - tabs -> spaces
 - drop #ifdef CONFIG_SMP ... #endif in fence.ha as it is simpler to handle NR_CPUS=1
   the same as NR_CPUS>1 with accepting less than ideal performance.
---
Changes in V3:
  - update the commit message
  - add SPDX for fence.h
  - code style fixes
  - Remove /* TODO: ... */ for add_sized macros. It looks correct to me.
  - re-order the patch
  - merge to this patch fence.h
---
Changes in V2:
 - Change an author of commit. I got this header from Bobby's old repo.
---
 xen/arch/riscv/include/asm/atomic.h  | 261 +++++++++++++++++++++++++++
 xen/include/asm-generic/atomic-ops.h |  97 ++++++++++
 2 files changed, 358 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/atomic.h
 create mode 100644 xen/include/asm-generic/atomic-ops.h

diff --git a/xen/arch/riscv/include/asm/atomic.h b/xen/arch/riscv/include/asm/atomic.h
new file mode 100644
index 0000000000..51574e7ce8
--- /dev/null
+++ b/xen/arch/riscv/include/asm/atomic.h
@@ -0,0 +1,261 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Taken and modified from Linux.
+ *
+ * The following changes were done:
+ * - * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated
+ *     to use__*xchg_generic()
+ * - drop casts in write_atomic() as they are unnecessary
+ * - drop introduction of WRITE_ONCE() and READ_ONCE().
+ *   Xen provides ACCESS_ONCE()
+ * - remove zero-length array access in read_atomic()
+ * - drop defines similar to pattern
+ *   #define atomic_add_return_relaxed   atomic_add_return_relaxed
+ * - move not RISC-V specific functions to asm-generic/atomics-ops.h
+ * 
+ * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
+ * Copyright (C) 2012 Regents of the University of California
+ * Copyright (C) 2017 SiFive
+ * Copyright (C) 2024 Vates SAS
+ */
+
+#ifndef _ASM_RISCV_ATOMIC_H
+#define _ASM_RISCV_ATOMIC_H
+
+#include <xen/atomic.h>
+
+#include <asm/cmpxchg.h>
+#include <asm/fence.h>
+#include <asm/io.h>
+#include <asm/system.h>
+
+void __bad_atomic_size(void);
+
+/*
+ * Legacy from Linux kernel. For some reason they wanted to have ordered
+ * read/write access. Thereby read* is used instead of read*_cpu()
+ */
+static always_inline void read_atomic_size(const volatile void *p,
+                                           void *res,
+                                           unsigned int size)
+{
+    switch ( size )
+    {
+    case 1: *(uint8_t *)res = readb(p); break;
+    case 2: *(uint16_t *)res = readw(p); break;
+    case 4: *(uint32_t *)res = readl(p); break;
+#ifndef CONFIG_RISCV_32
+    case 8: *(uint32_t *)res = readq(p); break;
+#endif
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define read_atomic(p) ({                                   \
+    union { typeof(*(p)) val; char c[sizeof(*(p))]; } x_;   \
+    read_atomic_size(p, x_.c, sizeof(*(p)));                \
+    x_.val;                                                 \
+})
+
+static always_inline void _write_atomic(volatile void *p,
+                                       unsigned long x, unsigned int size)
+{
+    switch ( size )
+    {
+    case 1: writeb(x, p); break;
+    case 2: writew(x, p); break;
+    case 4: writel(x, p); break;
+#ifndef CONFIG_RISCV_32
+    case 8: writeq(x, p); break;
+#endif
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define write_atomic(p, x)                              \
+({                                                      \
+    typeof(*(p)) x_ = (x);                              \
+    _write_atomic((p), x_, sizeof(*(p)));               \
+    x_;                                                 \
+})
+
+static always_inline void _add_sized(volatile void *p,
+                                     unsigned long x, unsigned int size)
+{
+    switch ( size )
+    {
+    case 1: writeb(read_atomic((volatile uint8_t *)p) + x, p); break;
+    case 2: writew(read_atomic((volatile uint16_t *)p) + x, p); break;
+    case 4: writel(read_atomic((volatile uint32_t *)p) + x, p); break;
+#ifndef CONFIG_RISCV_32
+    case 8: writeq(read_atomic((volatile uint64_t *)p) + x, p); break;
+#endif
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define add_sized(p, x)                                 \
+({                                                      \
+    typeof(*(p)) x_ = (x);                              \
+    _add_sized((p), x_, sizeof(*(p)));                  \
+})
+
+#define __atomic_acquire_fence() \
+    asm volatile ( RISCV_ACQUIRE_BARRIER "" ::: "memory" )
+
+#define __atomic_release_fence() \
+    asm volatile ( RISCV_RELEASE_BARRIER "" ::: "memory" )
+
+/*
+ * First, the atomic ops that have no ordering constraints and therefor don't
+ * have the AQ or RL bits set.  These don't return anything, so there's only
+ * one version to worry about.
+ */
+#define ATOMIC_OP(op, asm_op, unary_op, asm_type, c_type, prefix)  \
+static inline                                               \
+void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \
+{                                                           \
+    asm volatile (                                          \
+        "   amo" #asm_op "." #asm_type " zero, %1, %0"      \
+        : "+A" (v->counter)                                 \
+        : "r" (unary_op i)                                  \
+        : "memory" );                                       \
+}                                                           \
+
+/*
+ * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
+ * last argument for ATOMIC_OP isn't used.
+ */
+#define ATOMIC_OPS(op, asm_op, unary_op)                    \
+        ATOMIC_OP (op, asm_op, unary_op, w, int,   )
+
+ATOMIC_OPS(add, add, +)
+ATOMIC_OPS(sub, add, -)
+ATOMIC_OPS(and, and, +)
+ATOMIC_OPS( or,  or, +)
+ATOMIC_OPS(xor, xor, +)
+
+#undef ATOMIC_OP
+#undef ATOMIC_OPS
+
+#include <asm-generic/atomic-ops.h>
+
+/*
+ * Atomic ops that have ordered variant.
+ * There's two flavors of these: the arithmatic ops have both fetch and return
+ * versions, while the logical ops only have fetch versions.
+ */
+#define ATOMIC_FETCH_OP(op, asm_op, unary_op, asm_type, c_type, prefix) \
+static inline                                                       \
+c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t *v) \
+{                                                                   \
+    register c_type ret;                                            \
+    asm volatile (                                                  \
+        "   amo" #asm_op "." #asm_type ".aqrl  %1, %2, %0"          \
+        : "+A" (v->counter), "=r" (ret)                             \
+        : "r" (unary_op i)                                          \
+        : "memory" );                                               \
+    return ret;                                                     \
+}
+
+#define ATOMIC_OP_RETURN(op, asm_op, c_op, unary_op, asm_type, c_type, prefix) \
+static inline                                                           \
+c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t *v)  \
+{                                                                       \
+        return atomic##prefix##_fetch_##op(i, v) c_op (unary_op i);     \
+}
+
+/*
+ * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
+ * last argument of ATOMIC_FETCH_OP, ATOMIC_OP_RETURN isn't used.
+ */
+#define ATOMIC_OPS(op, asm_op, c_op, unary_op)                          \
+        ATOMIC_FETCH_OP( op, asm_op,       unary_op, w, int,   )        \
+        ATOMIC_OP_RETURN(op, asm_op, c_op, unary_op, w, int,   )
+
+ATOMIC_OPS(add, add, +, +)
+ATOMIC_OPS(sub, add, +, -)
+
+#undef ATOMIC_OPS
+
+#define ATOMIC_OPS(op, asm_op, unary_op) \
+        ATOMIC_FETCH_OP(op, asm_op, unary_op, w, int,   )
+
+ATOMIC_OPS(and, and, +)
+ATOMIC_OPS( or,  or, +)
+ATOMIC_OPS(xor, xor, +)
+
+#undef ATOMIC_OPS
+
+#undef ATOMIC_FETCH_OP
+#undef ATOMIC_OP_RETURN
+
+/* This is required to provide a full barrier on success. */
+static inline int atomic_add_unless(atomic_t *v, int a, int u)
+{
+    int prev, rc;
+
+    asm volatile (
+        "0: lr.w     %[p],  %[c]\n"
+        "   beq      %[p],  %[u], 1f\n"
+        "   add      %[rc], %[p], %[a]\n"
+        "   sc.w.aqrl  %[rc], %[rc], %[c]\n"
+        "   bnez     %[rc], 0b\n"
+        "1:\n"
+        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
+        : [a] "r" (a), [u] "r" (u)
+        : "memory");
+    return prev;
+}
+
+static inline int atomic_sub_if_positive(atomic_t *v, int offset)
+{
+    int prev, rc;
+
+    asm volatile (
+        "0: lr.w     %[p],  %[c]\n"
+        "   sub      %[rc], %[p], %[o]\n"
+        "   bltz     %[rc], 1f\n"
+        "   sc.w.aqrl  %[rc], %[rc], %[c]\n"
+        "   bnez     %[rc], 0b\n"
+        "1:\n"
+        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
+        : [o] "r" (offset)
+        : "memory" );
+    return prev - offset;
+}
+
+/*
+ * atomic_{cmp,}xchg is required to have exactly the same ordering semantics as
+ * {cmp,}xchg and the operations that return.
+ */
+#define ATOMIC_OP(c_t, prefix, size)                            \
+static inline                                                   \
+c_t atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n)         \
+{                                                               \
+    return __xchg(&v->counter, n, size);                        \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n) \
+{                                                               \
+    return __cmpxchg(&v->counter, o, n, size);                  \
+}
+
+#define ATOMIC_OPS() \
+    ATOMIC_OP(int,   , 4)
+
+ATOMIC_OPS()
+
+#undef ATOMIC_OPS
+#undef ATOMIC_OP
+
+#endif /* _ASM_RISCV_ATOMIC_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-generic/atomic-ops.h b/xen/include/asm-generic/atomic-ops.h
new file mode 100644
index 0000000000..98dd907942
--- /dev/null
+++ b/xen/include/asm-generic/atomic-ops.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * The header provides default implementations for every xen/atomic.h-provided
+ * forward inline declaration that can be synthesized from other atomic
+ * functions or being created from scratch.
+ */
+#ifndef _ASM_GENERIC_ATOMIC_OPS_H_
+#define _ASM_GENERIC_ATOMIC_OPS_H_
+
+#include <xen/atomic.h>
+#include <xen/lib.h>
+
+#ifndef ATOMIC_READ
+static inline int atomic_read(const atomic_t *v)
+{
+    return ACCESS_ONCE(v->counter);
+}
+#endif
+
+#ifndef _ATOMIC_READ
+static inline int _atomic_read(atomic_t v)
+{
+    return v.counter;
+}
+#endif
+
+#ifndef ATOMIC_SET
+static inline void atomic_set(atomic_t *v, int i)
+{
+    ACCESS_ONCE(v->counter) = i;
+}
+#endif
+
+#ifndef _ATOMIC_SET
+static inline void _atomic_set(atomic_t *v, int i)
+{
+    v->counter = i;
+}
+#endif
+
+#ifndef ATOMIC_SUB_AND_TEST
+static inline int atomic_sub_and_test(int i, atomic_t *v)
+{
+    return atomic_sub_return(i, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_INC_AND_TEST
+static inline int atomic_inc_and_test(atomic_t *v)
+{
+    return atomic_add_return(1, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_INC
+static inline void atomic_inc(atomic_t *v)
+{
+    atomic_add(1, v);
+}
+#endif
+
+#ifndef ATOMIC_INC_RETURN
+static inline int atomic_inc_return(atomic_t *v)
+{
+    return atomic_add_return(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC
+static inline void atomic_dec(atomic_t *v)
+{
+    atomic_sub(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC_RETURN
+static inline int atomic_dec_return(atomic_t *v)
+{
+    return atomic_sub_return(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC_AND_TEST
+static inline int atomic_dec_and_test(atomic_t *v)
+{
+    return atomic_sub_return(1, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_ADD_NEGATIVE
+static inline int atomic_add_negative(int i, atomic_t *v)
+{
+    return atomic_add_return(i, v) < 0;
+}
+#endif
+
+#endif /* _ASM_GENERIC_ATOMIC_OPS_H_ */
-- 
2.43.0

[PATCH v7 11/19] xen/riscv: introduce monitor.h

[Oleksii Kurochko]

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4-V7:
 - Nothing changed. Only rebase.
---
Changes in V3:
 - new patch.
---
 xen/arch/riscv/include/asm/monitor.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/monitor.h

diff --git a/xen/arch/riscv/include/asm/monitor.h b/xen/arch/riscv/include/asm/monitor.h
new file mode 100644
index 0000000000..f4fe2c0690
--- /dev/null
+++ b/xen/arch/riscv/include/asm/monitor.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_MONITOR_H__
+#define __ASM_RISCV_MONITOR_H__
+
+#include <xen/bug.h>
+
+#include <asm-generic/monitor.h>
+
+struct domain;
+
+static inline uint32_t arch_monitor_get_capabilities(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+#endif /* __ASM_RISCV_MONITOR_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0

[PATCH v7 12/19] xen/riscv: add definition of __read_mostly

[Oleksii Kurochko]

The definition of __read_mostly should be removed in:
https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

The patch introduces it in arch-specific header to not
block enabling of full Xen build for RISC-V.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.

[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/
---
Changes in V4-V7:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/cache.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/riscv/include/asm/cache.h b/xen/arch/riscv/include/asm/cache.h
index 69573eb051..94bd94db53 100644
--- a/xen/arch/riscv/include/asm/cache.h
+++ b/xen/arch/riscv/include/asm/cache.h
@@ -3,4 +3,6 @@
 #ifndef _ASM_RISCV_CACHE_H
 #define _ASM_RISCV_CACHE_H
 
+#define __read_mostly __section(".data.read_mostly")
+
 #endif /* _ASM_RISCV_CACHE_H */
-- 
2.43.0

[PATCH v7 13/19] xen/riscv: add required things to current.h

[Oleksii Kurochko]

Add minimal requied things to be able to build full Xen.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5/V6/V7:
 - Nothing changed. Only rebase.
---
Changes in V4:
 - BUG() was changed to BUG_ON("unimplemented");
 - Change "xen/bug.h" to "xen/lib.h" as BUG_ON is defined in xen/lib.h.
 - Add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V3:
 - add SPDX
 - drop a forward declaration of struct vcpu;
 - update guest_cpu_user_regs() macros
 - replace get_processor_id with smp_processor_id
 - update the commit message
 - code style fixes
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/current.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/xen/arch/riscv/include/asm/current.h b/xen/arch/riscv/include/asm/current.h
index d84f15dc50..aedb6dc732 100644
--- a/xen/arch/riscv/include/asm/current.h
+++ b/xen/arch/riscv/include/asm/current.h
@@ -3,6 +3,21 @@
 #ifndef __ASM_CURRENT_H
 #define __ASM_CURRENT_H
 
+#include <xen/lib.h>
+#include <xen/percpu.h>
+#include <asm/processor.h>
+
+#ifndef __ASSEMBLY__
+
+/* Which VCPU is "current" on this PCPU. */
+DECLARE_PER_CPU(struct vcpu *, curr_vcpu);
+
+#define current            this_cpu(curr_vcpu)
+#define set_current(vcpu)  do { current = (vcpu); } while (0)
+#define get_cpu_current(cpu)  per_cpu(curr_vcpu, cpu)
+
+#define guest_cpu_user_regs() ({ BUG_ON("unimplemented"); NULL; })
+
 #define switch_stack_and_jump(stack, fn) do {               \
     asm volatile (                                          \
             "mv sp, %0\n"                                   \
@@ -10,4 +25,8 @@
     unreachable();                                          \
 } while ( false )
 
+#define get_per_cpu_offset() __per_cpu_offset[smp_processor_id()]
+
+#endif /* __ASSEMBLY__ */
+
 #endif /* __ASM_CURRENT_H */
-- 
2.43.0

[PATCH v7 14/19] xen/riscv: add minimal stuff to page.h to build full Xen

[Oleksii Kurochko]

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5-V7:
 - Nothing changed. Only rebase.
---
Changes in V4:
---
 - Change message -> subject in "Changes in V3"
 - s/BUG/BUG_ON("...")
 - Do proper rebase ( pfn_to_paddr() and paddr_to_pfn() aren't removed ).
---
Changes in V3:
 - update the commit subject
 - add implemetation of PAGE_HYPERVISOR macros
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
 - drop definition of pfn_to_addr, and paddr_to_pfn in <asm/mm.h>
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/page.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/xen/arch/riscv/include/asm/page.h b/xen/arch/riscv/include/asm/page.h
index 95074e29b3..c831e16417 100644
--- a/xen/arch/riscv/include/asm/page.h
+++ b/xen/arch/riscv/include/asm/page.h
@@ -6,6 +6,7 @@
 #ifndef __ASSEMBLY__
 
 #include <xen/const.h>
+#include <xen/bug.h>
 #include <xen/types.h>
 
 #include <asm/mm.h>
@@ -32,6 +33,10 @@
 #define PTE_LEAF_DEFAULT            (PTE_VALID | PTE_READABLE | PTE_WRITABLE)
 #define PTE_TABLE                   (PTE_VALID)
 
+#define PAGE_HYPERVISOR_RW          (PTE_VALID | PTE_READABLE | PTE_WRITABLE)
+
+#define PAGE_HYPERVISOR             PAGE_HYPERVISOR_RW
+
 /* Calculate the offsets into the pagetables for a given VA */
 #define pt_linear_offset(lvl, va)   ((va) >> XEN_PT_LEVEL_SHIFT(lvl))
 
@@ -62,6 +67,20 @@ static inline bool pte_is_valid(pte_t p)
     return p.pte & PTE_VALID;
 }
 
+static inline void invalidate_icache(void)
+{
+    BUG_ON("unimplemented");
+}
+
+#define clear_page(page) memset((void *)(page), 0, PAGE_SIZE)
+#define copy_page(dp, sp) memcpy(dp, sp, PAGE_SIZE)
+
+/* TODO: Flush the dcache for an entire page. */
+static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache)
+{
+    BUG_ON("unimplemented");
+}
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_RISCV_PAGE_H */
-- 
2.43.0

[PATCH v7 15/19] xen/riscv: add minimal stuff to mm.h to build full Xen

[Oleksii Kurochko]

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V7:
 - update argument type of maddr_to_virt() function: unsigned long -> paddr_t
 - rename argument of PFN_ORDER(): pfn -> pg.
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V6:
 - drop __virt_to_maddr() ( transform to macro ) and __maddr_to_virt ( rename to maddr_to_virt ).
 - parenthesize va in definition of vmap_to_mfn().
 - Code style fixes. 
---
Changes in V5:
 - update the comment around "struct domain *domain;" : zero -> NULL
 - fix ident. for unsigned long val;
 - put page_to_virt() and virt_to_page() close to each other.
 - drop unnessary leading underscore
 - drop a space before the comment: /* Count of uses of this frame as its current type. */
 - drop comment about a page 'not as a shadow'. it is not necessary for RISC-V
---
Changes in V4:
 - update an argument name of PFN_ORDERN macros.
 - drop pad at the end of 'struct page_info'.
 - Change message -> subject in "Changes in V3"
 - delete duplicated macros from riscv/mm.h
 - fix identation in struct page_info
 - align comment for PGC_ macros
 - update definitions of domain_set_alloc_bitsize() and domain_clamp_alloc_bitsize()
 - drop unnessary comments.
 - s/BUG/BUG_ON("...")
 - define __virt_to_maddr, __maddr_to_virt as stubs
 - add inclusion of xen/mm-frame.h for mfn_x and others
 - include "xen/mm.h" instead of "asm/mm.h" to fix compilation issues:
	 In file included from arch/riscv/setup.c:7:
	./arch/riscv/include/asm/mm.h:60:28: error: field 'list' has incomplete type
	   60 |     struct page_list_entry list;
	      |                            ^~~~
	./arch/riscv/include/asm/mm.h:81:43: error: 'MAX_ORDER' undeclared here (not in a function)
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
	      |                                           ^~~~~~~~~
	./arch/riscv/include/asm/mm.h:81:31: error: bit-field 'first_dirty' width not an integer constant
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
 - Define __virt_to_mfn() and __mfn_to_virt() using maddr_to_mfn() and mfn_to_maddr().
---
Changes in V3:
 - update the commit title
 - introduce DIRECTMAP_VIRT_START.
 - drop changes related pfn_to_paddr() and paddr_to_pfn as they were remvoe in
   [PATCH v2 32/39] xen/riscv: add minimal stuff to asm/page.h to build full Xen
 - code style fixes.
 - drop get_page_nr  and put_page_nr as they don't need for time being
 - drop CONFIG_STATIC_MEMORY related things
 - code style fixes
---
Changes in V2:
 - define stub for arch_get_dma_bitsize(void)
---
 xen/arch/riscv/include/asm/mm.h | 240 ++++++++++++++++++++++++++++++++
 xen/arch/riscv/mm.c             |   2 +-
 xen/arch/riscv/setup.c          |   2 +-
 3 files changed, 242 insertions(+), 2 deletions(-)

diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
index 07c7a0abba..cc4a07a71c 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -3,11 +3,246 @@
 #ifndef _ASM_RISCV_MM_H
 #define _ASM_RISCV_MM_H
 
+#include <public/xen.h>
+#include <xen/bug.h>
+#include <xen/mm-frame.h>
+#include <xen/pdx.h>
+#include <xen/types.h>
+
 #include <asm/page-bits.h>
 
 #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
 #define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
 
+#define paddr_to_pdx(pa)    mfn_to_pdx(maddr_to_mfn(pa))
+#define gfn_to_gaddr(gfn)   pfn_to_paddr(gfn_x(gfn))
+#define gaddr_to_gfn(ga)    _gfn(paddr_to_pfn(ga))
+#define mfn_to_maddr(mfn)   pfn_to_paddr(mfn_x(mfn))
+#define maddr_to_mfn(ma)    _mfn(paddr_to_pfn(ma))
+#define vmap_to_mfn(va)     maddr_to_mfn(virt_to_maddr((vaddr_t)(va)))
+#define vmap_to_page(va)    mfn_to_page(vmap_to_mfn(va))
+
+static inline void *maddr_to_virt(paddr_t ma)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+#define virt_to_maddr(va) ({ BUG_ON("unimplemented"); 0; })
+
+/* Convert between Xen-heap virtual addresses and machine frame numbers. */
+#define __virt_to_mfn(va)  mfn_x(maddr_to_mfn(virt_to_maddr(va)))
+#define __mfn_to_virt(mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn)))
+
+/*
+ * We define non-underscored wrappers for above conversion functions.
+ * These are overriden in various source files while underscored version
+ * remain intact.
+ */
+#define virt_to_mfn(va)     __virt_to_mfn(va)
+#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+
+struct page_info
+{
+    /* Each frame can be threaded onto a doubly-linked list. */
+    struct page_list_entry list;
+
+    /* Reference count and various PGC_xxx flags and fields. */
+    unsigned long count_info;
+
+    /* Context-dependent fields follow... */
+    union {
+        /* Page is in use: ((count_info & PGC_count_mask) != 0). */
+        struct {
+            /* Type reference count and various PGT_xxx flags and fields. */
+            unsigned long type_info;
+        } inuse;
+
+        /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
+        union {
+            struct {
+                /*
+                 * Index of the first *possibly* unscrubbed page in the buddy.
+                 * One more bit than maximum possible order to accommodate
+                 * INVALID_DIRTY_IDX.
+                 */
+#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1)
+                unsigned long first_dirty:MAX_ORDER + 1;
+
+                /* Do TLBs need flushing for safety before next page use? */
+                bool need_tlbflush:1;
+
+#define BUDDY_NOT_SCRUBBING    0
+#define BUDDY_SCRUBBING        1
+#define BUDDY_SCRUB_ABORT      2
+                unsigned long scrub_state:2;
+            };
+
+            unsigned long val;
+        } free;
+    } u;
+
+    union {
+        /* Page is in use */
+        struct {
+            /* Owner of this page (NULL if page is anonymous). */
+            struct domain *domain;
+        } inuse;
+
+        /* Page is on a free list. */
+        struct {
+            /* Order-size of the free chunk this page is the head of. */
+            unsigned int order;
+        } free;
+    } v;
+
+    union {
+        /*
+         * Timestamp from 'TLB clock', used to avoid extra safety flushes.
+         * Only valid for: a) free pages, and b) pages with zero type count
+         */
+        uint32_t tlbflush_timestamp;
+    };
+};
+
+#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
+
+/* PDX of the first page in the frame table. */
+extern unsigned long frametable_base_pdx;
+
+/* Convert between machine frame numbers and page-info structures. */
+#define mfn_to_page(mfn)                                            \
+    (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
+#define page_to_mfn(pg)                                             \
+    pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)
+
+static inline void *page_to_virt(const struct page_info *pg)
+{
+    return mfn_to_virt(mfn_x(page_to_mfn(pg)));
+}
+
+/* Convert between Xen-heap virtual addresses and page-info structures. */
+static inline struct page_info *virt_to_page(const void *v)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+/*
+ * Common code requires get_page_type and put_page_type.
+ * We don't care about typecounts so we just do the minimum to make it
+ * happy.
+ */
+static inline int get_page_type(struct page_info *page, unsigned long type)
+{
+    return 1;
+}
+
+static inline void put_page_type(struct page_info *page)
+{
+}
+
+static inline void put_page_and_type(struct page_info *page)
+{
+    put_page_type(page);
+    put_page(page);
+}
+
+/*
+ * RISC-V does not have an M2P, but common code expects a handful of
+ * M2P-related defines and functions. Provide dummy versions of these.
+ */
+#define INVALID_M2P_ENTRY        (~0UL)
+#define SHARED_M2P_ENTRY         (~0UL - 1UL)
+#define SHARED_M2P(_e)           ((_e) == SHARED_M2P_ENTRY)
+
+#define set_gpfn_from_mfn(mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0)
+#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
+
+#define PDX_GROUP_SHIFT (PAGE_SHIFT + VPN_BITS)
+
+static inline unsigned long domain_get_maximum_gpfn(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+/*
+ * On RISCV, all the RAM is currently direct mapped in Xen.
+ * Hence return always true.
+ */
+static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
+{
+    return true;
+}
+
+#define PG_shift(idx)   (BITS_PER_LONG - (idx))
+#define PG_mask(x, idx) (x ## UL << PG_shift(idx))
+
+#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
+#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
+#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
+
+/* Count of uses of this frame as its current type. */
+#define PGT_count_width   PG_shift(2)
+#define PGT_count_mask    ((1UL << PGT_count_width) - 1)
+
+/*
+ * Page needs to be scrubbed. Since this bit can only be set on a page that is
+ * free (i.e. in PGC_state_free) we can reuse PGC_allocated bit.
+ */
+#define _PGC_need_scrub   _PGC_allocated
+#define PGC_need_scrub    PGC_allocated
+
+/* Cleared when the owning guest 'frees' this page. */
+#define _PGC_allocated    PG_shift(1)
+#define PGC_allocated     PG_mask(1, 1)
+/* Page is Xen heap? */
+#define _PGC_xen_heap     PG_shift(2)
+#define PGC_xen_heap      PG_mask(1, 2)
+/* Page is broken? */
+#define _PGC_broken       PG_shift(7)
+#define PGC_broken        PG_mask(1, 7)
+/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */
+#define PGC_state         PG_mask(3, 9)
+#define PGC_state_inuse   PG_mask(0, 9)
+#define PGC_state_offlining PG_mask(1, 9)
+#define PGC_state_offlined PG_mask(2, 9)
+#define PGC_state_free    PG_mask(3, 9)
+#define page_state_is(pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st)
+
+/* Count of references to this frame. */
+#define PGC_count_width   PG_shift(9)
+#define PGC_count_mask    ((1UL << PGC_count_width) - 1)
+
+#define _PGC_extra        PG_shift(10)
+#define PGC_extra         PG_mask(1, 10)
+
+#define is_xen_heap_page(page) ((page)->count_info & PGC_xen_heap)
+#define is_xen_heap_mfn(mfn) \
+    (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn)))
+
+#define is_xen_fixed_mfn(mfn)                                   \
+    ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) &&   \
+     (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1)))
+
+#define page_get_owner(p)    (p)->v.inuse.domain
+#define page_set_owner(p, d) ((p)->v.inuse.domain = (d))
+
+/* TODO: implement */
+#define mfn_valid(mfn) ({ (void)(mfn); 0; })
+
+#define domain_set_alloc_bitsize(d) ((void)(d))
+#define domain_clamp_alloc_bitsize(d, b) ((void)(d), (b))
+
+#define PFN_ORDER(pg) ((pg)->v.free.order)
+
 extern unsigned char cpu0_boot_stack[];
 
 void setup_initial_pagetables(void);
@@ -20,4 +255,9 @@ unsigned long calc_phys_offset(void);
 
 void turn_on_mmu(unsigned long ra);
 
+static inline unsigned int arch_get_dma_bitsize(void)
+{
+    return 32; /* TODO */
+}
+
 #endif /* _ASM_RISCV_MM_H */
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index 053f043a3d..fe3a43be20 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -5,12 +5,12 @@
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/macros.h>
+#include <xen/mm.h>
 #include <xen/pfn.h>
 
 #include <asm/early_printk.h>
 #include <asm/csr.h>
 #include <asm/current.h>
-#include <asm/mm.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 6593f601c1..98a94c4c48 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -2,9 +2,9 @@
 
 #include <xen/compile.h>
 #include <xen/init.h>
+#include <xen/mm.h>
 
 #include <asm/early_printk.h>
-#include <asm/mm.h>
 
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
-- 
2.43.0

[PATCH v7 16/19] xen/riscv: introduce vm_event_*() functions

[Oleksii Kurochko]

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5-V7:
 - Only rebase was done.
---
Changes in V4:
  - New patch.
---
 xen/arch/riscv/Makefile   |  1 +
 xen/arch/riscv/vm_event.c | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+)
 create mode 100644 xen/arch/riscv/vm_event.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 2fefe14e7c..1ed1a8369b 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
 obj-y += traps.o
+obj-y += vm_event.o
 
 $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
diff --git a/xen/arch/riscv/vm_event.c b/xen/arch/riscv/vm_event.c
new file mode 100644
index 0000000000..bb1fc73bc1
--- /dev/null
+++ b/xen/arch/riscv/vm_event.c
@@ -0,0 +1,19 @@
+#include <xen/bug.h>
+
+struct vm_event_st;
+struct vcpu;
+
+void vm_event_fill_regs(struct vm_event_st *req)
+{
+    BUG_ON("unimplemented");
+}
+
+void vm_event_set_registers(struct vcpu *v, struct vm_event_st *rsp)
+{
+    BUG_ON("unimplemented");
+}
+
+void vm_event_monitor_next_interrupt(struct vcpu *v)
+{
+    /* Not supported on RISCV. */
+}
-- 
2.43.0

[PATCH v7 17/19] xen/riscv: add minimal amount of stubs to build full Xen

[Oleksii Kurochko]

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V7:
 - Only rebase was done.
---
Changes in V6:
 - update the commit in stubs.c around /* ... common/irq.c ... */
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5:
 - drop unrelated changes
 - assert_failed("unimplmented...") change to BUG_ON()
---
Changes in V4:
  - added new stubs which are necessary for compilation after rebase: __cpu_up(), __cpu_disable(), __cpu_die()
    from smpboot.c
  - back changes related to printk() in early_printk() as they should be removed in the next patch to avoid
    compilation error.
  - update definition of cpu_khz: __read_mostly -> __ro_after_init.
  - drop vm_event_reset_vmtrace(). It is defibed in asm-generic/vm_event.h.
  - move vm_event_*() functions from stubs.c to riscv/vm_event.c.
  - s/BUG/BUG_ON("unimplemented") in stubs.c
  - back irq_actor_none() and irq_actor_none() as common/irq.c isn't compiled at this moment,
    so this function are needed to avoid compilation error.
  - defined max_page to avoid compilation error, it will be removed as soon as common/page_alloc.c will
    be compiled.
---
Changes in V3:
 - code style fixes.
 - update attribute for frametable_base_pdx  and frametable_virt_end to __ro_after_init.
   insteaf of read_mostly.
 - use BUG() instead of assert_failed/WARN for newly introduced stubs.
 - drop "#include <public/vm_event.h>" in stubs.c and use forward declaration instead.
 - drop ack_node() and end_node() as they aren't used now.
---
Changes in V2:
 - define udelay stub
 - remove 'select HAS_PDX' from RISC-V Kconfig because of
   https://lore.kernel.org/xen-devel/20231006144405.1078260-1-andrew.cooper3@citrix.com/
---
 xen/arch/riscv/Makefile |   1 +
 xen/arch/riscv/mm.c     |  50 +++++
 xen/arch/riscv/setup.c  |   8 +
 xen/arch/riscv/stubs.c  | 439 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/riscv/traps.c  |  25 +++
 5 files changed, 523 insertions(+)
 create mode 100644 xen/arch/riscv/stubs.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 1ed1a8369b..60afbc0ad9 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -4,6 +4,7 @@ obj-y += mm.o
 obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
+obj-y += stubs.o
 obj-y += traps.o
 obj-y += vm_event.o
 
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index fe3a43be20..2c3fb7d72e 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/cache.h>
 #include <xen/compiler.h>
 #include <xen/init.h>
@@ -14,6 +15,9 @@
 #include <asm/page.h>
 #include <asm/processor.h>
 
+unsigned long __ro_after_init frametable_base_pdx;
+unsigned long __ro_after_init frametable_virt_end;
+
 struct mmu_desc {
     unsigned int num_levels;
     unsigned int pgtbl_count;
@@ -294,3 +298,49 @@ unsigned long __init calc_phys_offset(void)
     phys_offset = load_start - XEN_VIRT_START;
     return phys_offset;
 }
+
+void put_page(struct page_info *page)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long get_upper_mfn_bound(void)
+{
+    /* No memory hotplug yet, so current memory limit is the final one. */
+    return max_page - 1;
+}
+
+void arch_dump_shared_mem_info(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int populate_pt_range(unsigned long virt, unsigned long nr_mfns)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int xenmem_add_to_physmap_one(struct domain *d, unsigned int space,
+                              union add_to_physmap_extra extra,
+                              unsigned long idx, gfn_t gfn)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
+
+int destroy_xen_mappings(unsigned long s, unsigned long e)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int map_pages_to_xen(unsigned long virt,
+                     mfn_t mfn,
+                     unsigned long nr_mfns,
+                     unsigned int flags)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 98a94c4c48..8bb5bdb2ae 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -1,11 +1,19 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/compile.h>
 #include <xen/init.h>
 #include <xen/mm.h>
 
+#include <public/version.h>
+
 #include <asm/early_printk.h>
 
+void arch_get_xen_caps(xen_capabilities_info_t *info)
+{
+    BUG_ON("unimplemented");
+}
+
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
     __aligned(STACK_SIZE);
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
new file mode 100644
index 0000000000..8285bcffef
--- /dev/null
+++ b/xen/arch/riscv/stubs.c
@@ -0,0 +1,439 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#include <xen/cpumask.h>
+#include <xen/domain.h>
+#include <xen/irq.h>
+#include <xen/nodemask.h>
+#include <xen/time.h>
+#include <public/domctl.h>
+
+#include <asm/current.h>
+
+/* smpboot.c */
+
+cpumask_t cpu_online_map;
+cpumask_t cpu_present_map;
+cpumask_t cpu_possible_map;
+
+/* ID of the PCPU we're running on */
+DEFINE_PER_CPU(unsigned int, cpu_id);
+/* XXX these seem awfully x86ish... */
+/* representing HT siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
+/* representing HT and core siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
+
+nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
+
+/*
+ * max_page is defined in page_alloc.c which isn't complied for now.
+ * definition of max_page will be remove as soon as page_alloc is built.
+ */
+unsigned long __read_mostly max_page;
+
+/* time.c */
+
+unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
+
+s_time_t get_s_time(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int reprogram_timer(s_time_t timeout)
+{
+    BUG_ON("unimplemented");
+}
+
+void send_timer_event(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void domain_set_time_offset(struct domain *d, int64_t time_offset_seconds)
+{
+    BUG_ON("unimplemented");
+}
+
+/* shutdown.c */
+
+void machine_restart(unsigned int delay_millisecs)
+{
+    BUG_ON("unimplemented");
+}
+
+void machine_halt(void)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domctl.c */
+
+long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
+                    XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_domain_info(const struct domain *d,
+                          struct xen_domctl_getdomaininfo *info)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+/* monitor.c */
+
+int arch_monitor_domctl_event(struct domain *d,
+                              struct xen_domctl_monitor_op *mop)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smp.c */
+
+void arch_flush_tlb_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_event_check_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_call_function_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+/* irq.c */
+
+struct pirq *alloc_pirq_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int pirq_guest_bind(struct vcpu *v, struct pirq *pirq, int will_share)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_guest_unbind(struct domain *d, struct pirq *pirq)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_set_affinity(struct domain *d, int pirq, const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+hw_irq_controller no_irq_type = {
+    .typename = "none",
+    .startup = irq_startup_none,
+    .shutdown = irq_shutdown_none,
+    .enable = irq_enable_none,
+    .disable = irq_disable_none,
+};
+
+int arch_init_one_irq_desc(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_state_dump(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domain.c */
+
+DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
+unsigned long __per_cpu_offset[NR_CPUS];
+
+void context_switch(struct vcpu *prev, struct vcpu *next)
+{
+    BUG_ON("unimplemented");
+}
+
+void continue_running(struct vcpu *same)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_local_execstate(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_vcpu_execstate(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void startup_cpu_idle_loop(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_domain_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void dump_pageframe_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_vcpu_struct(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_create(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_vcpu_destroy(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_switch_to_aarch64_mode(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_create(struct domain *d,
+                       struct xen_domctl_createdomain *config,
+                       unsigned int flags)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_teardown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_destroy(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_shutdown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_pause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_unpause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_soft_reset(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_creation_finished(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_set_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_initialise_vcpu(struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_reset(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int domain_relinquish_resources(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_domain_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_vcpu_info(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_mark_events_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_update_evtchn_irq(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_block_unless_event_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_kick(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+struct domain *alloc_domain_struct(void)
+{
+    BUG_ON("unimplemented");
+}
+
+struct vcpu *alloc_vcpu_struct(const struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long
+hypercall_create_continuation(unsigned int op, const char *format, ...)
+{
+    BUG_ON("unimplemented");
+}
+
+int __init parse_arch_dom0_param(const char *s, const char *e)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guestcopy.c */
+
+unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long raw_copy_from_guest(void *to, const void __user *from,
+                                  unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* sysctl.c */
+
+long arch_do_sysctl(struct xen_sysctl *sysctl,
+                    XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
+{
+    BUG_ON("unimplemented");
+}
+
+/* p2m.c */
+
+int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
+{
+    BUG_ON("unimplemented");
+}
+
+int unmap_mmio_regions(struct domain *d,
+                       gfn_t start_gfn,
+                       unsigned long nr,
+                       mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int map_mmio_regions(struct domain *d,
+                     gfn_t start_gfn,
+                     unsigned long nr,
+                     mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+                          unsigned long gfn, mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+/* Return the size of the pool, in bytes. */
+int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
+{
+    BUG_ON("unimplemented");
+}
+
+/* delay.c */
+
+void udelay(unsigned long usecs)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guest_access.h */ 
+
+static inline unsigned long raw_clear_guest(void *to, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smpboot.c */
+
+int __cpu_up(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_disable(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_die(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/*
+ * The following functions are defined in common/irq.c, but common/irq.c isn't
+ * built for now. These changes will be removed there when common/irq.c is
+ * ready.
+ */
+
+void cf_check irq_actor_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned int cf_check irq_startup_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c
index ccd3593f5a..5415cf8d90 100644
--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -4,6 +4,10 @@
  *
  * RISC-V Trap handlers
  */
+
+#include <xen/lib.h>
+#include <xen/sched.h>
+
 #include <asm/processor.h>
 #include <asm/traps.h>
 
@@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs)
 {
     die();
 }
+
+void vcpu_show_execution_state(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void show_execution_state(const struct cpu_user_regs *regs)
+{
+    printk("implement show_execution_state(regs)\n");
+}
+
+void arch_hypercall_tasklet_result(struct vcpu *v, long res)
+{
+    BUG_ON("unimplemented");
+}
+
+enum mc_disposition arch_do_multicall_call(struct mc_state *state)
+{
+    BUG_ON("unimplemented");
+    return mc_continue;
+}
-- 
2.43.0

[PATCH v7 18/19] xen/riscv: enable full Xen build

[Oleksii Kurochko]

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5-V7:
 - Nothing changed. Only rebase.
---
Changes in V4:
 - drop stubs for irq_actor_none() and irq_actor_none() as common/irq.c is compiled now.
 - drop defintion of max_page in stubs.c as common/page_alloc.c is compiled now.
 - drop printk() related changes in riscv/early_printk.c as common version will be used.
---
Changes in V3:
 - Reviewed-by: Jan Beulich <jbeulich@suse.com>
 - unrealted change dropped in tiny64_defconfig
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/Makefile       |  16 +++-
 xen/arch/riscv/arch.mk        |   4 -
 xen/arch/riscv/early_printk.c | 168 ----------------------------------
 xen/arch/riscv/stubs.c        |  24 -----
 4 files changed, 15 insertions(+), 197 deletions(-)

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 60afbc0ad9..81b77b13d6 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -12,10 +12,24 @@ $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
 
 $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
-	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(objtree)/common/symbols-dummy.o -o $(dot-target).0
+	$(NM) -pa --format=sysv $(dot-target).0 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).0.S
+	$(MAKE) $(build)=$(@D) $(dot-target).0.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(dot-target).0.o -o $(dot-target).1
+	$(NM) -pa --format=sysv $(dot-target).1 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).1.S
+	$(MAKE) $(build)=$(@D) $(dot-target).1.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
+	    $(dot-target).1.o -o $@
 	$(NM) -pa --format=sysv $@ \
 		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		> $@.map
+	rm -f $(@D)/.$(@F).[0-9]*
 
 $(obj)/xen.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed_dep,cpp_lds_S)
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index 24a7461bcc..517bb662e4 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -24,7 +24,3 @@ extensions := $(subst $(space),,$(extensions))
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += $(riscv-generic-flags)$(extensions) -mstrict-align -mcmodel=medany
-
-# TODO: Drop override when more of the build is working
-override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
-override ALL_LIBS-y =
diff --git a/xen/arch/riscv/early_printk.c b/xen/arch/riscv/early_printk.c
index 60742a042d..610c814f54 100644
--- a/xen/arch/riscv/early_printk.c
+++ b/xen/arch/riscv/early_printk.c
@@ -40,171 +40,3 @@ void early_printk(const char *str)
         str++;
     }
 }
-
-/*
- * The following #if 1 ... #endif should be removed after printk
- * and related stuff are ready.
- */
-#if 1
-
-#include <xen/stdarg.h>
-#include <xen/string.h>
-
-/**
- * strlen - Find the length of a string
- * @s: The string to be sized
- */
-size_t (strlen)(const char * s)
-{
-    const char *sc;
-
-    for (sc = s; *sc != '\0'; ++sc)
-        /* nothing */;
-    return sc - s;
-}
-
-/**
- * memcpy - Copy one area of memory to another
- * @dest: Where to copy to
- * @src: Where to copy from
- * @count: The size of the area.
- *
- * You should not use this function to access IO space, use memcpy_toio()
- * or memcpy_fromio() instead.
- */
-void *(memcpy)(void *dest, const void *src, size_t count)
-{
-    char *tmp = (char *) dest, *s = (char *) src;
-
-    while (count--)
-        *tmp++ = *s++;
-
-    return dest;
-}
-
-int vsnprintf(char* str, size_t size, const char* format, va_list args)
-{
-    size_t i = 0; /* Current position in the output string */
-    size_t written = 0; /* Total number of characters written */
-    char* dest = str;
-
-    while ( format[i] != '\0' && written < size - 1 )
-    {
-        if ( format[i] == '%' )
-        {
-            i++;
-
-            if ( format[i] == '\0' )
-                break;
-
-            if ( format[i] == '%' )
-            {
-                if ( written < size - 1 )
-                {
-                    dest[written] = '%';
-                    written++;
-                }
-                i++;
-                continue;
-            }
-
-            /*
-             * Handle format specifiers.
-             * For simplicity, only %s and %d are implemented here.
-             */
-
-            if ( format[i] == 's' )
-            {
-                char* arg = va_arg(args, char*);
-                size_t arglen = strlen(arg);
-
-                size_t remaining = size - written - 1;
-
-                if ( arglen > remaining )
-                    arglen = remaining;
-
-                memcpy(dest + written, arg, arglen);
-
-                written += arglen;
-                i++;
-            }
-            else if ( format[i] == 'd' )
-            {
-                int arg = va_arg(args, int);
-
-                /* Convert the integer to string representation */
-                char numstr[32]; /* Assumes a maximum of 32 digits */
-                int numlen = 0;
-                int num = arg;
-                size_t remaining;
-
-                if ( arg < 0 )
-                {
-                    if ( written < size - 1 )
-                    {
-                        dest[written] = '-';
-                        written++;
-                    }
-
-                    num = -arg;
-                }
-
-                do
-                {
-                    numstr[numlen] = '0' + num % 10;
-                    num = num / 10;
-                    numlen++;
-                } while ( num > 0 );
-
-                /* Reverse the string */
-                for (int j = 0; j < numlen / 2; j++)
-                {
-                    char tmp = numstr[j];
-                    numstr[j] = numstr[numlen - 1 - j];
-                    numstr[numlen - 1 - j] = tmp;
-                }
-
-                remaining = size - written - 1;
-
-                if ( numlen > remaining )
-                    numlen = remaining;
-
-                memcpy(dest + written, numstr, numlen);
-
-                written += numlen;
-                i++;
-            }
-        }
-        else
-        {
-            if ( written < size - 1 )
-            {
-                dest[written] = format[i];
-                written++;
-            }
-            i++;
-        }
-    }
-
-    if ( size > 0 )
-        dest[written] = '\0';
-
-    return written;
-}
-
-void printk(const char *format, ...)
-{
-    static char buf[1024];
-
-    va_list args;
-    va_start(args, format);
-
-    (void)vsnprintf(buf, sizeof(buf), format, args);
-
-    early_printk(buf);
-
-    va_end(args);
-}
-
-#endif
-
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
index 8285bcffef..bda35fc347 100644
--- a/xen/arch/riscv/stubs.c
+++ b/xen/arch/riscv/stubs.c
@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
 
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
 
-/*
- * max_page is defined in page_alloc.c which isn't complied for now.
- * definition of max_page will be remove as soon as page_alloc is built.
- */
-unsigned long __read_mostly max_page;
-
 /* time.c */
 
 unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu)
 {
     BUG_ON("unimplemented");
 }
-
-/*
- * The following functions are defined in common/irq.c, but common/irq.c isn't
- * built for now. These changes will be removed there when common/irq.c is
- * ready.
- */
-
-void cf_check irq_actor_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-}
-
-unsigned int cf_check irq_startup_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-
-    return 0;
-}
-- 
2.43.0

[PATCH v7 19/19] xen/README: add compiler and binutils versions for RISC-V64

[Oleksii Kurochko]

This patch doesn't represent a strict lower bound for GCC and
GNU Binutils; rather, these versions are specifically employed by
the Xen RISC-V container and are anticipated to undergo continuous
testing. Older GCC and GNU Binutils would work,
but this is not a guarantee.

While it is feasible to utilize Clang, it's important to note that,
currently, there is no Xen RISC-V CI job in place to verify the
seamless functioning of the build with Clang.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5-V7:
 - Nothing changed. Only rebase.
---
 Changes in V6:
  - update the message in README.
---
 Changes in V5:
  - update the commit message and README file with additional explanation about GCC and
    GNU Binutils version. Additionally, it was added information about Clang.
---
 Changes in V4:
  - Update version of GCC (12.2) and GNU Binutils (2.39) to the version
    which are in Xen's contrainter for RISC-V
---
 Changes in V3:
  - new patch
---
 README | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/README b/README
index c8a108449e..30da5ff9c0 100644
--- a/README
+++ b/README
@@ -48,6 +48,10 @@ provided by your OS distributor:
       - For ARM 64-bit:
         - GCC 5.1 or later
         - GNU Binutils 2.24 or later
+      - For RISC-V 64-bit:
+        - GCC 12.2 or later
+        - GNU Binutils 2.39 or later
+          Older GCC and GNU Binutils would work, but this is not a guarantee.
     * POSIX compatible awk
     * Development install of zlib (e.g., zlib-dev)
     * Development install of Python 2.7 or later (e.g., python-dev)
-- 
2.43.0

Re: [PATCH v7 02/19] xen/riscv: disable unnecessary configs

[Jan Beulich]

On 03.04.2024 12:19, Oleksii Kurochko wrote:
> This patch disables unnecessary configs for two cases:
> 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
> 2. By using tiny64_defconfig for non-randconfig builds.
> 
> Only configs which lead to compilation issues were disabled.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes in V7:
>  - Disable only configs which cause compilation issues.

Since the description doesn't go into details: While I can see that
PERF_COUNTERS and LIVEPATCH may require (a little / some more) extra
work, are HYPFS, ARGO, and XSM really causing issues?

> --- a/xen/arch/riscv/configs/tiny64_defconfig
> +++ b/xen/arch/riscv/configs/tiny64_defconfig
> @@ -1,12 +1,11 @@
> -# CONFIG_SCHED_CREDIT is not set
> -# CONFIG_SCHED_RTDS is not set
> -# CONFIG_SCHED_NULL is not set
> -# CONFIG_SCHED_ARINC653 is not set
> -# CONFIG_TRACEBUFFER is not set
>  # CONFIG_HYPFS is not set
>  # CONFIG_GRANT_TABLE is not set
> -# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
>  # CONFIG_MEM_ACCESS is not set
> +# CONFIG_ARGO is not set
> +# CONFIG_PERF_COUNTERS is not set
> +# CONFIG_COVERAGE is not set
> +# CONFIG_LIVEPATCH is not set
> +# CONFIG_XSM is not set
>  
>  CONFIG_RISCV_64=y
>  CONFIG_DEBUG=y

The description also says nothing about the items being removed.

Jan

Re: [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler

[Jan Beulich]

On 03.04.2024 12:19, Oleksii Kurochko wrote:
> Currently, RISC-V requires two extensions: _zbb and _zihintpause.
> 
> This patch introduces a compiler check to check if these extensions
> are supported.
> Additionally, it introduces the riscv/booting.txt file, which contains
> information about the extensions that should be supported by the platform.
> 
> In the future, a feature will be introduced to check whether an extension
> is supported at runtime.
> However, this feature requires functionality for parsing device tree
> source (DTS), which is not yet available.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

However, ...

> --- a/xen/arch/riscv/arch.mk
> +++ b/xen/arch/riscv/arch.mk
> @@ -3,16 +3,27 @@
>  
>  $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
>  
> -CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64
> +riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32
> +riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64
>  
>  riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
>  riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
>  
> +riscv-generic-flags := $(riscv-abi-y) -march=$(riscv-march-y)
> +
> +zbb := $(call as-insn,$(CC) $(riscv-generic-flags)_zbb,"",_zbb)
> +zihintpause := $(call as-insn,\
> +               $(CC) $(riscv-generic-flags)_zihintpause,"pause",_zihintpause)

... this would better be indented thus

zihintpause := $(call as-insn, \
                      $(CC) $(riscv-generic-flags)_zihintpause,"pause",_zihintpause)

to make nesting entirely obvious. I guess I'll table the liberty to adjust
while committing.

I'd also like to note that this is specifically not what I had suggested.
But it at least improves readability.

Jan

Re: [PATCH v7 02/19] xen/riscv: disable unnecessary configs

[Oleksii]

On Wed, 2024-04-03 at 12:28 +0200, Jan Beulich wrote:
> On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > This patch disables unnecessary configs for two cases:
> > 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds
> > (GitLab CI jobs).
> > 2. By using tiny64_defconfig for non-randconfig builds.
> > 
> > Only configs which lead to compilation issues were disabled.
> > 
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > Changes in V7:
> >  - Disable only configs which cause compilation issues.
> 
> Since the description doesn't go into details: While I can see that
> PERF_COUNTERS and LIVEPATCH may require (a little / some more) extra
> work, are HYPFS, ARGO, and XSM really causing issues?
For Argo, I recieved the following compilation errors:
   common/argo.c:1416:5: error: unknown type name 'p2m_type_t'; did you
   mean 'hvmmem_type_t'?
    1416 |     p2m_type_t p2mt;
         |     ^~~~~~~~~~
         |     hvmmem_type_t
   common/argo.c:1419:11: error: implicit declaration of function
   'check_get_page_from_gfn' [-Werror=implicit-function-declaration]
    1419 |     ret = check_get_page_from_gfn(d, gfn, false, &p2mt, &page);
         |           ^~~~~~~~~~~~~~~~~~~~~~~
   common/argo.c:1427:10: error: 'p2m_ram_rw' undeclared (first use in
   this function)
    1427 |     case p2m_ram_rw:
   
It seems it should be included xen/p2m-common.h and asm/p2m.h in
common/argo.c.

For CONFIG_HYPFS_CONFIG ( there is no issue with CONFIG_HYPFS,
overlooked that ):
   common/config_data.S:1:10: fatal error: asm/asm_defns.h: No such file
   or directory
       1 | #include <asm/asm_defns.h>
   

For XSM, I recieved the following error:

   xsm/xsm_core.c:79:19: error: 'xsm_core_init' defined but not used [-
   Werror=unused-function]
      79 | static int __init xsm_core_init(const void *policy_buffer,
   size_t policy_size)

I'll add an information with compilation errors to the commit message.

~ Oleksii

> 
> > --- a/xen/arch/riscv/configs/tiny64_defconfig
> > +++ b/xen/arch/riscv/configs/tiny64_defconfig
> > @@ -1,12 +1,11 @@
> > -# CONFIG_SCHED_CREDIT is not set
> > -# CONFIG_SCHED_RTDS is not set
> > -# CONFIG_SCHED_NULL is not set
> > -# CONFIG_SCHED_ARINC653 is not set
> > -# CONFIG_TRACEBUFFER is not set
> >  # CONFIG_HYPFS is not set
> >  # CONFIG_GRANT_TABLE is not set
> > -# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
> >  # CONFIG_MEM_ACCESS is not set
> > +# CONFIG_ARGO is not set
> > +# CONFIG_PERF_COUNTERS is not set
> > +# CONFIG_COVERAGE is not set
> > +# CONFIG_LIVEPATCH is not set
> > +# CONFIG_XSM is not set
> >  
> >  CONFIG_RISCV_64=y
> >  CONFIG_DEBUG=y
> 
> The description also says nothing about the items being removed.
> 
> Jan

Re: [PATCH v7 02/19] xen/riscv: disable unnecessary configs

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 1996 bytes --]

On 03.04.24 12:54, Oleksii wrote:
> On Wed, 2024-04-03 at 12:28 +0200, Jan Beulich wrote:
>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>> This patch disables unnecessary configs for two cases:
>>> 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds
>>> (GitLab CI jobs).
>>> 2. By using tiny64_defconfig for non-randconfig builds.
>>>
>>> Only configs which lead to compilation issues were disabled.
>>>
>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>> ---
>>> Changes in V7:
>>>   - Disable only configs which cause compilation issues.
>>
>> Since the description doesn't go into details: While I can see that
>> PERF_COUNTERS and LIVEPATCH may require (a little / some more) extra
>> work, are HYPFS, ARGO, and XSM really causing issues?
> For Argo, I recieved the following compilation errors:
>     common/argo.c:1416:5: error: unknown type name 'p2m_type_t'; did you
>     mean 'hvmmem_type_t'?
>      1416 |     p2m_type_t p2mt;
>           |     ^~~~~~~~~~
>           |     hvmmem_type_t
>     common/argo.c:1419:11: error: implicit declaration of function
>     'check_get_page_from_gfn' [-Werror=implicit-function-declaration]
>      1419 |     ret = check_get_page_from_gfn(d, gfn, false, &p2mt, &page);
>           |           ^~~~~~~~~~~~~~~~~~~~~~~
>     common/argo.c:1427:10: error: 'p2m_ram_rw' undeclared (first use in
>     this function)
>      1427 |     case p2m_ram_rw:
>     
> It seems it should be included xen/p2m-common.h and asm/p2m.h in
> common/argo.c.
> 
> For CONFIG_HYPFS_CONFIG ( there is no issue with CONFIG_HYPFS,
> overlooked that ):
>     common/config_data.S:1:10: fatal error: asm/asm_defns.h: No such file
>     or directory
>         1 | #include <asm/asm_defns.h>

Hmm, this seems to be needed for ASM_INT(), which is currently defined the same
way for arm and x86. Maybe we should move that macro to xen/linkage.h and
include that one instead of asm_defns.h?


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v7 02/19] xen/riscv: disable unnecessary configs

[Jan Beulich]

On 03.04.2024 13:18, Juergen Gross wrote:
> On 03.04.24 12:54, Oleksii wrote:
>> On Wed, 2024-04-03 at 12:28 +0200, Jan Beulich wrote:
>>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>>> This patch disables unnecessary configs for two cases:
>>>> 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds
>>>> (GitLab CI jobs).
>>>> 2. By using tiny64_defconfig for non-randconfig builds.
>>>>
>>>> Only configs which lead to compilation issues were disabled.
>>>>
>>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>>> ---
>>>> Changes in V7:
>>>>   - Disable only configs which cause compilation issues.
>>>
>>> Since the description doesn't go into details: While I can see that
>>> PERF_COUNTERS and LIVEPATCH may require (a little / some more) extra
>>> work, are HYPFS, ARGO, and XSM really causing issues?
>> For Argo, I recieved the following compilation errors:
>>     common/argo.c:1416:5: error: unknown type name 'p2m_type_t'; did you
>>     mean 'hvmmem_type_t'?
>>      1416 |     p2m_type_t p2mt;
>>           |     ^~~~~~~~~~
>>           |     hvmmem_type_t
>>     common/argo.c:1419:11: error: implicit declaration of function
>>     'check_get_page_from_gfn' [-Werror=implicit-function-declaration]
>>      1419 |     ret = check_get_page_from_gfn(d, gfn, false, &p2mt, &page);
>>           |           ^~~~~~~~~~~~~~~~~~~~~~~
>>     common/argo.c:1427:10: error: 'p2m_ram_rw' undeclared (first use in
>>     this function)
>>      1427 |     case p2m_ram_rw:
>>     
>> It seems it should be included xen/p2m-common.h and asm/p2m.h in
>> common/argo.c.
>>
>> For CONFIG_HYPFS_CONFIG ( there is no issue with CONFIG_HYPFS,
>> overlooked that ):
>>     common/config_data.S:1:10: fatal error: asm/asm_defns.h: No such file
>>     or directory
>>         1 | #include <asm/asm_defns.h>
> 
> Hmm, this seems to be needed for ASM_INT(), which is currently defined the same
> way for arm and x86. Maybe we should move that macro to xen/linkage.h and
> include that one instead of asm_defns.h?

Indeed while doing the entry annotation work (also touching the build logic
here iirc) I was thinking of doing so.

Jan

Re: [PATCH v7 02/19] xen/riscv: disable unnecessary configs

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 2347 bytes --]

On 03.04.24 13:47, Jan Beulich wrote:
> On 03.04.2024 13:18, Juergen Gross wrote:
>> On 03.04.24 12:54, Oleksii wrote:
>>> On Wed, 2024-04-03 at 12:28 +0200, Jan Beulich wrote:
>>>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>>>> This patch disables unnecessary configs for two cases:
>>>>> 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds
>>>>> (GitLab CI jobs).
>>>>> 2. By using tiny64_defconfig for non-randconfig builds.
>>>>>
>>>>> Only configs which lead to compilation issues were disabled.
>>>>>
>>>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>>>> ---
>>>>> Changes in V7:
>>>>>    - Disable only configs which cause compilation issues.
>>>>
>>>> Since the description doesn't go into details: While I can see that
>>>> PERF_COUNTERS and LIVEPATCH may require (a little / some more) extra
>>>> work, are HYPFS, ARGO, and XSM really causing issues?
>>> For Argo, I recieved the following compilation errors:
>>>      common/argo.c:1416:5: error: unknown type name 'p2m_type_t'; did you
>>>      mean 'hvmmem_type_t'?
>>>       1416 |     p2m_type_t p2mt;
>>>            |     ^~~~~~~~~~
>>>            |     hvmmem_type_t
>>>      common/argo.c:1419:11: error: implicit declaration of function
>>>      'check_get_page_from_gfn' [-Werror=implicit-function-declaration]
>>>       1419 |     ret = check_get_page_from_gfn(d, gfn, false, &p2mt, &page);
>>>            |           ^~~~~~~~~~~~~~~~~~~~~~~
>>>      common/argo.c:1427:10: error: 'p2m_ram_rw' undeclared (first use in
>>>      this function)
>>>       1427 |     case p2m_ram_rw:
>>>      
>>> It seems it should be included xen/p2m-common.h and asm/p2m.h in
>>> common/argo.c.
>>>
>>> For CONFIG_HYPFS_CONFIG ( there is no issue with CONFIG_HYPFS,
>>> overlooked that ):
>>>      common/config_data.S:1:10: fatal error: asm/asm_defns.h: No such file
>>>      or directory
>>>          1 | #include <asm/asm_defns.h>
>>
>> Hmm, this seems to be needed for ASM_INT(), which is currently defined the same
>> way for arm and x86. Maybe we should move that macro to xen/linkage.h and
>> include that one instead of asm_defns.h?
> 
> Indeed while doing the entry annotation work (also touching the build logic
> here iirc) I was thinking of doing so.

Okay, I'm preparing a patch.


Juergen


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH v7 02/19] xen/riscv: disable unnecessary configs

[Jan Beulich]

On 03.04.2024 12:54, Oleksii wrote:
> On Wed, 2024-04-03 at 12:28 +0200, Jan Beulich wrote:
>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>> This patch disables unnecessary configs for two cases:
>>> 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds
>>> (GitLab CI jobs).
>>> 2. By using tiny64_defconfig for non-randconfig builds.
>>>
>>> Only configs which lead to compilation issues were disabled.
>>>
>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>> ---
>>> Changes in V7:
>>>  - Disable only configs which cause compilation issues.
>>
>> Since the description doesn't go into details: While I can see that
>> PERF_COUNTERS and LIVEPATCH may require (a little / some more) extra
>> work, are HYPFS, ARGO, and XSM really causing issues?
> For Argo, I recieved the following compilation errors:
>    common/argo.c:1416:5: error: unknown type name 'p2m_type_t'; did you
>    mean 'hvmmem_type_t'?
>     1416 |     p2m_type_t p2mt;
>          |     ^~~~~~~~~~
>          |     hvmmem_type_t
>    common/argo.c:1419:11: error: implicit declaration of function
>    'check_get_page_from_gfn' [-Werror=implicit-function-declaration]
>     1419 |     ret = check_get_page_from_gfn(d, gfn, false, &p2mt, &page);
>          |           ^~~~~~~~~~~~~~~~~~~~~~~
>    common/argo.c:1427:10: error: 'p2m_ram_rw' undeclared (first use in
>    this function)
>     1427 |     case p2m_ram_rw:
>    
> It seems it should be included xen/p2m-common.h and asm/p2m.h in
> common/argo.c.
> 
> For CONFIG_HYPFS_CONFIG ( there is no issue with CONFIG_HYPFS,
> overlooked that ):
>    common/config_data.S:1:10: fatal error: asm/asm_defns.h: No such file
>    or directory
>        1 | #include <asm/asm_defns.h>
>    
> 
> For XSM, I recieved the following error:
> 
>    xsm/xsm_core.c:79:19: error: 'xsm_core_init' defined but not used [-
>    Werror=unused-function]
>       79 | static int __init xsm_core_init(const void *policy_buffer,
>    size_t policy_size)
> 
> I'll add an information with compilation errors to the commit message.

No need to quote full compiler diagnostics, but a hint at the problems
at least. That said, perhaps we want to rather sort the issues than
disable building stuff that sooner or later you will want to build
anyway. For hypfs we look to have an approach already. For Argo what
you suggest makes sense to me; it might be nice to understand where
the P2M headers needed are coming from on x86 and Arm. Ideally common
code .c files wouldn't include asm/*.h.

For XSM I'm a little puzzled: Shouldn't RISC-V have HAS_DEVICE_TREE=y?
Then xsm_core_init() would have a caller.

Jan

Re: [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler

[Jan Beulich]

On 03.04.2024 12:19, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/arch.mk
> +++ b/xen/arch/riscv/arch.mk
> @@ -3,16 +3,27 @@
>  
>  $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
>  
> -CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64
> +riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32
> +riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64
>  
>  riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
>  riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
>  
> +riscv-generic-flags := $(riscv-abi-y) -march=$(riscv-march-y)
> +
> +zbb := $(call as-insn,$(CC) $(riscv-generic-flags)_zbb,"",_zbb)

While committing another question popped up: Why "" (i.e. no insn) here, ...

> +zihintpause := $(call as-insn,\
> +               $(CC) $(riscv-generic-flags)_zihintpause,"pause",_zihintpause)

... but "pause" here?

Jan

Re: [PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Jan Beulich]

On 03.04.2024 12:19, Oleksii Kurochko wrote:
> The patch introduces the following generic functions:
> * test_bit
> * generic__test_and_set_bit
> * generic__test_and_clear_bit
> * generic__test_and_change_bit
> 
> Also, the patch introduces the following generics which are
> used by the functions mentioned above:
> * BITOP_BITS_PER_WORD
> * BITOP_MASK
> * BITOP_WORD
> * BITOP_TYPE
> 
> These functions and macros can be useful for architectures
> that don't have corresponding arch-specific instructions.
> 
> Because of that x86 has the following check in the macros test_bit(),
> __test_and_set_bit(), __test_and_clear_bit(), __test_and_change_bit():
>     if ( bitop_bad_size(addr) ) __bitop_bad_size();
> It was necessary to move the check to generic code and define as 0
> for other architectures as they do not require this check.

Hmm, yes, the checks need to be in the outermost wrapper macros. While
you're abstracting other stuff to arch_*(), wouldn't it make sense to
also abstract this to e.g. arch_check_bitop_size(), with the expansion
simply being (effectively) empty in the generic fallback case?

> --- a/xen/include/xen/bitops.h
> +++ b/xen/include/xen/bitops.h
> @@ -65,10 +65,164 @@ static inline int generic_flsl(unsigned long x)
>   * scope
>   */
>  
> +#define BITOP_BITS_PER_WORD 32
> +/* typedef uint32_t bitop_uint_t; */
> +#define bitop_uint_t uint32_t

So no arch overrides permitted anymore at all?

> +#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
> +
> +#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
> +
>  /* --------------------- Please tidy above here --------------------- */
>  
>  #include <asm/bitops.h>
>  
> +#ifndef bitop_bad_size
> +extern void __bitop_bad_size(void);

If not switching to arch_check_bitop_size() or alike as suggested above,
why exactly does this need duplicating here and in x86? Can't the decl
simply move ahead of the #include right above? (Sure, this will then
require that nothing needing any of the functions you move here would
still include asm/bitops.h; it would need to be xen/bitops.h everywhere.)

> +#define bitop_bad_size(addr) 0
> +#endif
> +
> +/**
> + * generic__test_and_set_bit - Set a bit and return its old value
> + * @nr: Bit to set
> + * @addr: Address to count from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed
> + * but actually fail.  You must protect multiple accesses with a lock.
> + */
> +static always_inline __pure bool
> +generic__test_and_set_bit(unsigned long nr, volatile void *addr)

Does __pure actually fit with the use of volatile? The former says multiple
accesses may be folded; the latter says they must not be.

> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = ((volatile bitop_uint_t *)addr) + BITOP_WORD(nr);

Nit: Slightly shorter line possible:

    volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr + BITOP_WORD(nr);

> +    bitop_uint_t old = *p;
> +
> +    *p = old | mask;
> +    return (old & mask);
> +}
> +
> +/**
> + * generic__test_and_clear_bit - Clear a bit and return its old value
> + * @nr: Bit to clear
> + * @addr: Address to count from
> + *
> + * This operation is non-atomic and can be reordered.
> + * If two examples of this operation race, one can appear to succeed
> + * but actually fail.  You must protect multiple accesses with a lock.
> + */
> +static always_inline __pure bool
> +generic__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = ((volatile bitop_uint_t *)addr) + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old & ~mask;
> +    return (old & mask);
> +}
> +
> +/* WARNING: non atomic and it can be reordered! */
> +static always_inline __pure bool
> +generic__test_and_change_bit(unsigned long nr, volatile void *addr)
> +{
> +    bitop_uint_t mask = BITOP_MASK(nr);
> +    volatile bitop_uint_t *p = ((volatile bitop_uint_t *)addr) + BITOP_WORD(nr);
> +    bitop_uint_t old = *p;
> +
> +    *p = old ^ mask;
> +    return (old & mask);
> +}
> +/**
> + * generic_test_bit - Determine whether a bit is set
> + * @nr: bit number to test
> + * @addr: Address to start counting from
> + */
> +static always_inline __pure int generic_test_bit(int nr, const volatile void *addr)

Further up you use bool; why int here?

> +{
> +    const volatile bitop_uint_t *p = addr;
> +    return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD - 1)));

And reason not to use BITOP_MASK() here as well (once having switched to
bool return type)?

> +}
> +
> +static always_inline __pure bool
> +__test_and_set_bit(unsigned long nr, volatile void *addr)
> +{
> +#ifndef arch__test_and_set_bit
> +#define arch__test_and_set_bit generic__test_and_set_bit
> +#endif
> +
> +    return arch__test_and_set_bit(nr, addr);
> +}
> +#define __test_and_set_bit(nr, addr) ({             \
> +    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
> +    __test_and_set_bit(nr, addr);                   \
> +})
> +
> +static always_inline __pure bool
> +__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
> +{
> +#ifndef arch__test_and_clear_bit
> +#define arch__test_and_clear_bit generic__test_and_clear_bit
> +#endif
> +
> +    return arch__test_and_clear_bit(nr, addr);
> +}
> +#define __test_and_clear_bit(nr, addr) ({           \
> +    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
> +    __test_and_clear_bit(nr, addr);                 \
> +})
> +
> +static always_inline __pure bool
> +__test_and_change_bit(unsigned long nr, volatile void *addr)
> +{
> +#ifndef arch__test_and_change_bit
> +#define arch__test_and_change_bit generic__test_and_change_bit
> +#endif
> +
> +    return arch__test_and_change_bit(nr, addr);
> +}
> +#define __test_and_change_bit(nr, addr) ({              \
> +    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
> +    __test_and_change_bit(nr, addr);                    \
> +})
> +
> +static always_inline __pure int test_bit(int nr, const volatile void *addr)

Further up you use bool; why int here?

> +{
> +#ifndef arch_test_bit
> +#define arch_test_bit generic_test_bit
> +#endif
> +
> +    return arch_test_bit(nr, addr);
> +}
> +#define test_bit(nr, addr) ({                           \
> +    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
> +    test_bit(nr, addr);                                 \
> +})

From here onwards, ...

> +static always_inline __pure int fls(unsigned int x)
> +{
> +    if (__builtin_constant_p(x))
> +        return generic_fls(x);
> +
> +#ifndef arch_fls
> +#define arch_fls generic_fls
> +#endif
> +
> +    return arch_fls(x);
> +}
> +
> +static always_inline __pure int flsl(unsigned long x)
> +{
> +    if (__builtin_constant_p(x))
> +        return generic_flsl(x);
> +
> +#ifndef arch_flsl
> +#define arch_flsl generic_flsl
> +#endif
> +
> +    return arch_flsl(x);
> +}

... does all of this really belong here? Neither title nor description have
any hint towards this.

>  /*
>   * Find First Set bit.  Bits are labelled from 1.
>   */

This context suggests there's a dependency on an uncommitted patch. Nothing
above says so. I guess you have a remark in the cover letter, yet imo that's
only partly helpful.

Jan

Re: [PATCH v7 05/19] xen/bitops: implement fls{l}() in common logic

[Jan Beulich]

On 03.04.2024 12:19, Oleksii Kurochko wrote:
> --- a/xen/include/xen/bitops.h
> +++ b/xen/include/xen/bitops.h
> @@ -201,7 +201,7 @@ static always_inline __pure int test_bit(int nr, const volatile void *addr)
>  
>  static always_inline __pure int fls(unsigned int x)
>  {
> -    if (__builtin_constant_p(x))
> +    if ( __builtin_constant_p(x) )
>          return generic_fls(x);
>  
>  #ifndef arch_fls
> @@ -213,7 +213,7 @@ static always_inline __pure int fls(unsigned int x)
>  
>  static always_inline __pure int flsl(unsigned long x)
>  {
> -    if (__builtin_constant_p(x))
> +    if ( __builtin_constant_p(x) )
>          return generic_flsl(x);
>  
>  #ifndef arch_flsl

As per this something is clearly wrong with the split between the previous
patch and this one.

Jan

Re: [PATCH v7 01/19] automation: introduce fixed randconfig for RISC-V

[Jan Beulich]

On 03.04.2024 12:19, Oleksii Kurochko wrote:
> This patch introduces the anchor riscv-fixed-randconfig,
> which includes all configurations that should be disabled for
> randconfig builds.

Oh, one thing I noticed while committing (and I think I said so before):
Can you please avoid phrases like "this patch" in descriptions? Just
say what is being done; before being committed it's a patch, yes, while
once committed it's no longer a patch, but a commit. In the case here
you'd simply have started "Introduce ...".

Jan

Re: [PATCH v7 06/19] xen/bitops: put __ffs() into linux compatible header

[Jan Beulich]

On 03.04.2024 12:19, Oleksii Kurochko wrote:
> --- a/xen/include/xen/linux-compat.h
> +++ b/xen/include/xen/linux-compat.h
> @@ -19,4 +19,6 @@ typedef int64_t __s64;
>  
>  typedef paddr_t phys_addr_t;
>  
> +#define __ffs(x) (ffsl(x) - 1)

To be truly Linux-compatible, the result type would better be unsigned long
here. Seeing that you build upon Andrew's work, the easiest might be

#define __ffs(x) (ffsl(x) - 1UL)

Then:
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan

Re: [PATCH v7 07/19] xen/riscv: introduce bitops.h

[Jan Beulich]

On 03.04.2024 12:20, Oleksii Kurochko wrote:
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/bitops.h
> @@ -0,0 +1,146 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/* Copyright (C) 2012 Regents of the University of California */
> +
> +#ifndef _ASM_RISCV_BITOPS_H
> +#define _ASM_RISCV_BITOPS_H
> +
> +#include <asm/system.h>
> +
> +#undef BITOP_BITS_PER_WORD
> +#undef bitop_uint_t
> +
> +#define BITOP_BITS_PER_WORD BITS_PER_LONG
> +#define bitop_uint_t unsigned long
> +
> +#if BITS_PER_LONG == 64
> +#define __AMO(op)   "amo" #op ".d"
> +#elif BITS_PER_LONG == 32
> +#define __AMO(op)   "amo" #op ".w"
> +#else
> +#error "Unexpected BITS_PER_LONG"
> +#endif
> +
> +#define __set_bit(n, p)      set_bit(n, p)
> +#define __clear_bit(n, p)    clear_bit(n, p)

First without comment and then ...

> +/* Based on linux/arch/include/asm/bitops.h */
> +
> +/*
> + * Non-atomic bit manipulation.
> + *
> + * Implemented using atomics to be interrupt safe. Could alternatively
> + * implement with local interrupt masking.
> + */
> +#define __set_bit(n, p)      set_bit(n, p)
> +#define __clear_bit(n, p)    clear_bit(n, p)

... with one?

> +/* Based on linux/arch/include/asm/bitops.h */

Does this really need repeating?

> +#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
> +({                                                      \
> +    unsigned long res, mask;                            \

bitop_uint_t?

> +    mask = BITOP_MASK(nr);                              \
> +    asm volatile (                                       \

Nit: One too many padding blanks.

> +        __AMO(op) #ord " %0, %2, %1"                    \
> +        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
> +        : "r" (mod(mask))                               \
> +        : "memory");                                    \
> +    ((res & mask) != 0);                                \
> +})
> +
> +#define op_bit_ord(op, mod, nr, addr, ord)      \
> +    asm volatile (                              \
> +        __AMO(op) #ord " zero, %1, %0"          \
> +        : "+A" (addr[BITOP_WORD(nr)])           \
> +        : "r" (mod(BITOP_MASK(nr)))             \
> +        : "memory");
> +
> +#define test_and_op_bit(op, mod, nr, addr)    \
> +    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
> +#define op_bit(op, mod, nr, addr) \
> +    op_bit_ord(op, mod, nr, addr, )
> +
> +/* Bitmask modifiers */
> +#define NOP(x)    (x)
> +#define NOT(x)    (~(x))
> +
> +/**
> + * test_and_set_bit - Set a bit and return its old value
> + * @nr: Bit to set
> + * @addr: Address to count from
> + */
> +static inline int test_and_set_bit(int nr, volatile void *p)

In patch 4 you switched to bool. Any reason you didn't here, too?

> +{
> +    volatile bitop_uint_t *addr = p;
> +
> +    return test_and_op_bit(or, NOP, nr, addr);
> +}
> +
> +/**
> + * test_and_clear_bit - Clear a bit and return its old value
> + * @nr: Bit to clear
> + * @addr: Address to count from
> + */
> +static inline int test_and_clear_bit(int nr, volatile void *p)
> +{
> +    volatile bitop_uint_t *addr = p;
> +
> +    return test_and_op_bit(and, NOT, nr, addr);
> +}
> +
> +/**
> + * set_bit - Atomically set a bit in memory
> + * @nr: the bit to set
> + * @addr: the address to start counting from
> + *
> + * Note that @nr may be almost arbitrarily large; this function is not
> + * restricted to acting on a single-word quantity.
> + */
> +static inline void set_bit(int nr, volatile void *p)
> +{
> +    volatile bitop_uint_t *addr = p;
> +
> +    op_bit(or, NOP, nr, addr);
> +}
> +
> +/**
> + * clear_bit - Clears a bit in memory
> + * @nr: Bit to clear
> + * @addr: Address to start counting from
> + */
> +static inline void clear_bit(int nr, volatile void *p)
> +{
> +    volatile bitop_uint_t *addr = p;
> +
> +    op_bit(and, NOT, nr, addr);
> +}
> +
> +/**
> + * test_and_change_bit - Toggle (change) a bit and return its old value
> + * @nr: Bit to change
> + * @addr: Address to count from
> + *
> + * This operation is atomic and cannot be reordered.
> + * It also implies a memory barrier.
> + */
> +static inline int test_and_change_bit(int nr, volatile unsigned long *addr)

unsigned long?

> +{
> +	return test_and_op_bit(xor, NOP, nr, addr);
> +}

Perhaps better to move up a little, next to the other test_and-s?

Also - nit: Hard tab used for indentation.

> +#undef test_and_op_bit
> +#undef __op_bit

This no longer has any effect in this shape.

Jan

Re: [PATCH v7 08/19] xen/riscv: introduce cmpxchg.h

[Jan Beulich]

On 03.04.2024 12:20, Oleksii Kurochko wrote:
> +#define emulate_cmpxchg_1_2(ptr, old, new, lr_sfx, sc_sfx) \
> +({ \
> +    uint32_t *aligned_ptr; \
> +    unsigned long alignment_mask = sizeof(*aligned_ptr) - sizeof(*(ptr)); \
> +    uint8_t new_val_bit = \
> +        ((unsigned long)(ptr) & alignment_mask) * BITS_PER_BYTE; \
> +    unsigned long mask = \
> +        GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_bit; \
> +    unsigned int old_ = (old) << new_val_bit; \
> +    unsigned int new_ = (new) << new_val_bit; \
> +    unsigned int old_val; \
> +    unsigned int scratch; \
> +    \
> +    aligned_ptr = (uint32_t *)((unsigned long)ptr & ~alignment_mask); \
> +    \
> +    asm volatile ( \
> +        "0: lr.w" lr_sfx " %[scratch], %[ptr_]\n" \
> +        "   and  %[old_val], %[scratch], %[mask]\n" \
> +        "   bne  %[old_val], %z[old_], 1f\n" \
> +        /* the following line is an equivalent to:
> +         *     scratch = old_val & ~mask;
> +         * And to elimanate one ( likely register ) input it was decided
> +         * to use:
> +         *     scratch = old_val ^ scratch
> +         */ \

I'm surprised this compiles without \-es inside the comment as well. Line
splicing happens ahead of comment recognition as per the spec.

Everything else okay-ish to me now, but I can't very well given an ack for
a patch depending on things that haven't been committed yet and may never be.

Jan

Re: [PATCH v7 02/19] xen/riscv: disable unnecessary configs

[Oleksii]

On Wed, 2024-04-03 at 13:53 +0200, Jan Beulich wrote:
> On 03.04.2024 12:54, Oleksii wrote:
> > On Wed, 2024-04-03 at 12:28 +0200, Jan Beulich wrote:
> > > On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > > > This patch disables unnecessary configs for two cases:
> > > > 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds
> > > > (GitLab CI jobs).
> > > > 2. By using tiny64_defconfig for non-randconfig builds.
> > > > 
> > > > Only configs which lead to compilation issues were disabled.
> > > > 
> > > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > > > ---
> > > > Changes in V7:
> > > >  - Disable only configs which cause compilation issues.
> > > 
> > > Since the description doesn't go into details: While I can see
> > > that
> > > PERF_COUNTERS and LIVEPATCH may require (a little / some more)
> > > extra
> > > work, are HYPFS, ARGO, and XSM really causing issues?
> > For Argo, I recieved the following compilation errors:
> >    common/argo.c:1416:5: error: unknown type name 'p2m_type_t'; did
> > you
> >    mean 'hvmmem_type_t'?
> >     1416 |     p2m_type_t p2mt;
> >          |     ^~~~~~~~~~
> >          |     hvmmem_type_t
> >    common/argo.c:1419:11: error: implicit declaration of function
> >    'check_get_page_from_gfn' [-Werror=implicit-function-
> > declaration]
> >     1419 |     ret = check_get_page_from_gfn(d, gfn, false, &p2mt,
> > &page);
> >          |           ^~~~~~~~~~~~~~~~~~~~~~~
> >    common/argo.c:1427:10: error: 'p2m_ram_rw' undeclared (first use
> > in
> >    this function)
> >     1427 |     case p2m_ram_rw:
> >    
> > It seems it should be included xen/p2m-common.h and asm/p2m.h in
> > common/argo.c.
> > 
> > For CONFIG_HYPFS_CONFIG ( there is no issue with CONFIG_HYPFS,
> > overlooked that ):
> >    common/config_data.S:1:10: fatal error: asm/asm_defns.h: No such
> > file
> >    or directory
> >        1 | #include <asm/asm_defns.h>
> >    
> > 
> > For XSM, I recieved the following error:
> > 
> >    xsm/xsm_core.c:79:19: error: 'xsm_core_init' defined but not
> > used [-
> >    Werror=unused-function]
> >       79 | static int __init xsm_core_init(const void
> > *policy_buffer,
> >    size_t policy_size)
> > 
> > I'll add an information with compilation errors to the commit
> > message.
> 
> No need to quote full compiler diagnostics, but a hint at the
> problems
> at least. That said, perhaps we want to rather sort the issues than
> disable building stuff that sooner or later you will want to build
> anyway. For hypfs we look to have an approach already. For Argo what
> you suggest makes sense to me; it might be nice to understand where
> the P2M headers needed are coming from on x86 and Arm. Ideally common
> code .c files wouldn't include asm/*.h.
I'll look at where P2M header came from on x86 and Arm.

> 
> For XSM I'm a little puzzled: Shouldn't RISC-V have
> HAS_DEVICE_TREE=y?
> Then xsm_core_init() would have a caller.
It should, but I haven't add "select HAS_DEVICE_TREE" for RISC-V as no
device tree functionality hasn't been introduced yet.

~ Oleksii

Re: [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler

[Oleksii]

On Thu, 2024-04-04 at 12:07 +0200, Jan Beulich wrote:
> On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > --- a/xen/arch/riscv/arch.mk
> > +++ b/xen/arch/riscv/arch.mk
> > @@ -3,16 +3,27 @@
> >  
> >  $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
> >  
> > -CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64
> > +riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32
> > +riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64
> >  
> >  riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
> >  riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
> >  
> > +riscv-generic-flags := $(riscv-abi-y) -march=$(riscv-march-y)
> > +
> > +zbb := $(call as-insn,$(CC) $(riscv-generic-flags)_zbb,"",_zbb)
> 
> While committing another question popped up: Why "" (i.e. no insn)
> here, ...
> 
> > +zihintpause := $(call as-insn,\
> > +               $(CC) $(riscv-generic-
> > flags)_zihintpause,"pause",_zihintpause)
> 
> ... but "pause" here?

In the case of the Zbb extension, we don't check for a specific
instruction, but with the Zihintpause, the idea was to verify if the
pause instruction is supported or not. However, in both checks, there
might be no instruction as an argument of as-insn.

~ Oleksii

Re: [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler

[Jan Beulich]

On 04.04.2024 17:18, Oleksii wrote:
> On Thu, 2024-04-04 at 12:07 +0200, Jan Beulich wrote:
>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>> --- a/xen/arch/riscv/arch.mk
>>> +++ b/xen/arch/riscv/arch.mk
>>> @@ -3,16 +3,27 @@
>>>  
>>>  $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
>>>  
>>> -CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64
>>> +riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32
>>> +riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64
>>>  
>>>  riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
>>>  riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
>>>  
>>> +riscv-generic-flags := $(riscv-abi-y) -march=$(riscv-march-y)
>>> +
>>> +zbb := $(call as-insn,$(CC) $(riscv-generic-flags)_zbb,"",_zbb)
>>
>> While committing another question popped up: Why "" (i.e. no insn)
>> here, ...
>>
>>> +zihintpause := $(call as-insn,\
>>> +               $(CC) $(riscv-generic-
>>> flags)_zihintpause,"pause",_zihintpause)
>>
>> ... but "pause" here?
> 
> In the case of the Zbb extension, we don't check for a specific
> instruction, but with the Zihintpause, the idea was to verify if the
> pause instruction is supported or not.

And why's this verification relevant here, but not for Zbb?

Jan

> However, in both checks, there
> might be no instruction as an argument of as-insn.
> 
> ~ Oleksii

Re: [PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Oleksii]

On Thu, 2024-04-04 at 15:22 +0200, Jan Beulich wrote:
> On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > The patch introduces the following generic functions:
> > * test_bit
> > * generic__test_and_set_bit
> > * generic__test_and_clear_bit
> > * generic__test_and_change_bit
> > 
> > Also, the patch introduces the following generics which are
> > used by the functions mentioned above:
> > * BITOP_BITS_PER_WORD
> > * BITOP_MASK
> > * BITOP_WORD
> > * BITOP_TYPE
> > 
> > These functions and macros can be useful for architectures
> > that don't have corresponding arch-specific instructions.
> > 
> > Because of that x86 has the following check in the macros
> > test_bit(),
> > __test_and_set_bit(), __test_and_clear_bit(),
> > __test_and_change_bit():
> >     if ( bitop_bad_size(addr) ) __bitop_bad_size();
> > It was necessary to move the check to generic code and define as 0
> > for other architectures as they do not require this check.
> 
> Hmm, yes, the checks need to be in the outermost wrapper macros.
> While
> you're abstracting other stuff to arch_*(), wouldn't it make sense to
> also abstract this to e.g. arch_check_bitop_size(), with the
> expansion
> simply being (effectively) empty in the generic fallback case?
Probably it would be better to be consistent and introduce
arch_check_bitop_size().

> 
> > --- a/xen/include/xen/bitops.h
> > +++ b/xen/include/xen/bitops.h
> > @@ -65,10 +65,164 @@ static inline int generic_flsl(unsigned long
> > x)
> >   * scope
> >   */
> >  
> > +#define BITOP_BITS_PER_WORD 32
> > +/* typedef uint32_t bitop_uint_t; */
> > +#define bitop_uint_t uint32_t
> 
> So no arch overrides permitted anymore at all?
Not really, I agree that it is ugly, but I expected that arch will use
undef to override.

> 
> > +#define BITOP_MASK(nr)  ((bitop_uint_t)1 << ((nr) %
> > BITOP_BITS_PER_WORD))
> > +
> > +#define BITOP_WORD(nr)  ((nr) / BITOP_BITS_PER_WORD)
> > +
> >  /* --------------------- Please tidy above here ------------------
> > --- */
> >  
> >  #include <asm/bitops.h>
> >  
> > +#ifndef bitop_bad_size
> > +extern void __bitop_bad_size(void);
> 
> If not switching to arch_check_bitop_size() or alike as suggested
> above,
> why exactly does this need duplicating here and in x86? Can't the
> decl
> simply move ahead of the #include right above? (Sure, this will then
> require that nothing needing any of the functions you move here would
> still include asm/bitops.h; it would need to be xen/bitops.h
> everywhere.)
It could be done in way you suggest, I just wanted to keep changes
minimal ( without going and switching asm/bitops.h -> xen/bitops.h ),
but we can consider such option.

> 
> > +#define bitop_bad_size(addr) 0
> > +#endif
> > +
> > +/**
> > + * generic__test_and_set_bit - Set a bit and return its old value
> > + * @nr: Bit to set
> > + * @addr: Address to count from
> > + *
> > + * This operation is non-atomic and can be reordered.
> > + * If two examples of this operation race, one can appear to
> > succeed
> > + * but actually fail.  You must protect multiple accesses with a
> > lock.
> > + */
> > +static always_inline __pure bool
> > +generic__test_and_set_bit(unsigned long nr, volatile void *addr)
> 
> Does __pure actually fit with the use of volatile? The former says
> multiple
> accesses may be folded; the latter says they must not be.
Overlooked that, __pure should be dropped.

> 
> > +{
> > +    bitop_uint_t mask = BITOP_MASK(nr);
> > +    volatile bitop_uint_t *p = ((volatile bitop_uint_t *)addr) +
> > BITOP_WORD(nr);
> 
> Nit: Slightly shorter line possible:
> 
>     volatile bitop_uint_t *p = (volatile bitop_uint_t *)addr +
> BITOP_WORD(nr);
> 
> > +    bitop_uint_t old = *p;
> > +
> > +    *p = old | mask;
> > +    return (old & mask);
> > +}
> > +
> > +/**
> > + * generic__test_and_clear_bit - Clear a bit and return its old
> > value
> > + * @nr: Bit to clear
> > + * @addr: Address to count from
> > + *
> > + * This operation is non-atomic and can be reordered.
> > + * If two examples of this operation race, one can appear to
> > succeed
> > + * but actually fail.  You must protect multiple accesses with a
> > lock.
> > + */
> > +static always_inline __pure bool
> > +generic__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
> > +{
> > +    bitop_uint_t mask = BITOP_MASK(nr);
> > +    volatile bitop_uint_t *p = ((volatile bitop_uint_t *)addr) +
> > BITOP_WORD(nr);
> > +    bitop_uint_t old = *p;
> > +
> > +    *p = old & ~mask;
> > +    return (old & mask);
> > +}
> > +
> > +/* WARNING: non atomic and it can be reordered! */
> > +static always_inline __pure bool
> > +generic__test_and_change_bit(unsigned long nr, volatile void
> > *addr)
> > +{
> > +    bitop_uint_t mask = BITOP_MASK(nr);
> > +    volatile bitop_uint_t *p = ((volatile bitop_uint_t *)addr) +
> > BITOP_WORD(nr);
> > +    bitop_uint_t old = *p;
> > +
> > +    *p = old ^ mask;
> > +    return (old & mask);
> > +}
> > +/**
> > + * generic_test_bit - Determine whether a bit is set
> > + * @nr: bit number to test
> > + * @addr: Address to start counting from
> > + */
> > +static always_inline __pure int generic_test_bit(int nr, const
> > volatile void *addr)
> 
> Further up you use bool; why int here?
No specific reason, I have to update everything to bool.

> 
> > +{
> > +    const volatile bitop_uint_t *p = addr;
> > +    return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD -
> > 1)));
> 
> And reason not to use BITOP_MASK() here as well (once having switched
> to
> bool return type)?
It seems we can use BITOP_MASK() this implementation was copied from
arch specific code.

> 
> > +}
> > +
> > +static always_inline __pure bool
> > +__test_and_set_bit(unsigned long nr, volatile void *addr)
> > +{
> > +#ifndef arch__test_and_set_bit
> > +#define arch__test_and_set_bit generic__test_and_set_bit
> > +#endif
> > +
> > +    return arch__test_and_set_bit(nr, addr);
> > +}
> > +#define __test_and_set_bit(nr, addr) ({             \
> > +    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
> > +    __test_and_set_bit(nr, addr);                   \
> > +})
> > +
> > +static always_inline __pure bool
> > +__test_and_clear_bit(bitop_uint_t nr, volatile void *addr)
> > +{
> > +#ifndef arch__test_and_clear_bit
> > +#define arch__test_and_clear_bit generic__test_and_clear_bit
> > +#endif
> > +
> > +    return arch__test_and_clear_bit(nr, addr);
> > +}
> > +#define __test_and_clear_bit(nr, addr) ({           \
> > +    if ( bitop_bad_size(addr) ) __bitop_bad_size(); \
> > +    __test_and_clear_bit(nr, addr);                 \
> > +})
> > +
> > +static always_inline __pure bool
> > +__test_and_change_bit(unsigned long nr, volatile void *addr)
> > +{
> > +#ifndef arch__test_and_change_bit
> > +#define arch__test_and_change_bit generic__test_and_change_bit
> > +#endif
> > +
> > +    return arch__test_and_change_bit(nr, addr);
> > +}
> > +#define __test_and_change_bit(nr, addr) ({              \
> > +    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
> > +    __test_and_change_bit(nr, addr);                    \
> > +})
> > +
> > +static always_inline __pure int test_bit(int nr, const volatile
> > void *addr)
> 
> Further up you use bool; why int here?
> 
> > +{
> > +#ifndef arch_test_bit
> > +#define arch_test_bit generic_test_bit
> > +#endif
> > +
> > +    return arch_test_bit(nr, addr);
> > +}
> > +#define test_bit(nr, addr) ({                           \
> > +    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
> > +    test_bit(nr, addr);                                 \
> > +})
> 
> From here onwards, ...
> 
> > +static always_inline __pure int fls(unsigned int x)
> > +{
> > +    if (__builtin_constant_p(x))
> > +        return generic_fls(x);
> > +
> > +#ifndef arch_fls
> > +#define arch_fls generic_fls
> > +#endif
> > +
> > +    return arch_fls(x);
> > +}
> > +
> > +static always_inline __pure int flsl(unsigned long x)
> > +{
> > +    if (__builtin_constant_p(x))
> > +        return generic_flsl(x);
> > +
> > +#ifndef arch_flsl
> > +#define arch_flsl generic_flsl
> > +#endif
> > +
> > +    return arch_flsl(x);
> > +}
> 
> ... does all of this really belong here? Neither title nor
> description have
> any hint towards this.
No, it should be a part of the 	[PATCH v7 05/19] xen/bitops: implement
fls{l}() in common logic. An issue during rebase. I'll update that.

> 
> >  /*
> >   * Find First Set bit.  Bits are labelled from 1.
> >   */
> 
> This context suggests there's a dependency on an uncommitted patch.
> Nothing
> above says so. I guess you have a remark in the cover letter, yet imo
> that's
> only partly helpful.
Is it really a hard dependency?
The current patch series really depends on ffs{l}() and that was
mentioned in the cover letter ( I'll reword the cover letter to explain
why exactly this dependency is needed ), but this patch isn't really
depends on Andrew's patch series, where ffs{l}() are introduced.

~ Oleksii

Re: [PATCH v7 07/19] xen/riscv: introduce bitops.h

[Oleksii]

On Thu, 2024-04-04 at 16:47 +0200, Jan Beulich wrote:
> On 03.04.2024 12:20, Oleksii Kurochko wrote:
> > --- /dev/null
> > +++ b/xen/arch/riscv/include/asm/bitops.h
> > @@ -0,0 +1,146 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/* Copyright (C) 2012 Regents of the University of California */
> > +
> > +#ifndef _ASM_RISCV_BITOPS_H
> > +#define _ASM_RISCV_BITOPS_H
> > +
> > +#include <asm/system.h>
> > +
> > +#undef BITOP_BITS_PER_WORD
> > +#undef bitop_uint_t
> > +
> > +#define BITOP_BITS_PER_WORD BITS_PER_LONG
> > +#define bitop_uint_t unsigned long
> > +
> > +#if BITS_PER_LONG == 64
> > +#define __AMO(op)   "amo" #op ".d"
> > +#elif BITS_PER_LONG == 32
> > +#define __AMO(op)   "amo" #op ".w"
> > +#else
> > +#error "Unexpected BITS_PER_LONG"
> > +#endif
> > +
> > +#define __set_bit(n, p)      set_bit(n, p)
> > +#define __clear_bit(n, p)    clear_bit(n, p)
> 
> First without comment and then ...
> 
> > +/* Based on linux/arch/include/asm/bitops.h */
> > +
> > +/*
> > + * Non-atomic bit manipulation.
> > + *
> > + * Implemented using atomics to be interrupt safe. Could
> > alternatively
> > + * implement with local interrupt masking.
> > + */
> > +#define __set_bit(n, p)      set_bit(n, p)
> > +#define __clear_bit(n, p)    clear_bit(n, p)
> 
> ... with one?
Hmm, it seems like rebasing issue with autoconflict resolving. It
should be only definitions with the comment.

> 
> > +/* Based on linux/arch/include/asm/bitops.h */
> 
> Does this really need repeating?
> 
> > +#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
> > +({                                                      \
> > +    unsigned long res, mask;                            \
> 
> bitop_uint_t?
> 
> > +    mask = BITOP_MASK(nr);                              \
> > +    asm volatile (                                       \
> 
> Nit: One too many padding blanks.
> 
> > +        __AMO(op) #ord " %0, %2, %1"                    \
> > +        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
> > +        : "r" (mod(mask))                               \
> > +        : "memory");                                    \
> > +    ((res & mask) != 0);                                \
> > +})
> > +
> > +#define op_bit_ord(op, mod, nr, addr, ord)      \
> > +    asm volatile (                              \
> > +        __AMO(op) #ord " zero, %1, %0"          \
> > +        : "+A" (addr[BITOP_WORD(nr)])           \
> > +        : "r" (mod(BITOP_MASK(nr)))             \
> > +        : "memory");
> > +
> > +#define test_and_op_bit(op, mod, nr, addr)    \
> > +    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
> > +#define op_bit(op, mod, nr, addr) \
> > +    op_bit_ord(op, mod, nr, addr, )
> > +
> > +/* Bitmask modifiers */
> > +#define NOP(x)    (x)
> > +#define NOT(x)    (~(x))
> > +
> > +/**
> > + * test_and_set_bit - Set a bit and return its old value
> > + * @nr: Bit to set
> > + * @addr: Address to count from
> > + */
> > +static inline int test_and_set_bit(int nr, volatile void *p)
> 
> In patch 4 you switched to bool. Any reason you didn't here, too?
Sure, it should return bool here too and probably some other functions
below.

> 
> > +{
> > +    volatile bitop_uint_t *addr = p;
> > +
> > +    return test_and_op_bit(or, NOP, nr, addr);
> > +}
> > +
> > +/**
> > + * test_and_clear_bit - Clear a bit and return its old value
> > + * @nr: Bit to clear
> > + * @addr: Address to count from
> > + */
> > +static inline int test_and_clear_bit(int nr, volatile void *p)
> > +{
> > +    volatile bitop_uint_t *addr = p;
> > +
> > +    return test_and_op_bit(and, NOT, nr, addr);
> > +}
> > +
> > +/**
> > + * set_bit - Atomically set a bit in memory
> > + * @nr: the bit to set
> > + * @addr: the address to start counting from
> > + *
> > + * Note that @nr may be almost arbitrarily large; this function is
> > not
> > + * restricted to acting on a single-word quantity.
> > + */
> > +static inline void set_bit(int nr, volatile void *p)
> > +{
> > +    volatile bitop_uint_t *addr = p;
> > +
> > +    op_bit(or, NOP, nr, addr);
> > +}
> > +
> > +/**
> > + * clear_bit - Clears a bit in memory
> > + * @nr: Bit to clear
> > + * @addr: Address to start counting from
> > + */
> > +static inline void clear_bit(int nr, volatile void *p)
> > +{
> > +    volatile bitop_uint_t *addr = p;
> > +
> > +    op_bit(and, NOT, nr, addr);
> > +}
> > +
> > +/**
> > + * test_and_change_bit - Toggle (change) a bit and return its old
> > value
> > + * @nr: Bit to change
> > + * @addr: Address to count from
> > + *
> > + * This operation is atomic and cannot be reordered.
> > + * It also implies a memory barrier.
> > + */
> > +static inline int test_and_change_bit(int nr, volatile unsigned
> > long *addr)
> 
> unsigned long?
It should be volatile void *. Something that was copied from Linux
kernel and I missed to change.

~ Oleksii

> 
> > +{
> > +	return test_and_op_bit(xor, NOP, nr, addr);
> > +}
> 
> Perhaps better to move up a little, next to the other test_and-s?
> 
> Also - nit: Hard tab used for indentation.
> 
> > +#undef test_and_op_bit
> > +#undef __op_bit
> 
> This no longer has any effect in this shape.
> 
> Jan

Re: [PATCH v7 08/19] xen/riscv: introduce cmpxchg.h

[Oleksii]

On Thu, 2024-04-04 at 17:01 +0200, Jan Beulich wrote:
> On 03.04.2024 12:20, Oleksii Kurochko wrote:
> > +#define emulate_cmpxchg_1_2(ptr, old, new, lr_sfx, sc_sfx) \
> > +({ \
> > +    uint32_t *aligned_ptr; \
> > +    unsigned long alignment_mask = sizeof(*aligned_ptr) -
> > sizeof(*(ptr)); \
> > +    uint8_t new_val_bit = \
> > +        ((unsigned long)(ptr) & alignment_mask) * BITS_PER_BYTE; \
> > +    unsigned long mask = \
> > +        GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) <<
> > new_val_bit; \
> > +    unsigned int old_ = (old) << new_val_bit; \
> > +    unsigned int new_ = (new) << new_val_bit; \
> > +    unsigned int old_val; \
> > +    unsigned int scratch; \
> > +    \
> > +    aligned_ptr = (uint32_t *)((unsigned long)ptr &
> > ~alignment_mask); \
> > +    \
> > +    asm volatile ( \
> > +        "0: lr.w" lr_sfx " %[scratch], %[ptr_]\n" \
> > +        "   and  %[old_val], %[scratch], %[mask]\n" \
> > +        "   bne  %[old_val], %z[old_], 1f\n" \
> > +        /* the following line is an equivalent to:
> > +         *     scratch = old_val & ~mask;
> > +         * And to elimanate one ( likely register ) input it was
> > decided
> > +         * to use:
> > +         *     scratch = old_val ^ scratch
> > +         */ \
> 
> I'm surprised this compiles without \-es inside the comment as well.
> Line
> splicing happens ahead of comment recognition as per the spec.
I'll add \-es inside the comment.

> 
> Everything else okay-ish to me now, but I can't very well given an
> ack for
> a patch depending on things that haven't been committed yet and may
> never be.
Except 3 dependency ( which should be dropped as nothing anymore is
used now from this headers ):
   #include <asm/fence.h>
   #include <asm/io.h>
   #include <asm/system.h>

It seems everything else doesn't depending on other things, does it?

~ Oleksii

Re: [PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Jan Beulich]

On 04.04.2024 17:45, Oleksii wrote:
> On Thu, 2024-04-04 at 15:22 +0200, Jan Beulich wrote:
>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>> --- a/xen/include/xen/bitops.h
>>> +++ b/xen/include/xen/bitops.h
>>> @@ -65,10 +65,164 @@ static inline int generic_flsl(unsigned long
>>> x)
>>>   * scope
>>>   */
>>>  
>>> +#define BITOP_BITS_PER_WORD 32
>>> +/* typedef uint32_t bitop_uint_t; */
>>> +#define bitop_uint_t uint32_t
>>
>> So no arch overrides permitted anymore at all?
> Not really, I agree that it is ugly, but I expected that arch will use
> undef to override.

Which would be fine in principle, just that Misra wants us to avoid #undef-s
(iirc).

>>>  /*
>>>   * Find First Set bit.  Bits are labelled from 1.
>>>   */
>>
>> This context suggests there's a dependency on an uncommitted patch.
>> Nothing
>> above says so. I guess you have a remark in the cover letter, yet imo
>> that's
>> only partly helpful.
> Is it really a hard dependency?
> The current patch series really depends on ffs{l}() and that was
> mentioned in the cover letter ( I'll reword the cover letter to explain
> why exactly this dependency is needed ), but this patch isn't really
> depends on Andrew's patch series, where ffs{l}() are introduced.

If anyone acked this patch, and if it otherwise looked independent, it would
be a candidate for committing. Just that it won't apply for a non-obvious
reason.

Jan

Re: [PATCH v7 08/19] xen/riscv: introduce cmpxchg.h

[Jan Beulich]

On 04.04.2024 18:04, Oleksii wrote:
> On Thu, 2024-04-04 at 17:01 +0200, Jan Beulich wrote:
>> On 03.04.2024 12:20, Oleksii Kurochko wrote:
>> Everything else okay-ish to me now, but I can't very well given an
>> ack for
>> a patch depending on things that haven't been committed yet and may
>> never be.
> Except 3 dependency ( which should be dropped as nothing anymore is
> used now from this headers ):
>    #include <asm/fence.h>
>    #include <asm/io.h>
>    #include <asm/system.h>
> 
> It seems everything else doesn't depending on other things, does it?

Didn't I see a STATIC_ASSERT_UNREACHABLE() somewhere?

Jan

Re: [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler

[Oleksii]

On Thu, 2024-04-04 at 17:43 +0200, Jan Beulich wrote:
> On 04.04.2024 17:18, Oleksii wrote:
> > On Thu, 2024-04-04 at 12:07 +0200, Jan Beulich wrote:
> > > On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > > > --- a/xen/arch/riscv/arch.mk
> > > > +++ b/xen/arch/riscv/arch.mk
> > > > @@ -3,16 +3,27 @@
> > > >  
> > > >  $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
> > > >  
> > > > -CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64
> > > > +riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32
> > > > +riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64
> > > >  
> > > >  riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
> > > >  riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
> > > >  
> > > > +riscv-generic-flags := $(riscv-abi-y) -march=$(riscv-march-y)
> > > > +
> > > > +zbb := $(call as-insn,$(CC) $(riscv-generic-
> > > > flags)_zbb,"",_zbb)
> > > 
> > > While committing another question popped up: Why "" (i.e. no
> > > insn)
> > > here, ...
> > > 
> > > > +zihintpause := $(call as-insn,\
> > > > +               $(CC) $(riscv-generic-
> > > > flags)_zihintpause,"pause",_zihintpause)
> > > 
> > > ... but "pause" here?
> > 
> > In the case of the Zbb extension, we don't check for a specific
> > instruction, but with the Zihintpause, the idea was to verify if
> > the
> > pause instruction is supported or not.
> 
> And why's this verification relevant here, but not for Zbb?
It is not relevant and can be dropped. I'll do a follow-up patch.

~ Oleksii

> 
> Jan
> 
> > However, in both checks, there
> > might be no instruction as an argument of as-insn.
> > 
> > ~ Oleksii
> 

Re: [PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Oleksii]

On Thu, 2024-04-04 at 18:12 +0200, Jan Beulich wrote:
> On 04.04.2024 17:45, Oleksii wrote:
> > On Thu, 2024-04-04 at 15:22 +0200, Jan Beulich wrote:
> > > On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > > > --- a/xen/include/xen/bitops.h
> > > > +++ b/xen/include/xen/bitops.h
> > > > @@ -65,10 +65,164 @@ static inline int generic_flsl(unsigned
> > > > long
> > > > x)
> > > >   * scope
> > > >   */
> > > >  
> > > > +#define BITOP_BITS_PER_WORD 32
> > > > +/* typedef uint32_t bitop_uint_t; */
> > > > +#define bitop_uint_t uint32_t
> > > 
> > > So no arch overrides permitted anymore at all?
> > Not really, I agree that it is ugly, but I expected that arch will
> > use
> > undef to override.
> 
> Which would be fine in principle, just that Misra wants us to avoid
> #undef-s
> (iirc).
Could you please give me a recommendation how to do that better?

The reason why I put this defintions before inclusion of asm/bitops.h
as RISC-V specific code uses these definitions inside it, so they
should be defined before asm/bitops.h; other option is to define these
definitions inside asm/bitops.h for each architecture.

> 
> > > >  /*
> > > >   * Find First Set bit.  Bits are labelled from 1.
> > > >   */
> > > 
> > > This context suggests there's a dependency on an uncommitted
> > > patch.
> > > Nothing
> > > above says so. I guess you have a remark in the cover letter, yet
> > > imo
> > > that's
> > > only partly helpful.
> > Is it really a hard dependency?
> > The current patch series really depends on ffs{l}() and that was
> > mentioned in the cover letter ( I'll reword the cover letter to
> > explain
> > why exactly this dependency is needed ), but this patch isn't
> > really
> > depends on Andrew's patch series, where ffs{l}() are introduced.
> 
> If anyone acked this patch, and if it otherwise looked independent,
> it would
> be a candidate for committing. Just that it won't apply for a non-
> obvious
> reason.
I didn't think about the it won't apply. In this I have to definitely
mention this moment in cover letter. Thanks.

~ Oleksii

Re: [PATCH v7 08/19] xen/riscv: introduce cmpxchg.h

[Oleksii]

On Thu, 2024-04-04 at 18:14 +0200, Jan Beulich wrote:
> On 04.04.2024 18:04, Oleksii wrote:
> > On Thu, 2024-04-04 at 17:01 +0200, Jan Beulich wrote:
> > > On 03.04.2024 12:20, Oleksii Kurochko wrote:
> > > Everything else okay-ish to me now, but I can't very well given
> > > an
> > > ack for
> > > a patch depending on things that haven't been committed yet and
> > > may
> > > never be.
> > Except 3 dependency ( which should be dropped as nothing anymore is
> > used now from this headers ):
> >    #include <asm/fence.h>
> >    #include <asm/io.h>
> >    #include <asm/system.h>
> > 
> > It seems everything else doesn't depending on other things, does
> > it?
> 
> Didn't I see a STATIC_ASSERT_UNREACHABLE() somewhere?
Sure, forgot about this macros. In this case, I think it would be
better to return back:

   extern void __bad_{xchg,...)(...);
   
   ....
   default:
       __bad_xchg(ptr, size), ret = 0;

And do a follow-up patch when it will be committed.

~ Oleksii

Re: [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler

[Jan Beulich]

On 04.04.2024 18:17, Oleksii wrote:
> On Thu, 2024-04-04 at 17:43 +0200, Jan Beulich wrote:
>> On 04.04.2024 17:18, Oleksii wrote:
>>> On Thu, 2024-04-04 at 12:07 +0200, Jan Beulich wrote:
>>>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>>>> --- a/xen/arch/riscv/arch.mk
>>>>> +++ b/xen/arch/riscv/arch.mk
>>>>> @@ -3,16 +3,27 @@
>>>>>  
>>>>>  $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
>>>>>  
>>>>> -CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64
>>>>> +riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32
>>>>> +riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64
>>>>>  
>>>>>  riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
>>>>>  riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
>>>>>  
>>>>> +riscv-generic-flags := $(riscv-abi-y) -march=$(riscv-march-y)
>>>>> +
>>>>> +zbb := $(call as-insn,$(CC) $(riscv-generic-
>>>>> flags)_zbb,"",_zbb)
>>>>
>>>> While committing another question popped up: Why "" (i.e. no
>>>> insn)
>>>> here, ...
>>>>
>>>>> +zihintpause := $(call as-insn,\
>>>>> +               $(CC) $(riscv-generic-
>>>>> flags)_zihintpause,"pause",_zihintpause)
>>>>
>>>> ... but "pause" here?
>>>
>>> In the case of the Zbb extension, we don't check for a specific
>>> instruction, but with the Zihintpause, the idea was to verify if
>>> the
>>> pause instruction is supported or not.
>>
>> And why's this verification relevant here, but not for Zbb?
> It is not relevant and can be dropped.

Is it really not? Aren't you checking two things for Zihintpause (compiler
and assembler support), while checking only one (compiler) for Zbb?

Jan

Re: [PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Jan Beulich]

On 04.04.2024 18:24, Oleksii wrote:
> On Thu, 2024-04-04 at 18:12 +0200, Jan Beulich wrote:
>> On 04.04.2024 17:45, Oleksii wrote:
>>> On Thu, 2024-04-04 at 15:22 +0200, Jan Beulich wrote:
>>>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>>>> --- a/xen/include/xen/bitops.h
>>>>> +++ b/xen/include/xen/bitops.h
>>>>> @@ -65,10 +65,164 @@ static inline int generic_flsl(unsigned
>>>>> long
>>>>> x)
>>>>>   * scope
>>>>>   */
>>>>>  
>>>>> +#define BITOP_BITS_PER_WORD 32
>>>>> +/* typedef uint32_t bitop_uint_t; */
>>>>> +#define bitop_uint_t uint32_t
>>>>
>>>> So no arch overrides permitted anymore at all?
>>> Not really, I agree that it is ugly, but I expected that arch will
>>> use
>>> undef to override.
>>
>> Which would be fine in principle, just that Misra wants us to avoid
>> #undef-s
>> (iirc).
> Could you please give me a recommendation how to do that better?
> 
> The reason why I put this defintions before inclusion of asm/bitops.h
> as RISC-V specific code uses these definitions inside it, so they
> should be defined before asm/bitops.h; other option is to define these
> definitions inside asm/bitops.h for each architecture.

Earlier on you had it that other way already (in a different header,
but the principle is the same): Move the generic definitions immediately
past inclusion of asm/bitops.h and frame them with #ifndef.

Jan

Re: [PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Oleksii]

On Fri, 2024-04-05 at 08:11 +0200, Jan Beulich wrote:
> On 04.04.2024 18:24, Oleksii wrote:
> > On Thu, 2024-04-04 at 18:12 +0200, Jan Beulich wrote:
> > > On 04.04.2024 17:45, Oleksii wrote:
> > > > On Thu, 2024-04-04 at 15:22 +0200, Jan Beulich wrote:
> > > > > On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > > > > > --- a/xen/include/xen/bitops.h
> > > > > > +++ b/xen/include/xen/bitops.h
> > > > > > @@ -65,10 +65,164 @@ static inline int
> > > > > > generic_flsl(unsigned
> > > > > > long
> > > > > > x)
> > > > > >   * scope
> > > > > >   */
> > > > > >  
> > > > > > +#define BITOP_BITS_PER_WORD 32
> > > > > > +/* typedef uint32_t bitop_uint_t; */
> > > > > > +#define bitop_uint_t uint32_t
> > > > > 
> > > > > So no arch overrides permitted anymore at all?
> > > > Not really, I agree that it is ugly, but I expected that arch
> > > > will
> > > > use
> > > > undef to override.
> > > 
> > > Which would be fine in principle, just that Misra wants us to
> > > avoid
> > > #undef-s
> > > (iirc).
> > Could you please give me a recommendation how to do that better?
> > 
> > The reason why I put this defintions before inclusion of
> > asm/bitops.h
> > as RISC-V specific code uses these definitions inside it, so they
> > should be defined before asm/bitops.h; other option is to define
> > these
> > definitions inside asm/bitops.h for each architecture.
> 
> Earlier on you had it that other way already (in a different header,
> but the principle is the same): Move the generic definitions
> immediately
> past inclusion of asm/bitops.h and frame them with #ifndef.
It can be done in this way:
xen/bitops.h:
   ...
   #include <asm/bitops.h>
   
   #ifndef BITOP_TYPE
   #define BITOP_BITS_PER_WORD 32
   /* typedef uint32_t bitop_uint_t; */
   #define bitop_uint_t uint32_t
   #endif
   ...
   
But then RISC-V will fail as it is using bitop_uint_t inside
asm/bitops.h.
So, at least, for RISC-V it will be needed to add asm/bitops.h:
   #define BITOP_BITS_PER_WORD 32
   /* typedef uint32_t bitop_uint_t; */
   #define bitop_uint_t uint32_t
   

It seems to me that this breaks the idea of having these macro
definitions generic, as RISC-V will redefine BITOP_BITS_PER_WORD and
bitop_uint_t with the same values as the generic ones.
   
~ Oleksii

> 
> Jan

Re: [PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Jan Beulich]

On 05.04.2024 09:56, Oleksii wrote:
> On Fri, 2024-04-05 at 08:11 +0200, Jan Beulich wrote:
>> On 04.04.2024 18:24, Oleksii wrote:
>>> On Thu, 2024-04-04 at 18:12 +0200, Jan Beulich wrote:
>>>> On 04.04.2024 17:45, Oleksii wrote:
>>>>> On Thu, 2024-04-04 at 15:22 +0200, Jan Beulich wrote:
>>>>>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>>>>>> --- a/xen/include/xen/bitops.h
>>>>>>> +++ b/xen/include/xen/bitops.h
>>>>>>> @@ -65,10 +65,164 @@ static inline int
>>>>>>> generic_flsl(unsigned
>>>>>>> long
>>>>>>> x)
>>>>>>>   * scope
>>>>>>>   */
>>>>>>>  
>>>>>>> +#define BITOP_BITS_PER_WORD 32
>>>>>>> +/* typedef uint32_t bitop_uint_t; */
>>>>>>> +#define bitop_uint_t uint32_t
>>>>>>
>>>>>> So no arch overrides permitted anymore at all?
>>>>> Not really, I agree that it is ugly, but I expected that arch
>>>>> will
>>>>> use
>>>>> undef to override.
>>>>
>>>> Which would be fine in principle, just that Misra wants us to
>>>> avoid
>>>> #undef-s
>>>> (iirc).
>>> Could you please give me a recommendation how to do that better?
>>>
>>> The reason why I put this defintions before inclusion of
>>> asm/bitops.h
>>> as RISC-V specific code uses these definitions inside it, so they
>>> should be defined before asm/bitops.h; other option is to define
>>> these
>>> definitions inside asm/bitops.h for each architecture.
>>
>> Earlier on you had it that other way already (in a different header,
>> but the principle is the same): Move the generic definitions
>> immediately
>> past inclusion of asm/bitops.h and frame them with #ifndef.
> It can be done in this way:
> xen/bitops.h:
>    ...
>    #include <asm/bitops.h>
>    
>    #ifndef BITOP_TYPE
>    #define BITOP_BITS_PER_WORD 32
>    /* typedef uint32_t bitop_uint_t; */
>    #define bitop_uint_t uint32_t
>    #endif
>    ...
>    
> But then RISC-V will fail as it is using bitop_uint_t inside
> asm/bitops.h.
> So, at least, for RISC-V it will be needed to add asm/bitops.h:
>    #define BITOP_BITS_PER_WORD 32
>    /* typedef uint32_t bitop_uint_t; */
>    #define bitop_uint_t uint32_t
>    
> 
> It seems to me that this breaks the idea of having these macro
> definitions generic, as RISC-V will redefine BITOP_BITS_PER_WORD and
> bitop_uint_t with the same values as the generic ones.

I don't follow. Right now patch 7 has

#undef BITOP_BITS_PER_WORD
#undef bitop_uint_t

#define BITOP_BITS_PER_WORD BITS_PER_LONG
#define bitop_uint_t unsigned long

You'd drop the #undef-s and keep the #define-s. You want to override them
both, after all.

A problem would arise for _another_ arch wanting to use these (default)
types in its asm/bitops.h. Which then could still be solved by having a
types-only header. Recall the discussion on the last summit of us meaning
to switch to such a model anyway (perhaps it being xen/types/bitops.h and
asm/types/bitops.h then), in a broader fashion? IOW for now you could use
the simple approach as long as no other arch needs the types in its
asm/bitops.h. Later we would introduce the types-only headers, thus
catering for possible future uses.

Jan

Re: [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler

[Oleksii]

On Fri, 2024-04-05 at 08:08 +0200, Jan Beulich wrote:
> On 04.04.2024 18:17, Oleksii wrote:
> > On Thu, 2024-04-04 at 17:43 +0200, Jan Beulich wrote:
> > > On 04.04.2024 17:18, Oleksii wrote:
> > > > On Thu, 2024-04-04 at 12:07 +0200, Jan Beulich wrote:
> > > > > On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > > > > > --- a/xen/arch/riscv/arch.mk
> > > > > > +++ b/xen/arch/riscv/arch.mk
> > > > > > @@ -3,16 +3,27 @@
> > > > > >  
> > > > > >  $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
> > > > > >  
> > > > > > -CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64
> > > > > > +riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32
> > > > > > +riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64
> > > > > >  
> > > > > >  riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
> > > > > >  riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-
> > > > > > y)c
> > > > > >  
> > > > > > +riscv-generic-flags := $(riscv-abi-y) -march=$(riscv-
> > > > > > march-y)
> > > > > > +
> > > > > > +zbb := $(call as-insn,$(CC) $(riscv-generic-
> > > > > > flags)_zbb,"",_zbb)
> > > > > 
> > > > > While committing another question popped up: Why "" (i.e. no
> > > > > insn)
> > > > > here, ...
> > > > > 
> > > > > > +zihintpause := $(call as-insn,\
> > > > > > +               $(CC) $(riscv-generic-
> > > > > > flags)_zihintpause,"pause",_zihintpause)
> > > > > 
> > > > > ... but "pause" here?
> > > > 
> > > > In the case of the Zbb extension, we don't check for a specific
> > > > instruction, but with the Zihintpause, the idea was to verify
> > > > if
> > > > the
> > > > pause instruction is supported or not.
> > > 
> > > And why's this verification relevant here, but not for Zbb?
> > It is not relevant and can be dropped.
> 
> Is it really not? Aren't you checking two things for Zihintpause
> (compiler
> and assembler support), while checking only one (compiler) for Zbb?
You are right.
I made again an assumption that binutils are as new as a compiler.

Then it makes sense to replace an argument if as-insn "" with andn t0,
t0, t0 for Zbb:

   -zbb := $(call as-insn,$(CC) $(riscv-generic-flags)_zbb,"",_zbb)
   +zbb_insn := "andn t0, t0, t0"
   +zbb := $(call as-insn,$(CC) $(riscv-generic-
   flags)_zbb,${zbb_insn},_zbb)

~ Oleksii

> 
> Jan

Re: [PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Oleksii]

On Fri, 2024-04-05 at 10:05 +0200, Jan Beulich wrote:
> On 05.04.2024 09:56, Oleksii wrote:
> > On Fri, 2024-04-05 at 08:11 +0200, Jan Beulich wrote:
> > > On 04.04.2024 18:24, Oleksii wrote:
> > > > On Thu, 2024-04-04 at 18:12 +0200, Jan Beulich wrote:
> > > > > On 04.04.2024 17:45, Oleksii wrote:
> > > > > > On Thu, 2024-04-04 at 15:22 +0200, Jan Beulich wrote:
> > > > > > > On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > > > > > > > --- a/xen/include/xen/bitops.h
> > > > > > > > +++ b/xen/include/xen/bitops.h
> > > > > > > > @@ -65,10 +65,164 @@ static inline int
> > > > > > > > generic_flsl(unsigned
> > > > > > > > long
> > > > > > > > x)
> > > > > > > >   * scope
> > > > > > > >   */
> > > > > > > >  
> > > > > > > > +#define BITOP_BITS_PER_WORD 32
> > > > > > > > +/* typedef uint32_t bitop_uint_t; */
> > > > > > > > +#define bitop_uint_t uint32_t
> > > > > > > 
> > > > > > > So no arch overrides permitted anymore at all?
> > > > > > Not really, I agree that it is ugly, but I expected that
> > > > > > arch
> > > > > > will
> > > > > > use
> > > > > > undef to override.
> > > > > 
> > > > > Which would be fine in principle, just that Misra wants us to
> > > > > avoid
> > > > > #undef-s
> > > > > (iirc).
> > > > Could you please give me a recommendation how to do that
> > > > better?
> > > > 
> > > > The reason why I put this defintions before inclusion of
> > > > asm/bitops.h
> > > > as RISC-V specific code uses these definitions inside it, so
> > > > they
> > > > should be defined before asm/bitops.h; other option is to
> > > > define
> > > > these
> > > > definitions inside asm/bitops.h for each architecture.
> > > 
> > > Earlier on you had it that other way already (in a different
> > > header,
> > > but the principle is the same): Move the generic definitions
> > > immediately
> > > past inclusion of asm/bitops.h and frame them with #ifndef.
> > It can be done in this way:
> > xen/bitops.h:
> >    ...
> >    #include <asm/bitops.h>
> >    
> >    #ifndef BITOP_TYPE
> >    #define BITOP_BITS_PER_WORD 32
> >    /* typedef uint32_t bitop_uint_t; */
> >    #define bitop_uint_t uint32_t
> >    #endif
> >    ...
> >    
> > But then RISC-V will fail as it is using bitop_uint_t inside
> > asm/bitops.h.
> > So, at least, for RISC-V it will be needed to add asm/bitops.h:
> >    #define BITOP_BITS_PER_WORD 32
> >    /* typedef uint32_t bitop_uint_t; */
> >    #define bitop_uint_t uint32_t
> >    
> > 
> > It seems to me that this breaks the idea of having these macro
> > definitions generic, as RISC-V will redefine BITOP_BITS_PER_WORD
> > and
> > bitop_uint_t with the same values as the generic ones.
> 
> I don't follow. Right now patch 7 has
> 
> #undef BITOP_BITS_PER_WORD
> #undef bitop_uint_t
> 
> #define BITOP_BITS_PER_WORD BITS_PER_LONG
> #define bitop_uint_t unsigned long
> 
> You'd drop the #undef-s and keep the #define-s. You want to override
> them
> both, after all.
> 
> A problem would arise for _another_ arch wanting to use these
> (default)
> types in its asm/bitops.h. Which then could still be solved by having
> a
> types-only header.
This problem arise now for Arm and PPC which use BITOP_BITS_PER_WORD
inside it. Then it is needed to define BITOP_BITS_PER_WORD=32 in
asm/bitops.h for Arm and PPC. If it is okay, then I will happy to
follow this approach.

>  Recall the discussion on the last summit of us meaning
> to switch to such a model anyway (perhaps it being xen/types/bitops.h
> and
> asm/types/bitops.h then), in a broader fashion? IOW for now you could
> use
> the simple approach as long as no other arch needs the types in its
> asm/bitops.h. Later we would introduce the types-only headers, thus
> catering for possible future uses.
Do we really need asm/types/bitops.h? Can't we just do the following in
asm/bitops.h:
  #ifndef BITOP_TYPE
  #include <xen/types/bitops.h>
  #endif

~ Oleksii

Re: [PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Oleksii]

On Fri, 2024-04-05 at 13:53 +0200, Oleksii wrote:
> On Fri, 2024-04-05 at 10:05 +0200, Jan Beulich wrote:
> > On 05.04.2024 09:56, Oleksii wrote:
> > > On Fri, 2024-04-05 at 08:11 +0200, Jan Beulich wrote:
> > > > On 04.04.2024 18:24, Oleksii wrote:
> > > > > On Thu, 2024-04-04 at 18:12 +0200, Jan Beulich wrote:
> > > > > > On 04.04.2024 17:45, Oleksii wrote:
> > > > > > > On Thu, 2024-04-04 at 15:22 +0200, Jan Beulich wrote:
> > > > > > > > On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > > > > > > > > --- a/xen/include/xen/bitops.h
> > > > > > > > > +++ b/xen/include/xen/bitops.h
> > > > > > > > > @@ -65,10 +65,164 @@ static inline int
> > > > > > > > > generic_flsl(unsigned
> > > > > > > > > long
> > > > > > > > > x)
> > > > > > > > >   * scope
> > > > > > > > >   */
> > > > > > > > >  
> > > > > > > > > +#define BITOP_BITS_PER_WORD 32
> > > > > > > > > +/* typedef uint32_t bitop_uint_t; */
> > > > > > > > > +#define bitop_uint_t uint32_t
> > > > > > > > 
> > > > > > > > So no arch overrides permitted anymore at all?
> > > > > > > Not really, I agree that it is ugly, but I expected that
> > > > > > > arch
> > > > > > > will
> > > > > > > use
> > > > > > > undef to override.
> > > > > > 
> > > > > > Which would be fine in principle, just that Misra wants us
> > > > > > to
> > > > > > avoid
> > > > > > #undef-s
> > > > > > (iirc).
> > > > > Could you please give me a recommendation how to do that
> > > > > better?
> > > > > 
> > > > > The reason why I put this defintions before inclusion of
> > > > > asm/bitops.h
> > > > > as RISC-V specific code uses these definitions inside it, so
> > > > > they
> > > > > should be defined before asm/bitops.h; other option is to
> > > > > define
> > > > > these
> > > > > definitions inside asm/bitops.h for each architecture.
> > > > 
> > > > Earlier on you had it that other way already (in a different
> > > > header,
> > > > but the principle is the same): Move the generic definitions
> > > > immediately
> > > > past inclusion of asm/bitops.h and frame them with #ifndef.
> > > It can be done in this way:
> > > xen/bitops.h:
> > >    ...
> > >    #include <asm/bitops.h>
> > >    
> > >    #ifndef BITOP_TYPE
> > >    #define BITOP_BITS_PER_WORD 32
> > >    /* typedef uint32_t bitop_uint_t; */
> > >    #define bitop_uint_t uint32_t
> > >    #endif
> > >    ...
> > >    
> > > But then RISC-V will fail as it is using bitop_uint_t inside
> > > asm/bitops.h.
> > > So, at least, for RISC-V it will be needed to add asm/bitops.h:
> > >    #define BITOP_BITS_PER_WORD 32
> > >    /* typedef uint32_t bitop_uint_t; */
> > >    #define bitop_uint_t uint32_t
> > >    
> > > 
> > > It seems to me that this breaks the idea of having these macro
> > > definitions generic, as RISC-V will redefine BITOP_BITS_PER_WORD
> > > and
> > > bitop_uint_t with the same values as the generic ones.
> > 
> > I don't follow. Right now patch 7 has
> > 
> > #undef BITOP_BITS_PER_WORD
> > #undef bitop_uint_t
> > 
> > #define BITOP_BITS_PER_WORD BITS_PER_LONG
> > #define bitop_uint_t unsigned long
> > 
> > You'd drop the #undef-s and keep the #define-s. You want to
> > override
> > them
> > both, after all.
> > 
> > A problem would arise for _another_ arch wanting to use these
> > (default)
> > types in its asm/bitops.h. Which then could still be solved by
> > having
> > a
> > types-only header.
> This problem arise now for Arm and PPC which use BITOP_BITS_PER_WORD
> inside it. Then it is needed to define BITOP_BITS_PER_WORD=32 in
> asm/bitops.h for Arm and PPC. If it is okay, then I will happy to
> follow this approach.
> 
> >  Recall the discussion on the last summit of us meaning
> > to switch to such a model anyway (perhaps it being
> > xen/types/bitops.h
> > and
> > asm/types/bitops.h then), in a broader fashion? IOW for now you
> > could
> > use
> > the simple approach as long as no other arch needs the types in its
> > asm/bitops.h. Later we would introduce the types-only headers, thus
> > catering for possible future uses.
> Do we really need asm/types/bitops.h? Can't we just do the following
> in
> asm/bitops.h:
>   #ifndef BITOP_TYPE
>   #include <xen/types/bitops.h>
>   #endif
Or as an options just update <xen/types.h> with after inclusion of
<asm/types.h>:
   #ifndef BITOP_TYPE
      #define BITOP_BITS_PER_WORD 32
      /* typedef uint32_t bitop_uint_t; */
      #define bitop_uint_t uint32_t
   #endif
   
And then just include <xen/types.h> to <<xen/bitops.h>.

~ Oleksii
> 
> ~ Oleksii

Re: [PATCH v7 04/19] xen: introduce generic non-atomic test_*bit()

[Jan Beulich]

On 05.04.2024 13:56, Oleksii wrote:
> On Fri, 2024-04-05 at 13:53 +0200, Oleksii wrote:
>> On Fri, 2024-04-05 at 10:05 +0200, Jan Beulich wrote:
>>> On 05.04.2024 09:56, Oleksii wrote:
>>>> On Fri, 2024-04-05 at 08:11 +0200, Jan Beulich wrote:
>>>>> On 04.04.2024 18:24, Oleksii wrote:
>>>>>> On Thu, 2024-04-04 at 18:12 +0200, Jan Beulich wrote:
>>>>>>> On 04.04.2024 17:45, Oleksii wrote:
>>>>>>>> On Thu, 2024-04-04 at 15:22 +0200, Jan Beulich wrote:
>>>>>>>>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>>>>>>>>> --- a/xen/include/xen/bitops.h
>>>>>>>>>> +++ b/xen/include/xen/bitops.h
>>>>>>>>>> @@ -65,10 +65,164 @@ static inline int
>>>>>>>>>> generic_flsl(unsigned
>>>>>>>>>> long
>>>>>>>>>> x)
>>>>>>>>>>   * scope
>>>>>>>>>>   */
>>>>>>>>>>  
>>>>>>>>>> +#define BITOP_BITS_PER_WORD 32
>>>>>>>>>> +/* typedef uint32_t bitop_uint_t; */
>>>>>>>>>> +#define bitop_uint_t uint32_t
>>>>>>>>>
>>>>>>>>> So no arch overrides permitted anymore at all?
>>>>>>>> Not really, I agree that it is ugly, but I expected that
>>>>>>>> arch
>>>>>>>> will
>>>>>>>> use
>>>>>>>> undef to override.
>>>>>>>
>>>>>>> Which would be fine in principle, just that Misra wants us
>>>>>>> to
>>>>>>> avoid
>>>>>>> #undef-s
>>>>>>> (iirc).
>>>>>> Could you please give me a recommendation how to do that
>>>>>> better?
>>>>>>
>>>>>> The reason why I put this defintions before inclusion of
>>>>>> asm/bitops.h
>>>>>> as RISC-V specific code uses these definitions inside it, so
>>>>>> they
>>>>>> should be defined before asm/bitops.h; other option is to
>>>>>> define
>>>>>> these
>>>>>> definitions inside asm/bitops.h for each architecture.
>>>>>
>>>>> Earlier on you had it that other way already (in a different
>>>>> header,
>>>>> but the principle is the same): Move the generic definitions
>>>>> immediately
>>>>> past inclusion of asm/bitops.h and frame them with #ifndef.
>>>> It can be done in this way:
>>>> xen/bitops.h:
>>>>    ...
>>>>    #include <asm/bitops.h>
>>>>    
>>>>    #ifndef BITOP_TYPE
>>>>    #define BITOP_BITS_PER_WORD 32
>>>>    /* typedef uint32_t bitop_uint_t; */
>>>>    #define bitop_uint_t uint32_t
>>>>    #endif
>>>>    ...
>>>>    
>>>> But then RISC-V will fail as it is using bitop_uint_t inside
>>>> asm/bitops.h.
>>>> So, at least, for RISC-V it will be needed to add asm/bitops.h:
>>>>    #define BITOP_BITS_PER_WORD 32
>>>>    /* typedef uint32_t bitop_uint_t; */
>>>>    #define bitop_uint_t uint32_t
>>>>    
>>>>
>>>> It seems to me that this breaks the idea of having these macro
>>>> definitions generic, as RISC-V will redefine BITOP_BITS_PER_WORD
>>>> and
>>>> bitop_uint_t with the same values as the generic ones.
>>>
>>> I don't follow. Right now patch 7 has
>>>
>>> #undef BITOP_BITS_PER_WORD
>>> #undef bitop_uint_t
>>>
>>> #define BITOP_BITS_PER_WORD BITS_PER_LONG
>>> #define bitop_uint_t unsigned long
>>>
>>> You'd drop the #undef-s and keep the #define-s. You want to
>>> override
>>> them
>>> both, after all.
>>>
>>> A problem would arise for _another_ arch wanting to use these
>>> (default)
>>> types in its asm/bitops.h. Which then could still be solved by
>>> having
>>> a
>>> types-only header.
>> This problem arise now for Arm and PPC which use BITOP_BITS_PER_WORD
>> inside it. Then it is needed to define BITOP_BITS_PER_WORD=32 in
>> asm/bitops.h for Arm and PPC. If it is okay, then I will happy to
>> follow this approach.
>>
>>>  Recall the discussion on the last summit of us meaning
>>> to switch to such a model anyway (perhaps it being
>>> xen/types/bitops.h
>>> and
>>> asm/types/bitops.h then), in a broader fashion? IOW for now you
>>> could
>>> use
>>> the simple approach as long as no other arch needs the types in its
>>> asm/bitops.h. Later we would introduce the types-only headers, thus
>>> catering for possible future uses.
>> Do we really need asm/types/bitops.h? Can't we just do the following
>> in
>> asm/bitops.h:
>>   #ifndef BITOP_TYPE
>>   #include <xen/types/bitops.h>
>>   #endif

This might do, yes.

> Or as an options just update <xen/types.h> with after inclusion of
> <asm/types.h>:
>    #ifndef BITOP_TYPE
>       #define BITOP_BITS_PER_WORD 32
>       /* typedef uint32_t bitop_uint_t; */
>       #define bitop_uint_t uint32_t
>    #endif
>    
> And then just include <xen/types.h> to <<xen/bitops.h>.

That's a (transient) option as well, I guess.

Jan

Re: [PATCH v7 09/19] xen/riscv: introduce io.h

[Jan Beulich]

On 03.04.2024 12:20, Oleksii Kurochko wrote:
> The header taken form Linux 6.4.0-rc1 and is based on
> arch/riscv/include/asm/mmio.h with the following changes:
> - drop forcing of endianess for read*(), write*() functions as
>   no matter what CPU endianness, what endianness a particular device
>   (and hence its MMIO region(s)) is using is entirely independent.
>   Hence conversion, where necessary, needs to occur at a layer up.
>   Another one reason to drop endianess conversion here is:
>   https://patchwork.kernel.org/project/linux-riscv/patch/20190411115623.5749-3-hch@lst.de/
>   One of the answers of the author of the commit:
>     And we don't know if Linux will be around if that ever changes.
>     The point is:
>      a) the current RISC-V spec is LE only
>      b) the current linux port is LE only except for this little bit
>     There is no point in leaving just this bitrotting code around.  It
>     just confuses developers, (very very slightly) slows down compiles
>     and will bitrot.  It also won't be any significant help to a future
>     developer down the road doing a hypothetical BE RISC-V Linux port.
> - drop unused argument of __io_ar() macros.
> - drop "#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}"
>   as they are unnecessary.
> - Adopt the Xen code style for this header, considering that significant changes
>   are not anticipated in the future.
>   In the event of any issues, adapting them to Xen style should be easily
>   manageable.
> - drop unnecessary  __r variables in macros read*_cpu()
> - update inline assembler constraints for addr argument for
>   __raw_read{b,w,l,q} and __raw_write{b,w,l,q} to tell a compiler that
>  *addr will be accessed.
> - add stubs for __raw_readq() and __raw_writeq() for RISCV_32
> 
> Addionally, to the header was added definions of ioremap_*().
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
despite ...

> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/io.h
> @@ -0,0 +1,168 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + *  The header taken form Linux 6.4.0-rc1 and is based on
> + *  arch/riscv/include/asm/mmio.h with the following changes:
> + *   - drop forcing of endianess for read*(), write*() functions as
> + *     no matter what CPU endianness, what endianness a particular device
> + *     (and hence its MMIO region(s)) is using is entirely independent.
> + *     Hence conversion, where necessary, needs to occur at a layer up.
> + *     Another one reason to drop endianess conversion is:
> + *     https://patchwork.kernel.org/project/linux-riscv/patch/20190411115623.5749-3-hch@lst.de/
> + *     One of the answers of the author of the commit:
> + *       And we don't know if Linux will be around if that ever changes.
> + *       The point is:
> + *        a) the current RISC-V spec is LE only
> + *        b) the current linux port is LE only except for this little bit
> + *       There is no point in leaving just this bitrotting code around.  It
> + *       just confuses developers, (very very slightly) slows down compiles
> + *      and will bitrot.  It also won't be any significant help to a future
> + *       developer down the road doing a hypothetical BE RISC-V Linux port.
> + *   - drop unused argument of __io_ar() macros.
> + *   - drop "#define _raw_{read,write}{b,w,l,q} _raw_{read,write}{b,w,l,q}"
> + *     as they are unnecessary.
> + *   - Adopt the Xen code style for this header, considering that significant
> + *     changes are not anticipated in the future.
> + *     In the event of any issues, adapting them to Xen style should be easily
> + *     manageable.
> + *   - drop unnecessary __r variables in macros read*_cpu()
> + *   - update inline assembler constraints for addr argument for
> + *     __raw_read{b,w,l,q} and __raw_write{b,w,l,q} to tell a compiler that
> + *     *addr will be accessed.
> + *
> + * Copyright (C) 1996-2000 Russell King
> + * Copyright (C) 2012 ARM Ltd.
> + * Copyright (C) 2014 Regents of the University of California
> + * Copyright (C) 2024 Vates
> + */
> +
> +#ifndef _ASM_RISCV_IO_H
> +#define _ASM_RISCV_IO_H
> +
> +#include <asm/byteorder.h>
> +
> +/*
> + * The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't
> + * change the properties of memory regions.  This should be fixed by the
> + * upcoming platform spec.
> + */
> +#define ioremap_nocache(addr, size) ioremap(addr, size)
> +#define ioremap_wc(addr, size) ioremap(addr, size)
> +#define ioremap_wt(addr, size) ioremap(addr, size)
> +
> +/* Generic IO read/write.  These perform native-endian accesses. */
> +static inline void __raw_writeb(uint8_t val, volatile void __iomem *addr)
> +{
> +    asm volatile ( "sb %1, %0"
> +                   : "=m" (*(volatile uint8_t __force *)addr) : "r" (val) );
> +}
> +
> +static inline void __raw_writew(uint16_t val, volatile void __iomem *addr)
> +{
> +    asm volatile ( "sh %1, %0"
> +                   : "=m" (*(volatile uint16_t __force *)addr) : "r" (val) );
> +}
> +
> +static inline void __raw_writel(uint32_t val, volatile void __iomem *addr)
> +{
> +    asm volatile ( "sw %1, %0"
> +                   : "=m" (*(volatile uint32_t __force *)addr) : "r" (val) );
> +}
> +
> +static inline void __raw_writeq(uint64_t val, volatile void __iomem *addr)
> +{
> +#ifdef CONFIG_RISCV_32
> +    BUILD_BUG_ON("unimplemented");
> +#else
> +    asm volatile ( "sd %1, %0"
> +                   : "=m" (*(volatile uint64_t __force *)addr) : "r" (val) );
> +#endif
> +}

... this and its read counterpart likely being in need of re-doing by anyone
wanting to enable RV32 support.

Jan

Re: [PATCH v7 10/19] xen/riscv: introduce atomic.h

[Jan Beulich]

On 03.04.2024 12:20, Oleksii Kurochko wrote:
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/atomic.h
> @@ -0,0 +1,261 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Taken and modified from Linux.
> + *
> + * The following changes were done:
> + * - * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated
> + *     to use__*xchg_generic()
> + * - drop casts in write_atomic() as they are unnecessary
> + * - drop introduction of WRITE_ONCE() and READ_ONCE().
> + *   Xen provides ACCESS_ONCE()
> + * - remove zero-length array access in read_atomic()
> + * - drop defines similar to pattern
> + *   #define atomic_add_return_relaxed   atomic_add_return_relaxed
> + * - move not RISC-V specific functions to asm-generic/atomics-ops.h
> + * 
> + * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
> + * Copyright (C) 2012 Regents of the University of California
> + * Copyright (C) 2017 SiFive
> + * Copyright (C) 2024 Vates SAS
> + */
> +
> +#ifndef _ASM_RISCV_ATOMIC_H
> +#define _ASM_RISCV_ATOMIC_H
> +
> +#include <xen/atomic.h>
> +
> +#include <asm/cmpxchg.h>
> +#include <asm/fence.h>
> +#include <asm/io.h>
> +#include <asm/system.h>
> +
> +void __bad_atomic_size(void);
> +
> +/*
> + * Legacy from Linux kernel. For some reason they wanted to have ordered
> + * read/write access. Thereby read* is used instead of read*_cpu()
> + */
> +static always_inline void read_atomic_size(const volatile void *p,
> +                                           void *res,
> +                                           unsigned int size)
> +{
> +    switch ( size )
> +    {
> +    case 1: *(uint8_t *)res = readb(p); break;
> +    case 2: *(uint16_t *)res = readw(p); break;
> +    case 4: *(uint32_t *)res = readl(p); break;
> +#ifndef CONFIG_RISCV_32
> +    case 8: *(uint32_t *)res = readq(p); break;
> +#endif
> +    default: __bad_atomic_size(); break;
> +    }
> +}
> +
> +#define read_atomic(p) ({                                   \
> +    union { typeof(*(p)) val; char c[sizeof(*(p))]; } x_;   \
> +    read_atomic_size(p, x_.c, sizeof(*(p)));                \
> +    x_.val;                                                 \
> +})
> +
> +static always_inline void _write_atomic(volatile void *p,
> +                                       unsigned long x, unsigned int size)
> +{
> +    switch ( size )
> +    {
> +    case 1: writeb(x, p); break;
> +    case 2: writew(x, p); break;
> +    case 4: writel(x, p); break;
> +#ifndef CONFIG_RISCV_32
> +    case 8: writeq(x, p); break;
> +#endif
> +    default: __bad_atomic_size(); break;
> +    }
> +}
> +
> +#define write_atomic(p, x)                              \
> +({                                                      \
> +    typeof(*(p)) x_ = (x);                              \
> +    _write_atomic((p), x_, sizeof(*(p)));               \
> +    x_;                                                 \
> +})
> +
> +static always_inline void _add_sized(volatile void *p,
> +                                     unsigned long x, unsigned int size)
> +{
> +    switch ( size )
> +    {
> +    case 1: writeb(read_atomic((volatile uint8_t *)p) + x, p); break;
> +    case 2: writew(read_atomic((volatile uint16_t *)p) + x, p); break;
> +    case 4: writel(read_atomic((volatile uint32_t *)p) + x, p); break;
> +#ifndef CONFIG_RISCV_32
> +    case 8: writeq(read_atomic((volatile uint64_t *)p) + x, p); break;
> +#endif

Any particular reason for using read_atomic() but write{b,w,l,q}() here?

> +    default: __bad_atomic_size(); break;
> +    }
> +}
> +
> +#define add_sized(p, x)                                 \
> +({                                                      \
> +    typeof(*(p)) x_ = (x);                              \
> +    _add_sized((p), x_, sizeof(*(p)));                  \
> +})
> +
> +#define __atomic_acquire_fence() \
> +    asm volatile ( RISCV_ACQUIRE_BARRIER "" ::: "memory" )
> +
> +#define __atomic_release_fence() \
> +    asm volatile ( RISCV_RELEASE_BARRIER "" ::: "memory" )

There isn't any need for the "" in these two, is there?

> +/*
> + * First, the atomic ops that have no ordering constraints and therefor don't
> + * have the AQ or RL bits set.  These don't return anything, so there's only
> + * one version to worry about.
> + */
> +#define ATOMIC_OP(op, asm_op, unary_op, asm_type, c_type, prefix)  \
> +static inline                                               \
> +void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \
> +{                                                           \
> +    asm volatile (                                          \
> +        "   amo" #asm_op "." #asm_type " zero, %1, %0"      \
> +        : "+A" (v->counter)                                 \
> +        : "r" (unary_op i)                                  \
> +        : "memory" );                                       \
> +}                                                           \
> +
> +/*
> + * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
> + * last argument for ATOMIC_OP isn't used.
> + */
> +#define ATOMIC_OPS(op, asm_op, unary_op)                    \
> +        ATOMIC_OP (op, asm_op, unary_op, w, int,   )
> +
> +ATOMIC_OPS(add, add, +)
> +ATOMIC_OPS(sub, add, -)
> +ATOMIC_OPS(and, and, +)
> +ATOMIC_OPS( or,  or, +)
> +ATOMIC_OPS(xor, xor, +)
> +
> +#undef ATOMIC_OP
> +#undef ATOMIC_OPS
> +
> +#include <asm-generic/atomic-ops.h>
> +
> +/*
> + * Atomic ops that have ordered variant.
> + * There's two flavors of these: the arithmatic ops have both fetch and return
> + * versions, while the logical ops only have fetch versions.
> + */
> +#define ATOMIC_FETCH_OP(op, asm_op, unary_op, asm_type, c_type, prefix) \
> +static inline                                                       \
> +c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t *v) \
> +{                                                                   \
> +    register c_type ret;                                            \
> +    asm volatile (                                                  \
> +        "   amo" #asm_op "." #asm_type ".aqrl  %1, %2, %0"          \
> +        : "+A" (v->counter), "=r" (ret)                             \
> +        : "r" (unary_op i)                                          \
> +        : "memory" );                                               \
> +    return ret;                                                     \
> +}
> +
> +#define ATOMIC_OP_RETURN(op, asm_op, c_op, unary_op, asm_type, c_type, prefix) \
> +static inline                                                           \
> +c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t *v)  \
> +{                                                                       \
> +        return atomic##prefix##_fetch_##op(i, v) c_op (unary_op i);     \

Nit: Too deep indentation.

> +}
> +
> +/*
> + * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
> + * last argument of ATOMIC_FETCH_OP, ATOMIC_OP_RETURN isn't used.
> + */
> +#define ATOMIC_OPS(op, asm_op, c_op, unary_op)                          \
> +        ATOMIC_FETCH_OP( op, asm_op,       unary_op, w, int,   )        \
> +        ATOMIC_OP_RETURN(op, asm_op, c_op, unary_op, w, int,   )
> +
> +ATOMIC_OPS(add, add, +, +)
> +ATOMIC_OPS(sub, add, +, -)
> +
> +#undef ATOMIC_OPS
> +
> +#define ATOMIC_OPS(op, asm_op, unary_op) \
> +        ATOMIC_FETCH_OP(op, asm_op, unary_op, w, int,   )
> +
> +ATOMIC_OPS(and, and, +)
> +ATOMIC_OPS( or,  or, +)
> +ATOMIC_OPS(xor, xor, +)

The + isn't really needed here as a macro argument; ATOMIC_OPS() itself could
pass it to ATOMIC_FETCH_OP(). I also wonder why ATOMIC_OPS() has both "op" and
"asm_op", when both are uniformly the same.

> +#undef ATOMIC_OPS
> +
> +#undef ATOMIC_FETCH_OP
> +#undef ATOMIC_OP_RETURN
> +
> +/* This is required to provide a full barrier on success. */
> +static inline int atomic_add_unless(atomic_t *v, int a, int u)
> +{
> +    int prev, rc;
> +
> +    asm volatile (
> +        "0: lr.w     %[p],  %[c]\n"
> +        "   beq      %[p],  %[u], 1f\n"
> +        "   add      %[rc], %[p], %[a]\n"
> +        "   sc.w.aqrl  %[rc], %[rc], %[c]\n"
> +        "   bnez     %[rc], 0b\n"
> +        "1:\n"
> +        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
> +        : [a] "r" (a), [u] "r" (u)
> +        : "memory");
> +    return prev;
> +}
> +
> +static inline int atomic_sub_if_positive(atomic_t *v, int offset)
> +{
> +    int prev, rc;
> +
> +    asm volatile (
> +        "0: lr.w     %[p],  %[c]\n"
> +        "   sub      %[rc], %[p], %[o]\n"
> +        "   bltz     %[rc], 1f\n"
> +        "   sc.w.aqrl  %[rc], %[rc], %[c]\n"
> +        "   bnez     %[rc], 0b\n"
> +        "1:\n"
> +        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
> +        : [o] "r" (offset)
> +        : "memory" );
> +    return prev - offset;
> +}
> +
> +/*
> + * atomic_{cmp,}xchg is required to have exactly the same ordering semantics as
> + * {cmp,}xchg and the operations that return.
> + */
> +#define ATOMIC_OP(c_t, prefix, size)                            \
> +static inline                                                   \
> +c_t atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n)         \
> +{                                                               \
> +    return __xchg(&v->counter, n, size);                        \
> +}                                                               \
> +static inline                                                   \
> +c_t atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n) \
> +{                                                               \
> +    return __cmpxchg(&v->counter, o, n, size);                  \
> +}
> +
> +#define ATOMIC_OPS() \
> +    ATOMIC_OP(int,   , 4)

Can't the two inline functions use sizeof(*v), sizeof(v->counter), sizeof(c_t),
or sizeof(n) instead of passing a literal 4 here?

Jan

Re: [PATCH v7 10/19] xen/riscv: introduce atomic.h

[Oleksii]

On Mon, 2024-04-08 at 10:23 +0200, Jan Beulich wrote:
> On 03.04.2024 12:20, Oleksii Kurochko wrote:
> > --- /dev/null
> > +++ b/xen/arch/riscv/include/asm/atomic.h
> > @@ -0,0 +1,261 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Taken and modified from Linux.
> > + *
> > + * The following changes were done:
> > + * - * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were
> > updated
> > + *     to use__*xchg_generic()
> > + * - drop casts in write_atomic() as they are unnecessary
> > + * - drop introduction of WRITE_ONCE() and READ_ONCE().
> > + *   Xen provides ACCESS_ONCE()
> > + * - remove zero-length array access in read_atomic()
> > + * - drop defines similar to pattern
> > + *   #define atomic_add_return_relaxed   atomic_add_return_relaxed
> > + * - move not RISC-V specific functions to asm-generic/atomics-
> > ops.h
> > + * 
> > + * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
> > + * Copyright (C) 2012 Regents of the University of California
> > + * Copyright (C) 2017 SiFive
> > + * Copyright (C) 2024 Vates SAS
> > + */
> > +
> > +#ifndef _ASM_RISCV_ATOMIC_H
> > +#define _ASM_RISCV_ATOMIC_H
> > +
> > +#include <xen/atomic.h>
> > +
> > +#include <asm/cmpxchg.h>
> > +#include <asm/fence.h>
> > +#include <asm/io.h>
> > +#include <asm/system.h>
> > +
> > +void __bad_atomic_size(void);
> > +
> > +/*
> > + * Legacy from Linux kernel. For some reason they wanted to have
> > ordered
> > + * read/write access. Thereby read* is used instead of read*_cpu()
> > + */
> > +static always_inline void read_atomic_size(const volatile void *p,
> > +                                           void *res,
> > +                                           unsigned int size)
> > +{
> > +    switch ( size )
> > +    {
> > +    case 1: *(uint8_t *)res = readb(p); break;
> > +    case 2: *(uint16_t *)res = readw(p); break;
> > +    case 4: *(uint32_t *)res = readl(p); break;
> > +#ifndef CONFIG_RISCV_32
> > +    case 8: *(uint32_t *)res = readq(p); break;
> > +#endif
> > +    default: __bad_atomic_size(); break;
> > +    }
> > +}
> > +
> > +#define read_atomic(p) ({                                   \
> > +    union { typeof(*(p)) val; char c[sizeof(*(p))]; } x_;   \
> > +    read_atomic_size(p, x_.c, sizeof(*(p)));                \
> > +    x_.val;                                                 \
> > +})
> > +
> > +static always_inline void _write_atomic(volatile void *p,
> > +                                       unsigned long x, unsigned
> > int size)
> > +{
> > +    switch ( size )
> > +    {
> > +    case 1: writeb(x, p); break;
> > +    case 2: writew(x, p); break;
> > +    case 4: writel(x, p); break;
> > +#ifndef CONFIG_RISCV_32
> > +    case 8: writeq(x, p); break;
> > +#endif
> > +    default: __bad_atomic_size(); break;
> > +    }
> > +}
> > +
> > +#define write_atomic(p, x)                              \
> > +({                                                      \
> > +    typeof(*(p)) x_ = (x);                              \
> > +    _write_atomic((p), x_, sizeof(*(p)));               \
> > +    x_;                                                 \
> > +})
> > +
> > +static always_inline void _add_sized(volatile void *p,
> > +                                     unsigned long x, unsigned int
> > size)
> > +{
> > +    switch ( size )
> > +    {
> > +    case 1: writeb(read_atomic((volatile uint8_t *)p) + x, p);
> > break;
> > +    case 2: writew(read_atomic((volatile uint16_t *)p) + x, p);
> > break;
> > +    case 4: writel(read_atomic((volatile uint32_t *)p) + x, p);
> > break;
> > +#ifndef CONFIG_RISCV_32
> > +    case 8: writeq(read_atomic((volatile uint64_t *)p) + x, p);
> > break;
> > +#endif
> 
> Any particular reason for using read_atomic() but write{b,w,l,q}()
> here?
There is no particular reason. I will use write_atomic for consistency.

> 
> > +    default: __bad_atomic_size(); break;
> > +    }
> > +}
> > +
> > +#define add_sized(p, x)                                 \
> > +({                                                      \
> > +    typeof(*(p)) x_ = (x);                              \
> > +    _add_sized((p), x_, sizeof(*(p)));                  \
> > +})
> > +
> > +#define __atomic_acquire_fence() \
> > +    asm volatile ( RISCV_ACQUIRE_BARRIER "" ::: "memory" )
> > +
> > +#define __atomic_release_fence() \
> > +    asm volatile ( RISCV_RELEASE_BARRIER "" ::: "memory" )
> 
> There isn't any need for the "" in these two, is there?
There is no really needed "" in this case.
> 
> > +/*
> > + * First, the atomic ops that have no ordering constraints and
> > therefor don't
> > + * have the AQ or RL bits set.  These don't return anything, so
> > there's only
> > + * one version to worry about.
> > + */
> > +#define ATOMIC_OP(op, asm_op, unary_op, asm_type, c_type, prefix) 
> > \
> > +static inline                                               \
> > +void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \
> > +{                                                           \
> > +    asm volatile (                                          \
> > +        "   amo" #asm_op "." #asm_type " zero, %1, %0"      \
> > +        : "+A" (v->counter)                                 \
> > +        : "r" (unary_op i)                                  \
> > +        : "memory" );                                       \
> > +}                                                           \
> > +
> > +/*
> > + * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the
> > reason why
> > + * last argument for ATOMIC_OP isn't used.
> > + */
> > +#define ATOMIC_OPS(op, asm_op, unary_op)                    \
> > +        ATOMIC_OP (op, asm_op, unary_op, w, int,   )
> > +
> > +ATOMIC_OPS(add, add, +)
> > +ATOMIC_OPS(sub, add, -)
> > +ATOMIC_OPS(and, and, +)
> > +ATOMIC_OPS( or,  or, +)
> > +ATOMIC_OPS(xor, xor, +)
> > +
> > +#undef ATOMIC_OP
> > +#undef ATOMIC_OPS
> > +
> > +#include <asm-generic/atomic-ops.h>
> > +
> > +/*
> > + * Atomic ops that have ordered variant.
> > + * There's two flavors of these: the arithmatic ops have both
> > fetch and return
> > + * versions, while the logical ops only have fetch versions.
> > + */
> > +#define ATOMIC_FETCH_OP(op, asm_op, unary_op, asm_type, c_type,
> > prefix) \
> > +static
> > inline                                                       \
> > +c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t
> > *v) \
> > +{                                                                 
> >   \
> > +    register c_type
> > ret;                                            \
> > +    asm volatile
> > (                                                  \
> > +        "   amo" #asm_op "." #asm_type ".aqrl  %1, %2,
> > %0"          \
> > +        : "+A" (v->counter), "=r"
> > (ret)                             \
> > +        : "r" (unary_op
> > i)                                          \
> > +        : "memory"
> > );                                               \
> > +    return
> > ret;                                                     \
> > +}
> > +
> > +#define ATOMIC_OP_RETURN(op, asm_op, c_op, unary_op, asm_type,
> > c_type, prefix) \
> > +static
> > inline                                                           \
> > +c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t
> > *v)  \
> > +{                                                                 
> >       \
> > +        return atomic##prefix##_fetch_##op(i, v) c_op (unary_op
> > i);     \
> 
> Nit: Too deep indentation.
> 
> > +}
> > +
> > +/*
> > + * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the
> > reason why
> > + * last argument of ATOMIC_FETCH_OP, ATOMIC_OP_RETURN isn't used.
> > + */
> > +#define ATOMIC_OPS(op, asm_op, c_op,
> > unary_op)                          \
> > +        ATOMIC_FETCH_OP( op, asm_op,       unary_op, w, int,  
> > )        \
> > +        ATOMIC_OP_RETURN(op, asm_op, c_op, unary_op, w, int,   )
> > +
> > +ATOMIC_OPS(add, add, +, +)
> > +ATOMIC_OPS(sub, add, +, -)
> > +
> > +#undef ATOMIC_OPS
> > +
> > +#define ATOMIC_OPS(op, asm_op, unary_op) \
> > +        ATOMIC_FETCH_OP(op, asm_op, unary_op, w, int,   )
> > +
> > +ATOMIC_OPS(and, and, +)
> > +ATOMIC_OPS( or,  or, +)
> > +ATOMIC_OPS(xor, xor, +)
> 
> The + isn't really needed here as a macro argument; ATOMIC_OPS()
> itself could
> pass it to ATOMIC_FETCH_OP(). I also wonder why ATOMIC_OPS() has both
> "op" and
> "asm_op", when both are uniformly the same.
It is needed for the case when sub operation is implemented using add
plus negative number:
+ATOMIC_OPS(sub, add, +, -)


> 
> > +#undef ATOMIC_OPS
> > +
> > +#undef ATOMIC_FETCH_OP
> > +#undef ATOMIC_OP_RETURN
> > +
> > +/* This is required to provide a full barrier on success. */
> > +static inline int atomic_add_unless(atomic_t *v, int a, int u)
> > +{
> > +    int prev, rc;
> > +
> > +    asm volatile (
> > +        "0: lr.w     %[p],  %[c]\n"
> > +        "   beq      %[p],  %[u], 1f\n"
> > +        "   add      %[rc], %[p], %[a]\n"
> > +        "   sc.w.aqrl  %[rc], %[rc], %[c]\n"
> > +        "   bnez     %[rc], 0b\n"
> > +        "1:\n"
> > +        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
> > +        : [a] "r" (a), [u] "r" (u)
> > +        : "memory");
> > +    return prev;
> > +}
> > +
> > +static inline int atomic_sub_if_positive(atomic_t *v, int offset)
> > +{
> > +    int prev, rc;
> > +
> > +    asm volatile (
> > +        "0: lr.w     %[p],  %[c]\n"
> > +        "   sub      %[rc], %[p], %[o]\n"
> > +        "   bltz     %[rc], 1f\n"
> > +        "   sc.w.aqrl  %[rc], %[rc], %[c]\n"
> > +        "   bnez     %[rc], 0b\n"
> > +        "1:\n"
> > +        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
> > +        : [o] "r" (offset)
> > +        : "memory" );
> > +    return prev - offset;
> > +}
> > +
> > +/*
> > + * atomic_{cmp,}xchg is required to have exactly the same ordering
> > semantics as
> > + * {cmp,}xchg and the operations that return.
> > + */
> > +#define ATOMIC_OP(c_t, prefix, size)                            \
> > +static inline                                                   \
> > +c_t atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n)         \
> > +{                                                               \
> > +    return __xchg(&v->counter, n, size);                        \
> > +}                                                               \
> > +static inline                                                   \
> > +c_t atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n)
> > \
> > +{                                                               \
> > +    return __cmpxchg(&v->counter, o, n, size);                  \
> > +}
> > +
> > +#define ATOMIC_OPS() \
> > +    ATOMIC_OP(int,   , 4)
> 
> Can't the two inline functions use sizeof(*v), sizeof(v->counter),
> sizeof(c_t),
> or sizeof(n) instead of passing a literal 4 here?
Agree, it would be better.

Thanks.

~ Oleksii

Re: [PATCH v7 02/19] xen/riscv: disable unnecessary configs

[Oleksii]

On Wed, 2024-04-03 at 13:53 +0200, Jan Beulich wrote:
> On 03.04.2024 12:54, Oleksii wrote:
> > On Wed, 2024-04-03 at 12:28 +0200, Jan Beulich wrote:
> > > On 03.04.2024 12:19, Oleksii Kurochko wrote:
> > > > This patch disables unnecessary configs for two cases:
> > > > 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds
> > > > (GitLab CI jobs).
> > > > 2. By using tiny64_defconfig for non-randconfig builds.
> > > > 
> > > > Only configs which lead to compilation issues were disabled.
> > > > 
> > > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > > > ---
> > > > Changes in V7:
> > > >  - Disable only configs which cause compilation issues.
> > > 
> > > Since the description doesn't go into details: While I can see
> > > that
> > > PERF_COUNTERS and LIVEPATCH may require (a little / some more)
> > > extra
> > > work, are HYPFS, ARGO, and XSM really causing issues?
> > For Argo, I recieved the following compilation errors:
> >    common/argo.c:1416:5: error: unknown type name 'p2m_type_t'; did
> > you
> >    mean 'hvmmem_type_t'?
> >     1416 |     p2m_type_t p2mt;
> >          |     ^~~~~~~~~~
> >          |     hvmmem_type_t
> >    common/argo.c:1419:11: error: implicit declaration of function
> >    'check_get_page_from_gfn' [-Werror=implicit-function-
> > declaration]
> >     1419 |     ret = check_get_page_from_gfn(d, gfn, false, &p2mt,
> > &page);
> >          |           ^~~~~~~~~~~~~~~~~~~~~~~
> >    common/argo.c:1427:10: error: 'p2m_ram_rw' undeclared (first use
> > in
> >    this function)
> >     1427 |     case p2m_ram_rw:
> >    
> > It seems it should be included xen/p2m-common.h and asm/p2m.h in
> > common/argo.c.
> > 
> > For CONFIG_HYPFS_CONFIG ( there is no issue with CONFIG_HYPFS,
> > overlooked that ):
> >    common/config_data.S:1:10: fatal error: asm/asm_defns.h: No such
> > file
> >    or directory
> >        1 | #include <asm/asm_defns.h>
> >    
> > 
> > For XSM, I recieved the following error:
> > 
> >    xsm/xsm_core.c:79:19: error: 'xsm_core_init' defined but not
> > used [-
> >    Werror=unused-function]
> >       79 | static int __init xsm_core_init(const void
> > *policy_buffer,
> >    size_t policy_size)
> > 
> > I'll add an information with compilation errors to the commit
> > message.
> 
> No need to quote full compiler diagnostics, but a hint at the
> problems
> at least. That said, perhaps we want to rather sort the issues than
> disable building stuff that sooner or later you will want to build
> anyway. For hypfs we look to have an approach already. For Argo what
> you suggest makes sense to me; it might be nice to understand where
> the P2M headers needed are coming from on x86 and Arm. Ideally common
> code .c files wouldn't include asm/*.h.
It seems to me that p2m.h comes for Arm from argo.c -> xen/domain.h ->
asm/domain.h and for x86 from argo.c -> xen/guest_access.h -> #include
<asm/hvm/support.h> -> asm/p2m.h.

So I can include asm/p2m.h to asm/domain.h as p2m will be used anyway
in asm/domain.h header and drop disablement of ARGO config from
*_defconfig and build.yaml for CI. Does it make sense?

~ Oleksii
> 
> For XSM I'm a little puzzled: Shouldn't RISC-V have
> HAS_DEVICE_TREE=y?
> Then xsm_core_init() would have a caller.
> 
> Jan

Re: [PATCH v7 10/19] xen/riscv: introduce atomic.h

[Oleksii]

On Mon, 2024-04-08 at 10:23 +0200, Jan Beulich wrote:
> > +static always_inline void _add_sized(volatile void *p,
> > +                                     unsigned long x, unsigned int
> > size)
> > +{
> > +    switch ( size )
> > +    {
> > +    case 1: writeb(read_atomic((volatile uint8_t *)p) + x, p);
> > break;
> > +    case 2: writew(read_atomic((volatile uint16_t *)p) + x, p);
> > break;
> > +    case 4: writel(read_atomic((volatile uint32_t *)p) + x, p);
> > break;
> > +#ifndef CONFIG_RISCV_32
> > +    case 8: writeq(read_atomic((volatile uint64_t *)p) + x, p);
> > break;
> > +#endif
> 
> Any particular reason for using read_atomic() but write{b,w,l,q}()
> here?
It was done because write_atomic() wants to have pointer as a first
argument, but read_atomic() returns a value.

As an option it can be used read{b,w,l,q}() instead of read_atomic() to
have the code consistent with write{b,w,l,q}.

Another option is to left as is and add the comment.

~ Oleksii

Re: [PATCH v7 10/19] xen/riscv: introduce atomic.h

[Oleksii]

On Fri, 2024-04-12 at 12:39 +0200, Oleksii wrote:
> On Mon, 2024-04-08 at 10:23 +0200, Jan Beulich wrote:
> > > +static always_inline void _add_sized(volatile void *p,
> > > +                                     unsigned long x, unsigned
> > > int
> > > size)
> > > +{
> > > +    switch ( size )
> > > +    {
> > > +    case 1: writeb(read_atomic((volatile uint8_t *)p) + x, p);
> > > break;
> > > +    case 2: writew(read_atomic((volatile uint16_t *)p) + x, p);
> > > break;
> > > +    case 4: writel(read_atomic((volatile uint32_t *)p) + x, p);
> > > break;
> > > +#ifndef CONFIG_RISCV_32
> > > +    case 8: writeq(read_atomic((volatile uint64_t *)p) + x, p);
> > > break;
> > > +#endif
> > 
> > Any particular reason for using read_atomic() but write{b,w,l,q}()
> > here?
> It was done because write_atomic() wants to have pointer as a first
> argument, but read_atomic() returns a value.
> 
> As an option it can be used read{b,w,l,q}() instead of read_atomic()
> to
> have the code consistent with write{b,w,l,q}.
> 
> Another option is to left as is and add the comment.
I decided to write it using write_atomic() in the next way:

    case 1:
    {
        uint8_t *t = (volatile uint8_t)p;
        write_atomic(t, read_atomic(t) + x);
        break;
    }
...

~ Oleksii

Re: [PATCH v7 06/19] xen/bitops: put __ffs() into linux compatible header

[Shawn Anastasio]

Hi Oleksii,

On 4/3/24 5:19 AM, Oleksii Kurochko wrote:
> The mentioned macros exist only because of Linux compatible purpose.
> 
> The patch defines __ffs() in terms of Xen bitops and it is safe
> to define in this way ( as __ffs() - 1 ) as considering that __ffs()
> was defined as __builtin_ctzl(x), which has undefined behavior when x=0,
> so it is assumed that such cases are not encountered in the current code.
> 
> To not include <xen/linux-compat.h> to Xen library files __ffs() and __ffz()
> were defined locally in find-next-bit.c.
> 
> Except __ffs() usage in find-next-bit.c only one usage of __ffs() leave
> in smmu-v3.c. It seems that it __ffs can be changed to ffsl(x)-1 in
> this file, but to keep smmu-v3.c looks close to linux it was deciced just
> to define __ffs() in xen/linux-compat.h and include it in smmu-v3.c
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

For the PPC part:
Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com>

Thanks,
Shawn

Re: [PATCH v7 02/19] xen/riscv: disable unnecessary configs

[Jan Beulich]

On 11.04.2024 16:39, Oleksii wrote:
> On Wed, 2024-04-03 at 13:53 +0200, Jan Beulich wrote:
>> On 03.04.2024 12:54, Oleksii wrote:
>>> On Wed, 2024-04-03 at 12:28 +0200, Jan Beulich wrote:
>>>> On 03.04.2024 12:19, Oleksii Kurochko wrote:
>>>>> This patch disables unnecessary configs for two cases:
>>>>> 1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds
>>>>> (GitLab CI jobs).
>>>>> 2. By using tiny64_defconfig for non-randconfig builds.
>>>>>
>>>>> Only configs which lead to compilation issues were disabled.
>>>>>
>>>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>>>> ---
>>>>> Changes in V7:
>>>>>  - Disable only configs which cause compilation issues.
>>>>
>>>> Since the description doesn't go into details: While I can see
>>>> that
>>>> PERF_COUNTERS and LIVEPATCH may require (a little / some more)
>>>> extra
>>>> work, are HYPFS, ARGO, and XSM really causing issues?
>>> For Argo, I recieved the following compilation errors:
>>>    common/argo.c:1416:5: error: unknown type name 'p2m_type_t'; did
>>> you
>>>    mean 'hvmmem_type_t'?
>>>     1416 |     p2m_type_t p2mt;
>>>          |     ^~~~~~~~~~
>>>          |     hvmmem_type_t
>>>    common/argo.c:1419:11: error: implicit declaration of function
>>>    'check_get_page_from_gfn' [-Werror=implicit-function-
>>> declaration]
>>>     1419 |     ret = check_get_page_from_gfn(d, gfn, false, &p2mt,
>>> &page);
>>>          |           ^~~~~~~~~~~~~~~~~~~~~~~
>>>    common/argo.c:1427:10: error: 'p2m_ram_rw' undeclared (first use
>>> in
>>>    this function)
>>>     1427 |     case p2m_ram_rw:
>>>    
>>> It seems it should be included xen/p2m-common.h and asm/p2m.h in
>>> common/argo.c.
>>>
>>> For CONFIG_HYPFS_CONFIG ( there is no issue with CONFIG_HYPFS,
>>> overlooked that ):
>>>    common/config_data.S:1:10: fatal error: asm/asm_defns.h: No such
>>> file
>>>    or directory
>>>        1 | #include <asm/asm_defns.h>
>>>    
>>>
>>> For XSM, I recieved the following error:
>>>
>>>    xsm/xsm_core.c:79:19: error: 'xsm_core_init' defined but not
>>> used [-
>>>    Werror=unused-function]
>>>       79 | static int __init xsm_core_init(const void
>>> *policy_buffer,
>>>    size_t policy_size)
>>>
>>> I'll add an information with compilation errors to the commit
>>> message.
>>
>> No need to quote full compiler diagnostics, but a hint at the
>> problems
>> at least. That said, perhaps we want to rather sort the issues than
>> disable building stuff that sooner or later you will want to build
>> anyway. For hypfs we look to have an approach already. For Argo what
>> you suggest makes sense to me; it might be nice to understand where
>> the P2M headers needed are coming from on x86 and Arm. Ideally common
>> code .c files wouldn't include asm/*.h.
> It seems to me that p2m.h comes for Arm from argo.c -> xen/domain.h ->
> asm/domain.h and for x86 from argo.c -> xen/guest_access.h -> #include
> <asm/hvm/support.h> -> asm/p2m.h.
> 
> So I can include asm/p2m.h to asm/domain.h as p2m will be used anyway
> in asm/domain.h header and drop disablement of ARGO config from
> *_defconfig and build.yaml for CI. Does it make sense?

Looks okay to ma, at a glance.

Jan

Re: [PATCH v7 10/19] xen/riscv: introduce atomic.h

[Jan Beulich]

On 09.04.2024 09:39, Oleksii wrote:
> On Mon, 2024-04-08 at 10:23 +0200, Jan Beulich wrote:
>> On 03.04.2024 12:20, Oleksii Kurochko wrote:
>>> +/*
>>> + * Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the
>>> reason why
>>> + * last argument of ATOMIC_FETCH_OP, ATOMIC_OP_RETURN isn't used.
>>> + */
>>> +#define ATOMIC_OPS(op, asm_op, c_op,
>>> unary_op)                          \
>>> +        ATOMIC_FETCH_OP( op, asm_op,       unary_op, w, int,  
>>> )        \
>>> +        ATOMIC_OP_RETURN(op, asm_op, c_op, unary_op, w, int,   )
>>> +
>>> +ATOMIC_OPS(add, add, +, +)
>>> +ATOMIC_OPS(sub, add, +, -)
>>> +
>>> +#undef ATOMIC_OPS
>>> +
>>> +#define ATOMIC_OPS(op, asm_op, unary_op) \
>>> +        ATOMIC_FETCH_OP(op, asm_op, unary_op, w, int,   )
>>> +
>>> +ATOMIC_OPS(and, and, +)
>>> +ATOMIC_OPS( or,  or, +)
>>> +ATOMIC_OPS(xor, xor, +)
>>
>> The + isn't really needed here as a macro argument; ATOMIC_OPS()
>> itself could
>> pass it to ATOMIC_FETCH_OP(). I also wonder why ATOMIC_OPS() has both
>> "op" and
>> "asm_op", when both are uniformly the same.
> It is needed for the case when sub operation is implemented using add
> plus negative number:
> +ATOMIC_OPS(sub, add, +, -)

Except there's no such case right here.

Jan