All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 0/3] Off-load TLB invalidations to host for !GTSE
@ 2020-06-26 13:09 Bharata B Rao
  2020-06-26 13:09 ` [PATCH v2 1/3] powerpc/mm: Enable radix GTSE only if supported Bharata B Rao
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Bharata B Rao @ 2020-06-26 13:09 UTC (permalink / raw)
  To: linuxppc-dev; +Cc: aneesh.kumar, Bharata B Rao, npiggin

Hypervisor may choose not to enable Guest Translation Shootdown Enable
(GTSE) option for the guest. When GTSE isn't ON, the guest OS isn't
permitted to use instructions like tblie and tlbsync directly, but is
expected to make hypervisor calls to get the TLB flushed.

This series enables the TLB flush routines in the radix code to
off-load TLB flushing to hypervisor via the newly proposed hcall
H_RPT_INVALIDATE. 

To easily check the availability of GTSE, it is made an MMU feature.
The OV5 handling and H_REGISTER_PROC_TBL hcall are changed to
handle GTSE as an optionally available feature and to not assume GTSE
when radix support is available.

The actual hcall implementation for KVM isn't included in this
patchset and will be posted separately.

Changes in v2
=============
- Dropped the patch that added H_RPT_INVALIDATE calls for the nested
  case. This patch will be posted separately along with KVM hcall
  implementation.
- Merged first two patches
- A few cleanups
- Rebased to powerpc/next

v1: https://lore.kernel.org/linuxppc-dev/20200618160930.26324-1-bharata@linux.ibm.com/

H_RPT_INVALIDATE
================
Syntax:
int64   /* H_Success: Return code on successful completion */
        /* H_Busy - repeat the call with the same */
        /* H_Parameter, H_P2, H_P3, H_P4, H_P5 : Invalid parameters */
        hcall(const uint64 H_RPT_INVALIDATE, /* Invalidate RPT translation lookaside information */
              uint64 pid,       /* PID/LPID to invalidate */
              uint64 target,    /* Invalidation target */
              uint64 type,      /* Type of lookaside information */
              uint64 pageSizes,     /* Page sizes */
              uint64 start,     /* Start of Effective Address (EA) range (inclusive) */
              uint64 end)       /* End of EA range (exclusive) */

Invalidation targets (target)
-----------------------------
Core MMU        0x01 /* All virtual processors in the partition */
Core local MMU  0x02 /* Current virtual processor */
Nest MMU        0x04 /* All nest/accelerator agents in use by the partition */

A combination of the above can be specified, except core and core local.

Type of translation to invalidate (type)
---------------------------------------
NESTED       0x0001  /* Invalidate nested guest partition-scope */
TLB          0x0002  /* Invalidate TLB */
PWC          0x0004  /* Invalidate Page Walk Cache */
PRT          0x0008  /* Invalidate Process Table Entries if NESTED is clear */
PAT          0x0008  /* Invalidate Partition Table Entries if NESTED is set */

A combination of the above can be specified.

Page size mask (pageSizes)
--------------------------
4K              0x01
64K             0x02
2M              0x04
1G              0x08
All sizes       (-1UL)

A combination of the above can be specified.
All page sizes can be selected with -1.

Semantics: Invalidate radix tree lookaside information
           matching the parameters given.
* Return H_P2, H_P3 or H_P4 if target, type, or pageSizes parameters are
  different from the defined values.
* Return H_PARAMETER if NESTED is set and pid is not a valid nested
  LPID allocated to this partition
* Return H_P5 if (start, end) doesn't form a valid range. Start and end
  should be a valid Quadrant address and  end > start.
* Return H_NotSupported if the partition is not in running in radix
  translation mode.
* May invalidate more translation information than requested.
* If start = 0 and end = -1, set the range to cover all valid addresses.
  Else start and end should be aligned to 4kB (lower 11 bits clear).
* If NESTED is clear, then invalidate process scoped lookaside information.
  Else pid specifies a nested LPID, and the invalidation is performed
  on nested guest partition table and nested guest partition scope real
  addresses.
* If pid = 0 and NESTED is clear, then valid addresses are quadrant 3 and
  quadrant 0 spaces, Else valid addresses are quadrant 0.
* Pages which are fully covered by the range are to be invalidated.
  Those which are partially covered are considered outside invalidation
  range, which allows a caller to optimally invalidate ranges that may
  contain mixed page sizes.
* Return H_SUCCESS on success.

Bharata B Rao (2):
  powerpc/mm: Enable radix GTSE only if supported.
  powerpc/pseries: H_REGISTER_PROC_TBL should ask for GTSE only if
    enabled

Nicholas Piggin (1):
  powerpc/mm/book3s64/radix: Off-load TLB invalidations to host when
    !GTSE

 .../include/asm/book3s/64/tlbflush-radix.h    | 15 ++++
 arch/powerpc/include/asm/hvcall.h             | 34 +++++++-
 arch/powerpc/include/asm/mmu.h                |  4 +
 arch/powerpc/include/asm/plpar_wrappers.h     | 50 +++++++++++
 arch/powerpc/kernel/dt_cpu_ftrs.c             |  1 +
 arch/powerpc/kernel/prom_init.c               | 13 +--
 arch/powerpc/mm/book3s64/radix_tlb.c          | 82 +++++++++++++++++--
 arch/powerpc/mm/init_64.c                     |  5 +-
 arch/powerpc/platforms/pseries/lpar.c         |  8 +-
 9 files changed, 195 insertions(+), 17 deletions(-)

-- 
2.21.3


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH v2 1/3] powerpc/mm: Enable radix GTSE only if supported.
  2020-06-26 13:09 [PATCH v2 0/3] Off-load TLB invalidations to host for !GTSE Bharata B Rao
@ 2020-06-26 13:09 ` Bharata B Rao
  2020-06-26 20:55   ` Murilo Opsfelder Araújo
  2020-06-30  5:26   ` Aneesh Kumar K.V
  2020-06-26 13:09 ` [PATCH v2 2/3] powerpc/pseries: H_REGISTER_PROC_TBL should ask for GTSE only if enabled Bharata B Rao
  2020-06-26 13:10 ` [PATCH v2 3/3] powerpc/mm/book3s64/radix: Off-load TLB invalidations to host when !GTSE Bharata B Rao
  2 siblings, 2 replies; 9+ messages in thread
From: Bharata B Rao @ 2020-06-26 13:09 UTC (permalink / raw)
  To: linuxppc-dev; +Cc: aneesh.kumar, Bharata B Rao, npiggin

Make GTSE an MMU feature and enable it by default for radix.
However for guest, conditionally enable it if hypervisor supports
it via OV5 vector. Let prom_init ask for radix GTSE only if the
support exists.

Having GTSE as an MMU feature will make it easy to enable radix
without GTSE. Currently radix assumes GTSE is enabled by default.

Signed-off-by: Bharata B Rao <bharata@linux.ibm.com>
---
 arch/powerpc/include/asm/mmu.h    |  4 ++++
 arch/powerpc/kernel/dt_cpu_ftrs.c |  1 +
 arch/powerpc/kernel/prom_init.c   | 13 ++++++++-----
 arch/powerpc/mm/init_64.c         |  5 ++++-
 4 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/arch/powerpc/include/asm/mmu.h b/arch/powerpc/include/asm/mmu.h
index f4ac25d4df05..884d51995934 100644
--- a/arch/powerpc/include/asm/mmu.h
+++ b/arch/powerpc/include/asm/mmu.h
@@ -28,6 +28,9 @@
  * Individual features below.
  */
 
+/* Guest Translation Shootdown Enable */
+#define MMU_FTR_GTSE			ASM_CONST(0x00001000)
+
 /*
  * Support for 68 bit VA space. We added that from ISA 2.05
  */
@@ -173,6 +176,7 @@ enum {
 #endif
 #ifdef CONFIG_PPC_RADIX_MMU
 		MMU_FTR_TYPE_RADIX |
+		MMU_FTR_GTSE |
 #ifdef CONFIG_PPC_KUAP
 		MMU_FTR_RADIX_KUAP |
 #endif /* CONFIG_PPC_KUAP */
diff --git a/arch/powerpc/kernel/dt_cpu_ftrs.c b/arch/powerpc/kernel/dt_cpu_ftrs.c
index a0edeb391e3e..ac650c233cd9 100644
--- a/arch/powerpc/kernel/dt_cpu_ftrs.c
+++ b/arch/powerpc/kernel/dt_cpu_ftrs.c
@@ -336,6 +336,7 @@ static int __init feat_enable_mmu_radix(struct dt_cpu_feature *f)
 #ifdef CONFIG_PPC_RADIX_MMU
 	cur_cpu_spec->mmu_features |= MMU_FTR_TYPE_RADIX;
 	cur_cpu_spec->mmu_features |= MMU_FTRS_HASH_BASE;
+	cur_cpu_spec->mmu_features |= MMU_FTR_GTSE;
 	cur_cpu_spec->cpu_user_features |= PPC_FEATURE_HAS_MMU;
 
 	return 1;
diff --git a/arch/powerpc/kernel/prom_init.c b/arch/powerpc/kernel/prom_init.c
index 90c604d00b7d..cbc605cfdec0 100644
--- a/arch/powerpc/kernel/prom_init.c
+++ b/arch/powerpc/kernel/prom_init.c
@@ -1336,12 +1336,15 @@ static void __init prom_check_platform_support(void)
 		}
 	}
 
-	if (supported.radix_mmu && supported.radix_gtse &&
-	    IS_ENABLED(CONFIG_PPC_RADIX_MMU)) {
-		/* Radix preferred - but we require GTSE for now */
-		prom_debug("Asking for radix with GTSE\n");
+	if (supported.radix_mmu && IS_ENABLED(CONFIG_PPC_RADIX_MMU)) {
+		/* Radix preferred - Check if GTSE is also supported */
+		prom_debug("Asking for radix\n");
 		ibm_architecture_vec.vec5.mmu = OV5_FEAT(OV5_MMU_RADIX);
-		ibm_architecture_vec.vec5.radix_ext = OV5_FEAT(OV5_RADIX_GTSE);
+		if (supported.radix_gtse)
+			ibm_architecture_vec.vec5.radix_ext =
+					OV5_FEAT(OV5_RADIX_GTSE);
+		else
+			prom_debug("Radix GTSE isn't supported\n");
 	} else if (supported.hash_mmu) {
 		/* Default to hash mmu (if we can) */
 		prom_debug("Asking for hash\n");
diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
index bc73abf0bc25..152aa0200cef 100644
--- a/arch/powerpc/mm/init_64.c
+++ b/arch/powerpc/mm/init_64.c
@@ -407,12 +407,15 @@ static void __init early_check_vec5(void)
 		if (!(vec5[OV5_INDX(OV5_RADIX_GTSE)] &
 						OV5_FEAT(OV5_RADIX_GTSE))) {
 			pr_warn("WARNING: Hypervisor doesn't support RADIX with GTSE\n");
-		}
+			cur_cpu_spec->mmu_features &= ~MMU_FTR_GTSE;
+		} else
+			cur_cpu_spec->mmu_features |= MMU_FTR_GTSE;
 		/* Do radix anyway - the hypervisor said we had to */
 		cur_cpu_spec->mmu_features |= MMU_FTR_TYPE_RADIX;
 	} else if (mmu_supported == OV5_FEAT(OV5_MMU_HASH)) {
 		/* Hypervisor only supports hash - disable radix */
 		cur_cpu_spec->mmu_features &= ~MMU_FTR_TYPE_RADIX;
+		cur_cpu_spec->mmu_features &= ~MMU_FTR_GTSE;
 	}
 }
 
-- 
2.21.3


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH v2 2/3] powerpc/pseries: H_REGISTER_PROC_TBL should ask for GTSE only if enabled
  2020-06-26 13:09 [PATCH v2 0/3] Off-load TLB invalidations to host for !GTSE Bharata B Rao
  2020-06-26 13:09 ` [PATCH v2 1/3] powerpc/mm: Enable radix GTSE only if supported Bharata B Rao
@ 2020-06-26 13:09 ` Bharata B Rao
  2020-06-30  5:26   ` Aneesh Kumar K.V
  2020-06-26 13:10 ` [PATCH v2 3/3] powerpc/mm/book3s64/radix: Off-load TLB invalidations to host when !GTSE Bharata B Rao
  2 siblings, 1 reply; 9+ messages in thread
From: Bharata B Rao @ 2020-06-26 13:09 UTC (permalink / raw)
  To: linuxppc-dev; +Cc: aneesh.kumar, Bharata B Rao, npiggin

H_REGISTER_PROC_TBL asks for GTSE by default. GTSE flag bit should
be set only when GTSE is supported.

Signed-off-by: Bharata B Rao <bharata@linux.ibm.com>
---
 arch/powerpc/platforms/pseries/lpar.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/lpar.c b/arch/powerpc/platforms/pseries/lpar.c
index fd26f3d21d7b..f82569a505f1 100644
--- a/arch/powerpc/platforms/pseries/lpar.c
+++ b/arch/powerpc/platforms/pseries/lpar.c
@@ -1680,9 +1680,11 @@ static int pseries_lpar_register_process_table(unsigned long base,
 
 	if (table_size)
 		flags |= PROC_TABLE_NEW;
-	if (radix_enabled())
-		flags |= PROC_TABLE_RADIX | PROC_TABLE_GTSE;
-	else
+	if (radix_enabled()) {
+		flags |= PROC_TABLE_RADIX;
+		if (mmu_has_feature(MMU_FTR_GTSE))
+			flags |= PROC_TABLE_GTSE;
+	} else
 		flags |= PROC_TABLE_HPT_SLB;
 	for (;;) {
 		rc = plpar_hcall_norets(H_REGISTER_PROC_TBL, flags, base,
-- 
2.21.3


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH v2 3/3] powerpc/mm/book3s64/radix: Off-load TLB invalidations to host when !GTSE
  2020-06-26 13:09 [PATCH v2 0/3] Off-load TLB invalidations to host for !GTSE Bharata B Rao
  2020-06-26 13:09 ` [PATCH v2 1/3] powerpc/mm: Enable radix GTSE only if supported Bharata B Rao
  2020-06-26 13:09 ` [PATCH v2 2/3] powerpc/pseries: H_REGISTER_PROC_TBL should ask for GTSE only if enabled Bharata B Rao
@ 2020-06-26 13:10 ` Bharata B Rao
  2020-06-30  5:26   ` Aneesh Kumar K.V
  2 siblings, 1 reply; 9+ messages in thread
From: Bharata B Rao @ 2020-06-26 13:10 UTC (permalink / raw)
  To: linuxppc-dev; +Cc: aneesh.kumar, Bharata B Rao, npiggin

From: Nicholas Piggin <npiggin@gmail.com>

When platform doesn't support GTSE, let TLB invalidation requests
for radix guests be off-loaded to the host using H_RPT_INVALIDATE
hcall.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Bharata B Rao <bharata@linux.ibm.com>
	[hcall wrapper, error path handling and renames]
---
 .../include/asm/book3s/64/tlbflush-radix.h    | 15 ++++
 arch/powerpc/include/asm/hvcall.h             | 34 +++++++-
 arch/powerpc/include/asm/plpar_wrappers.h     | 50 +++++++++++
 arch/powerpc/mm/book3s64/radix_tlb.c          | 82 +++++++++++++++++--
 4 files changed, 173 insertions(+), 8 deletions(-)

diff --git a/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h b/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h
index ca8db193ae38..e7cf50358411 100644
--- a/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h
+++ b/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h
@@ -2,10 +2,25 @@
 #ifndef _ASM_POWERPC_TLBFLUSH_RADIX_H
 #define _ASM_POWERPC_TLBFLUSH_RADIX_H
 
+#include <asm/hvcall.h>
+
 struct vm_area_struct;
 struct mm_struct;
 struct mmu_gather;
 
+static inline u64 psize_to_h_rpti(unsigned long psize)
+{
+	if (psize == MMU_PAGE_4K)
+		return H_RPTI_PAGE_4K;
+	if (psize == MMU_PAGE_64K)
+		return H_RPTI_PAGE_64K;
+	if (psize == MMU_PAGE_2M)
+		return H_RPTI_PAGE_2M;
+	if (psize == MMU_PAGE_1G)
+		return H_RPTI_PAGE_1G;
+	return H_RPTI_PAGE_ALL;
+}
+
 static inline int mmu_get_ap(int psize)
 {
 	return mmu_psize_defs[psize].ap;
diff --git a/arch/powerpc/include/asm/hvcall.h b/arch/powerpc/include/asm/hvcall.h
index e90c073e437e..43486e773bd6 100644
--- a/arch/powerpc/include/asm/hvcall.h
+++ b/arch/powerpc/include/asm/hvcall.h
@@ -305,7 +305,8 @@
 #define H_SCM_UNBIND_ALL        0x3FC
 #define H_SCM_HEALTH            0x400
 #define H_SCM_PERFORMANCE_STATS 0x418
-#define MAX_HCALL_OPCODE	H_SCM_PERFORMANCE_STATS
+#define H_RPT_INVALIDATE	0x448
+#define MAX_HCALL_OPCODE	H_RPT_INVALIDATE
 
 /* Scope args for H_SCM_UNBIND_ALL */
 #define H_UNBIND_SCOPE_ALL (0x1)
@@ -389,6 +390,37 @@
 #define PROC_TABLE_RADIX	0x04
 #define PROC_TABLE_GTSE		0x01
 
+/*
+ * Defines for
+ * H_RPT_INVALIDATE - Invalidate RPT translation lookaside information.
+ */
+
+/* Type of translation to invalidate (type) */
+#define H_RPTI_TYPE_NESTED	0x0001	/* Invalidate nested guest partition-scope */
+#define H_RPTI_TYPE_TLB		0x0002	/* Invalidate TLB */
+#define H_RPTI_TYPE_PWC		0x0004	/* Invalidate Page Walk Cache */
+/* Invalidate Process Table Entries if H_RPTI_TYPE_NESTED is clear */
+#define H_RPTI_TYPE_PRT		0x0008
+/* Invalidate Partition Table Entries if H_RPTI_TYPE_NESTED is set */
+#define H_RPTI_TYPE_PAT		0x0008
+#define H_RPTI_TYPE_ALL		(H_RPTI_TYPE_TLB | H_RPTI_TYPE_PWC | \
+				 H_RPTI_TYPE_PRT)
+#define H_RPTI_TYPE_NESTED_ALL	(H_RPTI_TYPE_TLB | H_RPTI_TYPE_PWC | \
+				 H_RPTI_TYPE_PAT)
+
+/* Invalidation targets (target) */
+#define H_RPTI_TARGET_CMMU		0x01 /* All virtual processors in the partition */
+#define H_RPTI_TARGET_CMMU_LOCAL	0x02 /* Current virtual processor */
+/* All nest/accelerator agents in use by the partition */
+#define H_RPTI_TARGET_NMMU		0x04
+
+/* Page size mask (page sizes) */
+#define H_RPTI_PAGE_4K	0x01
+#define H_RPTI_PAGE_64K	0x02
+#define H_RPTI_PAGE_2M	0x04
+#define H_RPTI_PAGE_1G	0x08
+#define H_RPTI_PAGE_ALL (-1UL)
+
 #ifndef __ASSEMBLY__
 #include <linux/types.h>
 
diff --git a/arch/powerpc/include/asm/plpar_wrappers.h b/arch/powerpc/include/asm/plpar_wrappers.h
index 4497c8afb573..a184923abd07 100644
--- a/arch/powerpc/include/asm/plpar_wrappers.h
+++ b/arch/powerpc/include/asm/plpar_wrappers.h
@@ -334,6 +334,49 @@ static inline long plpar_get_cpu_characteristics(struct h_cpu_char_result *p)
 	return rc;
 }
 
+/*
+ * Wrapper to H_RPT_INVALIDATE hcall that handles return values appropriately
+ *
+ * - Returns H_SUCCESS on success
+ * - For H_BUSY return value, we retry the hcall.
+ * - For any other hcall failures, attempt a full flush once before
+ *   resorting to BUG().
+ *
+ * Note: This hcall is expected to fail only very rarely. The correct
+ * error recovery of killing the process/guest will be eventually
+ * needed.
+ */
+static inline long pseries_rpt_invalidate(u32 pid, u64 target, u64 type,
+					  u64 page_sizes, u64 start, u64 end)
+{
+	long rc;
+	unsigned long all;
+
+	while (true) {
+		rc = plpar_hcall_norets(H_RPT_INVALIDATE, pid, target, type,
+					page_sizes, start, end);
+		if (rc == H_BUSY) {
+			cpu_relax();
+			continue;
+		} else if (rc == H_SUCCESS)
+			return rc;
+
+		/* Flush request failed, try with a full flush once */
+		all = (type & H_RPTI_TYPE_NESTED) ? H_RPTI_TYPE_NESTED_ALL :
+		      H_RPTI_TYPE_ALL;
+retry:
+		rc = plpar_hcall_norets(H_RPT_INVALIDATE, pid, target,
+					all, page_sizes, 0, -1UL);
+		if (rc == H_BUSY) {
+			cpu_relax();
+			goto retry;
+		} else if (rc == H_SUCCESS)
+			return rc;
+
+		BUG();
+	}
+}
+
 #else /* !CONFIG_PPC_PSERIES */
 
 static inline long plpar_set_ciabr(unsigned long ciabr)
@@ -346,6 +389,13 @@ static inline long plpar_pte_read_4(unsigned long flags, unsigned long ptex,
 {
 	return 0;
 }
+
+static inline long pseries_rpt_invalidate(u32 pid, u64 target, u64 type,
+					  u64 page_sizes, u64 start, u64 end)
+{
+	return 0;
+}
+
 #endif /* CONFIG_PPC_PSERIES */
 
 #endif /* _ASM_POWERPC_PLPAR_WRAPPERS_H */
diff --git a/arch/powerpc/mm/book3s64/radix_tlb.c b/arch/powerpc/mm/book3s64/radix_tlb.c
index b5cc9b23cf02..180d8ddcf6e3 100644
--- a/arch/powerpc/mm/book3s64/radix_tlb.c
+++ b/arch/powerpc/mm/book3s64/radix_tlb.c
@@ -16,6 +16,7 @@
 #include <asm/tlbflush.h>
 #include <asm/trace.h>
 #include <asm/cputhreads.h>
+#include <asm/plpar_wrappers.h>
 
 #define RIC_FLUSH_TLB 0
 #define RIC_FLUSH_PWC 1
@@ -694,7 +695,14 @@ void radix__flush_tlb_mm(struct mm_struct *mm)
 			goto local;
 		}
 
-		if (cputlb_use_tlbie()) {
+		if (!mmu_has_feature(MMU_FTR_GTSE)) {
+			unsigned long tgt = H_RPTI_TARGET_CMMU;
+
+			if (atomic_read(&mm->context.copros) > 0)
+				tgt |= H_RPTI_TARGET_NMMU;
+			pseries_rpt_invalidate(pid, tgt, H_RPTI_TYPE_TLB,
+					       H_RPTI_PAGE_ALL, 0, -1UL);
+		} else if (cputlb_use_tlbie()) {
 			if (mm_needs_flush_escalation(mm))
 				_tlbie_pid(pid, RIC_FLUSH_ALL);
 			else
@@ -727,7 +735,16 @@ static void __flush_all_mm(struct mm_struct *mm, bool fullmm)
 				goto local;
 			}
 		}
-		if (cputlb_use_tlbie())
+		if (!mmu_has_feature(MMU_FTR_GTSE)) {
+			unsigned long tgt = H_RPTI_TARGET_CMMU;
+			unsigned long type = H_RPTI_TYPE_TLB | H_RPTI_TYPE_PWC |
+					     H_RPTI_TYPE_PRT;
+
+			if (atomic_read(&mm->context.copros) > 0)
+				tgt |= H_RPTI_TARGET_NMMU;
+			pseries_rpt_invalidate(pid, tgt, type,
+					       H_RPTI_PAGE_ALL, 0, -1UL);
+		} else if (cputlb_use_tlbie())
 			_tlbie_pid(pid, RIC_FLUSH_ALL);
 		else
 			_tlbiel_pid_multicast(mm, pid, RIC_FLUSH_ALL);
@@ -760,7 +777,19 @@ void radix__flush_tlb_page_psize(struct mm_struct *mm, unsigned long vmaddr,
 			exit_flush_lazy_tlbs(mm);
 			goto local;
 		}
-		if (cputlb_use_tlbie())
+		if (!mmu_has_feature(MMU_FTR_GTSE)) {
+			unsigned long tgt, page_sizes, size;
+
+			tgt = H_RPTI_TARGET_CMMU;
+			page_sizes = psize_to_h_rpti(psize);
+			size = 1UL << mmu_psize_to_shift(psize);
+
+			if (atomic_read(&mm->context.copros) > 0)
+				tgt |= H_RPTI_TARGET_NMMU;
+			pseries_rpt_invalidate(pid, tgt, H_RPTI_TYPE_TLB,
+					       page_sizes, vmaddr,
+					       vmaddr + size);
+		} else if (cputlb_use_tlbie())
 			_tlbie_va(vmaddr, pid, psize, RIC_FLUSH_TLB);
 		else
 			_tlbiel_va_multicast(mm, vmaddr, pid, psize, RIC_FLUSH_TLB);
@@ -810,7 +839,14 @@ static inline void _tlbiel_kernel_broadcast(void)
  */
 void radix__flush_tlb_kernel_range(unsigned long start, unsigned long end)
 {
-	if (cputlb_use_tlbie())
+	if (!mmu_has_feature(MMU_FTR_GTSE)) {
+		unsigned long tgt = H_RPTI_TARGET_CMMU | H_RPTI_TARGET_NMMU;
+		unsigned long type = H_RPTI_TYPE_TLB | H_RPTI_TYPE_PWC |
+				     H_RPTI_TYPE_PRT;
+
+		pseries_rpt_invalidate(0, tgt, type, H_RPTI_PAGE_ALL,
+				       start, end);
+	} else if (cputlb_use_tlbie())
 		_tlbie_pid(0, RIC_FLUSH_ALL);
 	else
 		_tlbiel_kernel_broadcast();
@@ -864,7 +900,17 @@ static inline void __radix__flush_tlb_range(struct mm_struct *mm,
 				nr_pages > tlb_local_single_page_flush_ceiling);
 	}
 
-	if (full) {
+	if (!mmu_has_feature(MMU_FTR_GTSE) && !local) {
+		unsigned long tgt = H_RPTI_TARGET_CMMU;
+		unsigned long page_sizes = psize_to_h_rpti(mmu_virtual_psize);
+
+		if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
+			page_sizes |= psize_to_h_rpti(MMU_PAGE_2M);
+		if (atomic_read(&mm->context.copros) > 0)
+			tgt |= H_RPTI_TARGET_NMMU;
+		pseries_rpt_invalidate(pid, tgt, H_RPTI_TYPE_TLB, page_sizes,
+				       start, end);
+	} else if (full) {
 		if (local) {
 			_tlbiel_pid(pid, RIC_FLUSH_TLB);
 		} else {
@@ -1046,7 +1092,17 @@ static __always_inline void __radix__flush_tlb_range_psize(struct mm_struct *mm,
 				nr_pages > tlb_local_single_page_flush_ceiling);
 	}
 
-	if (full) {
+	if (!mmu_has_feature(MMU_FTR_GTSE) && !local) {
+		unsigned long tgt = H_RPTI_TARGET_CMMU;
+		unsigned long type = H_RPTI_TYPE_TLB;
+		unsigned long page_sizes = psize_to_h_rpti(psize);
+
+		if (also_pwc)
+			type |= H_RPTI_TYPE_PWC;
+		if (atomic_read(&mm->context.copros) > 0)
+			tgt |= H_RPTI_TARGET_NMMU;
+		pseries_rpt_invalidate(pid, tgt, type, page_sizes, start, end);
+	} else if (full) {
 		if (local) {
 			_tlbiel_pid(pid, also_pwc ? RIC_FLUSH_ALL : RIC_FLUSH_TLB);
 		} else {
@@ -1111,7 +1167,19 @@ void radix__flush_tlb_collapsed_pmd(struct mm_struct *mm, unsigned long addr)
 			exit_flush_lazy_tlbs(mm);
 			goto local;
 		}
-		if (cputlb_use_tlbie())
+		if (!mmu_has_feature(MMU_FTR_GTSE)) {
+			unsigned long tgt, type, page_sizes;
+
+			tgt = H_RPTI_TARGET_CMMU;
+			type = H_RPTI_TYPE_TLB | H_RPTI_TYPE_PWC |
+			       H_RPTI_TYPE_PRT;
+			page_sizes = psize_to_h_rpti(mmu_virtual_psize);
+
+			if (atomic_read(&mm->context.copros) > 0)
+				tgt |= H_RPTI_TARGET_NMMU;
+			pseries_rpt_invalidate(pid, tgt, type, page_sizes,
+					       addr, end);
+		} else if (cputlb_use_tlbie())
 			_tlbie_va_range(addr, end, pid, PAGE_SIZE, mmu_virtual_psize, true);
 		else
 			_tlbiel_va_range_multicast(mm,
-- 
2.21.3


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH v2 1/3] powerpc/mm: Enable radix GTSE only if supported.
  2020-06-26 13:09 ` [PATCH v2 1/3] powerpc/mm: Enable radix GTSE only if supported Bharata B Rao
@ 2020-06-26 20:55   ` Murilo Opsfelder Araújo
  2020-06-29  4:31     ` Bharata B Rao
  2020-06-30  5:26   ` Aneesh Kumar K.V
  1 sibling, 1 reply; 9+ messages in thread
From: Murilo Opsfelder Araújo @ 2020-06-26 20:55 UTC (permalink / raw)
  To: Bharata B Rao; +Cc: aneesh.kumar, linuxppc-dev, npiggin

Hi, Bharata.

On Fri, Jun 26, 2020 at 06:39:58PM +0530, Bharata B Rao wrote:
> Make GTSE an MMU feature and enable it by default for radix.
> However for guest, conditionally enable it if hypervisor supports
> it via OV5 vector. Let prom_init ask for radix GTSE only if the
> support exists.
>
> Having GTSE as an MMU feature will make it easy to enable radix
> without GTSE. Currently radix assumes GTSE is enabled by default.
>
> Signed-off-by: Bharata B Rao <bharata@linux.ibm.com>
> ---
>  arch/powerpc/include/asm/mmu.h    |  4 ++++
>  arch/powerpc/kernel/dt_cpu_ftrs.c |  1 +
>  arch/powerpc/kernel/prom_init.c   | 13 ++++++++-----
>  arch/powerpc/mm/init_64.c         |  5 ++++-
>  4 files changed, 17 insertions(+), 6 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/mmu.h b/arch/powerpc/include/asm/mmu.h
> index f4ac25d4df05..884d51995934 100644
> --- a/arch/powerpc/include/asm/mmu.h
> +++ b/arch/powerpc/include/asm/mmu.h
> @@ -28,6 +28,9 @@
>   * Individual features below.
>   */
>
> +/* Guest Translation Shootdown Enable */
> +#define MMU_FTR_GTSE			ASM_CONST(0x00001000)
> +
>  /*
>   * Support for 68 bit VA space. We added that from ISA 2.05
>   */
> @@ -173,6 +176,7 @@ enum {
>  #endif
>  #ifdef CONFIG_PPC_RADIX_MMU
>  		MMU_FTR_TYPE_RADIX |
> +		MMU_FTR_GTSE |
>  #ifdef CONFIG_PPC_KUAP
>  		MMU_FTR_RADIX_KUAP |
>  #endif /* CONFIG_PPC_KUAP */
> diff --git a/arch/powerpc/kernel/dt_cpu_ftrs.c b/arch/powerpc/kernel/dt_cpu_ftrs.c
> index a0edeb391e3e..ac650c233cd9 100644
> --- a/arch/powerpc/kernel/dt_cpu_ftrs.c
> +++ b/arch/powerpc/kernel/dt_cpu_ftrs.c
> @@ -336,6 +336,7 @@ static int __init feat_enable_mmu_radix(struct dt_cpu_feature *f)
>  #ifdef CONFIG_PPC_RADIX_MMU
>  	cur_cpu_spec->mmu_features |= MMU_FTR_TYPE_RADIX;
>  	cur_cpu_spec->mmu_features |= MMU_FTRS_HASH_BASE;
> +	cur_cpu_spec->mmu_features |= MMU_FTR_GTSE;
>  	cur_cpu_spec->cpu_user_features |= PPC_FEATURE_HAS_MMU;
>
>  	return 1;
> diff --git a/arch/powerpc/kernel/prom_init.c b/arch/powerpc/kernel/prom_init.c
> index 90c604d00b7d..cbc605cfdec0 100644
> --- a/arch/powerpc/kernel/prom_init.c
> +++ b/arch/powerpc/kernel/prom_init.c
> @@ -1336,12 +1336,15 @@ static void __init prom_check_platform_support(void)
>  		}
>  	}
>
> -	if (supported.radix_mmu && supported.radix_gtse &&
> -	    IS_ENABLED(CONFIG_PPC_RADIX_MMU)) {
> -		/* Radix preferred - but we require GTSE for now */
> -		prom_debug("Asking for radix with GTSE\n");
> +	if (supported.radix_mmu && IS_ENABLED(CONFIG_PPC_RADIX_MMU)) {
> +		/* Radix preferred - Check if GTSE is also supported */
> +		prom_debug("Asking for radix\n");
>  		ibm_architecture_vec.vec5.mmu = OV5_FEAT(OV5_MMU_RADIX);
> -		ibm_architecture_vec.vec5.radix_ext = OV5_FEAT(OV5_RADIX_GTSE);
> +		if (supported.radix_gtse)
> +			ibm_architecture_vec.vec5.radix_ext =
> +					OV5_FEAT(OV5_RADIX_GTSE);
> +		else
> +			prom_debug("Radix GTSE isn't supported\n");
>  	} else if (supported.hash_mmu) {
>  		/* Default to hash mmu (if we can) */
>  		prom_debug("Asking for hash\n");
> diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
> index bc73abf0bc25..152aa0200cef 100644
> --- a/arch/powerpc/mm/init_64.c
> +++ b/arch/powerpc/mm/init_64.c
> @@ -407,12 +407,15 @@ static void __init early_check_vec5(void)
>  		if (!(vec5[OV5_INDX(OV5_RADIX_GTSE)] &
>  						OV5_FEAT(OV5_RADIX_GTSE))) {
>  			pr_warn("WARNING: Hypervisor doesn't support RADIX with GTSE\n");
> -		}
> +			cur_cpu_spec->mmu_features &= ~MMU_FTR_GTSE;
> +		} else
> +			cur_cpu_spec->mmu_features |= MMU_FTR_GTSE;
>  		/* Do radix anyway - the hypervisor said we had to */
>  		cur_cpu_spec->mmu_features |= MMU_FTR_TYPE_RADIX;
>  	} else if (mmu_supported == OV5_FEAT(OV5_MMU_HASH)) {
>  		/* Hypervisor only supports hash - disable radix */
>  		cur_cpu_spec->mmu_features &= ~MMU_FTR_TYPE_RADIX;
> +		cur_cpu_spec->mmu_features &= ~MMU_FTR_GTSE;
>  	}
>  }

Is this a part of the code where mmu_clear_feature() cannot be used?

I'm just curious to understand the difference of clearing
cur_cpu_spec->mmu_features bits like above versus using
mmu_clear_feature() function.

--
Murilo

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH v2 1/3] powerpc/mm: Enable radix GTSE only if supported.
  2020-06-26 20:55   ` Murilo Opsfelder Araújo
@ 2020-06-29  4:31     ` Bharata B Rao
  0 siblings, 0 replies; 9+ messages in thread
From: Bharata B Rao @ 2020-06-29  4:31 UTC (permalink / raw)
  To: Murilo Opsfelder Araújo; +Cc: aneesh.kumar, linuxppc-dev, npiggin

On Fri, Jun 26, 2020 at 05:55:30PM -0300, Murilo Opsfelder Araújo wrote:
> > diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
> > index bc73abf0bc25..152aa0200cef 100644
> > --- a/arch/powerpc/mm/init_64.c
> > +++ b/arch/powerpc/mm/init_64.c
> > @@ -407,12 +407,15 @@ static void __init early_check_vec5(void)
> >  		if (!(vec5[OV5_INDX(OV5_RADIX_GTSE)] &
> >  						OV5_FEAT(OV5_RADIX_GTSE))) {
> >  			pr_warn("WARNING: Hypervisor doesn't support RADIX with GTSE\n");
> > -		}
> > +			cur_cpu_spec->mmu_features &= ~MMU_FTR_GTSE;
> > +		} else
> > +			cur_cpu_spec->mmu_features |= MMU_FTR_GTSE;
> >  		/* Do radix anyway - the hypervisor said we had to */
> >  		cur_cpu_spec->mmu_features |= MMU_FTR_TYPE_RADIX;
> >  	} else if (mmu_supported == OV5_FEAT(OV5_MMU_HASH)) {
> >  		/* Hypervisor only supports hash - disable radix */
> >  		cur_cpu_spec->mmu_features &= ~MMU_FTR_TYPE_RADIX;
> > +		cur_cpu_spec->mmu_features &= ~MMU_FTR_GTSE;
> >  	}
> >  }
> 
> Is this a part of the code where mmu_clear_feature() cannot be used?

Yes, it appears so. Jump label initialization isn't done yet.

Regards,
Bharata.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH v2 1/3] powerpc/mm: Enable radix GTSE only if supported.
  2020-06-26 13:09 ` [PATCH v2 1/3] powerpc/mm: Enable radix GTSE only if supported Bharata B Rao
  2020-06-26 20:55   ` Murilo Opsfelder Araújo
@ 2020-06-30  5:26   ` Aneesh Kumar K.V
  1 sibling, 0 replies; 9+ messages in thread
From: Aneesh Kumar K.V @ 2020-06-30  5:26 UTC (permalink / raw)
  To: Bharata B Rao, linuxppc-dev; +Cc: npiggin, Bharata B Rao

Bharata B Rao <bharata@linux.ibm.com> writes:

> Make GTSE an MMU feature and enable it by default for radix.
> However for guest, conditionally enable it if hypervisor supports
> it via OV5 vector. Let prom_init ask for radix GTSE only if the
> support exists.
>
> Having GTSE as an MMU feature will make it easy to enable radix
> without GTSE. Currently radix assumes GTSE is enabled by default.
>

Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>

> Signed-off-by: Bharata B Rao <bharata@linux.ibm.com>
> ---
>  arch/powerpc/include/asm/mmu.h    |  4 ++++
>  arch/powerpc/kernel/dt_cpu_ftrs.c |  1 +
>  arch/powerpc/kernel/prom_init.c   | 13 ++++++++-----
>  arch/powerpc/mm/init_64.c         |  5 ++++-
>  4 files changed, 17 insertions(+), 6 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/mmu.h b/arch/powerpc/include/asm/mmu.h
> index f4ac25d4df05..884d51995934 100644
> --- a/arch/powerpc/include/asm/mmu.h
> +++ b/arch/powerpc/include/asm/mmu.h
> @@ -28,6 +28,9 @@
>   * Individual features below.
>   */
>  
> +/* Guest Translation Shootdown Enable */
> +#define MMU_FTR_GTSE			ASM_CONST(0x00001000)
> +
>  /*
>   * Support for 68 bit VA space. We added that from ISA 2.05
>   */
> @@ -173,6 +176,7 @@ enum {
>  #endif
>  #ifdef CONFIG_PPC_RADIX_MMU
>  		MMU_FTR_TYPE_RADIX |
> +		MMU_FTR_GTSE |
>  #ifdef CONFIG_PPC_KUAP
>  		MMU_FTR_RADIX_KUAP |
>  #endif /* CONFIG_PPC_KUAP */
> diff --git a/arch/powerpc/kernel/dt_cpu_ftrs.c b/arch/powerpc/kernel/dt_cpu_ftrs.c
> index a0edeb391e3e..ac650c233cd9 100644
> --- a/arch/powerpc/kernel/dt_cpu_ftrs.c
> +++ b/arch/powerpc/kernel/dt_cpu_ftrs.c
> @@ -336,6 +336,7 @@ static int __init feat_enable_mmu_radix(struct dt_cpu_feature *f)
>  #ifdef CONFIG_PPC_RADIX_MMU
>  	cur_cpu_spec->mmu_features |= MMU_FTR_TYPE_RADIX;
>  	cur_cpu_spec->mmu_features |= MMU_FTRS_HASH_BASE;
> +	cur_cpu_spec->mmu_features |= MMU_FTR_GTSE;
>  	cur_cpu_spec->cpu_user_features |= PPC_FEATURE_HAS_MMU;
>  
>  	return 1;
> diff --git a/arch/powerpc/kernel/prom_init.c b/arch/powerpc/kernel/prom_init.c
> index 90c604d00b7d..cbc605cfdec0 100644
> --- a/arch/powerpc/kernel/prom_init.c
> +++ b/arch/powerpc/kernel/prom_init.c
> @@ -1336,12 +1336,15 @@ static void __init prom_check_platform_support(void)
>  		}
>  	}
>  
> -	if (supported.radix_mmu && supported.radix_gtse &&
> -	    IS_ENABLED(CONFIG_PPC_RADIX_MMU)) {
> -		/* Radix preferred - but we require GTSE for now */
> -		prom_debug("Asking for radix with GTSE\n");
> +	if (supported.radix_mmu && IS_ENABLED(CONFIG_PPC_RADIX_MMU)) {
> +		/* Radix preferred - Check if GTSE is also supported */
> +		prom_debug("Asking for radix\n");
>  		ibm_architecture_vec.vec5.mmu = OV5_FEAT(OV5_MMU_RADIX);
> -		ibm_architecture_vec.vec5.radix_ext = OV5_FEAT(OV5_RADIX_GTSE);
> +		if (supported.radix_gtse)
> +			ibm_architecture_vec.vec5.radix_ext =
> +					OV5_FEAT(OV5_RADIX_GTSE);
> +		else
> +			prom_debug("Radix GTSE isn't supported\n");
>  	} else if (supported.hash_mmu) {
>  		/* Default to hash mmu (if we can) */
>  		prom_debug("Asking for hash\n");
> diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
> index bc73abf0bc25..152aa0200cef 100644
> --- a/arch/powerpc/mm/init_64.c
> +++ b/arch/powerpc/mm/init_64.c
> @@ -407,12 +407,15 @@ static void __init early_check_vec5(void)
>  		if (!(vec5[OV5_INDX(OV5_RADIX_GTSE)] &
>  						OV5_FEAT(OV5_RADIX_GTSE))) {
>  			pr_warn("WARNING: Hypervisor doesn't support RADIX with GTSE\n");
> -		}
> +			cur_cpu_spec->mmu_features &= ~MMU_FTR_GTSE;
> +		} else
> +			cur_cpu_spec->mmu_features |= MMU_FTR_GTSE;
>  		/* Do radix anyway - the hypervisor said we had to */
>  		cur_cpu_spec->mmu_features |= MMU_FTR_TYPE_RADIX;
>  	} else if (mmu_supported == OV5_FEAT(OV5_MMU_HASH)) {
>  		/* Hypervisor only supports hash - disable radix */
>  		cur_cpu_spec->mmu_features &= ~MMU_FTR_TYPE_RADIX;
> +		cur_cpu_spec->mmu_features &= ~MMU_FTR_GTSE;
>  	}
>  }
>  
> -- 
> 2.21.3

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH v2 2/3] powerpc/pseries: H_REGISTER_PROC_TBL should ask for GTSE only if enabled
  2020-06-26 13:09 ` [PATCH v2 2/3] powerpc/pseries: H_REGISTER_PROC_TBL should ask for GTSE only if enabled Bharata B Rao
@ 2020-06-30  5:26   ` Aneesh Kumar K.V
  0 siblings, 0 replies; 9+ messages in thread
From: Aneesh Kumar K.V @ 2020-06-30  5:26 UTC (permalink / raw)
  To: Bharata B Rao, linuxppc-dev; +Cc: npiggin, Bharata B Rao

Bharata B Rao <bharata@linux.ibm.com> writes:

> H_REGISTER_PROC_TBL asks for GTSE by default. GTSE flag bit should
> be set only when GTSE is supported.
>
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>

> Signed-off-by: Bharata B Rao <bharata@linux.ibm.com>
> ---
>  arch/powerpc/platforms/pseries/lpar.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/arch/powerpc/platforms/pseries/lpar.c b/arch/powerpc/platforms/pseries/lpar.c
> index fd26f3d21d7b..f82569a505f1 100644
> --- a/arch/powerpc/platforms/pseries/lpar.c
> +++ b/arch/powerpc/platforms/pseries/lpar.c
> @@ -1680,9 +1680,11 @@ static int pseries_lpar_register_process_table(unsigned long base,
>  
>  	if (table_size)
>  		flags |= PROC_TABLE_NEW;
> -	if (radix_enabled())
> -		flags |= PROC_TABLE_RADIX | PROC_TABLE_GTSE;
> -	else
> +	if (radix_enabled()) {
> +		flags |= PROC_TABLE_RADIX;
> +		if (mmu_has_feature(MMU_FTR_GTSE))
> +			flags |= PROC_TABLE_GTSE;
> +	} else
>  		flags |= PROC_TABLE_HPT_SLB;
>  	for (;;) {
>  		rc = plpar_hcall_norets(H_REGISTER_PROC_TBL, flags, base,
> -- 
> 2.21.3

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH v2 3/3] powerpc/mm/book3s64/radix: Off-load TLB invalidations to host when !GTSE
  2020-06-26 13:10 ` [PATCH v2 3/3] powerpc/mm/book3s64/radix: Off-load TLB invalidations to host when !GTSE Bharata B Rao
@ 2020-06-30  5:26   ` Aneesh Kumar K.V
  0 siblings, 0 replies; 9+ messages in thread
From: Aneesh Kumar K.V @ 2020-06-30  5:26 UTC (permalink / raw)
  To: Bharata B Rao, linuxppc-dev; +Cc: npiggin, Bharata B Rao

Bharata B Rao <bharata@linux.ibm.com> writes:

> From: Nicholas Piggin <npiggin@gmail.com>
>
> When platform doesn't support GTSE, let TLB invalidation requests
> for radix guests be off-loaded to the host using H_RPT_INVALIDATE
> hcall.
>

Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>

> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> Signed-off-by: Bharata B Rao <bharata@linux.ibm.com>
> 	[hcall wrapper, error path handling and renames]
> ---
>  .../include/asm/book3s/64/tlbflush-radix.h    | 15 ++++
>  arch/powerpc/include/asm/hvcall.h             | 34 +++++++-
>  arch/powerpc/include/asm/plpar_wrappers.h     | 50 +++++++++++
>  arch/powerpc/mm/book3s64/radix_tlb.c          | 82 +++++++++++++++++--
>  4 files changed, 173 insertions(+), 8 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h b/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h
> index ca8db193ae38..e7cf50358411 100644
> --- a/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h
> +++ b/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h
> @@ -2,10 +2,25 @@
>  #ifndef _ASM_POWERPC_TLBFLUSH_RADIX_H
>  #define _ASM_POWERPC_TLBFLUSH_RADIX_H
>  
> +#include <asm/hvcall.h>
> +
>  struct vm_area_struct;
>  struct mm_struct;
>  struct mmu_gather;
>  
> +static inline u64 psize_to_h_rpti(unsigned long psize)
> +{
> +	if (psize == MMU_PAGE_4K)
> +		return H_RPTI_PAGE_4K;
> +	if (psize == MMU_PAGE_64K)
> +		return H_RPTI_PAGE_64K;
> +	if (psize == MMU_PAGE_2M)
> +		return H_RPTI_PAGE_2M;
> +	if (psize == MMU_PAGE_1G)
> +		return H_RPTI_PAGE_1G;
> +	return H_RPTI_PAGE_ALL;
> +}
> +
>  static inline int mmu_get_ap(int psize)
>  {
>  	return mmu_psize_defs[psize].ap;
> diff --git a/arch/powerpc/include/asm/hvcall.h b/arch/powerpc/include/asm/hvcall.h
> index e90c073e437e..43486e773bd6 100644
> --- a/arch/powerpc/include/asm/hvcall.h
> +++ b/arch/powerpc/include/asm/hvcall.h
> @@ -305,7 +305,8 @@
>  #define H_SCM_UNBIND_ALL        0x3FC
>  #define H_SCM_HEALTH            0x400
>  #define H_SCM_PERFORMANCE_STATS 0x418
> -#define MAX_HCALL_OPCODE	H_SCM_PERFORMANCE_STATS
> +#define H_RPT_INVALIDATE	0x448
> +#define MAX_HCALL_OPCODE	H_RPT_INVALIDATE
>  
>  /* Scope args for H_SCM_UNBIND_ALL */
>  #define H_UNBIND_SCOPE_ALL (0x1)
> @@ -389,6 +390,37 @@
>  #define PROC_TABLE_RADIX	0x04
>  #define PROC_TABLE_GTSE		0x01
>  
> +/*
> + * Defines for
> + * H_RPT_INVALIDATE - Invalidate RPT translation lookaside information.
> + */
> +
> +/* Type of translation to invalidate (type) */
> +#define H_RPTI_TYPE_NESTED	0x0001	/* Invalidate nested guest partition-scope */
> +#define H_RPTI_TYPE_TLB		0x0002	/* Invalidate TLB */
> +#define H_RPTI_TYPE_PWC		0x0004	/* Invalidate Page Walk Cache */
> +/* Invalidate Process Table Entries if H_RPTI_TYPE_NESTED is clear */
> +#define H_RPTI_TYPE_PRT		0x0008
> +/* Invalidate Partition Table Entries if H_RPTI_TYPE_NESTED is set */
> +#define H_RPTI_TYPE_PAT		0x0008
> +#define H_RPTI_TYPE_ALL		(H_RPTI_TYPE_TLB | H_RPTI_TYPE_PWC | \
> +				 H_RPTI_TYPE_PRT)
> +#define H_RPTI_TYPE_NESTED_ALL	(H_RPTI_TYPE_TLB | H_RPTI_TYPE_PWC | \
> +				 H_RPTI_TYPE_PAT)
> +
> +/* Invalidation targets (target) */
> +#define H_RPTI_TARGET_CMMU		0x01 /* All virtual processors in the partition */
> +#define H_RPTI_TARGET_CMMU_LOCAL	0x02 /* Current virtual processor */
> +/* All nest/accelerator agents in use by the partition */
> +#define H_RPTI_TARGET_NMMU		0x04
> +
> +/* Page size mask (page sizes) */
> +#define H_RPTI_PAGE_4K	0x01
> +#define H_RPTI_PAGE_64K	0x02
> +#define H_RPTI_PAGE_2M	0x04
> +#define H_RPTI_PAGE_1G	0x08
> +#define H_RPTI_PAGE_ALL (-1UL)
> +
>  #ifndef __ASSEMBLY__
>  #include <linux/types.h>
>  
> diff --git a/arch/powerpc/include/asm/plpar_wrappers.h b/arch/powerpc/include/asm/plpar_wrappers.h
> index 4497c8afb573..a184923abd07 100644
> --- a/arch/powerpc/include/asm/plpar_wrappers.h
> +++ b/arch/powerpc/include/asm/plpar_wrappers.h
> @@ -334,6 +334,49 @@ static inline long plpar_get_cpu_characteristics(struct h_cpu_char_result *p)
>  	return rc;
>  }
>  
> +/*
> + * Wrapper to H_RPT_INVALIDATE hcall that handles return values appropriately
> + *
> + * - Returns H_SUCCESS on success
> + * - For H_BUSY return value, we retry the hcall.
> + * - For any other hcall failures, attempt a full flush once before
> + *   resorting to BUG().
> + *
> + * Note: This hcall is expected to fail only very rarely. The correct
> + * error recovery of killing the process/guest will be eventually
> + * needed.
> + */
> +static inline long pseries_rpt_invalidate(u32 pid, u64 target, u64 type,
> +					  u64 page_sizes, u64 start, u64 end)
> +{
> +	long rc;
> +	unsigned long all;
> +
> +	while (true) {
> +		rc = plpar_hcall_norets(H_RPT_INVALIDATE, pid, target, type,
> +					page_sizes, start, end);
> +		if (rc == H_BUSY) {
> +			cpu_relax();
> +			continue;
> +		} else if (rc == H_SUCCESS)
> +			return rc;
> +
> +		/* Flush request failed, try with a full flush once */
> +		all = (type & H_RPTI_TYPE_NESTED) ? H_RPTI_TYPE_NESTED_ALL :
> +		      H_RPTI_TYPE_ALL;
> +retry:
> +		rc = plpar_hcall_norets(H_RPT_INVALIDATE, pid, target,
> +					all, page_sizes, 0, -1UL);
> +		if (rc == H_BUSY) {
> +			cpu_relax();
> +			goto retry;
> +		} else if (rc == H_SUCCESS)
> +			return rc;
> +
> +		BUG();
> +	}
> +}
> +
>  #else /* !CONFIG_PPC_PSERIES */
>  
>  static inline long plpar_set_ciabr(unsigned long ciabr)
> @@ -346,6 +389,13 @@ static inline long plpar_pte_read_4(unsigned long flags, unsigned long ptex,
>  {
>  	return 0;
>  }
> +
> +static inline long pseries_rpt_invalidate(u32 pid, u64 target, u64 type,
> +					  u64 page_sizes, u64 start, u64 end)
> +{
> +	return 0;
> +}
> +
>  #endif /* CONFIG_PPC_PSERIES */
>  
>  #endif /* _ASM_POWERPC_PLPAR_WRAPPERS_H */
> diff --git a/arch/powerpc/mm/book3s64/radix_tlb.c b/arch/powerpc/mm/book3s64/radix_tlb.c
> index b5cc9b23cf02..180d8ddcf6e3 100644
> --- a/arch/powerpc/mm/book3s64/radix_tlb.c
> +++ b/arch/powerpc/mm/book3s64/radix_tlb.c
> @@ -16,6 +16,7 @@
>  #include <asm/tlbflush.h>
>  #include <asm/trace.h>
>  #include <asm/cputhreads.h>
> +#include <asm/plpar_wrappers.h>
>  
>  #define RIC_FLUSH_TLB 0
>  #define RIC_FLUSH_PWC 1
> @@ -694,7 +695,14 @@ void radix__flush_tlb_mm(struct mm_struct *mm)
>  			goto local;
>  		}
>  
> -		if (cputlb_use_tlbie()) {
> +		if (!mmu_has_feature(MMU_FTR_GTSE)) {
> +			unsigned long tgt = H_RPTI_TARGET_CMMU;
> +
> +			if (atomic_read(&mm->context.copros) > 0)
> +				tgt |= H_RPTI_TARGET_NMMU;
> +			pseries_rpt_invalidate(pid, tgt, H_RPTI_TYPE_TLB,
> +					       H_RPTI_PAGE_ALL, 0, -1UL);
> +		} else if (cputlb_use_tlbie()) {
>  			if (mm_needs_flush_escalation(mm))
>  				_tlbie_pid(pid, RIC_FLUSH_ALL);
>  			else
> @@ -727,7 +735,16 @@ static void __flush_all_mm(struct mm_struct *mm, bool fullmm)
>  				goto local;
>  			}
>  		}
> -		if (cputlb_use_tlbie())
> +		if (!mmu_has_feature(MMU_FTR_GTSE)) {
> +			unsigned long tgt = H_RPTI_TARGET_CMMU;
> +			unsigned long type = H_RPTI_TYPE_TLB | H_RPTI_TYPE_PWC |
> +					     H_RPTI_TYPE_PRT;
> +
> +			if (atomic_read(&mm->context.copros) > 0)
> +				tgt |= H_RPTI_TARGET_NMMU;
> +			pseries_rpt_invalidate(pid, tgt, type,
> +					       H_RPTI_PAGE_ALL, 0, -1UL);
> +		} else if (cputlb_use_tlbie())
>  			_tlbie_pid(pid, RIC_FLUSH_ALL);
>  		else
>  			_tlbiel_pid_multicast(mm, pid, RIC_FLUSH_ALL);
> @@ -760,7 +777,19 @@ void radix__flush_tlb_page_psize(struct mm_struct *mm, unsigned long vmaddr,
>  			exit_flush_lazy_tlbs(mm);
>  			goto local;
>  		}
> -		if (cputlb_use_tlbie())
> +		if (!mmu_has_feature(MMU_FTR_GTSE)) {
> +			unsigned long tgt, page_sizes, size;
> +
> +			tgt = H_RPTI_TARGET_CMMU;
> +			page_sizes = psize_to_h_rpti(psize);
> +			size = 1UL << mmu_psize_to_shift(psize);
> +
> +			if (atomic_read(&mm->context.copros) > 0)
> +				tgt |= H_RPTI_TARGET_NMMU;
> +			pseries_rpt_invalidate(pid, tgt, H_RPTI_TYPE_TLB,
> +					       page_sizes, vmaddr,
> +					       vmaddr + size);
> +		} else if (cputlb_use_tlbie())
>  			_tlbie_va(vmaddr, pid, psize, RIC_FLUSH_TLB);
>  		else
>  			_tlbiel_va_multicast(mm, vmaddr, pid, psize, RIC_FLUSH_TLB);
> @@ -810,7 +839,14 @@ static inline void _tlbiel_kernel_broadcast(void)
>   */
>  void radix__flush_tlb_kernel_range(unsigned long start, unsigned long end)
>  {
> -	if (cputlb_use_tlbie())
> +	if (!mmu_has_feature(MMU_FTR_GTSE)) {
> +		unsigned long tgt = H_RPTI_TARGET_CMMU | H_RPTI_TARGET_NMMU;
> +		unsigned long type = H_RPTI_TYPE_TLB | H_RPTI_TYPE_PWC |
> +				     H_RPTI_TYPE_PRT;
> +
> +		pseries_rpt_invalidate(0, tgt, type, H_RPTI_PAGE_ALL,
> +				       start, end);
> +	} else if (cputlb_use_tlbie())
>  		_tlbie_pid(0, RIC_FLUSH_ALL);
>  	else
>  		_tlbiel_kernel_broadcast();
> @@ -864,7 +900,17 @@ static inline void __radix__flush_tlb_range(struct mm_struct *mm,
>  				nr_pages > tlb_local_single_page_flush_ceiling);
>  	}
>  
> -	if (full) {
> +	if (!mmu_has_feature(MMU_FTR_GTSE) && !local) {
> +		unsigned long tgt = H_RPTI_TARGET_CMMU;
> +		unsigned long page_sizes = psize_to_h_rpti(mmu_virtual_psize);
> +
> +		if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
> +			page_sizes |= psize_to_h_rpti(MMU_PAGE_2M);
> +		if (atomic_read(&mm->context.copros) > 0)
> +			tgt |= H_RPTI_TARGET_NMMU;
> +		pseries_rpt_invalidate(pid, tgt, H_RPTI_TYPE_TLB, page_sizes,
> +				       start, end);
> +	} else if (full) {
>  		if (local) {
>  			_tlbiel_pid(pid, RIC_FLUSH_TLB);
>  		} else {
> @@ -1046,7 +1092,17 @@ static __always_inline void __radix__flush_tlb_range_psize(struct mm_struct *mm,
>  				nr_pages > tlb_local_single_page_flush_ceiling);
>  	}
>  
> -	if (full) {
> +	if (!mmu_has_feature(MMU_FTR_GTSE) && !local) {
> +		unsigned long tgt = H_RPTI_TARGET_CMMU;
> +		unsigned long type = H_RPTI_TYPE_TLB;
> +		unsigned long page_sizes = psize_to_h_rpti(psize);
> +
> +		if (also_pwc)
> +			type |= H_RPTI_TYPE_PWC;
> +		if (atomic_read(&mm->context.copros) > 0)
> +			tgt |= H_RPTI_TARGET_NMMU;
> +		pseries_rpt_invalidate(pid, tgt, type, page_sizes, start, end);
> +	} else if (full) {
>  		if (local) {
>  			_tlbiel_pid(pid, also_pwc ? RIC_FLUSH_ALL : RIC_FLUSH_TLB);
>  		} else {
> @@ -1111,7 +1167,19 @@ void radix__flush_tlb_collapsed_pmd(struct mm_struct *mm, unsigned long addr)
>  			exit_flush_lazy_tlbs(mm);
>  			goto local;
>  		}
> -		if (cputlb_use_tlbie())
> +		if (!mmu_has_feature(MMU_FTR_GTSE)) {
> +			unsigned long tgt, type, page_sizes;
> +
> +			tgt = H_RPTI_TARGET_CMMU;
> +			type = H_RPTI_TYPE_TLB | H_RPTI_TYPE_PWC |
> +			       H_RPTI_TYPE_PRT;
> +			page_sizes = psize_to_h_rpti(mmu_virtual_psize);
> +
> +			if (atomic_read(&mm->context.copros) > 0)
> +				tgt |= H_RPTI_TARGET_NMMU;
> +			pseries_rpt_invalidate(pid, tgt, type, page_sizes,
> +					       addr, end);
> +		} else if (cputlb_use_tlbie())
>  			_tlbie_va_range(addr, end, pid, PAGE_SIZE, mmu_virtual_psize, true);
>  		else
>  			_tlbiel_va_range_multicast(mm,
> -- 
> 2.21.3

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2020-06-30  5:32 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-26 13:09 [PATCH v2 0/3] Off-load TLB invalidations to host for !GTSE Bharata B Rao
2020-06-26 13:09 ` [PATCH v2 1/3] powerpc/mm: Enable radix GTSE only if supported Bharata B Rao
2020-06-26 20:55   ` Murilo Opsfelder Araújo
2020-06-29  4:31     ` Bharata B Rao
2020-06-30  5:26   ` Aneesh Kumar K.V
2020-06-26 13:09 ` [PATCH v2 2/3] powerpc/pseries: H_REGISTER_PROC_TBL should ask for GTSE only if enabled Bharata B Rao
2020-06-30  5:26   ` Aneesh Kumar K.V
2020-06-26 13:10 ` [PATCH v2 3/3] powerpc/mm/book3s64/radix: Off-load TLB invalidations to host when !GTSE Bharata B Rao
2020-06-30  5:26   ` Aneesh Kumar K.V

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.