[PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[Wei Chen]

While we're doing apply_alternatives, we will generate new instructions
if required. The new instructions need to update the Xen text section,
but Xen text section is read-only. So we re-map Xen to a new virtual
address to enable write access.

The targets of the new generated instructions are located in this
re-mapped Xen area. But we haven't register this area as a virtual
region, so the checking code determines the targets are not in the
Xen text section, the new instructions could not be generated.

In this patch, we register the re-mapped Xen area as a temporary
virtual region to make the new instructions can be generated
successfully.

Signed-off-by: Wei Chen <Wei.Chen@arm.com>
---
 xen/arch/arm/alternative.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/alternative.c b/xen/arch/arm/alternative.c
index 1d10f51..96859fc 100644
--- a/xen/arch/arm/alternative.c
+++ b/xen/arch/arm/alternative.c
@@ -24,6 +24,7 @@
 #include <xen/vmap.h>
 #include <xen/smp.h>
 #include <xen/stop_machine.h>
+#include <xen/virtual_region.h>
 #include <asm/alternative.h>
 #include <asm/atomic.h>
 #include <asm/byteorder.h>
@@ -154,8 +155,12 @@ static int __apply_alternatives_multi_stop(void *unused)
         int ret;
         struct alt_region region;
         mfn_t xen_mfn = _mfn(virt_to_mfn(_start));
-        unsigned int xen_order = get_order_from_bytes(_end - _start);
+        unsigned int xen_size = _end - _start;
+        unsigned int xen_order = get_order_from_bytes(xen_size);
         void *xenmap;
+        struct virtual_region patch_region = {
+            .list = LIST_HEAD_INIT(patch_region.list),
+        };
 
         BUG_ON(patched);
 
@@ -169,6 +174,15 @@ static int __apply_alternatives_multi_stop(void *unused)
         BUG_ON(!xenmap);
 
         /*
+         * If we generate a new branch instruction, the target will be
+         * calculated in this re-mapped Xen region. So we have to register
+         * this re-mapped Xen region as a virtual region temporarily.
+         */
+        patch_region.start = xenmap;
+        patch_region.end = xenmap + xen_size;
+        register_virtual_region(&patch_region);
+
+        /*
          * Find the virtual address of the alternative region in the new
          * mapping.
          * alt_instr contains relative offset, so the function
@@ -182,6 +196,8 @@ static int __apply_alternatives_multi_stop(void *unused)
         /* The patching is not expected to fail during boot. */
         BUG_ON(ret != 0);
 
+        unregister_virtual_region(&patch_region);
+
         vunmap(xenmap);
 
         /* Barriers provided by the cache flushing */
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[Konrad Rzeszutek Wilk]

On Tue, Mar 14, 2017 at 05:27:17PM +0800, Wei Chen wrote:
> While we're doing apply_alternatives, we will generate new instructions
> if required. The new instructions need to update the Xen text section,
> but Xen text section is read-only. So we re-map Xen to a new virtual
> address to enable write access.
> 
> The targets of the new generated instructions are located in this
> re-mapped Xen area. But we haven't register this area as a virtual
> region, so the checking code determines the targets are not in the
> Xen text section, the new instructions could not be generated.

Could you expand on that please? Where is the checking code that determines
this? Are we talking about the traps handling and them scanning this
new region?

But you are saying 'new instructions'.. Hm, please enlighten!
> 
> In this patch, we register the re-mapped Xen area as a temporary
> virtual region to make the new instructions can be generated
> successfully.
> 
> Signed-off-by: Wei Chen <Wei.Chen@arm.com>
> ---
>  xen/arch/arm/alternative.c | 18 +++++++++++++++++-
>  1 file changed, 17 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/alternative.c b/xen/arch/arm/alternative.c
> index 1d10f51..96859fc 100644
> --- a/xen/arch/arm/alternative.c
> +++ b/xen/arch/arm/alternative.c
> @@ -24,6 +24,7 @@
>  #include <xen/vmap.h>
>  #include <xen/smp.h>
>  #include <xen/stop_machine.h>
> +#include <xen/virtual_region.h>
>  #include <asm/alternative.h>
>  #include <asm/atomic.h>
>  #include <asm/byteorder.h>
> @@ -154,8 +155,12 @@ static int __apply_alternatives_multi_stop(void *unused)
>          int ret;
>          struct alt_region region;
>          mfn_t xen_mfn = _mfn(virt_to_mfn(_start));
> -        unsigned int xen_order = get_order_from_bytes(_end - _start);
> +        unsigned int xen_size = _end - _start;
> +        unsigned int xen_order = get_order_from_bytes(xen_size);
>          void *xenmap;
> +        struct virtual_region patch_region = {
> +            .list = LIST_HEAD_INIT(patch_region.list),
> +        };
>  
>          BUG_ON(patched);
>  
> @@ -169,6 +174,15 @@ static int __apply_alternatives_multi_stop(void *unused)
>          BUG_ON(!xenmap);
>  
>          /*
> +         * If we generate a new branch instruction, the target will be
> +         * calculated in this re-mapped Xen region. So we have to register
> +         * this re-mapped Xen region as a virtual region temporarily.
> +         */
> +        patch_region.start = xenmap;
> +        patch_region.end = xenmap + xen_size;
> +        register_virtual_region(&patch_region);
> +
> +        /*
>           * Find the virtual address of the alternative region in the new
>           * mapping.
>           * alt_instr contains relative offset, so the function
> @@ -182,6 +196,8 @@ static int __apply_alternatives_multi_stop(void *unused)
>          /* The patching is not expected to fail during boot. */
>          BUG_ON(ret != 0);
>  
> +        unregister_virtual_region(&patch_region);
> +
>          vunmap(xenmap);
>  
>          /* Barriers provided by the cache flushing */
> -- 
> 2.7.4
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[Julien Grall]

Hi Konrad,

On 14/03/17 13:20, Konrad Rzeszutek Wilk wrote:
> On Tue, Mar 14, 2017 at 05:27:17PM +0800, Wei Chen wrote:
>> While we're doing apply_alternatives, we will generate new instructions
>> if required. The new instructions need to update the Xen text section,
>> but Xen text section is read-only. So we re-map Xen to a new virtual
>> address to enable write access.
>>
>> The targets of the new generated instructions are located in this
>> re-mapped Xen area. But we haven't register this area as a virtual
>> region, so the checking code determines the targets are not in the
>> Xen text section, the new instructions could not be generated.
>
> Could you expand on that please? Where is the checking code that determines
> this? Are we talking about the traps handling and them scanning this
> new region?
>
> But you are saying 'new instructions'.. Hm, please enlighten!

He is talking about the check in branch_insn_requires_update. There is 
some sanity checking about the branch offset. Because Xen text section 
is marked as read-only and we configure the hardware to not allow a 
region to be writable and executable at the same time, we need to re-map 
Xen in a temporary area. This means that the pc given in parameter of 
branch_insn_requires_update will point to a value in the re-mapped area. 
So the check is_active_kernel_text will always return false.

Registering the virtual region temporarily will solve the problem.

Wei, it would be worth to explain that you hit the BUG(); in 
branch_insn_* and show an example in the commit message.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[Konrad Rzeszutek Wilk]

On Tue, Mar 14, 2017 at 01:32:31PM +0000, Julien Grall wrote:
> Hi Konrad,
> 
> On 14/03/17 13:20, Konrad Rzeszutek Wilk wrote:
> > On Tue, Mar 14, 2017 at 05:27:17PM +0800, Wei Chen wrote:
> > > While we're doing apply_alternatives, we will generate new instructions
> > > if required. The new instructions need to update the Xen text section,
> > > but Xen text section is read-only. So we re-map Xen to a new virtual
> > > address to enable write access.
> > > 
> > > The targets of the new generated instructions are located in this
> > > re-mapped Xen area. But we haven't register this area as a virtual
> > > region, so the checking code determines the targets are not in the
> > > Xen text section, the new instructions could not be generated.
> > 
> > Could you expand on that please? Where is the checking code that determines
> > this? Are we talking about the traps handling and them scanning this
> > new region?
> > 
> > But you are saying 'new instructions'.. Hm, please enlighten!
> 
> He is talking about the check in branch_insn_requires_update. There is some

<lightbulb lights up>

> sanity checking about the branch offset. Because Xen text section is marked
> as read-only and we configure the hardware to not allow a region to be
> writable and executable at the same time, we need to re-map Xen in a
> temporary area. This means that the pc given in parameter of
> branch_insn_requires_update will point to a value in the re-mapped area. So
> the check is_active_kernel_text will always return false.

Yup.
> 
> Registering the virtual region temporarily will solve the problem.
> 
> Wei, it would be worth to explain that you hit the BUG(); in branch_insn_*
> and show an example in the commit message.

<nods>
> 
> Cheers,
> 
> -- 
> Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[Julien Grall]

Hello Wei,

Title: I would add alternative to make clear this is touching alternative.

On 14/03/17 09:27, Wei Chen wrote:
> While we're doing apply_alternatives, we will generate new instructions
> if required. The new instructions need to update the Xen text section,
> but Xen text section is read-only. So we re-map Xen to a new virtual
> address to enable write access.
>
> The targets of the new generated instructions are located in this
> re-mapped Xen area. But we haven't register this area as a virtual
> region, so the checking code determines the targets are not in the
> Xen text section, the new instructions could not be generated.
>
> In this patch, we register the re-mapped Xen area as a temporary
> virtual region to make the new instructions can be generated
> successfully.
>
> Signed-off-by: Wei Chen <Wei.Chen@arm.com>
> ---

This patch would need to be backported on Xen 4.8.

For the rest, see Konrad's comment.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[Julien Grall]

Hello Wei,

On 14/03/17 09:27, Wei Chen wrote:
> While we're doing apply_alternatives, we will generate new instructions

I didn't spot this on the first review. apply_alternatives does not call 
__apply_alternatives_multi_stop, so please clarify the problem.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[Wei Chen]

Hi Julien,

On 2017/3/14 21:32, Julien Grall wrote:
> Hi Konrad,
>
> On 14/03/17 13:20, Konrad Rzeszutek Wilk wrote:
>> On Tue, Mar 14, 2017 at 05:27:17PM +0800, Wei Chen wrote:
>>> While we're doing apply_alternatives, we will generate new instructions
>>> if required. The new instructions need to update the Xen text section,
>>> but Xen text section is read-only. So we re-map Xen to a new virtual
>>> address to enable write access.
>>>
>>> The targets of the new generated instructions are located in this
>>> re-mapped Xen area. But we haven't register this area as a virtual
>>> region, so the checking code determines the targets are not in the
>>> Xen text section, the new instructions could not be generated.
>>
>> Could you expand on that please? Where is the checking code that determines
>> this? Are we talking about the traps handling and them scanning this
>> new region?
>>
>> But you are saying 'new instructions'.. Hm, please enlighten!
>
> He is talking about the check in branch_insn_requires_update. There is
> some sanity checking about the branch offset. Because Xen text section
> is marked as read-only and we configure the hardware to not allow a
> region to be writable and executable at the same time, we need to re-map
> Xen in a temporary area. This means that the pc given in parameter of
> branch_insn_requires_update will point to a value in the re-mapped area.
> So the check is_active_kernel_text will always return false.
>
> Registering the virtual region temporarily will solve the problem.
>
> Wei, it would be worth to explain that you hit the BUG(); in
> branch_insn_* and show an example in the commit message.
>

Thanks for helping me to explain it. I will send a new version with
updated commit message. I think the new commit message would include:
1. Include alternative subsystem to the title.
2. Explain where the problem comes from.
3. An example to explain how to hit the BUG().
4. Which source trees and versions will be affected.

> Cheers,
>


-- 
Regards,
Wei Chen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[Wei Chen]

Hi Julien,

On 2017/3/14 23:35, Julien Grall wrote:
> Hello Wei,
>
> Title: I would add alternative to make clear this is touching alternative.
>
> On 14/03/17 09:27, Wei Chen wrote:
>> While we're doing apply_alternatives, we will generate new instructions
>> if required. The new instructions need to update the Xen text section,
>> but Xen text section is read-only. So we re-map Xen to a new virtual
>> address to enable write access.
>>
>> The targets of the new generated instructions are located in this
>> re-mapped Xen area. But we haven't register this area as a virtual
>> region, so the checking code determines the targets are not in the
>> Xen text section, the new instructions could not be generated.
>>
>> In this patch, we register the re-mapped Xen area as a temporary
>> virtual region to make the new instructions can be generated
>> successfully.
>>
>> Signed-off-by: Wei Chen <Wei.Chen@arm.com>
>> ---
>
> This patch would need to be backported on Xen 4.8.
>
> For the rest, see Konrad's comment.
>

Ok, I will do it.

> Regards,
>


-- 
Regards,
Wei Chen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[Wei Chen]

Hi Konrad,

On 2017/3/14 21:56, Konrad Rzeszutek Wilk wrote:
> On Tue, Mar 14, 2017 at 01:32:31PM +0000, Julien Grall wrote:
>> Hi Konrad,
>>
>> On 14/03/17 13:20, Konrad Rzeszutek Wilk wrote:
>>> On Tue, Mar 14, 2017 at 05:27:17PM +0800, Wei Chen wrote:
>>>> While we're doing apply_alternatives, we will generate new instructions
>>>> if required. The new instructions need to update the Xen text section,
>>>> but Xen text section is read-only. So we re-map Xen to a new virtual
>>>> address to enable write access.
>>>>
>>>> The targets of the new generated instructions are located in this
>>>> re-mapped Xen area. But we haven't register this area as a virtual
>>>> region, so the checking code determines the targets are not in the
>>>> Xen text section, the new instructions could not be generated.
>>>
>>> Could you expand on that please? Where is the checking code that determines
>>> this? Are we talking about the traps handling and them scanning this
>>> new region?
>>>
>>> But you are saying 'new instructions'.. Hm, please enlighten!
>>
>> He is talking about the check in branch_insn_requires_update. There is some
>
> <lightbulb lights up>
>
>> sanity checking about the branch offset. Because Xen text section is marked
>> as read-only and we configure the hardware to not allow a region to be
>> writable and executable at the same time, we need to re-map Xen in a
>> temporary area. This means that the pc given in parameter of
>> branch_insn_requires_update will point to a value in the re-mapped area. So
>> the check is_active_kernel_text will always return false.
>
> Yup.
>>
>> Registering the virtual region temporarily will solve the problem.
>>
>> Wei, it would be worth to explain that you hit the BUG(); in branch_insn_*
>> and show an example in the commit message.
>
> <nods>

Thanks for your comments, I sent this patch to quickly and hadn't
provided enough notes. I will update the commit message in the next
version.

>>
>> Cheers,
>>
>> --
>> Julien Grall
>


-- 
Regards,
Wei Chen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH] xen/arm: Register re-mapped Xen area as a temporary virtual region

[Wei Chen]

Hi Julien,

On 2017/3/15 0:01, Julien Grall wrote:
> Hello Wei,
>
> On 14/03/17 09:27, Wei Chen wrote:
>> While we're doing apply_alternatives, we will generate new instructions
>
> I didn't spot this on the first review. apply_alternatives does not call
> __apply_alternatives_multi_stop, so please clarify the problem.
>

Yes, it should be __apply_alternatives_multi_stop. I will clarify it.

> Cheers,
>


-- 
Regards,
Wei Chen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel