* Re: Qemu and ARM secure state. [not found] <07e63acb-b756-2586-2ba2-b54b837f7fc8@tribudubois.net> @ 2021-11-04 11:11 ` Peter Maydell 2021-11-04 21:11 ` Jean-Christophe DUBOIS 0 siblings, 1 reply; 12+ messages in thread From: Peter Maydell @ 2021-11-04 11:11 UTC (permalink / raw) To: Jean-Christophe DUBOIS; +Cc: qemu-arm, QEMU Developers On Wed, 3 Nov 2021 at 13:27, Jean-Christophe DUBOIS <jcd@tribudubois.net> wrote: > I have a little application that is designed to work on the i.MX6UL processor. > > I developed it and tested it on the mcimx6ul-evk platform emulated by Qemu. > > This application used to work "flawlessly" on Qemu 5.0.50 and is working on Qemu 6.0.0 (available as a pre-built package on the latest Ubuntu). > > But when I try to run the exact same command line on a Qemu version I compile myself from main/latest of github (Qemu 6.1.50), my application fails to start. > > So a little background: > > My application expects to start in "secure" state and supervisor mode (which is the default state of i.MX6UL when booting barebone [without u-boot]). > > From this state the application tries to get to "non secure" / hypervisor mode which imply going to the "secure" / monitor state before being able to drop to "non secure" / hypervisor. To do so is runs a "smc 0" operand (from "secure" / supervisor). > > This "smc" instruction is processed "as expected" by Qemu 5.0.50 and Qemu 6.0.0 (getting to "secure" / monitor mode) but on Qemu 6.1.50 (latest from github) it is as if the smc operand was a no-op. It doesn't trigger any exception and the processor just get to the next instruction after the "smc" instruction. So I am a bit puzzled. > > Is there something that changed in Qemu (since Qemu 6.0.0) when it comes to the "secure" world/state? > Is there some additional command line parameters to use (I search in the documentation but without luck) to get secure world behavior ? > Is it necessary to "adapt" the emulated platform (i.MX6UL/mcimx6ul-evk) in some way (it looks like the "virt" machine with "secure=on" does work for arm platform)? Could you try doing a bisect to find the QEMU commit that caused your guest to stop working ? thanks -- PMM ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-04 11:11 ` Qemu and ARM secure state Peter Maydell @ 2021-11-04 21:11 ` Jean-Christophe DUBOIS 2021-11-06 10:04 ` Jean-Christophe DUBOIS 0 siblings, 1 reply; 12+ messages in thread From: Jean-Christophe DUBOIS @ 2021-11-04 21:11 UTC (permalink / raw) To: Peter Maydell; +Cc: qemu-arm, QEMU Developers [-- Attachment #1: Type: text/plain, Size: 2244 bytes --] Le 04/11/2021 à 12:11, Peter Maydell a écrit : > On Wed, 3 Nov 2021 at 13:27, Jean-Christophe DUBOIS<jcd@tribudubois.net> wrote: >> I have a little application that is designed to work on the i.MX6UL processor. >> >> I developed it and tested it on the mcimx6ul-evk platform emulated by Qemu. >> >> This application used to work "flawlessly" on Qemu 5.0.50 and is working on Qemu 6.0.0 (available as a pre-built package on the latest Ubuntu). >> >> But when I try to run the exact same command line on a Qemu version I compile myself from main/latest of github (Qemu 6.1.50), my application fails to start. >> >> So a little background: >> >> My application expects to start in "secure" state and supervisor mode (which is the default state of i.MX6UL when booting barebone [without u-boot]). >> >> From this state the application tries to get to "non secure" / hypervisor mode which imply going to the "secure" / monitor state before being able to drop to "non secure" / hypervisor. To do so is runs a "smc 0" operand (from "secure" / supervisor). >> >> This "smc" instruction is processed "as expected" by Qemu 5.0.50 and Qemu 6.0.0 (getting to "secure" / monitor mode) but on Qemu 6.1.50 (latest from github) it is as if the smc operand was a no-op. It doesn't trigger any exception and the processor just get to the next instruction after the "smc" instruction. So I am a bit puzzled. >> >> Is there something that changed in Qemu (since Qemu 6.0.0) when it comes to the "secure" world/state? >> Is there some additional command line parameters to use (I search in the documentation but without luck) to get secure world behavior ? >> Is it necessary to "adapt" the emulated platform (i.MX6UL/mcimx6ul-evk) in some way (it looks like the "virt" machine with "secure=on" does work for arm platform)? > Could you try doing a bisect to find the QEMU commit that caused > your guest to stop working ? OK, I did the bisect and the commit that break the i.MX6UL behavior for my program is commit 9fcd15b9193e819b6cc2fd0a45e3506148812bb4 (arm: tcg: Adhere to SMCCC 1.3 section 5.2). Before it the SMC instruction would trigger a monitor exception. After it the SMC instruction is acting like a no-op. Thanks JC > > thanks > -- PMM > [-- Attachment #2: Type: text/html, Size: 3309 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-04 21:11 ` Jean-Christophe DUBOIS @ 2021-11-06 10:04 ` Jean-Christophe DUBOIS 2021-11-06 13:04 ` Jean-Christophe DUBOIS 0 siblings, 1 reply; 12+ messages in thread From: Jean-Christophe DUBOIS @ 2021-11-06 10:04 UTC (permalink / raw) To: Peter Maydell; +Cc: qemu-arm, QEMU Developers [-- Attachment #1: Type: text/plain, Size: 3843 bytes --] So, I am trying to understand: Contrary to what I said, In my case the SMC instruction is not really a "no-op" as it sets R0 to -1 (0xffffffff) to indicate an unknown PSCI service (by the Qemu internal PSCI handler). With the new code introduced by the "arm: tcg: Adhere to SMCCC 1.3 section 5.2" commit, once a processor/platform configure things to have PSCI requests handled by Qemu code (with "psci-conduit" attribute set to QEMU_PSCI_CONDUIT_SMC for example), then any exception raised by an "SMC" instruction will be only handled by the Qemu internal code and will no call the monitor related code in the guest/OS application. This seems to be why my SMC monitor handler is not called anymore in my case. As my i.MX6UL is a mono-processor platform I don't really need to set the "psci-conduit" attribute (which really makes sense when you have a cluster of 2 or more cores I guess). As a matter of fact if I remove the "psci-conduit" attribute setting from the i.MX6UL processor file, my application is working again on main/latest. But this still raises the question to know if the current behavior for processors with "psci-conduit" set to SMC or HVC is correct. For example an i.MX7 based platform (with up to 4 cortex A7 cores) would not be able to trigger OS SMC handler as the exception would be entirely processed by the Qemu internal code (with CR generally set to -1 in R0 to indicate unknown PSCI request). Is there something I am missing? Regards JC Le 04/11/2021 à 22:11, Jean-Christophe DUBOIS a écrit : > Le 04/11/2021 à 12:11, Peter Maydell a écrit : >> On Wed, 3 Nov 2021 at 13:27, Jean-Christophe DUBOIS<jcd@tribudubois.net> wrote: >>> I have a little application that is designed to work on the i.MX6UL processor. >>> >>> I developed it and tested it on the mcimx6ul-evk platform emulated by Qemu. >>> >>> This application used to work "flawlessly" on Qemu 5.0.50 and is working on Qemu 6.0.0 (available as a pre-built package on the latest Ubuntu). >>> >>> But when I try to run the exact same command line on a Qemu version I compile myself from main/latest of github (Qemu 6.1.50), my application fails to start. >>> >>> So a little background: >>> >>> My application expects to start in "secure" state and supervisor mode (which is the default state of i.MX6UL when booting barebone [without u-boot]). >>> >>> >From this state the application tries to get to "non secure" / hypervisor mode which imply going to the "secure" / monitor state before being able to drop to "non secure" / hypervisor. To do so is runs a "smc 0" operand (from "secure" / supervisor). >>> >>> This "smc" instruction is processed "as expected" by Qemu 5.0.50 and Qemu 6.0.0 (getting to "secure" / monitor mode) but on Qemu 6.1.50 (latest from github) it is as if the smc operand was a no-op. It doesn't trigger any exception and the processor just get to the next instruction after the "smc" instruction. So I am a bit puzzled. >>> >>> Is there something that changed in Qemu (since Qemu 6.0.0) when it comes to the "secure" world/state? >>> Is there some additional command line parameters to use (I search in the documentation but without luck) to get secure world behavior ? >>> Is it necessary to "adapt" the emulated platform (i.MX6UL/mcimx6ul-evk) in some way (it looks like the "virt" machine with "secure=on" does work for arm platform)? >> Could you try doing a bisect to find the QEMU commit that caused >> your guest to stop working ? > > OK, I did the bisect and the commit that break the i.MX6UL behavior > for my program is commit 9fcd15b9193e819b6cc2fd0a45e3506148812bb4 > (arm: tcg: Adhere to SMCCC 1.3 section 5.2). > > Before it the SMC instruction would trigger a monitor exception. > > After it the SMC instruction is acting like a no-op. > > Thanks > > JC > > >> thanks >> -- PMM >> > [-- Attachment #2: Type: text/html, Size: 6052 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-06 10:04 ` Jean-Christophe DUBOIS @ 2021-11-06 13:04 ` Jean-Christophe DUBOIS 2021-11-06 18:11 ` Jean-Christophe DUBOIS 0 siblings, 1 reply; 12+ messages in thread From: Jean-Christophe DUBOIS @ 2021-11-06 13:04 UTC (permalink / raw) To: Peter Maydell; +Cc: qemu-arm, QEMU Developers [-- Attachment #1: Type: text/plain, Size: 4399 bytes --] So it seems that what is needed is a way to choose on the command line if we want to enable the Qemu built-in PSCI implementation (because we are booting linux for example) or if we really want a bare metal processor (because we are booting a trustedOS like optee). The "virt" platform allows to dynamically choose one or the other. Other platforms seems to need the same feature. JC Le 06/11/2021 à 11:04, Jean-Christophe DUBOIS a écrit : > So, I am trying to understand: > > Contrary to what I said, In my case the SMC instruction is not really > a "no-op" as it sets R0 to -1 (0xffffffff) to indicate an unknown PSCI > service (by the Qemu internal PSCI handler). > > With the new code introduced by the "arm: tcg: Adhere to SMCCC 1.3 > section 5.2" commit, once a processor/platform configure things to > have PSCI requests handled by Qemu code (with "psci-conduit" attribute > set to QEMU_PSCI_CONDUIT_SMC for example), then any exception raised > by an "SMC" instruction will be only handled by the Qemu internal code > and will no call the monitor related code in the guest/OS application. > This seems to be why my SMC monitor handler is not called anymore in > my case. > > As my i.MX6UL is a mono-processor platform I don't really need to set > the "psci-conduit" attribute (which really makes sense when you have a > cluster of 2 or more cores I guess). As a matter of fact if I remove > the "psci-conduit" attribute setting from the i.MX6UL processor file, > my application is working again on main/latest. > > But this still raises the question to know if the current behavior for > processors with "psci-conduit" set to SMC or HVC is correct. For > example an i.MX7 based platform (with up to 4 cortex A7 cores) would > not be able to trigger OS SMC handler as the exception would be > entirely processed by the Qemu internal code (with CR generally set to > -1 in R0 to indicate unknown PSCI request). > > Is there something I am missing? > > Regards > > JC > > Le 04/11/2021 à 22:11, Jean-Christophe DUBOIS a écrit : >> Le 04/11/2021 à 12:11, Peter Maydell a écrit : >>> On Wed, 3 Nov 2021 at 13:27, Jean-Christophe DUBOIS<jcd@tribudubois.net> wrote: >>>> I have a little application that is designed to work on the i.MX6UL processor. >>>> >>>> I developed it and tested it on the mcimx6ul-evk platform emulated by Qemu. >>>> >>>> This application used to work "flawlessly" on Qemu 5.0.50 and is working on Qemu 6.0.0 (available as a pre-built package on the latest Ubuntu). >>>> >>>> But when I try to run the exact same command line on a Qemu version I compile myself from main/latest of github (Qemu 6.1.50), my application fails to start. >>>> >>>> So a little background: >>>> >>>> My application expects to start in "secure" state and supervisor mode (which is the default state of i.MX6UL when booting barebone [without u-boot]). >>>> >>>> >From this state the application tries to get to "non secure" / hypervisor mode which imply going to the "secure" / monitor state before being able to drop to "non secure" / hypervisor. To do so is runs a "smc 0" operand (from "secure" / supervisor). >>>> >>>> This "smc" instruction is processed "as expected" by Qemu 5.0.50 and Qemu 6.0.0 (getting to "secure" / monitor mode) but on Qemu 6.1.50 (latest from github) it is as if the smc operand was a no-op. It doesn't trigger any exception and the processor just get to the next instruction after the "smc" instruction. So I am a bit puzzled. >>>> >>>> Is there something that changed in Qemu (since Qemu 6.0.0) when it comes to the "secure" world/state? >>>> Is there some additional command line parameters to use (I search in the documentation but without luck) to get secure world behavior ? >>>> Is it necessary to "adapt" the emulated platform (i.MX6UL/mcimx6ul-evk) in some way (it looks like the "virt" machine with "secure=on" does work for arm platform)? >>> Could you try doing a bisect to find the QEMU commit that caused >>> your guest to stop working ? >> >> OK, I did the bisect and the commit that break the i.MX6UL behavior >> for my program is commit 9fcd15b9193e819b6cc2fd0a45e3506148812bb4 >> (arm: tcg: Adhere to SMCCC 1.3 section 5.2). >> >> Before it the SMC instruction would trigger a monitor exception. >> >> After it the SMC instruction is acting like a no-op. >> >> Thanks >> >> JC >> >> >>> thanks >>> -- PMM >>> >> > [-- Attachment #2: Type: text/html, Size: 7245 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-06 13:04 ` Jean-Christophe DUBOIS @ 2021-11-06 18:11 ` Jean-Christophe DUBOIS 2021-11-08 14:14 ` Alex Bennée 2021-11-08 14:50 ` Peter Maydell 0 siblings, 2 replies; 12+ messages in thread From: Jean-Christophe DUBOIS @ 2021-11-06 18:11 UTC (permalink / raw) To: Peter Maydell; +Cc: qemu-arm, QEMU Developers [-- Attachment #1: Type: text/plain, Size: 4959 bytes --] One small question/remark: According to the the "Arm Power State Coordinate Interface" (DEN0022D.b) document (chapter 5) PSCI calls can only be issued by "normal world" (EL1 or EL2). Therefore, should we be adding a test for the current secure state in the arm_is_psci_call() function? This would prevent calling the built-in Qemu PSCI function if SMC is issued from secure state. Regards. JC Le 06/11/2021 à 14:04, Jean-Christophe DUBOIS a écrit : > So it seems that what is needed is a way to choose on the command line > if we want to enable the Qemu built-in PSCI implementation (because we > are booting linux for example) or if we really want a bare metal > processor (because we are booting a trustedOS like optee). > > The "virt" platform allows to dynamically choose one or the other. > Other platforms seems to need the same feature. > > JC > > Le 06/11/2021 à 11:04, Jean-Christophe DUBOIS a écrit : >> So, I am trying to understand: >> >> Contrary to what I said, In my case the SMC instruction is not really >> a "no-op" as it sets R0 to -1 (0xffffffff) to indicate an unknown >> PSCI service (by the Qemu internal PSCI handler). >> >> With the new code introduced by the "arm: tcg: Adhere to SMCCC 1.3 >> section 5.2" commit, once a processor/platform configure things to >> have PSCI requests handled by Qemu code (with "psci-conduit" >> attribute set to QEMU_PSCI_CONDUIT_SMC for example), then any >> exception raised by an "SMC" instruction will be only handled by the >> Qemu internal code and will no call the monitor related code in the >> guest/OS application. This seems to be why my SMC monitor handler is >> not called anymore in my case. >> >> As my i.MX6UL is a mono-processor platform I don't really need to set >> the "psci-conduit" attribute (which really makes sense when you have >> a cluster of 2 or more cores I guess). As a matter of fact if I >> remove the "psci-conduit" attribute setting from the i.MX6UL >> processor file, my application is working again on main/latest. >> >> But this still raises the question to know if the current behavior >> for processors with "psci-conduit" set to SMC or HVC is correct. For >> example an i.MX7 based platform (with up to 4 cortex A7 cores) would >> not be able to trigger OS SMC handler as the exception would be >> entirely processed by the Qemu internal code (with CR generally set >> to -1 in R0 to indicate unknown PSCI request). >> >> Is there something I am missing? >> >> Regards >> >> JC >> >> Le 04/11/2021 à 22:11, Jean-Christophe DUBOIS a écrit : >>> Le 04/11/2021 à 12:11, Peter Maydell a écrit : >>>> On Wed, 3 Nov 2021 at 13:27, Jean-Christophe DUBOIS<jcd@tribudubois.net> wrote: >>>>> I have a little application that is designed to work on the i.MX6UL processor. >>>>> >>>>> I developed it and tested it on the mcimx6ul-evk platform emulated by Qemu. >>>>> >>>>> This application used to work "flawlessly" on Qemu 5.0.50 and is working on Qemu 6.0.0 (available as a pre-built package on the latest Ubuntu). >>>>> >>>>> But when I try to run the exact same command line on a Qemu version I compile myself from main/latest of github (Qemu 6.1.50), my application fails to start. >>>>> >>>>> So a little background: >>>>> >>>>> My application expects to start in "secure" state and supervisor mode (which is the default state of i.MX6UL when booting barebone [without u-boot]). >>>>> >>>>> >From this state the application tries to get to "non secure" / hypervisor mode which imply going to the "secure" / monitor state before being able to drop to "non secure" / hypervisor. To do so is runs a "smc 0" operand (from "secure" / supervisor). >>>>> >>>>> This "smc" instruction is processed "as expected" by Qemu 5.0.50 and Qemu 6.0.0 (getting to "secure" / monitor mode) but on Qemu 6.1.50 (latest from github) it is as if the smc operand was a no-op. It doesn't trigger any exception and the processor just get to the next instruction after the "smc" instruction. So I am a bit puzzled. >>>>> >>>>> Is there something that changed in Qemu (since Qemu 6.0.0) when it comes to the "secure" world/state? >>>>> Is there some additional command line parameters to use (I search in the documentation but without luck) to get secure world behavior ? >>>>> Is it necessary to "adapt" the emulated platform (i.MX6UL/mcimx6ul-evk) in some way (it looks like the "virt" machine with "secure=on" does work for arm platform)? >>>> Could you try doing a bisect to find the QEMU commit that caused >>>> your guest to stop working ? >>> >>> OK, I did the bisect and the commit that break the i.MX6UL behavior >>> for my program is commit 9fcd15b9193e819b6cc2fd0a45e3506148812bb4 >>> (arm: tcg: Adhere to SMCCC 1.3 section 5.2). >>> >>> Before it the SMC instruction would trigger a monitor exception. >>> >>> After it the SMC instruction is acting like a no-op. >>> >>> Thanks >>> >>> JC >>> >>> >>>> thanks >>>> -- PMM >>>> >>> >> > [-- Attachment #2: Type: text/html, Size: 8634 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-06 18:11 ` Jean-Christophe DUBOIS @ 2021-11-08 14:14 ` Alex Bennée 2021-11-08 22:06 ` Jean-Christophe DUBOIS 2021-11-08 14:50 ` Peter Maydell 1 sibling, 1 reply; 12+ messages in thread From: Alex Bennée @ 2021-11-08 14:14 UTC (permalink / raw) To: Jean-Christophe DUBOIS; +Cc: Peter Maydell, qemu-arm, QEMU Developers Jean-Christophe DUBOIS <jcd@tribudubois.net> writes: > One small question/remark: > > According to the the "Arm Power State Coordinate Interface" (DEN0022D.b) document (chapter 5) PSCI calls can only be issued by > "normal world" (EL1 or EL2). Therefore, should we be adding a test for the current secure state in the arm_is_psci_call() function? This > would prevent calling the built-in Qemu PSCI function if SMC is issued > from secure state. All that should be handled in: void HELPER(pre_smc)(CPUARMState *env, uint32_t syndrome) which should cause things to be trapped if the CPU is in an invalid state to execute the SMC instruction. If the exception is a valid SMC we end up in arm_cpu_do_interrupt where we may divert to arm_handle_psci_call. -- Alex Bennée ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-08 14:14 ` Alex Bennée @ 2021-11-08 22:06 ` Jean-Christophe DUBOIS 0 siblings, 0 replies; 12+ messages in thread From: Jean-Christophe DUBOIS @ 2021-11-08 22:06 UTC (permalink / raw) To: Alex Bennée; +Cc: Peter Maydell, qemu-arm, QEMU Developers Le 08/11/2021 à 15:14, Alex Bennée a écrit : > Jean-Christophe DUBOIS <jcd@tribudubois.net> writes: > >> One small question/remark: >> >> According to the the "Arm Power State Coordinate Interface" (DEN0022D.b) document (chapter 5) PSCI calls can only be issued by >> "normal world" (EL1 or EL2). Therefore, should we be adding a test for the current secure state in the arm_is_psci_call() function? This >> would prevent calling the built-in Qemu PSCI function if SMC is issued >> from secure state. > All that should be handled in: > > void HELPER(pre_smc)(CPUARMState *env, uint32_t syndrome) > > which should cause things to be trapped if the CPU is in an invalid > state to execute the SMC instruction. If the exception is a valid SMC we > end up in arm_cpu_do_interrupt where we may divert to > arm_handle_psci_call. The problem I have is that is seems that once the "psci-conduit" is set to SMC, all SMC exception would be handled only by the diverted arm_handle_psci_call(). It seems there is no way to handle SMC exception in my software once "psci-conduit" is set to SMC on a platform. It used to be that only some PSCI services were diverted and other SMC services were handled by the EL3 software if any (and this split was actually also a bit awkward). > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-06 18:11 ` Jean-Christophe DUBOIS 2021-11-08 14:14 ` Alex Bennée @ 2021-11-08 14:50 ` Peter Maydell 2021-11-08 22:09 ` Jean-Christophe DUBOIS 1 sibling, 1 reply; 12+ messages in thread From: Peter Maydell @ 2021-11-08 14:50 UTC (permalink / raw) To: Jean-Christophe DUBOIS; +Cc: qemu-arm, QEMU Developers On Sat, 6 Nov 2021 at 18:11, Jean-Christophe DUBOIS <jcd@tribudubois.net> wrote: > > One small question/remark: > > According to the the "Arm Power State Coordinate Interface" (DEN0022D.b) document (chapter 5) PSCI calls can only be issued by "normal world" (EL1 or EL2). Therefore, should we be adding a test for the current secure state in the arm_is_psci_call() function? This would prevent calling the built-in Qemu PSCI function if SMC is issued from secure state. This shouldn't matter, because if the machine model is configured to execute guest code in EL3 at all then it should not be enabling QEMU's internal PSCI support. The internal PSCI stuff is only there as a kind of "emulated firmware" for when we're running guest code that starts at EL2 (notably, when directly booting a Linux kernel). The problem seems to be that fsl_imx6ul_realize() and fsl_imx7_realize() unconditionally enable PSCI-via-SMC. The imx7 code also puts all the secondaries into PSCI-powered-off mode -- this should be checked to work out what the right thing is if we're not doing emulated PSCI and instead starting the guest at EL3. -- PMM ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-08 14:50 ` Peter Maydell @ 2021-11-08 22:09 ` Jean-Christophe DUBOIS 2021-11-09 10:55 ` Peter Maydell 0 siblings, 1 reply; 12+ messages in thread From: Jean-Christophe DUBOIS @ 2021-11-08 22:09 UTC (permalink / raw) To: Peter Maydell; +Cc: qemu-arm, QEMU Developers Le 08/11/2021 à 15:50, Peter Maydell a écrit : > On Sat, 6 Nov 2021 at 18:11, Jean-Christophe DUBOIS <jcd@tribudubois.net> wrote: >> One small question/remark: >> >> According to the the "Arm Power State Coordinate Interface" (DEN0022D.b) document (chapter 5) PSCI calls can only be issued by "normal world" (EL1 or EL2). Therefore, should we be adding a test for the current secure state in the arm_is_psci_call() function? This would prevent calling the built-in Qemu PSCI function if SMC is issued from secure state. > This shouldn't matter, because if the machine model is configured > to execute guest code in EL3 at all then it should not be enabling > QEMU's internal PSCI support. The internal PSCI stuff is only > there as a kind of "emulated firmware" for when we're running > guest code that starts at EL2 (notably, when directly booting > a Linux kernel). > > The problem seems to be that fsl_imx6ul_realize() and > fsl_imx7_realize() unconditionally enable PSCI-via-SMC. > The imx7 code also puts all the secondaries into > PSCI-powered-off mode -- this should be checked to > work out what the right thing is if we're not doing > emulated PSCI and instead starting the guest at EL3. OK, so one problem seems to be that PSCI-via-SMC is enabled on i.MX6UL when there is no built in PSCI related function on this processor. According the Linux DTS, i.MX7 (solo and dual) processors have a somewhat PSCI related "entry-method" (https://github.com/torvalds/linux/blob/master/arch/arm/boot/dts/imx7s.dtsi). But it is not clear to me how this is used and this seems a bit strange as "entry-method" seems to be mostly used on arm64 and there is no other PSCI related information in the i.MX7 DTS files. As a matter of fact previous quad or dual i.MX6 were not supporting PSCI. Instead they were using a proprietary method through the internal SRC device (and i.MX7 also has a similar internal SRC device). But let's assume Linux on i.mx7 is actually using PSCI to handle processors. Thinking about it, I guess this might be u-boot that sets an EL3 monitor software that is able to handle PSCI requests for the Linux kernel. If this is the case, it make sense that Qemu emulates the PSCI services normally provided by u-boot to be able to boot linux directly (without booting a real u-boot prior to linux). All is well and nice. But then if I want to boot and test the u-boot binary (or any trusted OS for the matter) on a Qemu emulated i.MX7 (to later boot an hypervisor or an OS), it would be rather strange that any PSCI related service requested by the hypervisor/OS would be handled by Qemu directly and not by the u-boot code (or any other EL3 code) running on the processor. How is it supposed to work? How can I tell Qemu (dynamically?) if I want it to provide (or not) the PSCI services (and more generally SMC/HVC services). How can I tell it that I want to handle all SMC/EL3 services by myself even if the "psci-conduit" is already set to SMC in Qemu? Am I missing something? thanks. JC > > -- PMM > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-08 22:09 ` Jean-Christophe DUBOIS @ 2021-11-09 10:55 ` Peter Maydell 2021-11-09 19:06 ` Jean-Christophe DUBOIS 0 siblings, 1 reply; 12+ messages in thread From: Peter Maydell @ 2021-11-09 10:55 UTC (permalink / raw) To: Jean-Christophe DUBOIS; +Cc: qemu-arm, QEMU Developers On Mon, 8 Nov 2021 at 22:09, Jean-Christophe DUBOIS <jcd@tribudubois.net> wrote: > OK, so one problem seems to be that PSCI-via-SMC is enabled on i.MX6UL > when there is no built in PSCI related function on this processor. > > According the Linux DTS, i.MX7 (solo and dual) processors have a > somewhat PSCI related "entry-method" > (https://github.com/torvalds/linux/blob/master/arch/arm/boot/dts/imx7s.dtsi). > But it is not clear to me how this is used and this seems a bit strange > as "entry-method" seems to be mostly used on arm64 and there is no other > PSCI related information in the i.MX7 DTS files. Yeah, PSCI was an interface introduced mostly with aarch64. In the 32-bit world bringing up multiple CPUs was complete anarchy -- the way the kernel told the secure firmware that it should start up a secondary core was entirely determined by the firmware, and the kernel had to have board-specific code to do this. (For the 32-bit imx boards I think this is in arch/arm/mach-imx/src.c.) For aarch64 we had a clean slate and took the opportunity to insist that all boards did it the same way, ie using PSCI. (There are other useful things PSCI allows, but standardising secondary boot is the one that matters for this discussion.) So if a platform's firmware implements PSCI, all the dts file has to do is say so, and then there's no need for board-specific "start secondary CPUs" code. PSCI does define an aarch32 interface, but there are a lot of legacy older boards (and new flavours of boards in long-standing design families) which still do things the old way in aarch32 land. Typically the top-level "PSCI is available" node is added by the firmware. (QEMU will do this too when it's emulating PSCI firmware) -- if the board code enables the psci-conduit it will add an appropriate psci node in the hw/arm/boot.c code.) > As a matter of fact > previous quad or dual i.MX6 were not supporting PSCI. Instead they were > using a proprietary method through the internal SRC device (and i.MX7 > also has a similar internal SRC device). But let's assume Linux on i.mx7 > is actually using PSCI to handle processors. > > Thinking about it, I guess this might be u-boot that sets an EL3 monitor > software that is able to handle PSCI requests for the Linux kernel. If > this is the case, it make sense that Qemu emulates the PSCI services > normally provided by u-boot to be able to boot linux directly (without > booting a real u-boot prior to linux). All is well and nice. Yes, that's the way it works. The EL3 firmware is supposed to provide PSCI. For aarch64 the kernel is never entered in EL3 -- it will always run at EL2 or EL1. (This is unlike aarch32, where in some cases you might run the kernel in secure-SVC, although even there starting the kernel in NS-SVC or NS-Hyp is more common.) > But then if I want to boot and test the u-boot binary (or any trusted OS > for the matter) on a Qemu emulated i.MX7 (to later boot an hypervisor or > an OS), it would be rather strange that any PSCI related service > requested by the hypervisor/OS would be handled by Qemu directly and > not by the u-boot code (or any other EL3 code) running on the processor. Exactly. This is why the board code is supposed to set things up so that if we are starting the guest code in EL1 or EL2 then we enable the PSCI-via-SMC support, and if we're starting the guest code in EL3 then we do not. > How is it supposed to work? How can I tell Qemu (dynamically?) if I want > it to provide (or not) the PSCI services (and more generally SMC/HVC > services). If you want PSCI via SMC or HVC, then set the psci-conduit property on the CPUs to QEMU_PSCI_CONDUIT_SMC or QEMU_PSCI_CONDUIT_HVC. If you do not want QEMU to provide PSCI, then leave psci-conduit at its default (which is QEMU_PSCI_CONDUIT_DISABLED). How can I tell it that I want to handle all SMC/EL3 services > by myself even if the "psci-conduit" is already set to SMC in Qemu? It's the imx7 code that's setting psci-conduit, so it should not do that if it doesn't want that (and also should either start or not start the secondary cpus powered off, depending on what the hardware-to-firmware interface is supposed to be.) This is a bit awkward, because the boards we initially wanted PSCI for (notably virt) don't have an SoC object, so the code creating CPUs is in the same source file as the code that knows whether it's booting a kernel directly or not, and so it just open-codes the decision logic. With the imx, the CPU creation is in the source code for the SoC object which is abstracted away from the board model code. So we'd need to sort out how to plumb that information into the SoC object (or have the SoC object's code that creates the CPUs call some function to find out). -- PMM ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-09 10:55 ` Peter Maydell @ 2021-11-09 19:06 ` Jean-Christophe DUBOIS 2021-11-09 19:20 ` Peter Maydell 0 siblings, 1 reply; 12+ messages in thread From: Jean-Christophe DUBOIS @ 2021-11-09 19:06 UTC (permalink / raw) To: Peter Maydell; +Cc: qemu-arm, QEMU Developers [-- Attachment #1: Type: text/plain, Size: 7122 bytes --] Le 09/11/2021 à 11:55, Peter Maydell a écrit : > On Mon, 8 Nov 2021 at 22:09, Jean-Christophe DUBOIS<jcd@tribudubois.net> wrote: >> OK, so one problem seems to be that PSCI-via-SMC is enabled on i.MX6UL >> when there is no built in PSCI related function on this processor. >> >> According the Linux DTS, i.MX7 (solo and dual) processors have a >> somewhat PSCI related "entry-method" >> (https://github.com/torvalds/linux/blob/master/arch/arm/boot/dts/imx7s.dtsi). >> But it is not clear to me how this is used and this seems a bit strange >> as "entry-method" seems to be mostly used on arm64 and there is no other >> PSCI related information in the i.MX7 DTS files. > Yeah, PSCI was an interface introduced mostly with aarch64. In the > 32-bit world bringing up multiple CPUs was complete anarchy -- the > way the kernel told the secure firmware that it should start up > a secondary core was entirely determined by the firmware, and > the kernel had to have board-specific code to do this. (For the > 32-bit imx boards I think this is in arch/arm/mach-imx/src.c.) > For aarch64 we had a clean slate and took the opportunity to insist > that all boards did it the same way, ie using PSCI. (There are other > useful things PSCI allows, but standardising secondary boot is the > one that matters for this discussion.) So if a platform's firmware > implements PSCI, all the dts file has to do is say so, and then > there's no need for board-specific "start secondary CPUs" code. > PSCI does define an aarch32 interface, but there are a lot of > legacy older boards (and new flavours of boards in long-standing > design families) which still do things the old way in aarch32 land. > > Typically the top-level "PSCI is available" node is added by the firmware. > (QEMU will do this too when it's emulating PSCI firmware) -- if the > board code enables the psci-conduit it will add an appropriate psci > node in the hw/arm/boot.c code.) > >> As a matter of fact >> previous quad or dual i.MX6 were not supporting PSCI. Instead they were >> using a proprietary method through the internal SRC device (and i.MX7 >> also has a similar internal SRC device). But let's assume Linux on i.mx7 >> is actually using PSCI to handle processors. >> >> Thinking about it, I guess this might be u-boot that sets an EL3 monitor >> software that is able to handle PSCI requests for the Linux kernel. If >> this is the case, it make sense that Qemu emulates the PSCI services >> normally provided by u-boot to be able to boot linux directly (without >> booting a real u-boot prior to linux). All is well and nice. > Yes, that's the way it works. The EL3 firmware is supposed to provide > PSCI. > > For aarch64 the kernel is never entered in EL3 -- it will always run at EL2 > or EL1. (This is unlike aarch32, where in some cases you might run the > kernel in secure-SVC, although even there starting the kernel in > NS-SVC or NS-Hyp is more common.) > >> But then if I want to boot and test the u-boot binary (or any trusted OS >> for the matter) on a Qemu emulated i.MX7 (to later boot an hypervisor or >> an OS), it would be rather strange that any PSCI related service >> requested by the hypervisor/OS would be handled by Qemu directly and >> not by the u-boot code (or any other EL3 code) running on the processor. > Exactly. This is why the board code is supposed to set things > up so that if we are starting the guest code in EL1 or EL2 > then we enable the PSCI-via-SMC support, and if we're starting > the guest code in EL3 then we do not. > >> How is it supposed to work? How can I tell Qemu (dynamically?) if I want >> it to provide (or not) the PSCI services (and more generally SMC/HVC >> services). > If you want PSCI via SMC or HVC, then set the psci-conduit > property on the CPUs to QEMU_PSCI_CONDUIT_SMC or QEMU_PSCI_CONDUIT_HVC. > If you do not want QEMU to provide PSCI, then leave psci-conduit > at its default (which is QEMU_PSCI_CONDUIT_DISABLED). > > How can I tell it that I want to handle all SMC/EL3 services >> by myself even if the "psci-conduit" is already set to SMC in Qemu? > It's the imx7 code that's setting psci-conduit, so it should > not do that if it doesn't want that (and also should either > start or not start the secondary cpus powered off, depending > on what the hardware-to-firmware interface is supposed to be.) > This is a bit awkward, because the boards we initially wanted > PSCI for (notably virt) don't have an SoC object, so the code > creating CPUs is in the same source file as the code that knows > whether it's booting a kernel directly or not, and so it just > open-codes the decision logic. With the imx, the CPU creation > is in the source code for the SoC object which is abstracted > away from the board model code. So we'd need to sort out how > to plumb that information into the SoC object (or have the SoC > object's code that creates the CPUs call some function to find out). Thanks Peter, So basically the Qemu i.MX7 processor code needs to set psci-conduit to SMC because we want to be able to boot the Linux kernel directly (without u-boot) with Qemu emulating the PSCI services when an SMC instruction is triggered. As I see it we also need a way to disable this Qemu PSCI emulation in case we want to boot an EL3 software (u-boot, optee or other). We could certainly pass a command line option to explicitly tell Qemu to disable the PCSI emulation. But this would be a bit cumbersome as the board level code would then need to call to the SOC code to disable the psci setting on all cores. So I am wondering if we could be more "clever" solution and have things sorted out automatically. I would like to suggest 2 solutions; 1. The first solution could be based on the mode of the processor when it starts executing the provided software. If I remember correctly what you explained, if an ELF file is provided then the processor boots in the highest level (EL3 for i.MX6/i.MX7) while if a non ELF file is provided (uImage ...) then the processor boots in EL2 (or lower depending on the processor) to emulate the uboot behavior. 1. if the processor starts the software in EL3 then we need to disable the "psci-conduit" through SMC (if set) because the EL3 software should install/provide the necessary firmware. 2. If the processor starts the software in EL2 then we need to disable the "psci-conduit" through HVC (if set) because the EL2 software should install/provide the necessary firmware. 2. The other solution is to check if the booted software has installed an EL3 or EL2 exception vector table. 1. if the software has installed an EL3 (monitor) exception vector table then we need to disable the "psci-conduit" through SMC (if set). 2. if the software has installed an EL2 (hypervisor) exception vector table then we need to disable the "psci-conduit" through HVC (if set). Is any of these proposals reasonable in your point of view? This would apply for any (ARM) target and this seems generic enough. Thanks JC > > -- PMM > [-- Attachment #2: Type: text/html, Size: 8746 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Qemu and ARM secure state. 2021-11-09 19:06 ` Jean-Christophe DUBOIS @ 2021-11-09 19:20 ` Peter Maydell 0 siblings, 0 replies; 12+ messages in thread From: Peter Maydell @ 2021-11-09 19:20 UTC (permalink / raw) To: Jean-Christophe DUBOIS; +Cc: qemu-arm, QEMU Developers On Tue, 9 Nov 2021 at 19:06, Jean-Christophe DUBOIS <jcd@tribudubois.net> wrote: > So basically the Qemu i.MX7 processor code needs to set psci-conduit to SMC > because we want to be able to boot the Linux kernel directly (without u-boot) > with Qemu emulating the PSCI services when an SMC instruction is triggered. > > As I see it we also need a way to disable this Qemu PSCI emulation in case we > want to boot an EL3 software (u-boot, optee or other). Yes; we definitely want to support both of these use cases. > We could certainly pass a command line option to explicitly tell Qemu to disable > the PCSI emulation. But this would be a bit cumbersome as the board level > code would then need to call to the SOC code to disable the psci setting on all cores. We definitely don't want a new command line option. We just want to use the information we already have (whether we're going to start the guest code in secure/at EL3 or not) to make this decision. > So I am wondering if we could be more "clever" solution and have things sorted out automatically. I would like to suggest 2 solutions; > > The first solution could be based on the mode of the processor when it starts executing the provided software. If I remember correctly what you explained, if an ELF file is provided then the processor boots in the highest level (EL3 for i.MX6/i.MX7) while if a non ELF file is provided (uImage ...) then the processor boots in EL2 (or lower depending on the processor) to emulate the uboot behavior. > > if the processor starts the software in EL3 then we need to disable the "psci-conduit" through SMC (if set) because the EL3 software should install/provide the necessary firmware. > If the processor starts the software in EL2 then we need to disable the "psci-conduit" through HVC (if set) because the EL2 software should install/provide the necessary firmware. This is the right approach. The idea I suggested in the previous email was that rather than having the SoC object code setting psci-conduit always, and then something else coming along later and changing it, we should instead not set it in the first place. It's a simple QOM property which means it's not writable after the CPU object has been realized, I think, so we'd have to jump through some hoops to get it to be writable later. On the other hand looking in hw/arm/boot.c we don't decide whether this is a Linux boot (which means it will be to non-secure) or a non-Linux boot (which means it will be to EL3) until quite late, so that's awkward too (plus there's potentially duplication between board/SoC code of the logic to decide whether to set psci-conduit or not). Let me have a think about the cleanest way to arrange this. -- PMM ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2021-11-09 19:21 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <07e63acb-b756-2586-2ba2-b54b837f7fc8@tribudubois.net> 2021-11-04 11:11 ` Qemu and ARM secure state Peter Maydell 2021-11-04 21:11 ` Jean-Christophe DUBOIS 2021-11-06 10:04 ` Jean-Christophe DUBOIS 2021-11-06 13:04 ` Jean-Christophe DUBOIS 2021-11-06 18:11 ` Jean-Christophe DUBOIS 2021-11-08 14:14 ` Alex Bennée 2021-11-08 22:06 ` Jean-Christophe DUBOIS 2021-11-08 14:50 ` Peter Maydell 2021-11-08 22:09 ` Jean-Christophe DUBOIS 2021-11-09 10:55 ` Peter Maydell 2021-11-09 19:06 ` Jean-Christophe DUBOIS 2021-11-09 19:20 ` Peter Maydell
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.