Semihosting, arm, riscv, ppc and common code

Semihosting, arm, riscv, ppc and common code

[Benjamin Herrenschmidt]

Hi Folks !

So I started "porting" over (read: copying) the arm semihosting code to
ppc to mimmic what Keith did for risv (mostly for picolibc support).

I noticed that the bulk of arm-semi.c (or riscv-semi.c) is trivially
made completely generic by doing a couple of changes:

 - Make most functions take a CPUState instead of the architecture
specific "env"

 - Provide arch helpers to retreive the op, set the result, do the
flen bug hack and possibly a couple of others (I'm not done yet).

There are other archs who do semihosting completely differently but at
least those 3 can share code.

Any objection/comment on the approach ? What I'll probably do is get
things going first with my ppc version (which I made more/less generic
but still located in target/ppc) at which point I can post an RFC so
you get an idea, and we find a good location for it.

From thre we might consider fixing some of the worst crap in there
doing backwards compatible extensions if we care enough :-)

Cheers,
Ben.

Re: Semihosting, arm, riscv, ppc and common code

[Liviu Ionescu]

> On 14 Jan 2020, at 08:25, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:
> 
> I noticed that the bulk of arm-semi.c (or riscv-semi.c) is trivially
> made completely generic by doing a couple of changes:

Last year I did a similar exercise in OpenOCD, where I took the Arm semihosting code from the Arm specific location, extracted the common part, updated it for latest Arm 64-bit specs, and then used the common code for both Arm and RISC-V.

If you think useful, you might take a look there too.


Regards,

Liviu

Re: Semihosting, arm, riscv, ppc and common code

[Benjamin Herrenschmidt]

On Tue, 2020-01-14 at 09:32 +0200, Liviu Ionescu wrote:
> > On 14 Jan 2020, at 08:25, Benjamin Herrenschmidt <
> > benh@kernel.crashing.org> wrote:
> > 
> > I noticed that the bulk of arm-semi.c (or riscv-semi.c) is
> > trivially
> > made completely generic by doing a couple of changes:
> 
> Last year I did a similar exercise in OpenOCD, where I took the Arm
> semihosting code from the Arm specific location, extracted the common
> part, updated it for latest Arm 64-bit specs, and then used the
> common code for both Arm and RISC-V.
> 
> If you think useful, you might take a look there too.

Well, one of the LCA talks wasn't that interesting so I started doing
it and am almost done :-)

I'll look at doing something for arm, riscv and ppc and send patches
once I get it working.

Cheers,
Ben.

> 
> Regards,
> 
> Liviu

Re: Semihosting, arm, riscv, ppc and common code

[Alex Bennée]

Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:

> On Tue, 2020-01-14 at 09:32 +0200, Liviu Ionescu wrote:
>> > On 14 Jan 2020, at 08:25, Benjamin Herrenschmidt <
>> > benh@kernel.crashing.org> wrote:
>> > 
>> > I noticed that the bulk of arm-semi.c (or riscv-semi.c) is
>> > trivially
>> > made completely generic by doing a couple of changes:
>> 
>> Last year I did a similar exercise in OpenOCD, where I took the Arm
>> semihosting code from the Arm specific location, extracted the common
>> part, updated it for latest Arm 64-bit specs, and then used the
>> common code for both Arm and RISC-V.
>> 
>> If you think useful, you might take a look there too.
>
> Well, one of the LCA talks wasn't that interesting so I started doing
> it and am almost done :-)
>
> I'll look at doing something for arm, riscv and ppc and send patches
> once I get it working.

Cool. Are you considering linux-user as well or are these only things
that make sense for system emulation?

>
> Cheers,
> Ben.
>
>> 
>> Regards,
>> 
>> Liviu


-- 
Alex Bennée

Re: Semihosting, arm, riscv, ppc and common code

[Benjamin Herrenschmidt]

On Tue, 2020-01-14 at 09:51 +0000, Alex Bennée wrote:
> > Well, one of the LCA talks wasn't that interesting so I started
> > doing
> > it and am almost done :-)
> > 
> > I'll look at doing something for arm, riscv and ppc and send
> > patches
> > once I get it working.
> 
> Cool. Are you considering linux-user as well or are these only things
> that make sense for system emulation?

There seem to be some linux-user stuff in there, I'm mostly considering
whatever ARM does today but we can certainly extend later.

The idea is to make sure ARM, RiscV and POWER use the same protocol and
code base to make picolibc (and others) life easier. Bug compatible :-)

Cheers,
Ben.

Re: Semihosting, arm, riscv, ppc and common code

[Alex Bennée]

Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:

> On Tue, 2020-01-14 at 09:51 +0000, Alex Bennée wrote:
>> > Well, one of the LCA talks wasn't that interesting so I started
>> > doing
>> > it and am almost done :-)
>> > 
>> > I'll look at doing something for arm, riscv and ppc and send
>> > patches
>> > once I get it working.
>> 
>> Cool. Are you considering linux-user as well or are these only things
>> that make sense for system emulation?
>
> There seem to be some linux-user stuff in there, I'm mostly considering
> whatever ARM does today but we can certainly extend later.

Depends on if it is to be used. AFAIK the main users of arm linux user
are compiler test cases for M-profile CPUs. 

> The idea is to make sure ARM, RiscV and POWER use the same protocol and
> code base to make picolibc (and others) life easier. Bug compatible
> :-)

Hmm, I'm not so sure. QEMU tries to emulate real HW although I
appreciate that is a somewhat loose definition once you get to things
like -M virt and other such SW like abstractions. Is semihosting even
going to be a thing on real RiscV/Power silicon?

>
> Cheers,
> Ben.


-- 
Alex Bennée

Re: Semihosting, arm, riscv, ppc and common code

[Liviu Ionescu]

> On 15 Jan 2020, at 14:01, Alex Bennée <alex.bennee@linaro.org> wrote:
> 
> ... AFAIK the main users of arm linux user
> are compiler test cases for M-profile CPUs. 

I confirm, generally unit tests.

But not necessarily, I used QEMU as the main development platform for the Cortex-M port of µOS++, a C/C++ framework/RTOS, and it was very convenient.

> ... Is semihosting even
> going to be a thing on real RiscV/Power silicon?

Even for Arm it is a thing of debuggers and development environments, not of silicon itself, since it requires no special hardware support, a BRK instruction is enough.

From a software point of view it would be beneficial to use the same semihosting protocol on other non-Arm architectures too, even if it is far from perfect.

I pushed hard for adopting the Arm semihosting for RISC-V, and even contributed the semihosting code for OpenOCD RISC-V, but officially the only mention of semihosting in the RISC-V specs is in the notes sections, where it is mentioned how to use the BRK instruction to invoke semihosting calls (RISC-V has a single BREAK instructions, and to make it unique it must be surrounded by some hint NOPs).


Regards,

Liviu

Re: Semihosting, arm, riscv, ppc and common code

[Richard Henderson]

On 1/15/20 2:01 AM, Alex Bennée wrote:
> Hmm, I'm not so sure. QEMU tries to emulate real HW although I
> appreciate that is a somewhat loose definition once you get to things
> like -M virt and other such SW like abstractions. Is semihosting even
> going to be a thing on real RiscV/Power silicon?

For risc-v, the odd nop-full semi-hosting call sequence was chosen to work with
jtag debuggers on real silicon.

As for the rest of the abi, the stuff after the debugger/emulator has gotten
control, they did have the opportunity to do better, and did not.


r~

Re: Semihosting, arm, riscv, ppc and common code

[Liviu Ionescu]

> On 15 Jan 2020, at 23:28, Richard Henderson <richard.henderson@linaro.org> wrote:
> 
> For risc-v, the odd nop-full semi-hosting call sequence

That unfortunate call sequence was the least worst compromise that the RISC-V design team could agree on. :-(

The actual problem was that the RISC-V instruction set has a single BREAK op code, without any way to parametrise it, and they refused to spend another op code for an extra BREAK.

> was chosen to work with
> jtag debuggers on real silicon.

Yes, I know at least two, SEGGER J-Link of OpenOCD. 

But again, there is nothing in the silicon related to the odd call sequence or the ABI, everything is implemented in the debuggers. The silicon has only to break to the debugger, then it's up to the debugger to decide if this is a semihosting call or a regular break.

> ... they did have the opportunity to do better, and did not.

I don't know why you present Arm semihosting as a disaster. It is not perfect, but it is functional, and common unit tests use only a small subset of the calls.

And there is no 'window of opportunity', if the RISC-V guys will ever want to reinvent the wheel and come with an official 'RISC-V semihosting' specs, they can do it at any time, and this will have no impact on existing devices, everything will continue to work as before, only the debuggers/emulators will need to be upgraded.

But the only immediate effect such a move will have is that software efforts in test frameworks will be increased, to support another protocol, while the advantages will be minimal.


Regards,

Liviu

Re: Semihosting, arm, riscv, ppc and common code

[Benjamin Herrenschmidt]

On Thu, 2020-01-16 at 00:02 +0200, Liviu Ionescu wrote:
> > ... they did have the opportunity to do better, and did not.
> 
> I don't know why you present Arm semihosting as a disaster. It is not
> perfect, but it is functional, and common unit tests use only a small
> subset of the calls.
> 
> And there is no 'window of opportunity', if the RISC-V guys will ever
> want to reinvent the wheel and come with an official 'RISC-V
> semihosting' specs, they can do it at any time, and this will have no
> impact on existing devices, everything will continue to work as
> before, only the debuggers/emulators will need to be upgraded.
> 
> But the only immediate effect such a move will have is that software
> efforts in test frameworks will be increased, to support another
> protocol, while the advantages will be minimal.

I agree, which is also why I want to use the same interface for
powerpc, it will simply make life easier for everybody. The calls
aren't perfect but they do work sufficiently well to be useful and I
haven't yet been convinced that it can't be extended if necessary.

Cheers,
Ben.

Re: Semihosting, arm, riscv, ppc and common code

[Liviu Ionescu]

> On 16 Jan 2020, at 04:04, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:
> 
> ... I agree, which is also why I want to use the same interface for
> powerpc, it will simply make life easier for everybody.


I don't know the current implementation in QEMU, but I suggest you also take a look at the current Arm specs:

	https://static.docs.arm.com/100863/0200/semihosting.pdf

The implementation in OpenOCD is:

	https://github.com/xpack-dev-tools/openocd/blob/xpack/src/target/semihosting_common.c

> The calls
> aren't perfect but they do work sufficiently well to be useful and I
> haven't yet been convinced that it can't be extended if necessary.

In theory it can, in practice it'll not be easy.

Imagine that you decide to use Arm semihosting for your architecture, and implement it in QEMU.

Then you try to convince other tool developers to do the same. If you ask them nicely, they might copy/paste the Arm code, and you're done, major commercial tools will have Arm semihosting support for your architecture and it becomes a de facto standard.

If at a later time you decide to extend the protocol, you have to go again to all commercial tool vendors and ask them to implement the extension. You'll need good reasons to convince them.

In my case it was easier to convince SEGGER to add Arm semihosting support to their RISC-V JTAG, because we had a long relationship, but even so I doubt they'll be happy to implement extensions.

---

Anyway, from my experience, for practical cases the Arm semihosting is more than enough, once you have a functional implementation for the client side you simply forget about it, and you can use it for any architecture.

My implementation is the following:

	https://github.com/micro-os-plus/semihosting-xpack/blob/xpack/src/syscalls-semihosting.cpp


Regards,

Liviu

Re: Semihosting, arm, riscv, ppc and common code

[Benjamin Herrenschmidt]

On Wed, 2020-01-15 at 12:01 +0000, Alex Bennée wrote:
> 
> > There seem to be some linux-user stuff in there, I'm mostly considering
> > whatever ARM does today but we can certainly extend later.
> 
> Depends on if it is to be used. AFAIK the main users of arm linux user
> are compiler test cases for M-profile CPUs. 

For microwatt I was going to implement HW support as well via JTAG
but the user linux-user bits are less obviously useful to me.

That said, most of that code can trivially be made arch neutral by
replacing the use of the arch specific CPU type with CPUState as
the first argument to most functions. There are only a handful of arch
specific helpers needed from there to extract the op & arg, set the
result etc..

> > The idea is to make sure ARM, RiscV and POWER use the same protocol and
> > code base to make picolibc (and others) life easier. Bug compatible
> > :-)
> 
> Hmm, I'm not so sure. QEMU tries to emulate real HW although I
> appreciate that is a somewhat loose definition once you get to things
> like -M virt and other such SW like abstractions. Is semihosting even
> going to be a thing on real RiscV/Power silicon?

It will be on microwatt once I add support for it. We could probably
make it work on real power9 if the systems give access to the external
debug facilities of the processor as well. I'm no longer involved with
powerpc in a professional capacity but I can ask Anton or Paul to help
there.

Cheers,
Ben.

Re: Semihosting, arm, riscv, ppc and common code

[Peter Maydell]

On Tue, 14 Jan 2020 at 06:29, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
>
> Hi Folks !
>
> So I started "porting" over (read: copying) the arm semihosting code to
> ppc to mimmic what Keith did for risv (mostly for picolibc support).
>
> I noticed that the bulk of arm-semi.c (or riscv-semi.c) is trivially
> made completely generic by doing a couple of changes:

Note that semihosting is not a "here's a handy QEMU feature"
thing. It's an architecture-specific API and ABI, which should
be defined somewhere in a standard external to QEMU.
You need to start by having a definition for PPC of what
semihosting is. If you're starting from scratch there, there
are some important things you should do differently to Arm --
there is no benefit to repeating the mistakes of API definition
that we made! Most notably, you want to specify and require
that any unrecognized semihosting call function fails in a
clean and detectable way; you also should have a semihosting
function for "ask for a feature bit mask" so you don't need
the silly magic-filename approach Arm had to go for. You
also want to standardize what the errno values are, which Arm
forgot to do and which makes the errno handling in the spec
pretty useless.

TLDR: don't start by writing code, start by writing the *API/ABI spec*.
I tried to push the RISCV folks in this direction as well.

thanks
-- PMM

Re: Semihosting, arm, riscv, ppc and common code

[Benjamin Herrenschmidt]

On Tue, 2020-01-14 at 09:59 +0000, Peter Maydell wrote:
> Note that semihosting is not a "here's a handy QEMU feature"
> thing. It's an architecture-specific API and ABI, which should
> be defined somewhere in a standard external to QEMU.

There is no such standard for powerpc today that I know of.

> You need to start by having a definition for PPC of what
> semihosting is. If you're starting from scratch there, there
> are some important things you should do differently to Arm --
> there is no benefit to repeating the mistakes of API definition
> that we made! Most notably, you want to specify and require
> that any unrecognized semihosting call function fails in a
> clean and detectable way; you also should have a semihosting
> function for "ask for a feature bit mask" so you don't need
> the silly magic-filename approach Arm had to go for. You
> also want to standardize what the errno values are, which Arm
> forgot to do and which makes the errno handling in the spec
> pretty useless.

Keith and I are somewhat of a different mind here. From the perspective
of the user of that API (picolibc is one), it's easier to deal with a
single one and have everybody inherit the same bugs.

Now I understand the point of wanting to fix the mistakes made but I
would suggest we do so by proposing extensions to the existing one to
do so.

> TLDR: don't start by writing code, start by writing the *API/ABI
> spec*.
> I tried to push the RISCV folks in this direction as well.

AFAIK they are still just doing what ARM does for the above reason.

Cheers,
Ben.

Re: Semihosting, arm, riscv, ppc and common code

[Peter Maydell]

On Wed, 15 Jan 2020 at 01:17, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Tue, 2020-01-14 at 09:59 +0000, Peter Maydell wrote:
> > Note that semihosting is not a "here's a handy QEMU feature"
> > thing. It's an architecture-specific API and ABI, which should
> > be defined somewhere in a standard external to QEMU.
>
> There is no such standard for powerpc today that I know of.

So you need to write one down somewhere. You're proposing
an ABI which will be implemented by multiple implementors
and used by multiple consumers. That needs a spec, not
just code. I don't want to see more semihosting implementations
added to QEMU that don't have a specification written
down somewhere.

The riscv single paragraph in their arch spec that vaguely
says "works like arm semihosting" is not sufficient detail, incidentally.

> Keith and I are somewhat of a different mind here. From the perspective
> of the user of that API (picolibc is one), it's easier to deal with a
> single one and have everybody inherit the same bugs.
>
> Now I understand the point of wanting to fix the mistakes made but I
> would suggest we do so by proposing extensions to the existing one to
> do so.

The point about the mistakes is that you can't easily fix
them by adding extensions, because the core of the biggest
mistake is "we didn't provide a good way to allow extensions to
be added and probed for by the user". So we had to implement
an ugly and hard to implement mechanism instead. New
architectures could mandate providing the good way from the start
and avoid the painful-to-implement approach entirely.
(I think RISCV has already missed this window of opportunity,
which is a shame.)

thanks
-- PMM

Re: Semihosting, arm, riscv, ppc and common code

[Benjamin Herrenschmidt]

On Wed, 2020-01-15 at 13:32 +0000, Peter Maydell wrote:
> On Wed, 15 Jan 2020 at 01:17, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > On Tue, 2020-01-14 at 09:59 +0000, Peter Maydell wrote:
> > > Note that semihosting is not a "here's a handy QEMU feature"
> > > thing. It's an architecture-specific API and ABI, which should
> > > be defined somewhere in a standard external to QEMU.
> > 
> > There is no such standard for powerpc today that I know of.
> 
> So you need to write one down somewhere. You're proposing
> an ABI which will be implemented by multiple implementors
> and used by multiple consumers. That needs a spec, not
> just code. I don't want to see more semihosting implementations
> added to QEMU that don't have a specification written
> down somewhere.

That's ok, I can probably get openpower to put a link to the ARM one
somewhere :-)

> The point about the mistakes is that you can't easily fix
> them by adding extensions, because the core of the biggest
> mistake is "we didn't provide a good way to allow extensions to
> be added and probed for by the user". So we had to implement
> an ugly and hard to implement mechanism instead.
>  New
> architectures could mandate providing the good way from the start
> and avoid the painful-to-implement approach entirely.
> (I think RISCV has already missed this window of opportunity,
> which is a shame.)

It is done and so now we have two architectures using that standard, I
think the value in using the same overweight the value in fixing it but
yes, we should try to agree on a method of extending at least. Is it
really that hard ?

IE. We could add a new call for capabilities that takes a pointer to
a region which we pre-zero before calling in the client and if remains
zero after the call, then the new stuff isn't there for example. That
sort of stuff is easy, or am I missing something ?

Cheers,
Ben.

Re: Semihosting, arm, riscv, ppc and common code

[Peter Maydell]

On Thu, 16 Jan 2020 at 02:01, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
>
> On Wed, 2020-01-15 at 13:32 +0000, Peter Maydell wrote:
> > On Wed, 15 Jan 2020 at 01:17, Benjamin Herrenschmidt
> > <benh@kernel.crashing.org> wrote:
> > > On Tue, 2020-01-14 at 09:59 +0000, Peter Maydell wrote:
> > > > Note that semihosting is not a "here's a handy QEMU feature"
> > > > thing. It's an architecture-specific API and ABI, which should
> > > > be defined somewhere in a standard external to QEMU.
> > >
> > > There is no such standard for powerpc today that I know of.
> >
> > So you need to write one down somewhere. You're proposing
> > an ABI which will be implemented by multiple implementors
> > and used by multiple consumers. That needs a spec, not
> > just code. I don't want to see more semihosting implementations
> > added to QEMU that don't have a specification written
> > down somewhere.
>
> That's ok, I can probably get openpower to put a link to the ARM one
> somewhere :-)

Minimally, you need to say:
 * this architecture follows version x of the Arm semihosting
   spec [and give URL]
 * specify the equivalent things to the 'The semihosting interface'
   part of the Arm spec:
    - instruction that causes the trap
    - what the OPERATION NUMBER REGISTER, PARAMETER REGISTER
      and RETURN REGISTER are for your architecture
    - what the data block field size is (32 or 64 bit)
 * say whether your SYS_EXIT is doing the silly 32-bit Arm
   thing or the more sensible allow-app-to-specify-exit-status
   64-bit Arm thing (you should choose the latter!)
 * if you want to make either of the 'semihosting extensions'
   (which fix missing features) mandatory, say so here
 * does your architecture need to define some SYS_EXIT
   codes? The Arm ones have a lot of arm-specific stuff
   like "stopped because of prefetch abort"...

> > The point about the mistakes is that you can't easily fix
> > them by adding extensions, because the core of the biggest
> > mistake is "we didn't provide a good way to allow extensions to
> > be added and probed for by the user". So we had to implement
> > an ugly and hard to implement mechanism instead.
> >  New
> > architectures could mandate providing the good way from the start
> > and avoid the painful-to-implement approach entirely.
> > (I think RISCV has already missed this window of opportunity,
> > which is a shame.)
>
> It is done and so now we have two architectures using that standard, I
> think the value in using the same overweight the value in fixing it but
> yes, we should try to agree on a method of extending at least. Is it
> really that hard ?

It's hard if you don't do it from the start. This is why the
Arm spec's feature-query API is "open a magic filename" --
we couldn't guarantee that existing implementations wouldn't
blow up/crash/etc if the guest code attempted a new "query
feature" semihosting call that didn't exist. If you can
reliably say "all implementations of semihosting for this
arch will fail ENOSYS on unknown semihosting calls", or if
you put in a get-feature-flags call from the start, you don't
have that problem. Otherwise you're stuck with doing it the
way the Arm version does (which requires your implementation
to actually track file handles so it can distinguish "magic
feature-flag file" from "normal host file"; it can't just
pass through host fds/handles to the guest).

Similarly, actually specifying errnos is only easy if you do it
before any implementations.

> IE. We could add a new call for capabilities that takes a pointer to
> a region which we pre-zero before calling in the client and if remains
> zero after the call, then the new stuff isn't there for example. That
> sort of stuff is easy, or am I missing something ?

The problem is if you do "call" and then the implementation
does "kill the VM" or "drop the user into the debugger"...

thanks
-- PMM