[Qemu-devel] AVX support for TCG

[Qemu-devel] AVX support for TCG

[Nick Renieris]

Hello,

What's the current status on AVX support for TCG? Is there anyone working on it?
Is there interest for it?

Effort was put into making 'TCGv_vec', as elaborated in this talk:

"[2017] Vectoring in on QEMU's TCG Engine by Alex Bennée"
(https://www.youtube.com/watch?v=IYHTwnde0g8)

,yet there doesn't seem to be a front-end (except for VEX prefix
parsing) nor a backend for AVX, a pretty common x86 extension.

I'd appreciate any info on this,

Regards,
Nick Renieris.

Re: [Qemu-devel] AVX support for TCG

[Richard Henderson]

On 12/26/18 10:43 AM, Nick Renieris wrote:
> Hello,
> 
> What's the current status on AVX support for TCG? Is there anyone working on it?
> Is there interest for it?

A couple of people, including myself, have started on it, but no results posted
so far.  It's a large amount of work to clean up the exiting sse support before
avx could be added.


r~

Re: [Qemu-devel] AVX support for TCG

[Nick Renieris]

Hi Richard,

I did know about https://github.com/andikleen/qemu-avx but didn't
mention it as it seems abandoned and quite old (also it doesn't use
`TCGv_vec`).
Do you think this could work as a GSoC project? I'm potentially
interested in working on it this summer.

Thanks,
Nick R.

Στις Τετ, 26 Δεκ 2018 στις 3:07 π.μ., ο/η Richard Henderson
<richard.henderson@linaro.org> έγραψε:
>
> On 12/26/18 10:43 AM, Nick Renieris wrote:
> > Hello,
> >
> > What's the current status on AVX support for TCG? Is there anyone working on it?
> > Is there interest for it?
>
> A couple of people, including myself, have started on it, but no results posted
> so far.  It's a large amount of work to clean up the exiting sse support before
> avx could be added.
>
>
> r~

Re: [Qemu-devel] AVX support for TCG

[Richard Henderson]

On 12/26/18 12:28 PM, Nick Renieris wrote:
> Hi Richard,
> 
> I did know about https://github.com/andikleen/qemu-avx but didn't
> mention it as it seems abandoned and quite old (also it doesn't use
> `TCGv_vec`).

Yep.  Mine pre-dates tcg-op-gvec.h as well.

> Do you think this could work as a GSoC project? I'm potentially
> interested in working on it this summer.

Could be.  My first guess is something like 4 months work for this.


r~

Re: [Qemu-devel] AVX support for TCG

[Nick Renieris]

>> Do you think this could work as a GSoC project? I'm potentially
>> interested in working on it this summer.

>Could be.  My first guess is something like 4 months work for this.

Four months full-time? If so I would say it's not viable for a GSoC
project (it's 3 months), I've done the 12-hours-a-day-crunch thing for
a week or so in GSoC 2017 and it was _not_ fun.
Also, I hope you meant four months for me, not for you - I'm
completely new to the QEMU codebase. I expect it will take me weeks
just to understand x86's 'translate.c' (who thought it'd be a good
idea to put all this stuff in _one_ file?).

Another question, are there existing discussions about this
refactoring effort or specifically AVX? I asked a similar question on
IRC a few days ago and got no answers.

Στις Τετ, 26 Δεκ 2018 στις 4:12 π.μ., ο/η Richard Henderson
<richard.henderson@linaro.org> έγραψε:
>
> On 12/26/18 12:28 PM, Nick Renieris wrote:
> > Hi Richard,
> >
> > I did know about https://github.com/andikleen/qemu-avx but didn't
> > mention it as it seems abandoned and quite old (also it doesn't use
> > `TCGv_vec`).
>
> Yep.  Mine pre-dates tcg-op-gvec.h as well.
>
> > Do you think this could work as a GSoC project? I'm potentially
> > interested in working on it this summer.
>
> Could be.  My first guess is something like 4 months work for this.
>
>
> r~

Re: [Qemu-devel] AVX support for TCG

[Peter Maydell]

On Fri, 28 Dec 2018 at 13:45, Nick Renieris <makren67@gmail.com> wrote:
> Also, I hope you meant four months for me, not for you - I'm
> completely new to the QEMU codebase. I expect it will take me weeks
> just to understand x86's 'translate.c' (who thought it'd be a good
> idea to put all this stuff in _one_ file?).

x86 suffers from being one of the first and oldest frontends,
and on top of that from not having had a great deal of active
attention. So it has a lot of remnants from older styles of
implementation, as well as the kind of "one big function in
one huge file" that organic growth of a codebase tends to give you.
It doesn't make that much difference whether you have one file
or several, though -- target/arm/translate-a64.c is pretty
well structured and easy (IMHO) to comprehend, but it's
5000 lines longer than target/i386/translate.c. The comprehensibility
improvements come from better breakdown into separate functions
and much clearer commenting of the extent of the decode at any
particular point (plus not having any legacy baggage to deal with).

thanks
-- PMM

Re: [Qemu-devel] AVX support for TCG

[Nick Renieris]

Right, thanks, that file looks better, though I still think splitting
to multiple files would absolutely be of value, if only for the
practicality of being able to have several parts of it open at the
same in a code editor (instead of having to jump back and forth or
find workarounds to open the same file multiple times). Of course
there are more important, code structure related reasons beyond that,
but let's not digress. I'm guessing current devs are so used to this
that they wouldn't accept splitting-up changes anyway.

Στις Παρ, 28 Δεκ 2018 στις 4:13 μ.μ., ο/η Peter Maydell
<peter.maydell@linaro.org> έγραψε:
>
> On Fri, 28 Dec 2018 at 13:45, Nick Renieris <makren67@gmail.com> wrote:
> > Also, I hope you meant four months for me, not for you - I'm
> > completely new to the QEMU codebase. I expect it will take me weeks
> > just to understand x86's 'translate.c' (who thought it'd be a good
> > idea to put all this stuff in _one_ file?).
>
> x86 suffers from being one of the first and oldest frontends,
> and on top of that from not having had a great deal of active
> attention. So it has a lot of remnants from older styles of
> implementation, as well as the kind of "one big function in
> one huge file" that organic growth of a codebase tends to give you.
> It doesn't make that much difference whether you have one file
> or several, though -- target/arm/translate-a64.c is pretty
> well structured and easy (IMHO) to comprehend, but it's
> 5000 lines longer than target/i386/translate.c. The comprehensibility
> improvements come from better breakdown into separate functions
> and much clearer commenting of the extent of the decode at any
> particular point (plus not having any legacy baggage to deal with).
>
> thanks
> -- PMM

Re: [Qemu-devel] AVX support for TCG

[Peter Maydell]

On Fri, 28 Dec 2018 at 14:28, Nick Renieris <makren67@gmail.com> wrote:
> Right, thanks, that file looks better, though I still think splitting
> to multiple files would absolutely be of value, if only for the
> practicality of being able to have several parts of it open at the
> same in a code editor (instead of having to jump back and forth or
> find workarounds to open the same file multiple times).

If your editor can't show multiple views onto one file with
the same simplicity and UI as it has for multiple different
files then I would suggest getting a better editor :-)
Unless you want to restrict all your files to 100 lines or
shorter then you need to be able to see multiple parts of them
at once -- one 10,000 line file is no worse than 4 2500 line
files for this.

> Of course
> there are more important, code structure related reasons beyond that,
> but let's not digress. I'm guessing current devs are so used to this
> that they wouldn't accept splitting-up changes anyway.

There are definitely improvements that could be made to
the x86 code, and where splitting of files makes conceptual
sense it's certainly worthwhile. The trick is finding the
right logical splitting points.

thanks
-- PMM

Re: [Qemu-devel] AVX support for TCG

[Nick Renieris]

Στις Παρ, 28 Δεκ 2018 στις 4:39 μ.μ., ο/η Peter Maydell
<peter.maydell@linaro.org> έγραψε:
> If your editor can't show multiple views onto one file with
> the same simplicity and UI as it has for multiple different
> files then I would suggest getting a better editor :-)

Apparently I just didn't know how to use my editor :) In my defense,
I've never had to do this before.

> Unless you want to restrict all your files to 100 lines or
> shorter then you need to be able to see multiple parts of them
> at once -- one 10,000 line file is no worse than 4 2500 line
> files for this.

As you mention below, logical separation is key here, I definitely
didn't mean that there should be arbitrary LoC limits for each file.

> There are definitely improvements that could be made to
> the x86 code, and where splitting of files makes conceptual
> sense it's certainly worthwhile. The trick is finding the
> right logical splitting points.

Agreed, though it's not something I can personally help on, unless I
spend quite a long time getting acquainted with _this_ code first.

I'd like to get some answers to my previous question about estimates
of the total amount of work required before considering diving into
it.

Re: [Qemu-devel] AVX support for TCG

[Alex Bennée]

On Fri, 28 Dec 2018, 13:46 Nick Renieris <makren67@gmail.com wrote:

>
> Another question, are there existing discussions about this
> refactoring effort or specifically AVX? I asked a similar question on
> IRC a few days ago and got no answers.
>

You might want to try again next week. Most QEMU hackers are away for the
holidays and also tend to be on IRC during working hours.

>

Re: [Qemu-devel] AVX support for TCG

[Richard Henderson]

On 12/29/18 12:43 AM, Nick Renieris wrote:
>>> Do you think this could work as a GSoC project? I'm potentially
>>> interested in working on it this summer.
> 
>> Could be.  My first guess is something like 4 months work for this.
> 
> Four months full-time? If so I would say it's not viable for a GSoC
> project (it's 3 months), I've done the 12-hours-a-day-crunch thing for
> a week or so in GSoC 2017 and it was _not_ fun.
> Also, I hope you meant four months for me, not for you - I'm
> completely new to the QEMU codebase. I expect it will take me weeks
> just to understand x86's 'translate.c' (who thought it'd be a good
> idea to put all this stuff in _one_ file?).

I did have a beginner in mind when guessing 4 months.  Don't take that as a
fully speced out answer, but it may well be that full avx2 support cannot be
done within the 3 months of gsoc.  I would certainly expect avx512 to take even
longer.

> Another question, are there existing discussions about this
> refactoring effort or specifically AVX? I asked a similar question on
> IRC a few days ago and got no answers.

Not that I recall.  I have some code at

https://github.com/rth7680/qemu/commits/i386-avx

that attempts to remove the sse_op_table(s).  However, it also splits up the
sse operations into units of uint64_t.  Which seemed sort of reasonable at the
time, considering that a lot of sse is 2*mmx.

But in the intervening 2.5 years since I worked on that branch, we have learned
that calls to helpers dominate.  It's better to have a single call that does 4x
the work than 4 separate calls.

The tcg-op-gvec.h infrastructure allows for the different modes that avx+mmx
allows:

(1) 64-bit operations,
(2) 128-bit operations, modifying only the low 128 bits,
(3) 128-bit operations, zeroing bits beyond the first 128,
(4) N*128-bit operations, zeroing bits beyond the first N*128.

so we should not need a great proliferation of helper functions, merely a
re-organization of what we have now.


r~

Re: [Qemu-devel] AVX support for TCG

[Nick Renieris]

Στις Σάβ, 29 Δεκ 2018 στις 10:24 μ.μ., ο/η Richard Henderson
<richard.henderson@linaro.org> έγραψε:
> I did have a beginner in mind when guessing 4 months.  Don't take that as a
> fully speced out answer, but it may well be that full avx2 support cannot be
> done within the 3 months of gsoc.  I would certainly expect avx512 to take even
> longer.

Right, that's ok.
Bit of context, the reason I'm interested in this feature (other than
GSoC potential and general interest) is Orbital, a Sony PlayStation 4
emulator that implements a QEMU fork. Normally we use virtualization
as anything else would be too slow, but TCG would help with debugging
and such.
The PS4's APU doesn't support AVX2 or AVX-512 so I'd be fine if I
didn't have enough time to implement them.

> The tcg-op-gvec.h infrastructure allows for the different modes that avx+mmx
> allows:
>
> (1) 64-bit operations,
> (2) 128-bit operations, modifying only the low 128 bits,
> (3) 128-bit operations, zeroing bits beyond the first 128,
> (4) N*128-bit operations, zeroing bits beyond the first N*128.

I assume you mean 256-bit ops on (2) and (3), and N*256 on (4)? Low
128 bits of a 128-bit number is just the number.

> so we should not need a great proliferation of helper functions, merely a
> re-organization of what we have now.

So, I would need to implement every SSE instruction that isn't
SSE_SPECIAL at the moment, using tcg-op-gvec.h? Or more instructions
than that?

Assuming I do this for SSE and AVX, I would not need to touch anything
else like the TCG back-end, as every gvec/vec op is already
implemented for i386, correct?

PS: I'm 'vel0city' on IRC, if that'd be more convenient.

Re: [Qemu-devel] AVX support for TCG

[Richard Henderson]

On 12/31/18 7:51 AM, Nick Renieris wrote:
> The PS4's APU doesn't support AVX2 or AVX-512 so I'd be fine if I
> didn't have enough time to implement them.

Fair enough.  A goal like this is a good thing.

>> The tcg-op-gvec.h infrastructure allows for the different modes that avx+mmx
>> allows:
>>
>> (1) 64-bit operations,
>> (2) 128-bit operations, modifying only the low 128 bits,
>> (3) 128-bit operations, zeroing bits beyond the first 128,
>> (4) N*128-bit operations, zeroing bits beyond the first N*128.
> 
> I assume you mean 256-bit ops on (2) and (3), and N*256 on (4)? Low
> 128 bits of a 128-bit number is just the number.

No, I mean

 0FFCC8   	paddb	%mm0, %mm1		(1)
 660FFCC8 	paddb	%xmm0, %xmm1		(2)
 C5F1FCC8 	vpaddb	%xmm0, %xmm1, %xmm1	(3)
 C5F5FCC8 	vpaddb	%ymm0, %ymm1, %ymm1	(4)
 62F17548FCC8 	vpaddb	%zmm0, %zmm1, %zmm1	(4)

On a system that supports AVX, (2) and (3), while computing 128-bit inputs and
producing a 128-bit output, have different effects on the rest of the 256-bit
register.


> So, I would need to implement every SSE instruction that isn't
> SSE_SPECIAL at the moment, using tcg-op-gvec.h? Or more instructions
> than that?

You'd want to do all of the SSE instructions, SSE_SPECIAL and otherwise.

I believe that we want to eliminate sse_op_table* and implement all insns
within a switch statement, like SSE_SPECIAL.  Note that this does not mean one
gigantic 5000 line function; appropriate use of helper functions should make
the code for each switch entry fairly small.

You'd want to re-organize the code generated by ops_sse.h using the (ptr, ptr,
..., desc) signature of gen_helper_gvec_{2,2i,3,...} and expand them using
tcg_gen_gvec_{2,2i,3,...}_ool.

Examples of these are in accel/tcg/tcg-runtime-gvec.c and
target/arm/vec_helper.c.  Use simd_oprsz to find out how much data should be
operated upon.  The clear_high function should be moved somewhere that it can
be shared.

Once all of this has been done for SSE, then AVX is implemented simply
adjusting the oprsz and maxsz arguments to tcg_gen_gvec_*.

> Assuming I do this for SSE and AVX, I would not need to touch anything
> else like the TCG back-end, as every gvec/vec op is already
> implemented for i386, correct?

Correct.


r~

Re: [Qemu-devel] AVX support for TCG

[Nick Renieris]

Right, that makes sense, thanks for the explanations. As someone with
very little x86 experience (zero experience from this perspective)
it's kind of daunting that I'd have to refactor all this stuff. All
these helpers via macros to get around C's 'minimalism' also seem like
something I'd have to get accustomed to. I will think about it.

Just curious, why is gvec-desc a bitfield instead of a normal struct?
Surely it'd be more readable that way. Also this is C, so it's not
even a typed bitfield, just a uint32. I'm guessing there's a reason
behind this?

Re: [Qemu-devel] AVX support for TCG

[Richard Henderson]

On 1/5/19 7:43 AM, Nick Renieris wrote:
> Just curious, why is gvec-desc a bitfield instead of a normal struct?
> Surely it'd be more readable that way. Also this is C, so it's not
> even a typed bitfield, just a uint32. I'm guessing there's a reason
> behind this?

As an integer it is always passed by value.  As a structure some host abis pass
it by reference, and the TCG compiler doesn't know about that.


r~

Re: [Qemu-devel] AVX support for TCG

[Nick Renieris]

Στις Παρ, 4 Ιαν 2019 στις 11:51 μ.μ., ο/η Richard Henderson
<richard.henderson@linaro.org> έγραψε:
> As an integer it is always passed by value.  As a structure some host abis pass
> it by reference, and the TCG compiler doesn't know about that.

Ah so they modify it? If so it could surely be worked around with
explicit stack copies, right?

Re: [Qemu-devel] AVX support for TCG

[Richard Henderson]

On 1/5/19 8:09 AM, Nick Renieris wrote:
> Στις Παρ, 4 Ιαν 2019 στις 11:51 μ.μ., ο/η Richard Henderson
> <richard.henderson@linaro.org> έγραψε:
>> As an integer it is always passed by value.  As a structure some host abis pass
>> it by reference, and the TCG compiler doesn't know about that.
> 
> Ah so they modify it? If so it could surely be worked around with
> explicit stack copies, right?

No, it's just calling conventions.  And it could be worked around, but I think
what we have is convenient enough.

Especially since the sizes are encoded as (n+1)*8, which also shows the
compiler that the size is positive, so the for loop must iterate at least once.


r~

Re: [Qemu-devel] AVX support for TCG

[Nick Renieris]

Στις Σάβ, 5 Ιαν 2019 στις 12:14 π.μ., ο/η Richard Henderson
<richard.henderson@linaro.org> έγραψε:
> No, it's just calling conventions.  And it could be worked around, but I think
> what we have is convenient enough.
>
> Especially since the sizes are encoded as (n+1)*8, which also shows the
> compiler that the size is positive, so the for loop must iterate at least once.

I know host ABI's can differ like that, but I don't understand why
that should matter. Everything (TCG compiler included) is compiled
with the same way, right? For the host arch.
Or is that a host ABI vs guest ABI thing? Though I don't understand
why that would matter either. All this is stuff that runs on the host,
right? Oh or does it have to do with JIT'ted tcg helper functions
where ABI would matter?

No real need to explain, I'm just curious.

Re: [Qemu-devel] AVX support for TCG

[Richard Henderson]

On 1/5/19 8:33 AM, Nick Renieris wrote:
> I know host ABI's can differ like that, but I don't understand why
> that should matter. Everything (TCG compiler included) is compiled
> with the same way, right? For the host arch.

No, not all of the pieces are compiled the same way.  TCG generates machine
instructions to perform a call from generated code into compiler generated
code.  So TCG needs to know about the host compiler ABI.


r~

Re: [Qemu-devel] AVX support for TCG

[Nick Renieris]

Ohh got it, thanks.

Στις Σάβ, 5 Ιαν 2019 στις 12:38 π.μ., ο/η Richard Henderson
<richard.henderson@linaro.org> έγραψε:
>
> On 1/5/19 8:33 AM, Nick Renieris wrote:
> > I know host ABI's can differ like that, but I don't understand why
> > that should matter. Everything (TCG compiler included) is compiled
> > with the same way, right? For the host arch.
>
> No, not all of the pieces are compiled the same way.  TCG generates machine
> instructions to perform a call from generated code into compiler generated
> code.  So TCG needs to know about the host compiler ABI.
>
>
> r~

Re: [Qemu-devel] AVX support for TCG

[Stefan Hajnoczi]

On Wed, Dec 26, 2018 at 1:29 AM Nick Renieris <makren67@gmail.com> wrote:
> Do you think this could work as a GSoC project? I'm potentially
> interested in working on it this summer.

Richard and Nick,
You are welcome to create a GSoC project idea and post it here:
https://wiki.qemu.org/Google_Summer_of_Code_2019

Good project ideas have the following characteristics:
 * Well-defined - the scope is clear
 * Self-contained - there are few dependencies
 * Uncontroversial - they are acceptable to the community
 * Incremental - they produce deliverables along the way

Can you break down the project into tasks?
1. Test cases for SSE and AVX
2. SSE refactoring that needs to be done first
3. AVX instructions that will be covered
4. Stretch goals (AVX2, AVX512, etc)

The project idea template looks like this:

=== TITLE ===

 '''Summary:''' Short description of the project

 Detailed description of the project.

 '''Links:'''
 * Wiki links to relevant material
 * External links to mailing lists or web sites

 '''Details:'''
 * Skill level: beginner or intermediate or advanced
 * Language: C
 * Mentor: Email address and IRC nick
 * Suggested by: Person who suggested the idea


Note that project ideas are open to all applicants in order to ensure fairness.

Stefan

Re: [Qemu-devel] AVX support for TCG

[Nick Renieris]

Thanks Stefan,

Just a heads up, I don't know if I want to go for this myself yet.
I'm still not sure I'll even be able to deliver the refactoring in
time (Step 2 in your list), and I hadn't even considered tests for SSE
as something I'd have to do since they're already implemented.
In addition, there's been talks with a different project that I might apply for.
I'll keep in touch if that changes, of course.

Cheers,
Nick Renieris.

Στις Πέμ, 17 Ιαν 2019 στις 11:29 π.μ., ο/η Stefan Hajnoczi
<stefanha@gmail.com> έγραψε:
>
> On Wed, Dec 26, 2018 at 1:29 AM Nick Renieris <makren67@gmail.com> wrote:
> > Do you think this could work as a GSoC project? I'm potentially
> > interested in working on it this summer.
>
> Richard and Nick,
> You are welcome to create a GSoC project idea and post it here:
> https://wiki.qemu.org/Google_Summer_of_Code_2019
>
> Good project ideas have the following characteristics:
>  * Well-defined - the scope is clear
>  * Self-contained - there are few dependencies
>  * Uncontroversial - they are acceptable to the community
>  * Incremental - they produce deliverables along the way
>
> Can you break down the project into tasks?
> 1. Test cases for SSE and AVX
> 2. SSE refactoring that needs to be done first
> 3. AVX instructions that will be covered
> 4. Stretch goals (AVX2, AVX512, etc)
>
> The project idea template looks like this:
>
> === TITLE ===
>
>  '''Summary:''' Short description of the project
>
>  Detailed description of the project.
>
>  '''Links:'''
>  * Wiki links to relevant material
>  * External links to mailing lists or web sites
>
>  '''Details:'''
>  * Skill level: beginner or intermediate or advanced
>  * Language: C
>  * Mentor: Email address and IRC nick
>  * Suggested by: Person who suggested the idea
>
>
> Note that project ideas are open to all applicants in order to ensure fairness.
>
> Stefan

Re: [Qemu-devel] AVX support for TCG

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 675 bytes --]

On Thu, Jan 17, 2019 at 03:57:08PM +0200, Nick Renieris wrote:
> Just a heads up, I don't know if I want to go for this myself yet.
> I'm still not sure I'll even be able to deliver the refactoring in
> time (Step 2 in your list), and I hadn't even considered tests for SSE
> as something I'd have to do since they're already implemented.
> In addition, there's been talks with a different project that I might apply for.
> I'll keep in touch if that changes, of course.

No worries.  It's still an interesting project idea and if Richard wants
to mentor it then maybe we can find an intern to work on it this summer.
You're always welcome to apply to QEMU projects!

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]