Looking for architecture papers

Looking for architecture papers

[Raz]

Hello

I want to learn about powerpc architecture, mainly hypervisor and
partioning.  I download the books (1,2, and 3 ) but I feel it lacks
a lot of information. Are there other books ?

Kind regards

Re: Looking for architecture papers

[Segher Boessenkool]

Hi Raz,

On Sun, Sep 30, 2018 at 12:24:35AM +0300, Raz wrote:
> I want to learn about powerpc architecture, mainly hypervisor and
> partioning.  I download the books (1,2, and 3 ) but I feel it lacks
> a lot of information. Are there other books ?

I suspect you are looking at an ancient version of the architecture.
The current version is 3.0B, available here:
https://openpowerfoundation.org/?resource_lib=power-isa-version-3-0

Good luck and have fun,


Segher

Re: Looking for architecture papers

[Raz]

First, thank you.
And another question, is there a development environment that
resembles Fixed Virtual Platform for ARM ( FVP ARM) ?





On Sun, Sep 30, 2018 at 1:13 AM Segher Boessenkool
<segher@kernel.crashing.org> wrote:
>
> Hi Raz,
>
> On Sun, Sep 30, 2018 at 12:24:35AM +0300, Raz wrote:
> > I want to learn about powerpc architecture, mainly hypervisor and
> > partioning.  I download the books (1,2, and 3 ) but I feel it lacks
> > a lot of information. Are there other books ?
>
> I suspect you are looking at an ancient version of the architecture.
> The current version is 3.0B, available here:
> https://openpowerfoundation.org/?resource_lib=power-isa-version-3-0
>
> Good luck and have fun,
>
>
> Segher

Re: Looking for architecture papers

[Segher Boessenkool]

On Sun, Sep 30, 2018 at 10:05:01AM +0300, Raz wrote:
> First, thank you.
> And another question, is there a development environment that
> resembles Fixed Virtual Platform for ARM ( FVP ARM) ?

I don't know FVP, but that seems similar to systemsim?
https://www14.software.ibm.com/webapp/set2/sas/f/pwrfs/home.html
(p8 version)
https://www14.software.ibm.com/webapp/set2/sas/f/pwrfs/pwr9/home.html
(p9 version)


Segher

Re: Looking for architecture papers

[Michael Ellerman]

Raz <raziebe@gmail.com> writes:

> Hello
>
> I want to learn about powerpc architecture, mainly hypervisor and
> partioning.  I download the books (1,2, and 3 ) but I feel it lacks
> a lot of information. Are there other books ?

The ISA describes how the CPU works to allow you to implement a
hypervisor, but it doesn't describe any of the HV/Kernel APIs.

That's mostly covered in "LoPAPR":

  https://members.openpowerfoundation.org/document/dl/469


Although that's not been updated for Power9.

cheers

Re: Looking for architecture papers

[Raz]

Frankly, the more I read the more perplexed I get. For example,
according to BOOK III-S, chapter 3,
the MSR bits are differ from the ones described in
arch/powerpc/include/asm/reg.h.
Bit zero, is LE, but in the book it is 64-bit mode.

Would someone be kind to explain what I do not understand?

Thank you











On Wed, Oct 3, 2018 at 7:07 AM Michael Ellerman <mpe@ellerman.id.au> wrote:
>
> Raz <raziebe@gmail.com> writes:
>
> > Hello
> >
> > I want to learn about powerpc architecture, mainly hypervisor and
> > partioning.  I download the books (1,2, and 3 ) but I feel it lacks
> > a lot of information. Are there other books ?
>
> The ISA describes how the CPU works to allow you to implement a
> hypervisor, but it doesn't describe any of the HV/Kernel APIs.
>
> That's mostly covered in "LoPAPR":
>
>   https://members.openpowerfoundation.org/document/dl/469
>
>
> Although that's not been updated for Power9.
>
> cheers

Re: Looking for architecture papers

[Segher Boessenkool]

On Thu, Oct 04, 2018 at 10:41:13AM +0300, Raz wrote:
> Frankly, the more I read the more perplexed I get. For example,
> according to BOOK III-S, chapter 3,
> the MSR bits are differ from the ones described in
> arch/powerpc/include/asm/reg.h.
> Bit zero, is LE, but in the book it is 64-bit mode.

LE is bit 63, the least significant bit, value 0x0000000000000001.
SF is bit 0, the most significant bit, value 0x8000000000000000.

Some people wrongly start counting at the end, calling the least
significant bit "bit 0".  That isn't its name.


Segher

Re: Looking for architecture papers

[Gabriel Paubert]

On Thu, Oct 04, 2018 at 10:41:13AM +0300, Raz wrote:
> Frankly, the more I read the more perplexed I get. For example,
> according to BOOK III-S, chapter 3,
> the MSR bits are differ from the ones described in
> arch/powerpc/include/asm/reg.h.
> Bit zero, is LE, but in the book it is 64-bit mode.

Just a problem of bit order definitions: IBM hardware definitions use
big-endian bit ordering, where bit 0 is the most significant bit.

> 
> Would someone be kind to explain what I do not understand?
> 
> Thank you

	Gabriel

Re: Looking for architecture papers

[Raz]

Hey
How does HVSC works ?
I looked in the code and LoPAR documentation. It looks like there is
vector called
system_call_pSeries ( at 0xc00 ) that is supposed to be called when we
invoke HVSC from kernel
mode.
Now, I wrote a NULL call HSVC and patched the exceptions-64s.S to
return RFID immediately.
This does not work.
Would you be so kind to explain how HVSC works ?
thank you








On Thu, Oct 4, 2018 at 1:16 PM Gabriel Paubert <paubert@iram.es> wrote:
>
> On Thu, Oct 04, 2018 at 10:41:13AM +0300, Raz wrote:
> > Frankly, the more I read the more perplexed I get. For example,
> > according to BOOK III-S, chapter 3,
> > the MSR bits are differ from the ones described in
> > arch/powerpc/include/asm/reg.h.
> > Bit zero, is LE, but in the book it is 64-bit mode.
>
> Just a problem of bit order definitions: IBM hardware definitions use
> big-endian bit ordering, where bit 0 is the most significant bit.
>
> >
> > Would someone be kind to explain what I do not understand?
> >
> > Thank you
>
>         Gabriel

Re: Looking for architecture papers

[Segher Boessenkool]

On Sat, Oct 06, 2018 at 12:19:45PM +0300, Raz wrote:
> Hey
> How does HVSC works ?
> I looked in the code and LoPAR documentation. It looks like there is
> vector called
> system_call_pSeries ( at 0xc00 ) that is supposed to be called when we
> invoke HVSC from kernel
> mode.
> Now, I wrote a NULL call HSVC and patched the exceptions-64s.S to
> return RFID immediately.
> This does not work.
> Would you be so kind to explain how HVSC works ?
> thank you

If your kernel is not running in hypervisor mode, sc 1 does not call the
kernel (but the hypervisor, instead).  If your kernel _is_ running in
hypervisor mode, sc 1 does the same as sc 0, a normal system call.

I don't know which it is for you; you didn't say.

I have no idea what "a NULL call HSVC" means.  If you make exception c00
return immediately (as you suggest) then you have made all system calls
non-functional, which indeed is unlikely to work as you want.


Segher

Re: Looking for architecture papers

[Raz]

Both systemsim and my powerpc server boots with MSR_HV=1, i.e, hypervisor state.
Is there away to fix that ? writing to the MSR cannot work according
the documentation ( and reality ).



On Sat, Oct 6, 2018 at 3:27 PM Segher Boessenkool
<segher@kernel.crashing.org> wrote:
>
> On Sat, Oct 06, 2018 at 12:19:45PM +0300, Raz wrote:
> > Hey
> > How does HVSC works ?
> > I looked in the code and LoPAR documentation. It looks like there is
> > vector called
> > system_call_pSeries ( at 0xc00 ) that is supposed to be called when we
> > invoke HVSC from kernel
> > mode.
> > Now, I wrote a NULL call HSVC and patched the exceptions-64s.S to
> > return RFID immediately.
> > This does not work.
> > Would you be so kind to explain how HVSC works ?
> > thank you
>
> If your kernel is not running in hypervisor mode, sc 1 does not call the
> kernel (but the hypervisor, instead).  If your kernel _is_ running in
> hypervisor mode, sc 1 does the same as sc 0, a normal system call.
>
> I don't know which it is for you; you didn't say.
>
> I have no idea what "a NULL call HSVC" means.  If you make exception c00
> return immediately (as you suggest) then you have made all system calls
> non-functional, which indeed is unlikely to work as you want.
>
>
> Segher

Re: Looking for architecture papers

[Segher Boessenkool]

On Mon, Oct 08, 2018 at 07:44:12PM +0300, Raz wrote:
> Both systemsim and my powerpc server boots with MSR_HV=1, i.e, hypervisor state.
> Is there away to fix that ? writing to the MSR cannot work according
> the documentation ( and reality ).

But that is what you do: you write HV=0 in MSR.  After doing other setup,
of course.

On some hardware you cannot set HV=0.  You cannot do logical partitioning
on such hardware.  PowerMac G5 comes to mind.


Segher

Re: Looking for architecture papers

[Gustavo Romero]

Hi Raz,

On 10/04/2018 04:41 AM, Raz wrote:
> Frankly, the more I read the more perplexed I get. For example,
> according to BOOK III-S, chapter 3,
> the MSR bits are differ from the ones described in
> arch/powerpc/include/asm/reg.h.
> Bit zero, is LE, but in the book it is 64-bit mode.
> 
> Would someone be kind to explain what I do not understand?

Yes, I know that can be confusing at the first sight when one is used to, for
instance, x86.

x86 documents use LSB 0 notation, which means (as others already pointed out)
that the least significant bit of a value is marked as being bit 0.

On the other hand Power documents use MSB 0 notation, which means that the most
significant bit of a value is marked as being bit 0 and as a consequence the
least significant bit in that notation in a 64-bit platform is bit 63, not bit
0. MSB 0 notation is also known as IBM bit notation/bit numbering.

Historically LSB 0 notation tend to be used on docs about little-endian
architectures (for instance, x86), whilst MSB 0 notation tend to be used on docs
about big-endian architectures (for instance, Power - Power is actually a little
different because it's now bi-endian actually).

However LSB 0 and MSB 0 are only different notations, so LSB 0 can be employed
on a big-endian architecture documentation, and vice versa.

It happens that kernel code is written in C and for shifts, etc, it's convenient
the LSB 0 notation, not the MSB 0 notation, so it's convenient to use LSB 0
notation when creating a mask, like in arch/powerpc/include/asm/reg.h), i.e.
it's convenient to employ bit positions as '63 - <bit position in PowerISA>'.

So, as another example, in the following gcc macro '_TEXASR_EXTRACT_BITS' takes
a bit position 'BITNUM' as found in the PowerISA documentation but then for the
shift right it uses '63 - BITNUM':

https://github.com/gcc-mirror/gcc/blob/master/gcc/config/rs6000/htmintrin.h#L44-L45

I think it's also important to mention that on PowerISA the elements also follow
the MSB 0 notation. So byte, word, and dword elements in a register found in the
instruction descriptions when referred to 0 are the element "at the left tip",
i.e. "the most significant elements", so to speak. For instance, take
instruction "vperm": doc says 'index' takes bits 3:7 of a byte from [byte]
element 'i'. So for a byte element i=0 it means the most significant byte
("on the left tip") of vector register operand 'VRC'. Moreover, specified bits
in that byte element, i.e. bits 3:7,  also follow the MSB 0, so for the
little-endian addicted thought they are bits 4:0 (LSB 0 notation).

Now, if bits 4:0 = 0b00011 (decimal 3), we grab byte element 3 from 'src'
(256-bit). However byte element 3 is also in MSB 0 notation, so it means third
byte of 'src' but starting counting bytes from 0 from the left to the right
(which in IMO looks indeed more natural since we count, for instance, Natural
Numbers on the 'x' axis similarly).

Hence, it's like to say that 'vperm' instruction in a certain sense has a
"big-endian semantics" for the byte indices. The 'vpermr' instruction introduced
by PowerISA v3.0 is meant to cope with that, so 'vpermr' byte indices have a
"little-endian semantics", so for bits 3:7 MSB 0 (or bits 4:0 in LSB 0 notation) =
0b00011 (decimal 3), on the 'vpermr' instruction it really means we must count
bytes starting from right to left as in the LSB 0 notation and grab the third byte
element from right to left.

So, for instance:

vr0            uint128 = 0x00000000000000000000000000000000
vr1            uint128 = 0x00102030405060708090a0b0c0d0e0f0
vr2            uint128 = 0x01112233445566779999aabbccddeeff
vr3            uint128 = 0x03000000000000000000000000000000

we have 'src' as:

MSB 0:             v--- byte 0, 1, 2, 3, ...
LSB 0:                                                                                  ...  3, 2, 1, byte 0 ---v
src = vr1 || vr2 = 00 10 20 30 40 50 60 70 80 90 A0 B0 C0 D0 E0 F0 01 11 22 33 44 55 66 77 99 99 AA BB CC DD EE FF

vperm   vr0, vr1, vr2, vr3 result is:
vr0            uint128 = 0x30000000000000000000000000000000
byte 3 in MSB 0  = 0x30 ---^ and 0x00 (byte 0 in MSB 0) copied to the remaining bytes

whilst with vpermr (PowerISA v3.0 / POWER9):
vpermr   vr0, vr1, vr2, vr3 result is:
vr0            uint128 = 0xccffffffffffffffffffffffffffffff
byte 3 in LSB 0  = 0xCC----^ and 0xFF (byte 0 in LSB 0) copied to the remaining bytes


Anyway, vperm/vpermr was just an example about notation not being restricted to
bits on Power ISA. So read the docs carefully :) GDB is always useful for checking
if one's understanding about a given Power instruction is correct.

HTH.


Regards,
Gustavo

Re: Looking for architecture papers

[Raz]

So,if we're talk about address...
EA0 is actually bit 63 for any kernel address. This means[1] the
effective address is
the real address ( the physical address), because MSR_HV=1.
1. What does it mean PA=EA ? How does the translation work now ?
2. in interrupts, the program counter is set to EA 0c00, ==>  EA=0.
So, in interrupt there's a different addressing model ?

[1]. Section 5.7.3, PowerISA 2.07B.


On Mon, Oct 8, 2018 at 10:59 PM Gustavo Romero
<gromero@linux.vnet.ibm.com> wrote:
>
> Hi Raz,
>
> On 10/04/2018 04:41 AM, Raz wrote:
> > Frankly, the more I read the more perplexed I get. For example,
> > according to BOOK III-S, chapter 3,
> > the MSR bits are differ from the ones described in
> > arch/powerpc/include/asm/reg.h.
> > Bit zero, is LE, but in the book it is 64-bit mode.
> >
> > Would someone be kind to explain what I do not understand?
>
> Yes, I know that can be confusing at the first sight when one is used to, for
> instance, x86.
>
> x86 documents use LSB 0 notation, which means (as others already pointed out)
> that the least significant bit of a value is marked as being bit 0.
>
> On the other hand Power documents use MSB 0 notation, which means that the most
> significant bit of a value is marked as being bit 0 and as a consequence the
> least significant bit in that notation in a 64-bit platform is bit 63, not bit
> 0. MSB 0 notation is also known as IBM bit notation/bit numbering.
>
> Historically LSB 0 notation tend to be used on docs about little-endian
> architectures (for instance, x86), whilst MSB 0 notation tend to be used on docs
> about big-endian architectures (for instance, Power - Power is actually a little
> different because it's now bi-endian actually).
>
> However LSB 0 and MSB 0 are only different notations, so LSB 0 can be employed
> on a big-endian architecture documentation, and vice versa.
>
> It happens that kernel code is written in C and for shifts, etc, it's convenient
> the LSB 0 notation, not the MSB 0 notation, so it's convenient to use LSB 0
> notation when creating a mask, like in arch/powerpc/include/asm/reg.h), i.e.
> it's convenient to employ bit positions as '63 - <bit position in PowerISA>'.
>
> So, as another example, in the following gcc macro '_TEXASR_EXTRACT_BITS' takes
> a bit position 'BITNUM' as found in the PowerISA documentation but then for the
> shift right it uses '63 - BITNUM':
>
> https://github.com/gcc-mirror/gcc/blob/master/gcc/config/rs6000/htmintrin.h#L44-L45
>
> I think it's also important to mention that on PowerISA the elements also follow
> the MSB 0 notation. So byte, word, and dword elements in a register found in the
> instruction descriptions when referred to 0 are the element "at the left tip",
> i.e. "the most significant elements", so to speak. For instance, take
> instruction "vperm": doc says 'index' takes bits 3:7 of a byte from [byte]
> element 'i'. So for a byte element i=0 it means the most significant byte
> ("on the left tip") of vector register operand 'VRC'. Moreover, specified bits
> in that byte element, i.e. bits 3:7,  also follow the MSB 0, so for the
> little-endian addicted thought they are bits 4:0 (LSB 0 notation).
>
> Now, if bits 4:0 = 0b00011 (decimal 3), we grab byte element 3 from 'src'
> (256-bit). However byte element 3 is also in MSB 0 notation, so it means third
> byte of 'src' but starting counting bytes from 0 from the left to the right
> (which in IMO looks indeed more natural since we count, for instance, Natural
> Numbers on the 'x' axis similarly).
>
> Hence, it's like to say that 'vperm' instruction in a certain sense has a
> "big-endian semantics" for the byte indices. The 'vpermr' instruction introduced
> by PowerISA v3.0 is meant to cope with that, so 'vpermr' byte indices have a
> "little-endian semantics", so for bits 3:7 MSB 0 (or bits 4:0 in LSB 0 notation) =
> 0b00011 (decimal 3), on the 'vpermr' instruction it really means we must count
> bytes starting from right to left as in the LSB 0 notation and grab the third byte
> element from right to left.
>
> So, for instance:
>
> vr0            uint128 = 0x00000000000000000000000000000000
> vr1            uint128 = 0x00102030405060708090a0b0c0d0e0f0
> vr2            uint128 = 0x01112233445566779999aabbccddeeff
> vr3            uint128 = 0x03000000000000000000000000000000
>
> we have 'src' as:
>
> MSB 0:             v--- byte 0, 1, 2, 3, ...
> LSB 0:                                                                                  ...  3, 2, 1, byte 0 ---v
> src = vr1 || vr2 = 00 10 20 30 40 50 60 70 80 90 A0 B0 C0 D0 E0 F0 01 11 22 33 44 55 66 77 99 99 AA BB CC DD EE FF
>
> vperm   vr0, vr1, vr2, vr3 result is:
> vr0            uint128 = 0x30000000000000000000000000000000
> byte 3 in MSB 0  = 0x30 ---^ and 0x00 (byte 0 in MSB 0) copied to the remaining bytes
>
> whilst with vpermr (PowerISA v3.0 / POWER9):
> vpermr   vr0, vr1, vr2, vr3 result is:
> vr0            uint128 = 0xccffffffffffffffffffffffffffffff
> byte 3 in LSB 0  = 0xCC----^ and 0xFF (byte 0 in LSB 0) copied to the remaining bytes
>
>
> Anyway, vperm/vpermr was just an example about notation not being restricted to
> bits on Power ISA. So read the docs carefully :) GDB is always useful for checking
> if one's understanding about a given Power instruction is correct.
>
> HTH.
>
>
> Regards,
> Gustavo
>