qemu-user: avoid allocations to convert stuff when not necessary

qemu-user: avoid allocations to convert stuff when not necessary

[Michael Tokarev]

Hi!

In the qemu-user case, we allocate various structures and arrays
for conversion of data between host and guest byte orders and sizes.
But it is actually not necessary to do such allocation when the
*size* is the same, and only byte order is different, because the
conversion can be done in-place.  Does it make any sense to avoid'
allocations in such cases?

There are 2 issues with this though. First is that in some cases,
the data being converted is const, and we may end up writing to a
data resides in a read-only segment, is it ever possible?  And
second - it is not entirely clear what to do in case the syscall
returned error.

Thanks,

/mjt

Re: qemu-user: avoid allocations to convert stuff when not necessary

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]

On Sun, Apr 9, 2023 at 2:53 AM Michael Tokarev <mjt@tls.msk.ru> wrote:

> Hi!
>
> In the qemu-user case, we allocate various structures and arrays
> for conversion of data between host and guest byte orders and sizes.
> But it is actually not necessary to do such allocation when the
> *size* is the same, and only byte order is different, because the
> conversion can be done in-place.  Does it make any sense to avoid'
> allocations in such cases?
>
> There are 2 issues with this though. First is that in some cases,
> the data being converted is const, and we may end up writing to a
> data resides in a read-only segment, is it ever possible?  And
> second - it is not entirely clear what to do in case the syscall
> returned error.
>

I don't think you can reliably do it in place. What if another thread in the
guest reads the data after you've converted it? It will get the wrong
answer.
I think you have to copy when endian mismatches, just like when alignment,
data size or layout differences are present. You'd need to convert it back
after the system call as well, which can cause problems, especially
if the system call needs multiple steps to emulate for whatever reason.

Warner

[-- Attachment #2: Type: text/html, Size: 1662 bytes --]

Re: qemu-user: avoid allocations to convert stuff when not necessary

[Michael Tokarev]

09.04.2023 17:04, Warner Losh пишет:
> On Sun, Apr 9, 2023 at 2:53 AM Michael Tokarev <mjt@tls.msk.ru <mailto:mjt@tls.msk.ru>> wrote:
> 
>     Hi!
> 
>     In the qemu-user case, we allocate various structures and arrays
>     for conversion of data between host and guest byte orders and sizes.
>     But it is actually not necessary to do such allocation when the
>     *size* is the same, and only byte order is different, because the
>     conversion can be done in-place.  Does it make any sense to avoid'
>     allocations in such cases?
> 
>     There are 2 issues with this though. First is that in some cases,
>     the data being converted is const, and we may end up writing to a
>     data resides in a read-only segment, is it ever possible?  And
>     second - it is not entirely clear what to do in case the syscall
>     returned error.
> 
> 
> I don't think you can reliably do it in place. What if another thread in the
> guest reads the data after you've converted it? It will get the wrong answer.
> I think you have to copy when endian mismatches, just like when alignment,
> data size or layout differences are present. You'd need to convert it back
> after the system call as well, which can cause problems, especially
> if the system call needs multiple steps to emulate for whatever reason.

Han. I thought more about syscalls which accept in-out parameters.
Such as poll(fd, n, pfd) which uses pfd array for both input and
output. Or just for output, like getgroups() - for this one, if
sizeof(gid_t) == sizeof(target_gid_t), there's no need to allocate
anything, the supplied array can be used, and we need just a loop
after syscall returned, to convert each gid_t to target_gid_t.

For pure-input syscalls, like setgroups(), it is better to perform
allocation IMHO. If not only to avoid a case when RO-segment is
used.

/mjt

Re: qemu-user: avoid allocations to convert stuff when not necessary

[Richard Henderson]

On 4/9/23 01:52, Michael Tokarev wrote:
> Hi!
> 
> In the qemu-user case, we allocate various structures and arrays
> for conversion of data between host and guest byte orders and sizes.
> But it is actually not necessary to do such allocation when the
> *size* is the same, and only byte order is different, because the
> conversion can be done in-place.  Does it make any sense to avoid'
> allocations in such cases?

Alignment can also change.  This is especially visible with m68k guest, where even 'int' 
is 2-byte aligned.

So, no.  It's best to just allocate and convert always.


r~

Re: qemu-user: avoid allocations to convert stuff when not necessary

[Michael Tokarev]

09.04.2023 19:27, Richard Henderson пишет:
> On 4/9/23 01:52, Michael Tokarev wrote:
>> Hi!
>>
>> In the qemu-user case, we allocate various structures and arrays
>> for conversion of data between host and guest byte orders and sizes.
>> But it is actually not necessary to do such allocation when the
>> *size* is the same, and only byte order is different, because the
>> conversion can be done in-place.  Does it make any sense to avoid'
>> allocations in such cases?
> 
> Alignment can also change.  This is especially visible with m68k guest, where even 'int' is 2-byte aligned.

I didn't think about the alignment. So such optimization has less and
less chances to happen. I see.

> So, no.  It's best to just allocate and convert always.

Yes, this makes good sense.

Thanks!

/mjt