[GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Aleksandar Markovic]

[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]

*Extend support for ioctls in QEMU linux-user mode*



*PLANNED ACTIVITIES*

BACKGROUND

There is currently 2500+ ioctls defined in Linux kernel. QEMU linux-user
currently supports only several hundred. There is a constant need for
expanding ioctl support in QEMU. Users use Linux-user mode in variety of
setups (for example, building and testing tools and applications under
chroot environment), and, on a regular basis, efforts by multiple people
are made to fill in missing support. However, these efforts have been
usually done on a piece-by-piece basis, i a limited way covering a
partucular need. This project will take more proactive stance, and try to
improve QEMU before users start complaining.

PART I:

   a) Add strace support for outputing ioctl IDs (the second argument of
ioctl()) as strings rather than numbers - for all platform independant
ioctls.
   b) Add strace support for printing the third argument of ioctl() (be it
int, string, structure or array) - limited to selected ioctls that are
frequently used.

PART II:

   a) Amend support for existing groups of ioctls that are not completed
100% (let's say, filesystem ioctls)
   b) Add support for a selected group of ioctls that are not currently
supported (for example, dm ioctls, Bluetooth ioctls, or Radeon DRM ioctls)

PART III:

  a) Develop unit tests for selected ioctls that are already supported in
QEMU.


*DELIVERABLES*

The deliverables are in the form of source code for each part, intended to
be upstreamed, and time needed for upstreaming (addressing reviews, etc.)
process is included int this project.

The delivery of results can and should be distributed over larger period of
time 2-3 months.


Montor: open (I propose Laurent Vivier)

Student: open

[-- Attachment #2: Type: text/html, Size: 2450 bytes --]

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2413 bytes --]

On Thu, Jan 23, 2020 at 02:34:01PM +0100, Aleksandar Markovic wrote:
> *Extend support for ioctls in QEMU linux-user mode*
> 
> 
> 
> *PLANNED ACTIVITIES*
> 
> BACKGROUND
> 
> There is currently 2500+ ioctls defined in Linux kernel. QEMU linux-user
> currently supports only several hundred. There is a constant need for
> expanding ioctl support in QEMU. Users use Linux-user mode in variety of
> setups (for example, building and testing tools and applications under
> chroot environment), and, on a regular basis, efforts by multiple people
> are made to fill in missing support. However, these efforts have been
> usually done on a piece-by-piece basis, i a limited way covering a

s/ i / in /

> partucular need. This project will take more proactive stance, and try to

s/partucular/particular/

> improve QEMU before users start complaining.
> 
> PART I:
> 
>    a) Add strace support for outputing ioctl IDs (the second argument of
> ioctl()) as strings rather than numbers - for all platform independant
> ioctls.
>    b) Add strace support for printing the third argument of ioctl() (be it
> int, string, structure or array) - limited to selected ioctls that are
> frequently used.
> 
> PART II:
> 
>    a) Amend support for existing groups of ioctls that are not completed
> 100% (let's say, filesystem ioctls)
>    b) Add support for a selected group of ioctls that are not currently
> supported (for example, dm ioctls, Bluetooth ioctls, or Radeon DRM ioctls)
> 
> PART III:
> 
>   a) Develop unit tests for selected ioctls that are already supported in
> QEMU.
> 
> 
> *DELIVERABLES*
> 
> The deliverables are in the form of source code for each part, intended to
> be upstreamed, and time needed for upstreaming (addressing reviews, etc.)
> process is included int this project.
> 
> The delivery of results can and should be distributed over larger period of
> time 2-3 months.

Good project idea.  Please choose concrete ioctls.  Applicants may not
have the necessary experience to choose a set of ioctls that are useful.

I wonder if it's possible to use something like the Debian popularity
contest (https://popcon.debian.org/) and then scan the source of the
most popular N packages for ioctl() calls.  Maybe this is overthinking
it, but it would give an idea of which ioctl() calls are relevant and
missing from QEMU.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Aleksandar Markovic]

[-- Attachment #1: Type: text/plain, Size: 3327 bytes --]

On Mon, Jan 27, 2020 at 10:30 AM Stefan Hajnoczi <stefanha@redhat.com>
wrote:
>
> On Thu, Jan 23, 2020 at 02:34:01PM +0100, Aleksandar Markovic wrote:
> > *Extend support for ioctls in QEMU linux-user mode*
> >
> >
> >
> > *PLANNED ACTIVITIES*
> >
> > BACKGROUND
> >
> > There is currently 2500+ ioctls defined in Linux kernel. QEMU linux-user
> > currently supports only several hundred. There is a constant need for
> > expanding ioctl support in QEMU. Users use Linux-user mode in variety of
> > setups (for example, building and testing tools and applications under
> > chroot environment), and, on a regular basis, efforts by multiple people
> > are made to fill in missing support. However, these efforts have been
> > usually done on a piece-by-piece basis, i a limited way covering a
>
> s/ i / in /
>
> > partucular need. This project will take more proactive stance, and try
to
>
> s/partucular/particular/
>
> > improve QEMU before users start complaining.
> >
> > PART I:
> >
> >    a) Add strace support for outputing ioctl IDs (the second argument of
> > ioctl()) as strings rather than numbers - for all platform independant
> > ioctls.
> >    b) Add strace support for printing the third argument of ioctl() (be
it
> > int, string, structure or array) - limited to selected ioctls that are
> > frequently used.
> >
> > PART II:
> >
> >    a) Amend support for existing groups of ioctls that are not completed
> > 100% (let's say, filesystem ioctls)
> >    b) Add support for a selected group of ioctls that are not currently
> > supported (for example, dm ioctls, Bluetooth ioctls, or Radeon DRM
ioctls)
> >
> > PART III:
> >
> >   a) Develop unit tests for selected ioctls that are already supported
in
> > QEMU.
> >
> >
> > *DELIVERABLES*
> >
> > The deliverables are in the form of source code for each part, intended
to
> > be upstreamed, and time needed for upstreaming (addressing reviews,
etc.)
> > process is included int this project.
> >
> > The delivery of results can and should be distributed over larger
period of
> > time 2-3 months.
>
> Good project idea.  Please choose concrete ioctls.  Applicants may not
> have the necessary experience to choose a set of ioctls that are useful.
>

PART I should not be that difficult. PART II is, however, a minefield. An
implementation of support of a ioctl range from 15 minutes to two months.
The least we wont to happen is that the student is stuck with a problem for
months. Therefore I suggest first some "low hanging fruit" for a student to
get self-confidence and experience. One such group is DM ioctl group ( link
<https://github.com/torvalds/linux/blob/master/include/uapi/linux/dm-ioctl.h#L251>
) (Laurent may confirm, or "unconfirm" that). The next shoudl be something
a little harder, but useful in terms of end user. PART III should be
developed on the fly, but we need to provide a guideline/framework prior to
starting working with a student, IMHO..

> I wonder if it's possible to use something like the Debian popularity
> contest (https://popcon.debian.org/) and then scan the source of the
> most popular N packages for ioctl() calls.

Great! I'll try. A very interesting site and method.

Aleksandar

>  Maybe this is overthinking
> it, but it would give an idea of which ioctl() calls are relevant and
> missing from QEMU.
>
> Stefan

[-- Attachment #2: Type: text/html, Size: 4157 bytes --]

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Laurent Vivier]

Le 27/01/2020 à 17:30, Aleksandar Markovic a écrit :
> 
> 
> On Mon, Jan 27, 2020 at 10:30 AM Stefan Hajnoczi <stefanha@redhat.com
> <mailto:stefanha@redhat.com>> wrote:
>>
>> On Thu, Jan 23, 2020 at 02:34:01PM +0100, Aleksandar Markovic wrote:
>> > *Extend support for ioctls in QEMU linux-user mode*
>> >
>> >
>> >
>> > *PLANNED ACTIVITIES*
>> >
>> > BACKGROUND
>> >
>> > There is currently 2500+ ioctls defined in Linux kernel. QEMU linux-user
>> > currently supports only several hundred. There is a constant need for
>> > expanding ioctl support in QEMU. Users use Linux-user mode in variety of
>> > setups (for example, building and testing tools and applications under
>> > chroot environment), and, on a regular basis, efforts by multiple people
>> > are made to fill in missing support. However, these efforts have been
>> > usually done on a piece-by-piece basis, i a limited way covering a
>>
>> s/ i / in /
>>
>> > partucular need. This project will take more proactive stance, and
> try to
>>
>> s/partucular/particular/
>>
>> > improve QEMU before users start complaining.
>> >
>> > PART I:
>> >
>> >    a) Add strace support for outputing ioctl IDs (the second argument of
>> > ioctl()) as strings rather than numbers - for all platform independant
>> > ioctls.
>> >    b) Add strace support for printing the third argument of ioctl()
> (be it
>> > int, string, structure or array) - limited to selected ioctls that are
>> > frequently used.
>> >
>> > PART II:
>> >
>> >    a) Amend support for existing groups of ioctls that are not completed
>> > 100% (let's say, filesystem ioctls)
>> >    b) Add support for a selected group of ioctls that are not currently
>> > supported (for example, dm ioctls, Bluetooth ioctls, or Radeon DRM
> ioctls)
>> >
>> > PART III:
>> >
>> >   a) Develop unit tests for selected ioctls that are already
> supported in
>> > QEMU.
>> >
>> >
>> > *DELIVERABLES*
>> >
>> > The deliverables are in the form of source code for each part,
> intended to
>> > be upstreamed, and time needed for upstreaming (addressing reviews,
> etc.)
>> > process is included int this project.
>> >
>> > The delivery of results can and should be distributed over larger
> period of
>> > time 2-3 months.
>>
>> Good project idea.  Please choose concrete ioctls.  Applicants may not
>> have the necessary experience to choose a set of ioctls that are useful.
>>
> 
> PART I should not be that difficult. PART II is, however, a minefield.
> An implementation of support of a ioctl range from 15 minutes to two
> months. The least we wont to happen is that the student is stuck with a
> problem for months. Therefore I suggest first some "low hanging fruit"
> for a student to get self-confidence and experience. One such group is
> DM ioctl group ( link
> <https://github.com/torvalds/linux/blob/master/include/uapi/linux/dm-ioctl.h#L251>
> ) (Laurent may confirm, or "unconfirm" that). The next shoudl be

Well, ioctl are generally implemented on demand when we see there is one
missing. DM can be interresting, but do we have an easy way to test them?

> something a little harder, but useful in terms of end user. PART III
> should be developed on the fly, but we need to provide a
> guideline/framework prior to starting working with a student, IMHO..
> 
>> I wonder if it's possible to use something like the Debian popularity
>> contest (https://popcon.debian.org/) and then scan the source of the
>> most popular N packages for ioctl() calls.
> 
> Great! I'll try. A very interesting site and method.
> 

The other point to implement ioctl that we know are used by a given
package is we can run this package to see if it works or not before and
after the implementation of the missing ioctl.

We can also rely on some test suites provided by the packages.

We can also detect missing ioctl by looking at "Unsupported ioctl: cmd="
error message while we run chroot.

It could be interesting to run a gnome-session from inside a chroot with
qemu-linux-user to trigger more multimedia related ioctl.

Thanks,
Laurent

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Peter Maydell]

On Thu, 23 Jan 2020 at 15:33, Aleksandar Markovic
<aleksandar.m.mail@gmail.com> wrote:
> PART III:
>
>   a) Develop unit tests for selected ioctls that are already supported in QEMU.

These should probably just be part of the external Linux Test Project
test suite.

Incidentally a related topic would be to look at recent
new syscalls added to the kernel and see which ones we
could/should add support for. But maybe you're interested
in ioctls specifically.

thanks
-- PMM

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Aleksandar Markovic]

[-- Attachment #1: Type: text/plain, Size: 5257 bytes --]

15:28 Uto, 28.01.2020. Laurent Vivier <laurent@vivier.eu> је написао/ла:
>
> Le 27/01/2020 à 17:30, Aleksandar Markovic a écrit :
> >
> >
> > On Mon, Jan 27, 2020 at 10:30 AM Stefan Hajnoczi <stefanha@redhat.com
> > <mailto:stefanha@redhat.com>> wrote:
> >>
> >> On Thu, Jan 23, 2020 at 02:34:01PM +0100, Aleksandar Markovic wrote:
> >> > *Extend support for ioctls in QEMU linux-user mode*
> >> >
> >> >
> >> >
> >> > *PLANNED ACTIVITIES*
> >> >
> >> > BACKGROUND
> >> >
> >> > There is currently 2500+ ioctls defined in Linux kernel. QEMU
linux-user
> >> > currently supports only several hundred. There is a constant need for
> >> > expanding ioctl support in QEMU. Users use Linux-user mode in
variety of
> >> > setups (for example, building and testing tools and applications
under
> >> > chroot environment), and, on a regular basis, efforts by multiple
people
> >> > are made to fill in missing support. However, these efforts have been
> >> > usually done on a piece-by-piece basis, i a limited way covering a
> >>
> >> s/ i / in /
> >>
> >> > partucular need. This project will take more proactive stance, and
> > try to
> >>
> >> s/partucular/particular/
> >>
> >> > improve QEMU before users start complaining.
> >> >
> >> > PART I:
> >> >
> >> >    a) Add strace support for outputing ioctl IDs (the second
argument of
> >> > ioctl()) as strings rather than numbers - for all platform
independant
> >> > ioctls.
> >> >    b) Add strace support for printing the third argument of ioctl()
> > (be it
> >> > int, string, structure or array) - limited to selected ioctls that
are
> >> > frequently used.
> >> >
> >> > PART II:
> >> >
> >> >    a) Amend support for existing groups of ioctls that are not
completed
> >> > 100% (let's say, filesystem ioctls)
> >> >    b) Add support for a selected group of ioctls that are not
currently
> >> > supported (for example, dm ioctls, Bluetooth ioctls, or Radeon DRM
> > ioctls)
> >> >
> >> > PART III:
> >> >
> >> >   a) Develop unit tests for selected ioctls that are already
> > supported in
> >> > QEMU.
> >> >
> >> >
> >> > *DELIVERABLES*
> >> >
> >> > The deliverables are in the form of source code for each part,
> > intended to
> >> > be upstreamed, and time needed for upstreaming (addressing reviews,
> > etc.)
> >> > process is included int this project.
> >> >
> >> > The delivery of results can and should be distributed over larger
> > period of
> >> > time 2-3 months.
> >>
> >> Good project idea.  Please choose concrete ioctls.  Applicants may not
> >> have the necessary experience to choose a set of ioctls that are
useful.
> >>
> >
> > PART I should not be that difficult. PART II is, however, a minefield.
> > An implementation of support of a ioctl range from 15 minutes to two
> > months. The least we wont to happen is that the student is stuck with a
> > problem for months. Therefore I suggest first some "low hanging fruit"
> > for a student to get self-confidence and experience. One such group is
> > DM ioctl group ( link
> > <
https://github.com/torvalds/linux/blob/master/include/uapi/linux/dm-ioctl.h#L251
>
> > ) (Laurent may confirm, or "unconfirm" that). The next shoudl be
>
> Well, ioctl are generally implemented on demand when we see there is one
> missing. DM can be interresting, but do we have an easy way to test them?
>

The main difficilty, as I see this now, would be establishing a test bed.
This should be possible using one or two USB flash drives (if one doesn't
want to mess with own hard drives and partitions). After that, having
dm-enabled devices available, writing unit test should be straightforward.

I don't have a particular prefference towards dm ioctls, they just look to
me simple enough to be a good and gentle starting point for a beginner.

Giving a student a hard case at the beginning could be quite discouraging
for him/her, and probably even counterproductive for the project as a whole.

Again, this is just my opinion, I don't have the ambition to impose such
details. After all, this was and is just a proposal.

> > something a little harder, but useful in terms of end user. PART III
> > should be developed on the fly, but we need to provide a
> > guideline/framework prior to starting working with a student, IMHO..
> >
> >> I wonder if it's possible to use something like the Debian popularity
> >> contest (https://popcon.debian.org/) and then scan the source of the
> >> most popular N packages for ioctl() calls.
> >
> > Great! I'll try. A very interesting site and method.
> >
>
> The other point to implement ioctl that we know are used by a given
> package is we can run this package to see if it works or not before and
> after the implementation of the missing ioctl.
>
> We can also rely on some test suites provided by the packages.
>
> We can also detect missing ioctl by looking at "Unsupported ioctl: cmd="
> error message while we run chroot.
>
> It could be interesting to run a gnome-session from inside a chroot with
> qemu-linux-user to trigger more multimedia related ioctl.
>

One more intriguing idea!

Regards,
Aleksandar

> Thanks,
> Laurent
>

[-- Attachment #2: Type: text/html, Size: 7117 bytes --]

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Laurent Vivier]

Le 28/01/2020 à 18:32, Aleksandar Markovic a écrit :
> 15:28 Uto, 28.01.2020. Laurent Vivier <laurent@vivier.eu
> <mailto:laurent@vivier.eu>> је написао/ла:
>>
>> Le 27/01/2020 à 17:30, Aleksandar Markovic a écrit :
>> >
>> >
>> > On Mon, Jan 27, 2020 at 10:30 AM Stefan Hajnoczi
> <stefanha@redhat.com <mailto:stefanha@redhat.com>
>> > <mailto:stefanha@redhat.com <mailto:stefanha@redhat.com>>> wrote:
>> >>
>> >> On Thu, Jan 23, 2020 at 02:34:01PM +0100, Aleksandar Markovic wrote:
>> >> > *Extend support for ioctls in QEMU linux-user mode*
>> >> >
>> >> >
>> >> >
>> >> > *PLANNED ACTIVITIES*
>> >> >
>> >> > BACKGROUND
>> >> >
>> >> > There is currently 2500+ ioctls defined in Linux kernel. QEMU
> linux-user
>> >> > currently supports only several hundred. There is a constant need for
>> >> > expanding ioctl support in QEMU. Users use Linux-user mode in
> variety of
>> >> > setups (for example, building and testing tools and applications
> under
>> >> > chroot environment), and, on a regular basis, efforts by multiple
> people
>> >> > are made to fill in missing support. However, these efforts have been
>> >> > usually done on a piece-by-piece basis, i a limited way covering a
>> >>
>> >> s/ i / in /
>> >>
>> >> > partucular need. This project will take more proactive stance, and
>> > try to
>> >>
>> >> s/partucular/particular/
>> >>
>> >> > improve QEMU before users start complaining.
>> >> >
>> >> > PART I:
>> >> >
>> >> >    a) Add strace support for outputing ioctl IDs (the second
> argument of
>> >> > ioctl()) as strings rather than numbers - for all platform
> independant
>> >> > ioctls.
>> >> >    b) Add strace support for printing the third argument of ioctl()
>> > (be it
>> >> > int, string, structure or array) - limited to selected ioctls
> that are
>> >> > frequently used.
>> >> >
>> >> > PART II:
>> >> >
>> >> >    a) Amend support for existing groups of ioctls that are not
> completed
>> >> > 100% (let's say, filesystem ioctls)
>> >> >    b) Add support for a selected group of ioctls that are not
> currently
>> >> > supported (for example, dm ioctls, Bluetooth ioctls, or Radeon DRM
>> > ioctls)
>> >> >
>> >> > PART III:
>> >> >
>> >> >   a) Develop unit tests for selected ioctls that are already
>> > supported in
>> >> > QEMU.
>> >> >
>> >> >
>> >> > *DELIVERABLES*
>> >> >
>> >> > The deliverables are in the form of source code for each part,
>> > intended to
>> >> > be upstreamed, and time needed for upstreaming (addressing reviews,
>> > etc.)
>> >> > process is included int this project.
>> >> >
>> >> > The delivery of results can and should be distributed over larger
>> > period of
>> >> > time 2-3 months.
>> >>
>> >> Good project idea.  Please choose concrete ioctls.  Applicants may not
>> >> have the necessary experience to choose a set of ioctls that are
> useful.
>> >>
>> >
>> > PART I should not be that difficult. PART II is, however, a minefield.
>> > An implementation of support of a ioctl range from 15 minutes to two
>> > months. The least we wont to happen is that the student is stuck with a
>> > problem for months. Therefore I suggest first some "low hanging fruit"
>> > for a student to get self-confidence and experience. One such group is
>> > DM ioctl group ( link
>> >
> <https://github.com/torvalds/linux/blob/master/include/uapi/linux/dm-ioctl.h#L251>
>> > ) (Laurent may confirm, or "unconfirm" that). The next shoudl be
>>
>> Well, ioctl are generally implemented on demand when we see there is one
>> missing. DM can be interresting, but do we have an easy way to test them?
>>
> 
> The main difficilty, as I see this now, would be establishing a test
> bed. This should be possible using one or two USB flash drives (if one
> doesn't want to mess with own hard drives and partitions). After that,
> having dm-enabled devices available, writing unit test should be
> straightforward.

The easiest way to test DM is to use a qemu-system-x86_64/kvm VM with
several storage files and to run qemu-user inside. And a VM is easier to
restore when we break it... and it will happen.

Thanks,
Laurent

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Aleksandar Markovic]

[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]

15:38 Uto, 28.01.2020. Peter Maydell <peter.maydell@linaro.org> је
написао/ла:
>
> On Thu, 23 Jan 2020 at 15:33, Aleksandar Markovic
> <aleksandar.m.mail@gmail.com> wrote:
> > PART III:
> >
> >   a) Develop unit tests for selected ioctls that are already supported
in QEMU.
>
> These should probably just be part of the external Linux Test Project
> test suite.
>

I like the idea! This would free the student from reinventing the wheel
(ioctl test framework) - most probably resulting in more work done during
the project.

I am just not sure how this stands with GSoC/Outreachy rules. The project
would touch two open source project. Stefan?

> Incidentally a related topic would be to look at recent
> new syscalls added to the kernel and see which ones we
> could/should add support for. But maybe you're interested
> in ioctls specifically.
>

I am personally interested in both. However, learning about syscalls and
implementation of support for them in QEMU would require more ramp-up time
from student (as is the case for any new employee in a similar situation
for that matter). In the worst case, the student would spend much more time
on learning than on productive work. My impression is at it is better to
leave the student focused on just one area - ioctl support.

Thanks, Peter, for comments and ideas!

Aleksandar

> thanks
> -- PMM

[-- Attachment #2: Type: text/html, Size: 1752 bytes --]

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Peter Maydell]

On Tue, 28 Jan 2020 at 17:51, Aleksandar Markovic
<aleksandar.m.mail@gmail.com> wrote:
>
> I am personally interested in both. However, learning about syscalls and implementation of support for them in QEMU would require more ramp-up time from student (as is the case for any new employee in a similar situation for that matter). In the worst case, the student would spend much more time on learning than on productive work. My impression is at it is better to leave the student focused on just one area - ioctl support.

I think they're about the same, really. The level of difficulty is more
in what the syscall or ioctl does and what its arguments are (ie does
QEMU have to do much mangling on the way past) rather than whether
it happens to be a syscall or an ioctl. Some syscalls are hard, but
some are trivial; same with ioctls.

thanks
-- PMM

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Aleksandar Markovic]

[-- Attachment #1: Type: text/plain, Size: 3956 bytes --]

On Tue, Jan 28, 2020 at 7:00 PM Peter Maydell <peter.maydell@linaro.org>
wrote:

> On Tue, 28 Jan 2020 at 17:51, Aleksandar Markovic
> <aleksandar.m.mail@gmail.com> wrote:
> >
> > I am personally interested in both. However, learning about syscalls and
> implementation of support for them in QEMU would require more ramp-up time
> from student (as is the case for any new employee in a similar situation
> for that matter). In the worst case, the student would spend much more time
> on learning than on productive work. My impression is at it is better to
> leave the student focused on just one area - ioctl support.
>
> I think they're about the same, really. The level of difficulty is more
> in what the syscall or ioctl does and what its arguments are (ie does
> QEMU have to do much mangling on the way past) rather than whether
> it happens to be a syscall or an ioctl. Some syscalls are hard, but
> some are trivial; same with ioctls.
>
> thanks
> -- PMM
>


How about this:

- I removed the first item (a) of PART I (since I already have an idea how
to do this in, I think, a very good way), to free the student time. I will
send my solution to the list before this summer, I hope.
- I amended all parts to include some syscall-related activities - and the
end result is this:


*Extend support for ioctls and syscalls in QEMU linux-user mode*



*PLANNED ACTIVITIES*

BACKGROUND

There are currently 2500+ ioctls defined in Linux kernel. QEMU linux-user
currently supports only several hundred. There is a constant need for
expanding ioctl support in QEMU. Users use Linux-user mode in variety of
setups (for example, building and testing tools and applications under
chroot environment), and, on a regular basis, efforts by multiple people
are made to fill in missing support.

Regarding syscall support in QEMU linux-user, the coverage is much better
than in case of ioctls. However, kernel syscall interface continuously
develops and grows, and QEMU linux-user support usually lags considerably.
The support for new syscalls is usually left unimplemented, until an end
user reports that it is missing in hers/his usage scenario.

In conclusion, the efforts for supporting ioctls and syscalls in QEMU have
usually been done on a piece-by-piece basis, in a limited way covering a
particular need. This project will take more proactive stance, and try to
improve QEMU before users start complaining.

This contributions of this project will be mostly to QEMU, but some parts
would be also contirbutions to LTP (Linux Test Project).

PART I:

   a) Add strace support for printing the third argument of ioctl() (be it
int, string, structure or array) - limited to selected ioctls that are
frequently used.
   b) Add strace support for printing the arguments of selected syscalls
that are frequently used, and not covered in QEMU strace module so far.

PART II:

   a) Amend support for existing groups of ioctls that are not completed
100% (let's say, filesystem ioctls)
   b) Add support for a selected group of ioctls that are not currently
supported (for example, DM ioctls, Bluetooth ioctls, or Radeon DRM ioctls)
   c) Add support for a selected group of syscalls that were recently
introduced in kernel.

PART III:

  a) Within LTP (Linux Test Project), develop unit tests for selected
ioctls that are supported in QEMU (including some whose support is
developed in PART II).
  b) Within LTP (Linux Test Project), develop unit tests for selected
syscalls that are supported in QEMU (including some whose support is
developed in PART II).



*DELIVERABLES*

The deliverables are in the form of source code for each part, intended to
be upstreamed to either QEMU or LTP open source projects. The time needed
for upstreaming (addressing reviews, etc.) process is included into this
project. The delivery of results can and should be distributed over larger
period of time (2-3 months).


Mentor: open (I propose Laurent Vivier)
Student: open

[-- Attachment #2: Type: text/html, Size: 6094 bytes --]

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Aleksandar Markovic]

[-- Attachment #1: Type: text/plain, Size: 3181 bytes --]

14:34 Čet, 23.01.2020. Aleksandar Markovic <aleksandar.m.mail@gmail.com> је
написао/ла:
>
> Extend support for ioctls in QEMU linux-user mode
>
>
> PLANNED ACTIVITIES
>
> BACKGROUND
>
> There is currently 2500+ ioctls defined in Linux kernel. QEMU linux-user
currently supports only several hundred. There is a constant need for
expanding ioctl support in QEMU. Users use Linux-user mode in variety of
setups (for example, building and testing tools and applications under
chroot environment), and, on a regular basis, efforts by multiple people
are made to fill in missing support. However, these efforts have been
usually done on a piece-by-piece basis, i a limited way covering a
partucular need. This project will take more proactive stance, and try to
improve QEMU before users start complaining.
>
> PART I:
>
>    a) Add strace support for outputing ioctl IDs (the second argument of
ioctl()) as strings rather than numbers - for all platform independant
ioctls.
>    b) Add strace support for printing the third argument of ioctl() (be
it int, string, structure or array) - limited to selected ioctls that are
frequently used.
>
> PART II:
>
>    a) Amend support for existing groups of ioctls that are not completed
100% (let's say, filesystem ioctls)
>    b) Add support for a selected group of ioctls that are not currently
supported (for example, dm ioctls, Bluetooth ioctls, or Radeon DRM ioctls)
>
> PART III:
>
>   a) Develop unit tests for selected ioctls that are already supported in
QEMU.
>
> DELIVERABLES
>
> The deliverables are in the form of source code for each part, intended
to be upstreamed, and time needed for upstreaming (addressing reviews,
etc.) process is included int this project.
>
> The delivery of results can and should be distributed over larger period
of time 2-3 months.
>
>
> Montor: open (I propose Laurent Vivier)
>
> Student: open

Hello, Peter, Laurent, Stefan.

I presented in this thread two variants of a potential linux-user-related
project for GSoC/Outreachy. The first variant is more focused on a
particular area (ioctl support), while the second one covers wider set of
current issues within linux-user. The pros and cons of both should be
carefully assesed. I will leave to Peter and Laurent the final judgement if
we want to go or not with this project and also the final formulation of
the project.

Stefan, there was an idea in this thread that this project contributes
(apart to QEMU) to another ooen source project (LTP). In my layman view,
this is an advantage. But, how does that fit into GSoC/Outreachy rules?

Laurent, all this seems to be dependant on whether you are ready to mentor
the project. Are you?

The deadline for submitting GSoC/Outreachy projects (within QEMU) is just
around the corner (Feb 1). I leave to Laurent or Peter (should they give
"go" to this proposal) to officially submit the project on our wiki page
created for that purpose, in the form they deem the best.

Thanks to everyone for expressing the interest in this topic, and
especially coming up with additional ideas!

Truly yours,
Aleksandar

[-- Attachment #2: Type: text/html, Size: 3616 bytes --]

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Laurent Vivier]

Le 30/01/2020 à 12:09, Aleksandar Markovic a écrit :
> 14:34 Čet, 23.01.2020. Aleksandar Markovic <aleksandar.m.mail@gmail.com
> <mailto:aleksandar.m.mail@gmail.com>> је написао/ла:
>>
>> Extend support for ioctls in QEMU linux-user mode
>>
>>
>> PLANNED ACTIVITIES
>>
>> BACKGROUND
>>
>> There is currently 2500+ ioctls defined in Linux kernel. QEMU
> linux-user currently supports only several hundred. There is a constant
> need for expanding ioctl support in QEMU. Users use Linux-user mode in
> variety of setups (for example, building and testing tools and
> applications under chroot environment), and, on a regular basis, efforts
> by multiple people are made to fill in missing support. However, these
> efforts have been usually done on a piece-by-piece basis, i a limited
> way covering a partucular need. This project will take more proactive
> stance, and try to improve QEMU before users start complaining.
>>
>> PART I:
>>
>>    a) Add strace support for outputing ioctl IDs (the second argument
> of ioctl()) as strings rather than numbers - for all platform
> independant ioctls.
>>    b) Add strace support for printing the third argument of ioctl()
> (be it int, string, structure or array) - limited to selected ioctls
> that are frequently used.
>>
>> PART II:
>>
>>    a) Amend support for existing groups of ioctls that are not
> completed 100% (let's say, filesystem ioctls)
>>    b) Add support for a selected group of ioctls that are not
> currently supported (for example, dm ioctls, Bluetooth ioctls, or Radeon
> DRM ioctls)
>>
>> PART III:
>>
>>   a) Develop unit tests for selected ioctls that are already supported
> in QEMU.
>>
>> DELIVERABLES
>>
>> The deliverables are in the form of source code for each part,
> intended to be upstreamed, and time needed for upstreaming (addressing
> reviews, etc.) process is included int this project.
>>
>> The delivery of results can and should be distributed over larger
> period of time 2-3 months.
>>
>>
>> Montor: open (I propose Laurent Vivier)
>>
>> Student: open
> 
> Hello, Peter, Laurent, Stefan.
> 
> I presented in this thread two variants of a potential
> linux-user-related project for GSoC/Outreachy. The first variant is more
> focused on a particular area (ioctl support), while the second one
> covers wider set of current issues within linux-user. The pros and cons
> of both should be carefully assesed. I will leave to Peter and Laurent
> the final judgement if we want to go or not with this project and also
> the final formulation of the project.

I think the second variant (that includes new syscalls) is more
interesting for us and for the student.

> Stefan, there was an idea in this thread that this project contributes
> (apart to QEMU) to another ooen source project (LTP). In my layman view,
> this is an advantage. But, how does that fit into GSoC/Outreachy rules?
> 
> Laurent, all this seems to be dependant on whether you are ready to
> mentor the project. Are you?

Yes, of course.

> The deadline for submitting GSoC/Outreachy projects (within QEMU) is
> just around the corner (Feb 1). I leave to Laurent or Peter (should they
> give "go" to this proposal) to officially submit the project on our wiki
> page created for that purpose, in the form they deem the best.

Peter, is it ok for you?

Thanks,
Laurent

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Peter Maydell]

On Thu, 30 Jan 2020 at 11:20, Laurent Vivier <laurent@vivier.eu> wrote:
>
> Le 30/01/2020 à 12:09, Aleksandar Markovic a écrit :
> > Laurent, all this seems to be dependant on whether you are ready to
> > mentor the project. Are you?
>
> Yes, of course.
>
> > The deadline for submitting GSoC/Outreachy projects (within QEMU) is
> > just around the corner (Feb 1). I leave to Laurent or Peter (should they
> > give "go" to this proposal) to officially submit the project on our wiki
> > page created for that purpose, in the form they deem the best.
>
> Peter, is it ok for you?

I don't really have the time to do GSoC mentoring projects;
if you are willing to do the mentoring then that's great.

thanks
-- PMM

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Laurent Vivier]

Le 30/01/2020 à 12:52, Peter Maydell a écrit :
> On Thu, 30 Jan 2020 at 11:20, Laurent Vivier <laurent@vivier.eu> wrote:
>>
>> Le 30/01/2020 à 12:09, Aleksandar Markovic a écrit :
>>> Laurent, all this seems to be dependant on whether you are ready to
>>> mentor the project. Are you?
>>
>> Yes, of course.
>>
>>> The deadline for submitting GSoC/Outreachy projects (within QEMU) is
>>> just around the corner (Feb 1). I leave to Laurent or Peter (should they
>>> give "go" to this proposal) to officially submit the project on our wiki
>>> page created for that purpose, in the form they deem the best.
>>
>> Peter, is it ok for you?
> 
> I don't really have the time to do GSoC mentoring projects;
> if you are willing to do the mentoring then that's great.

No problem. I will. I asked only to be sure you don't see any problem.

Stefan,  I think it's your role to add the project to the wiki page.
Could you?

Thanks,
Laurent

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On Thu, Jan 30, 2020 at 12:09:58PM +0100, Aleksandar Markovic wrote:
> Stefan, there was an idea in this thread that this project contributes
> (apart to QEMU) to another ooen source project (LTP). In my layman view,
> this is an advantage. But, how does that fit into GSoC/Outreachy rules?

That is fine for GSoC.  It will benefit the open source community and
the code being written will be under an approved open source license.

For Outreachy we focus on the QEMU codebase because the funding comes
for sponsors that may wish their donations to be used for QEMU
development.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]

On Fri, Jan 31, 2020 at 09:37:04AM +0100, Laurent Vivier wrote:
> Le 30/01/2020 à 12:52, Peter Maydell a écrit :
> > On Thu, 30 Jan 2020 at 11:20, Laurent Vivier <laurent@vivier.eu> wrote:
> >>
> >> Le 30/01/2020 à 12:09, Aleksandar Markovic a écrit :
> >>> Laurent, all this seems to be dependant on whether you are ready to
> >>> mentor the project. Are you?
> >>
> >> Yes, of course.
> >>
> >>> The deadline for submitting GSoC/Outreachy projects (within QEMU) is
> >>> just around the corner (Feb 1). I leave to Laurent or Peter (should they
> >>> give "go" to this proposal) to officially submit the project on our wiki
> >>> page created for that purpose, in the form they deem the best.
> >>
> >> Peter, is it ok for you?
> > 
> > I don't really have the time to do GSoC mentoring projects;
> > if you are willing to do the mentoring then that's great.
> 
> No problem. I will. I asked only to be sure you don't see any problem.
> 
> Stefan,  I think it's your role to add the project to the wiki page.
> Could you?

Great, I'll add it now.

You are welcome to add project ideas to the wiki in the future.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 225 bytes --]

On Tue, Jan 28, 2020 at 09:48:33PM +0100, Aleksandar Markovic wrote:
> How about this:

Thank you!  I have posted the idea here:

https://wiki.qemu.org/Google_Summer_of_Code_2020#Extend_linux-user_syscalls_and_ioctls

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [GSoC/Outreachy QEMU proposal] Extend support for ioctls in QEMU linux-user mode

[Laurent Vivier]

[-- Attachment #1.1: Type: text/plain, Size: 321 bytes --]

Le 05/02/2020 à 12:16, Stefan Hajnoczi a écrit :
> On Tue, Jan 28, 2020 at 09:48:33PM +0100, Aleksandar Markovic wrote:
>> How about this:
> 
> Thank you!  I have posted the idea here:
> 
> https://wiki.qemu.org/Google_Summer_of_Code_2020#Extend_linux-user_syscalls_and_ioctls

Thank you Stefan.

Laurent


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]