Re: [RFC] Unify KVM kernel-space and user-space code into a single project

[Avi Kivity <avi@redhat.com>]

On 03/22/2010 04:54 PM, Ingo Molnar wrote:
> * Pekka Enberg<penberg@cs.helsinki.fi>  wrote:
>
>    
>> Hi Avi,
>>
>> On Mon, Mar 22, 2010 at 2:49 PM, Avi Kivity<avi@redhat.com>  wrote:
>>      
>>> Seems like perf is also split, with sysprof being developed outside the
>>> kernel. ?Will you bring sysprof into the kernel? ?Will every feature be
>>> duplicated in prof and sysprof?
>>>        
>> I am glad you brought it up! Sysprof was historically outside of the kernel
>> (with it's own kernel module, actually). While the GUI was nice, it was much
>> harder to set up compared to oprofile so it wasn't all that popular. Things
>> improved slightly when Ingo merged the custom kernel module but the
>> _userspace_ part of sysprof was lagging behind a bit. I don't know what's
>> the situation now that they've switched over to perf syscalls but you
>> probably get my point.
>>
>> It would be nice if the two projects merged but I honestly don't see any
>> fundamental problem with two (or more) co-existing projects. Friendly
>> competition will ultimately benefit the users (think KDE and Gnome here).
>>      
> See my previous mail - what i see as the most healthy project model is to have
> a full solution reference implementation, connected to a flexible halo of
> plugins or sub-apps.
>
> Firefox does that, KDE does that, and Gnome as well to a certain degree.
>
> The 'halo' provides a constant feedback of new features, and it also provides
> competition and pressure on the 'main' code to be top-notch.
>
> The problem i see with KVM is that there's no reference implementation! There
> is _only_ the KVM kernel part which is not functional in itself. Surrounded by
> a 'halo' - where none of the entities is really 'the' reference implementation
> we call 'KVM'.
>    

The reference implementation is qemu-kvm.git, in the future qemu.git.  
Like the reference implementation of device-mapper is 
lvm2/device-mapper, not tools/device-mapper.

> This causes constant quality problems as the developers of the main project
> dont have constant pressure towards good quality (it is not their
> responsibility to care about user-space bits after all),

The developers of the main project are very much aware that users don't 
call the ioctls directly but instead use qemu.

>   plus it causes a lack
> of focus as well: integration between (friendly) competing user-space
> components is a lot harder than integration within a single framework such as
> Firefox.
>    

We are very focused, just not on what you think we should be focused.

> I hope this explains my points about modularization a bit better! I suggested
> KVM to grow a user-space tool component in the kernel repo in tools/kvm/,
> which would become the reference implementation for tooling. User-space
> projects can still provide alternative tooling or can plug into this tooling,
> just like they are doing it now. So the main effect isnt even on those
> projects but on the kernel developers. The ABI remains and all the user-space
> packages and projects remain.
>    

Seems like wanton duplication of effort.  Can we throw so many 
developer-years away on duplicate projects?  Assuming not all are true 
volunteers (85% for 2.6.33) who will fund this duplicate effort?

> Yes, i thought Qemu would be a prime candidate to be the baseline for
> tools/kvm/, but i guess that has become socially impossible now after this
> flamewar. It's not a big problem in the big scheme of things: tools/kvm/ is
> best grown up from a small towards larger size anyway ...
>    

Qemu is open source, you can cp it into tools/kvm.  Rewriting it from 
scratch is a mammoth effort, there's a reason kvm, Xen, and virtualbox 
all use qemu.  Qemu itself copied code from bochs.  Writing this stuff 
is hard, especially if there is something already working.

You'll probably get much better threading (the qemu device model is 
still single threaded), but it will take years to reach where qemu is 
already at.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.