[PATCH] multiprocess: move feature to meson_options.txt

[PATCH] multiprocess: move feature to meson_options.txt

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure         | 12 ++++--------
 meson.build       |  9 +++++++--
 meson_options.txt |  2 ++
 3 files changed, 13 insertions(+), 10 deletions(-)

diff --git a/configure b/configure
index 19f2b88589..bdc96d0831 100755
--- a/configure
+++ b/configure
@@ -463,7 +463,7 @@ skip_meson=no
 gettext="auto"
 fuse="auto"
 fuse_lseek="auto"
-multiprocess="no"
+multiprocess="auto"
 
 malloc_trim="auto"
 
@@ -798,7 +798,6 @@ Linux)
   linux="yes"
   linux_user="yes"
   vhost_user=${default_feature:-yes}
-  multiprocess=${default_feature:-yes}
 ;;
 esac
 
@@ -1558,9 +1557,9 @@ for opt do
   ;;
   --disable-fuse-lseek) fuse_lseek="disabled"
   ;;
-  --enable-multiprocess) multiprocess="yes"
+  --enable-multiprocess) multiprocess="enabled"
   ;;
-  --disable-multiprocess) multiprocess="no"
+  --disable-multiprocess) multiprocess="disabled"
   ;;
   *)
       echo "ERROR: unknown option $opt"
@@ -6089,9 +6088,6 @@ fi
 if test "$have_mlockall" = "yes" ; then
   echo "HAVE_MLOCKALL=y" >> $config_host_mak
 fi
-if test "$multiprocess" = "yes" ; then
-  echo "CONFIG_MULTIPROCESS_ALLOWED=y" >> $config_host_mak
-fi
 if test "$fuzzing" = "yes" ; then
   # If LIB_FUZZING_ENGINE is set, assume we are running on OSS-Fuzz, and the
   # needed CFLAGS have already been provided
@@ -6434,7 +6430,7 @@ NINJA=$ninja $meson setup \
         -Dzstd=$zstd -Dseccomp=$seccomp -Dvirtfs=$virtfs -Dcap_ng=$cap_ng \
         -Dattr=$attr -Ddefault_devices=$default_devices \
         -Ddocs=$docs -Dsphinx_build=$sphinx_build -Dinstall_blobs=$blobs \
-        -Dvhost_user_blk_server=$vhost_user_blk_server \
+        -Dvhost_user_blk_server=$vhost_user_blk_server -Dmultiprocess=$multiprocess \
         -Dfuse=$fuse -Dfuse_lseek=$fuse_lseek -Dguest_agent_msi=$guest_agent_msi \
         $(if test "$default_features" = no; then echo "-Dauto_features=disabled"; fi) \
 	-Dtcg_interpreter=$tcg_interpreter \
diff --git a/meson.build b/meson.build
index 05a67c20d9..cb9420a99e 100644
--- a/meson.build
+++ b/meson.build
@@ -157,6 +157,11 @@ if targetos != 'linux' and get_option('mpath').enabled()
   error('Multipath is supported only on Linux')
 endif
 
+if targetos != 'linux' and get_option('multiprocess').enabled()
+  error('Multiprocess QEMU is supported only on Linux')
+endif
+multiprocess_allowed = targetos == 'linux' and not get_option('multiprocess').disabled()
+
 m = cc.find_library('m', required: false)
 util = cc.find_library('util', required: false)
 winmm = []
@@ -1228,7 +1233,7 @@ host_kconfig = \
   (have_virtfs ? ['CONFIG_VIRTFS=y'] : []) + \
   ('CONFIG_LINUX' in config_host ? ['CONFIG_LINUX=y'] : []) + \
   ('CONFIG_PVRDMA' in config_host ? ['CONFIG_PVRDMA=y'] : []) + \
-  ('CONFIG_MULTIPROCESS_ALLOWED' in config_host ? ['CONFIG_MULTIPROCESS_ALLOWED=y'] : [])
+  (multiprocess_allowed ? ['CONFIG_MULTIPROCESS_ALLOWED=y'] : [])
 
 ignored = [ 'TARGET_XML_FILES', 'TARGET_ABI_DIR', 'TARGET_ARCH' ]
 
@@ -2535,6 +2540,7 @@ endif
 summary_info += {'target list':       ' '.join(target_dirs)}
 if have_system
   summary_info += {'default devices':   get_option('default_devices')}
+  summary_info += {'Multiprocess QEMU': multiprocess_allowed}
 endif
 summary(summary_info, bool_yn: true, section: 'Targets and accelerators')
 
@@ -2655,7 +2661,6 @@ summary_info += {'libpmem support':   config_host.has_key('CONFIG_LIBPMEM')}
 summary_info += {'libdaxctl support': config_host.has_key('CONFIG_LIBDAXCTL')}
 summary_info += {'libudev':           libudev.found()}
 summary_info += {'FUSE lseek':        fuse_lseek.found()}
-summary_info += {'Multiprocess QEMU': config_host.has_key('CONFIG_MULTIPROCESS_ALLOWED')}
 summary(summary_info, bool_yn: true, section: 'Dependencies')
 
 if not supported_cpus.contains(cpu)
diff --git a/meson_options.txt b/meson_options.txt
index 675a9c500a..bf11de7bb2 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -45,6 +45,8 @@ option('cfi', type: 'boolean', value: 'false',
        description: 'Control-Flow Integrity (CFI)')
 option('cfi_debug', type: 'boolean', value: 'false',
        description: 'Verbose errors in case of CFI violation')
+option('multiprocess', type: 'feature', value: 'auto',
+       description: 'Multiprocess QEMU support')
 
 option('attr', type : 'feature', value : 'auto',
        description: 'attr/xattr support')
-- 
2.29.2

Re: [PATCH] multiprocess: move feature to meson_options.txt

[Philippe Mathieu-Daudé]

On 2/24/21 1:23 PM, Paolo Bonzini wrote:
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  configure         | 12 ++++--------
>  meson.build       |  9 +++++++--
>  meson_options.txt |  2 ++
>  3 files changed, 13 insertions(+), 10 deletions(-)
...

> @@ -2535,6 +2540,7 @@ endif
>  summary_info += {'target list':       ' '.join(target_dirs)}
>  if have_system
>    summary_info += {'default devices':   get_option('default_devices')}
> +  summary_info += {'Multiprocess QEMU': multiprocess_allowed}

Since you are changing this, it is a good opportunity to find a
better description to this feature (similarly how we recently clarified
the TCI description).

The current description is confusing with multiprocessing (which is
by default on QEMU and every developer want to exploit that).

So the main multiprocess code resides in hw/remote/mpqemu*.

I have the impression "monolithic application" is common in
software engineering. What about "polylithic QEMU"?

Stefan once described it as "out of (main) process device emulation".

Relevant links:
https://english.stackexchange.com/questions/112633/whats-an-antonym-of-monolithic-as-in-monolithic-architecture/119212#119212
https://infovis-wiki.net/wiki/Polylithic_design

...
>  if not supported_cpus.contains(cpu)
> diff --git a/meson_options.txt b/meson_options.txt
> index 675a9c500a..bf11de7bb2 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -45,6 +45,8 @@ option('cfi', type: 'boolean', value: 'false',
>         description: 'Control-Flow Integrity (CFI)')
>  option('cfi_debug', type: 'boolean', value: 'false',
>         description: 'Verbose errors in case of CFI violation')
> +option('multiprocess', type: 'feature', value: 'auto',
> +       description: 'Multiprocess QEMU support')

Re: [PATCH] multiprocess: move feature to meson_options.txt

[Paolo Bonzini]

On 25/02/21 11:38, Philippe Mathieu-Daudé wrote:
> On 2/24/21 1:23 PM, Paolo Bonzini wrote:
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> ---
>>   configure         | 12 ++++--------
>>   meson.build       |  9 +++++++--
>>   meson_options.txt |  2 ++
>>   3 files changed, 13 insertions(+), 10 deletions(-)
> ...
> 
>> @@ -2535,6 +2540,7 @@ endif
>>   summary_info += {'target list':       ' '.join(target_dirs)}
>>   if have_system
>>     summary_info += {'default devices':   get_option('default_devices')}
>> +  summary_info += {'Multiprocess QEMU': multiprocess_allowed}
> 
> Since you are changing this, it is a good opportunity to find a
> better description to this feature (similarly how we recently clarified
> the TCI description).
> 
> The current description is confusing with multiprocessing (which is
> by default on QEMU and every developer want to exploit that).
> 
> So the main multiprocess code resides in hw/remote/mpqemu*.
> 
> I have the impression "monolithic application" is common in
> software engineering. What about "polylithic QEMU"?
> 
> Stefan once described it as "out of (main) process device emulation".

Out of process emulation?

Paolo

> Relevant links:
> https://english.stackexchange.com/questions/112633/whats-an-antonym-of-monolithic-as-in-monolithic-architecture/119212#119212
> https://infovis-wiki.net/wiki/Polylithic_design
> 
> ...
>>   if not supported_cpus.contains(cpu)
>> diff --git a/meson_options.txt b/meson_options.txt
>> index 675a9c500a..bf11de7bb2 100644
>> --- a/meson_options.txt
>> +++ b/meson_options.txt
>> @@ -45,6 +45,8 @@ option('cfi', type: 'boolean', value: 'false',
>>          description: 'Control-Flow Integrity (CFI)')
>>   option('cfi_debug', type: 'boolean', value: 'false',
>>          description: 'Verbose errors in case of CFI violation')
>> +option('multiprocess', type: 'feature', value: 'auto',
>> +       description: 'Multiprocess QEMU support')
> 

Re: [PATCH] multiprocess: move feature to meson_options.txt

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1478 bytes --]

On Thu, Feb 25, 2021 at 01:15:53PM +0100, Paolo Bonzini wrote:
> On 25/02/21 11:38, Philippe Mathieu-Daudé wrote:
> > On 2/24/21 1:23 PM, Paolo Bonzini wrote:
> > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > > ---
> > >   configure         | 12 ++++--------
> > >   meson.build       |  9 +++++++--
> > >   meson_options.txt |  2 ++
> > >   3 files changed, 13 insertions(+), 10 deletions(-)
> > ...
> > 
> > > @@ -2535,6 +2540,7 @@ endif
> > >   summary_info += {'target list':       ' '.join(target_dirs)}
> > >   if have_system
> > >     summary_info += {'default devices':   get_option('default_devices')}
> > > +  summary_info += {'Multiprocess QEMU': multiprocess_allowed}
> > 
> > Since you are changing this, it is a good opportunity to find a
> > better description to this feature (similarly how we recently clarified
> > the TCI description).
> > 
> > The current description is confusing with multiprocessing (which is
> > by default on QEMU and every developer want to exploit that).
> > 
> > So the main multiprocess code resides in hw/remote/mpqemu*.
> > 
> > I have the impression "monolithic application" is common in
> > software engineering. What about "polylithic QEMU"?
> > 
> > Stefan once described it as "out of (main) process device emulation".
> 
> Out of process emulation?

When Multiprocess QEMU switches to the vfio-user protocol the feature
could be renamed to "vfio-user device backends".

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH] multiprocess: move feature to meson_options.txt

[Jag Raman]

> On Feb 25, 2021, at 11:35 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> 
> On Thu, Feb 25, 2021 at 01:15:53PM +0100, Paolo Bonzini wrote:
>> On 25/02/21 11:38, Philippe Mathieu-Daudé wrote:
>>> On 2/24/21 1:23 PM, Paolo Bonzini wrote:
>>>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>>>> ---
>>>>  configure         | 12 ++++--------
>>>>  meson.build       |  9 +++++++--
>>>>  meson_options.txt |  2 ++
>>>>  3 files changed, 13 insertions(+), 10 deletions(-)
>>> ...
>>> 
>>>> @@ -2535,6 +2540,7 @@ endif
>>>>  summary_info += {'target list':       ' '.join(target_dirs)}
>>>>  if have_system
>>>>    summary_info += {'default devices':   get_option('default_devices')}
>>>> +  summary_info += {'Multiprocess QEMU': multiprocess_allowed}
>>> 
>>> Since you are changing this, it is a good opportunity to find a
>>> better description to this feature (similarly how we recently clarified
>>> the TCI description).
>>> 
>>> The current description is confusing with multiprocessing (which is
>>> by default on QEMU and every developer want to exploit that).
>>> 
>>> So the main multiprocess code resides in hw/remote/mpqemu*.
>>> 
>>> I have the impression "monolithic application" is common in
>>> software engineering. What about "polylithic QEMU"?
>>> 
>>> Stefan once described it as "out of (main) process device emulation".
>> 
>> Out of process emulation?
> 
> When Multiprocess QEMU switches to the vfio-user protocol the feature
> could be renamed to "vfio-user device backends".

I personally don’t have any preference for the name.

Like Stefan pointed out, this would be merged with the
vfio-user model for emulating devices in a separate
process. We could probably rename this during that change.

Thank you very much!
—
Jag

> 
> Stefan

Re: [PATCH] multiprocess: move feature to meson_options.txt

[Philippe Mathieu-Daudé]

On 2/25/21 6:50 PM, Jag Raman wrote:
> 
> 
>> On Feb 25, 2021, at 11:35 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>>
>> On Thu, Feb 25, 2021 at 01:15:53PM +0100, Paolo Bonzini wrote:
>>> On 25/02/21 11:38, Philippe Mathieu-Daudé wrote:
>>>> On 2/24/21 1:23 PM, Paolo Bonzini wrote:
>>>>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>>>>> ---
>>>>>  configure         | 12 ++++--------
>>>>>  meson.build       |  9 +++++++--
>>>>>  meson_options.txt |  2 ++
>>>>>  3 files changed, 13 insertions(+), 10 deletions(-)
>>>> ...
>>>>
>>>>> @@ -2535,6 +2540,7 @@ endif
>>>>>  summary_info += {'target list':       ' '.join(target_dirs)}
>>>>>  if have_system
>>>>>    summary_info += {'default devices':   get_option('default_devices')}
>>>>> +  summary_info += {'Multiprocess QEMU': multiprocess_allowed}
>>>>
>>>> Since you are changing this, it is a good opportunity to find a
>>>> better description to this feature (similarly how we recently clarified
>>>> the TCI description).
>>>>
>>>> The current description is confusing with multiprocessing (which is
>>>> by default on QEMU and every developer want to exploit that).
>>>>
>>>> So the main multiprocess code resides in hw/remote/mpqemu*.
>>>>
>>>> I have the impression "monolithic application" is common in
>>>> software engineering. What about "polylithic QEMU"?
>>>>
>>>> Stefan once described it as "out of (main) process device emulation".
>>>
>>> Out of process emulation?
>>
>> When Multiprocess QEMU switches to the vfio-user protocol the feature
>> could be renamed to "vfio-user device backends".
> 
> I personally don’t have any preference for the name.

Great.

So with the summary/description updated as:

summary_info += {'Multiprocess QEMU (vfio-user device backends)':
multiprocess_allowed}

option('multiprocess', type: 'feature', value: 'auto',
       description: 'Multiprocess QEMU (vfio-user device backends) support')

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

> 
> Like Stefan pointed out, this would be merged with the
> vfio-user model for emulating devices in a separate
> process. We could probably rename this during that change.
> 
> Thank you very much!
> —
> Jag
> 
>>
>> Stefan
> 

Re: [PATCH] multiprocess: move feature to meson_options.txt

[Paolo Bonzini]

On 26/02/21 00:16, Philippe Mathieu-Daudé wrote:
>> I personally don’t have any preference for the name.
> Great.
> 
> So with the summary/description updated as:
> 
> summary_info += {'Multiprocess QEMU (vfio-user device backends)':
> multiprocess_allowed}
> 
> option('multiprocess', type: 'feature', value: 'auto',
>         description: 'Multiprocess QEMU (vfio-user device backends) support')
> 
> Reviewed-by: Philippe Mathieu-Daudé<philmd@redhat.com>
> 

It's not yet vfio-user.  For now I can put "out of process device 
emulation"; however, if the protocol is going to change, I wonder if it 
should be disabled by default.

Paolo

Re: [PATCH] multiprocess: move feature to meson_options.txt

[Philippe Mathieu-Daudé]

On 2/26/21 8:48 AM, Paolo Bonzini wrote:
> On 26/02/21 00:16, Philippe Mathieu-Daudé wrote:
>>> I personally don’t have any preference for the name.
>> Great.
>>
>> So with the summary/description updated as:
>>
>> summary_info += {'Multiprocess QEMU (vfio-user device backends)':
>> multiprocess_allowed}
>>
>> option('multiprocess', type: 'feature', value: 'auto',
>>         description: 'Multiprocess QEMU (vfio-user device backends)
>> support')
>>
>> Reviewed-by: Philippe Mathieu-Daudé<philmd@redhat.com>
>>
> 
> It's not yet vfio-user.  For now I can put "out of process device
> emulation";

OK.

> however, if the protocol is going to change, I wonder if it
> should be disabled by default.

Sounds safer indeed. We need to add --enable-multiprocess in CI to
keep testing the feature.

Re: [PATCH] multiprocess: move feature to meson_options.txt

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1266 bytes --]

On Fri, Feb 26, 2021 at 09:50:59AM +0100, Philippe Mathieu-Daudé wrote:
> On 2/26/21 8:48 AM, Paolo Bonzini wrote:
> > On 26/02/21 00:16, Philippe Mathieu-Daudé wrote:
> >>> I personally don’t have any preference for the name.
> >> Great.
> >>
> >> So with the summary/description updated as:
> >>
> >> summary_info += {'Multiprocess QEMU (vfio-user device backends)':
> >> multiprocess_allowed}
> >>
> >> option('multiprocess', type: 'feature', value: 'auto',
> >>         description: 'Multiprocess QEMU (vfio-user device backends)
> >> support')
> >>
> >> Reviewed-by: Philippe Mathieu-Daudé<philmd@redhat.com>
> >>
> > 
> > It's not yet vfio-user.  For now I can put "out of process device
> > emulation";
> 
> OK.
> 
> > however, if the protocol is going to change, I wonder if it
> > should be disabled by default.
> 
> Sounds safer indeed. We need to add --enable-multiprocess in CI to
> keep testing the feature.

Package maintainers tend to disable optional features explicitly, while
developers and CIs may not notice new features that are disabled by
default.

In the interest of preventing bitrot and catching failures early (before
CI!), I suggest leaving it enabled for maximum build coverage.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]