* Re: [RFC] Unicore Subproject Proposal
[not found] <D5D6EB26.39481%felipe.huici@neclab.eu-0>
@ 2017-09-07 13:41 ` Lars Kurth
2017-09-07 21:16 ` Stefano Stabellini
[not found] ` <alpine.DEB.2.10.1709071414310.4600@sstabellini-ThinkPad-X260>
2017-09-20 0:20 ` Lars Kurth
[not found] ` <776E12BC-8D5A-4B90-AF99-BFDDEBEECCE4@citrix.com>
2 siblings, 2 replies; 25+ messages in thread
From: Lars Kurth @ 2017-09-07 13:41 UTC (permalink / raw)
To: Felipe Huici, committers, xen-devel, xen-api, mirageos-devel,
minios-devel
Cc: julian.chesterfield, rshaposhnik, Saverio Niccolini, stefano,
alexander.dubinin, julien.grall, Simon Kuenzer,
volodymyr_babchuk
Hi all,
there is a technical issue which still needs resolving: we need a Sponsor. I am thinking of Wei – he would qualify as a Hypervisor Leadership team member and it would have the benefit of making sure that the MiniOS angle is covered. I asked Wei, and he will get back to us once he read the proposal.
I also want to highlight this proposal at the next AB board meeting, Sept 19th. It would be good, if most feedback could be given in the next week, such that a) we have time to make mods, b) I have a good baseline to share with the AB. I would need to share an updated proposal on the 18th at the latest.
Technically, the subproject does not need AB approval, as there is no financial impact, but it is always good to have it.
Regards
Lars
On 07/09/2017, 11:26, "Felipe Huici" <Felipe.Huici@neclab.eu> wrote:
Dear all,
Following up on discussions that Simon Kuenzer had with several of you at
the last Xen summit, we’re now submitting a Xen subproject proposal based
on our Unicore work. Could you please review it?
Thanks,
Felipe Huici & Simon Kuenzer - NEC Labs Heidelberg.
PROPOSAL: Unicore
=================
Roles
-----
Project Leads: Simon Kuenzer <simon.kuenzer@neclab.eu> (main lead)
Felipe Huici <felipe.huici@neclab.eu> (co-lead)
Florian Schmidt <florian.schmidt@neclab.eu> (co-lead)
Project Mentor: Lars Kurth <lars.kurth@citrix.com>
Project Sponsor: -To be found-
Background
----------
In recent years, several papers and projects dedicated to unikernels have
shown the immense potential for performance gains that these have. By
leveraging specialization and the use of minimalistic OSes, unikernels are
able to yield impressive numbers, including fast instantiation times (tens
of milliseconds or less), tiny memory footprints (a few MBs or even KBs),
high network throughput (10-40 Gb/s), and high consolidation (e.g., being
able to run thousands of instances on a single commodity server), not to
mention a reduced attack surface and the potential for easier
certification. Unikernel projects worthy of mention include MirageOS,
ClickOS, Erlang on Xen, OSv, HALVM, and Minicache, among others.
The fundamental drawback of unikernels is that they require that
applications be manually ported to the underlying minimalistic OS (e.g.
having to port nginx, snort, mysql or memcached to MiniOS or OSv); this
requires both expert work and often considerable amount of time. In
essence, we need to pick between either high performance with unikernels,
or no porting effort but decreased performance and decreased efficiency
with standard OS/VM images. The goal of this proposal is to change this
status quo by providing a highly configurable unikernel code base; we call
this base Unicore.
This project also aims to concentrate the various efforts currently going
on in the Xen community regarding minimalistic OSes (essentially different
variants of MiniOS). We think that splitting the community across these
variants is counter-productive and hope that Unicore will provide a common
place for all or most improvements and customizations of minimalistic
OSes. The long term goal is to replace something like MiniOS with a tool
that can automatically build such a minimalistic OS.
Unicore - The "Unikernel Core"
---------------------------------
The high level goal of Unicore is to be able to build unikernels targeted
at specific applications without requiring the time-consuming, expert work
that building such a unikernel requires today. An additional goal (or
hope) of Unicore is that all developers interested in unikernel
development would contribute by supplying libraries rather than working on
independent projects with different code bases as it is done now. The main
idea behind Unicore is depicted in Figure 1 and consists of two basic
components:
[Attachment: unicore-oneslider.pdf]
Figure 1. Unicore architecture.
Library pools would contain libraries that the user of Unicore can select
from to create the unikernel. From the bottom up, library pools are
organized into (1) the architecture library tool, containing libraries
specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the
platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no
virtualization) and user-space Linux; and (3) the main library pool,
containing a rich set of functionality to build the unikernel from. This
last library includes drivers (both virtual such as netback/netfront and
physical such as ixgbe), filesystems, memory allocators, schedulers,
network stacks, standard libs (e.g. libc, openssl, etc.), runtimes (e.g. a
Python interpreter and debugging and profiling tools. These pools of
libraries constitute a code base for creating unikernels. As shown, a
library can be relatively large (e.g libc) or quite small (a scheduler),
which should allow for a fair amount of customization for the unikernel.
The Unicore build tool is in charge of compiling the application and the
selected libraries together to create a binary for a specific platform and
architecture (e.g., Xen on x86_64). The tool is currently inspired by
Linux’s kconfig system and consists of a set of Makefiles. It allows users
to select libraries, to configure them, and to warn them when library
dependencies are not met. In addition, the tool can also simultaneously
generate binaries for multiple platforms.
As an example, imagine a user wanting to generate a network driver domain
unikernel. In this case, we would assume the “application” to be the
netback driver. To select this application, the user would first run “make
menuconfig” from within the netback application folder. The Makefile there
would set a variable to indicate what the application is, and would
include the main Unicore Makefiles so that the unikernel can be built
(Step 1 in the figure). Using the menu-based system, the user chooses the
relevant libraries; for a Xen driver domain this would include a physical
network driver, the netback driver, the libxenplat library and a library
from the architecture library pool such as libx86_64arch (Step 2 in the
figure). With this in place, the user saves the configuration and types
“make” to build the unikernel (Step 3) and xl create to run it (Step 4).
A note on the ABI/API: because Unicore allows for customization of the
unikernels, the ABI (or API since there is no kernel) would be custom,
that is, defined by the libraries the user selected. Having said that, it
would be perfectly possible, for instance, to build POSIX-compliant
unikernels with it.
Relevance to Xen and its Community
-----------------------------------
Unikernels are important to a number of areas relevant to the Xen
community, including IoT, automotive, stub domains and driver domain
disaggregation. Unicore could help boost the progress in all of these
areas by quickly providing the necessary tools to create unikernels for
them. For instance, for a driver domain, the user would include the
“library” containing the relevant hardware driver and corresponding
back-end driver, and in principle Unicore would take care of the rest.
In addition, Unicore could eventually replace Mini-OS, providing a
cleaner, more stable and flexible base from which to build unikernels for
projects (the modularization of Mini-OS is in fact already taking place).
Current Status
--------------
Unicore is at an early stage. For now it includes some base libraries with
code extracted from Mini-OS as well as a build tool inspired by
Linux's KConfig system. Unicore is currently able to build "hello world"
unikernels for Xen and Linux user space on x86_64 and ARMv7.
Incubation
----------
The reason behind making Unicore a Xen sub-project project is to (1) bring
the existence of Unicore to the attention of the Xen community
and to outside world; (2) to attempt to harness interest and potentially
development cycles from people and companies interested in
unikernels; and (3) to concentrate maintenance resources from people
interested in unikernels within the community.
License
-------
The main license of the run-time components of Unicore will be a 3-clause
BSD license, unless there is a good reason not to use it (e.g. we may
import 2-clause BSD licensed code from Mini-OS, which we would *not*
anticipate to change). The Makefile system would be licensed under GPL v2
or later as we want to be able to use KConfig functionality from
Buildroot/Linux.
Required Infrastructure
-----------------------
The official repositories should be created on
[http://xenbits.xenproject.org/] under `unicore.git`. There should be a
main repository for the core unicore implementation and additional
repositories for some more advanced extension libraries (e.g., lwIP,
newlib).
### Main repository
`unicore.git`
### Repositories for extension libraries
Repositories for additional libraries that are supported by the Unicore
project should exist under a separate directory:
`unicore-libs/`
For example:
`unicore-libs/lwip.git`
`unicore-libs/newlib.git`
### Mailing list
In the beginning we would use the MiniOS mailing list
(minios-devel@lists.xenproject.org). When we get traction with Unicore we
could consider splitting that traffic onto a unicore mailing list.
============================================================
Dr. Felipe Huici
Chief Researcher, Networked Systems and Data
Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49
(0)6221 4342-241
Fax: +49
(0)6221 4342-155
e-mail:
felipe.huici@neclab.eu
============================================================
NEC Europe Limited Registered Office: NEC House, 1
Victoria Road, London W3 6BL Registered in England 2832014
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
2017-09-07 13:41 ` [RFC] Unicore Subproject Proposal Lars Kurth
@ 2017-09-07 21:16 ` Stefano Stabellini
[not found] ` <alpine.DEB.2.10.1709071414310.4600@sstabellini-ThinkPad-X260>
1 sibling, 0 replies; 25+ messages in thread
From: Stefano Stabellini @ 2017-09-07 21:16 UTC (permalink / raw)
To: Lars Kurth
Cc: Felipe Huici, stefano, julian.chesterfield, rshaposhnik, xen-api,
Saverio Niccolini, minios-devel, alexander.dubinin, julien.grall,
committers, mirageos-devel, xen-devel, Simon Kuenzer,
volodymyr_babchuk
[-- Attachment #1: Type: TEXT/PLAIN, Size: 11273 bytes --]
Hi all,
I would be glad to sponsor this proposal. I think it will be of great
benefit to the ecosystem. Let me know if I need to do anything specific.
Cheers,
Stefano
On Thu, 7 Sep 2017, Lars Kurth wrote:
> Hi all,
>
> there is a technical issue which still needs resolving: we need a Sponsor. I am thinking of Wei – he would qualify as a Hypervisor Leadership team member and it would have the benefit of making sure that the MiniOS angle is covered. I asked Wei, and he will get back to us once he read the proposal.
>
> I also want to highlight this proposal at the next AB board meeting, Sept 19th. It would be good, if most feedback could be given in the next week, such that a) we have time to make mods, b) I have a good baseline to share with the AB. I would need to share an updated proposal on the 18th at the latest.
>
> Technically, the subproject does not need AB approval, as there is no financial impact, but it is always good to have it.
>
> Regards
> Lars
>
> On 07/09/2017, 11:26, "Felipe Huici" <Felipe.Huici@neclab.eu> wrote:
>
> Dear all,
>
> Following up on discussions that Simon Kuenzer had with several of you at
> the last Xen summit, we’re now submitting a Xen subproject proposal based
> on our Unicore work. Could you please review it?
>
> Thanks,
>
> Felipe Huici & Simon Kuenzer - NEC Labs Heidelberg.
>
>
> PROPOSAL: Unicore
> =================
>
> Roles
> -----
> Project Leads: Simon Kuenzer <simon.kuenzer@neclab.eu> (main lead)
> Felipe Huici <felipe.huici@neclab.eu> (co-lead)
> Florian Schmidt <florian.schmidt@neclab.eu> (co-lead)
> Project Mentor: Lars Kurth <lars.kurth@citrix.com>
> Project Sponsor: -To be found-
>
> Background
> ----------
> In recent years, several papers and projects dedicated to unikernels have
> shown the immense potential for performance gains that these have. By
> leveraging specialization and the use of minimalistic OSes, unikernels are
> able to yield impressive numbers, including fast instantiation times (tens
> of milliseconds or less), tiny memory footprints (a few MBs or even KBs),
> high network throughput (10-40 Gb/s), and high consolidation (e.g., being
> able to run thousands of instances on a single commodity server), not to
> mention a reduced attack surface and the potential for easier
> certification. Unikernel projects worthy of mention include MirageOS,
> ClickOS, Erlang on Xen, OSv, HALVM, and Minicache, among others.
>
> The fundamental drawback of unikernels is that they require that
> applications be manually ported to the underlying minimalistic OS (e.g.
> having to port nginx, snort, mysql or memcached to MiniOS or OSv); this
> requires both expert work and often considerable amount of time. In
> essence, we need to pick between either high performance with unikernels,
> or no porting effort but decreased performance and decreased efficiency
> with standard OS/VM images. The goal of this proposal is to change this
> status quo by providing a highly configurable unikernel code base; we call
> this base Unicore.
>
> This project also aims to concentrate the various efforts currently going
> on in the Xen community regarding minimalistic OSes (essentially different
> variants of MiniOS). We think that splitting the community across these
> variants is counter-productive and hope that Unicore will provide a common
> place for all or most improvements and customizations of minimalistic
> OSes. The long term goal is to replace something like MiniOS with a tool
> that can automatically build such a minimalistic OS.
>
> Unicore - The "Unikernel Core"
> ---------------------------------
> The high level goal of Unicore is to be able to build unikernels targeted
> at specific applications without requiring the time-consuming, expert work
> that building such a unikernel requires today. An additional goal (or
> hope) of Unicore is that all developers interested in unikernel
> development would contribute by supplying libraries rather than working on
> independent projects with different code bases as it is done now. The main
> idea behind Unicore is depicted in Figure 1 and consists of two basic
> components:
>
>
> [Attachment: unicore-oneslider.pdf]
>
>
> Figure 1. Unicore architecture.
>
>
> Library pools would contain libraries that the user of Unicore can select
> from to create the unikernel. From the bottom up, library pools are
> organized into (1) the architecture library tool, containing libraries
> specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the
> platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no
> virtualization) and user-space Linux; and (3) the main library pool,
> containing a rich set of functionality to build the unikernel from. This
> last library includes drivers (both virtual such as netback/netfront and
> physical such as ixgbe), filesystems, memory allocators, schedulers,
> network stacks, standard libs (e.g. libc, openssl, etc.), runtimes (e.g. a
> Python interpreter and debugging and profiling tools. These pools of
> libraries constitute a code base for creating unikernels. As shown, a
> library can be relatively large (e.g libc) or quite small (a scheduler),
> which should allow for a fair amount of customization for the unikernel.
>
>
> The Unicore build tool is in charge of compiling the application and the
> selected libraries together to create a binary for a specific platform and
> architecture (e.g., Xen on x86_64). The tool is currently inspired by
> Linux’s kconfig system and consists of a set of Makefiles. It allows users
> to select libraries, to configure them, and to warn them when library
> dependencies are not met. In addition, the tool can also simultaneously
> generate binaries for multiple platforms.
>
>
> As an example, imagine a user wanting to generate a network driver domain
> unikernel. In this case, we would assume the “application” to be the
> netback driver. To select this application, the user would first run “make
> menuconfig” from within the netback application folder. The Makefile there
> would set a variable to indicate what the application is, and would
> include the main Unicore Makefiles so that the unikernel can be built
> (Step 1 in the figure). Using the menu-based system, the user chooses the
> relevant libraries; for a Xen driver domain this would include a physical
> network driver, the netback driver, the libxenplat library and a library
> from the architecture library pool such as libx86_64arch (Step 2 in the
> figure). With this in place, the user saves the configuration and types
> “make” to build the unikernel (Step 3) and xl create to run it (Step 4).
>
>
> A note on the ABI/API: because Unicore allows for customization of the
> unikernels, the ABI (or API since there is no kernel) would be custom,
> that is, defined by the libraries the user selected. Having said that, it
> would be perfectly possible, for instance, to build POSIX-compliant
> unikernels with it.
>
>
> Relevance to Xen and its Community
> -----------------------------------
> Unikernels are important to a number of areas relevant to the Xen
> community, including IoT, automotive, stub domains and driver domain
> disaggregation. Unicore could help boost the progress in all of these
> areas by quickly providing the necessary tools to create unikernels for
> them. For instance, for a driver domain, the user would include the
> “library” containing the relevant hardware driver and corresponding
> back-end driver, and in principle Unicore would take care of the rest.
>
> In addition, Unicore could eventually replace Mini-OS, providing a
> cleaner, more stable and flexible base from which to build unikernels for
> projects (the modularization of Mini-OS is in fact already taking place).
>
>
> Current Status
> --------------
> Unicore is at an early stage. For now it includes some base libraries with
> code extracted from Mini-OS as well as a build tool inspired by
> Linux's KConfig system. Unicore is currently able to build "hello world"
> unikernels for Xen and Linux user space on x86_64 and ARMv7.
>
> Incubation
> ----------
> The reason behind making Unicore a Xen sub-project project is to (1) bring
> the existence of Unicore to the attention of the Xen community
> and to outside world; (2) to attempt to harness interest and potentially
> development cycles from people and companies interested in
> unikernels; and (3) to concentrate maintenance resources from people
> interested in unikernels within the community.
>
> License
> -------
> The main license of the run-time components of Unicore will be a 3-clause
> BSD license, unless there is a good reason not to use it (e.g. we may
> import 2-clause BSD licensed code from Mini-OS, which we would *not*
> anticipate to change). The Makefile system would be licensed under GPL v2
> or later as we want to be able to use KConfig functionality from
> Buildroot/Linux.
>
> Required Infrastructure
> -----------------------
> The official repositories should be created on
> [http://xenbits.xenproject.org/] under `unicore.git`. There should be a
> main repository for the core unicore implementation and additional
> repositories for some more advanced extension libraries (e.g., lwIP,
> newlib).
>
> ### Main repository
>
> `unicore.git`
>
> ### Repositories for extension libraries
>
> Repositories for additional libraries that are supported by the Unicore
> project should exist under a separate directory:
>
> `unicore-libs/`
>
> For example:
>
> `unicore-libs/lwip.git`
> `unicore-libs/newlib.git`
>
> ### Mailing list
>
> In the beginning we would use the MiniOS mailing list
> (minios-devel@lists.xenproject.org). When we get traction with Unicore we
> could consider splitting that traffic onto a unicore mailing list.
>
>
>
>
> ============================================================
> Dr. Felipe Huici
> Chief Researcher, Networked Systems and Data
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49
> (0)6221 4342-241
> Fax: +49
> (0)6221 4342-155
>
> e-mail:
> felipe.huici@neclab.eu
> ============================================================
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
>
>
>
>
>
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
[not found] ` <alpine.DEB.2.10.1709071414310.4600@sstabellini-ThinkPad-X260>
@ 2017-09-08 11:00 ` Lars Kurth
2017-09-08 12:31 ` Felipe Huici
0 siblings, 1 reply; 25+ messages in thread
From: Lars Kurth @ 2017-09-08 11:00 UTC (permalink / raw)
To: Stefano Stabellini, Wei Liu
Cc: Felipe Huici, stefano, julian.chesterfield, rshaposhnik, xen-api,
Saverio Niccolini, minios-devel, alexander.dubinin, julien.grall,
committers, mirageos-devel, xen-devel, Simon Kuenzer,
volodymyr_babchuk
@Wei, @Stefano,
On 07/09/2017, 22:16, "Stefano Stabellini" <sstabellini@kernel.org> wrote:
Hi all,
I would be glad to sponsor this proposal. I think it will be of great
benefit to the ecosystem. Let me know if I need to do anything specific.
Basically, all which is needed is an agreement. Which we have from you both. Felipe, can then add your names to the proposal.
Looking out for the evolving project and helping (e.g. through advice) is not strictly necessary, but always welcome.
Lars
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
2017-09-08 11:00 ` Lars Kurth
@ 2017-09-08 12:31 ` Felipe Huici
2017-09-10 20:48 ` Alexander Dubinin
0 siblings, 1 reply; 25+ messages in thread
From: Felipe Huici @ 2017-09-08 12:31 UTC (permalink / raw)
To: Lars Kurth, Stefano Stabellini, Wei Liu
Cc: stefano, julian.chesterfield, rshaposhnik, xen-api,
Saverio Niccolini, minios-devel, alexander.dubinin, julien.grall,
committers, mirageos-devel, xen-devel, Simon Kuenzer,
volodymyr_babchuk
Hi Wei, Stefano,
Thank you so much for agreeing to be sponsors! I’ll update the document.
— Felipe
============================================================
Dr. Felipe Huici
Chief Researcher, Networked Systems and Data
Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49
(0)6221 4342-241
Fax: +49
(0)6221 4342-155
e-mail:
felipe.huici@neclab.eu
============================================================
NEC Europe Limited Registered Office: NEC House, 1
Victoria Road, London W3 6BL Registered in England 2832014
On 9/8/17, 1:00 PM, "Lars Kurth" <lars.kurth@citrix.com> wrote:
>@Wei, @Stefano,
>
>On 07/09/2017, 22:16, "Stefano Stabellini" <sstabellini@kernel.org> wrote:
>
> Hi all,
>
> I would be glad to sponsor this proposal. I think it will be of great
> benefit to the ecosystem. Let me know if I need to do anything
>specific.
>
>Basically, all which is needed is an agreement. Which we have from you
>both. Felipe, can then add your names to the proposal.
>
>Looking out for the evolving project and helping (e.g. through advice) is
>not strictly necessary, but always welcome.
>
>Lars
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
2017-09-08 12:31 ` Felipe Huici
@ 2017-09-10 20:48 ` Alexander Dubinin
2017-09-11 12:08 ` Simon Kuenzer
0 siblings, 1 reply; 25+ messages in thread
From: Alexander Dubinin @ 2017-09-10 20:48 UTC (permalink / raw)
To: Felipe Huici
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, julian.chesterfield,
rshaposhnik, xen-api, Saverio Niccolini, minios-devel,
julien.grall, committers, mirageos-devel, stefano, xen-devel,
Simon Kuenzer, volodymyr_babchuk
[-- Attachment #1.1: Type: text/plain, Size: 2253 bytes --]
Hi Felipe, all,
Great that it's going to start :) Looking forward to join :)
Just my 2 cents:
1. Is this academic project, or it have specific goals and areas of
application? Would be good to have some practical use-cases and well
formulated list of problems (we all feel these by guts, but...), it aiming
to solve. IMHO that will help to prioritize functionality and get usable
result faster :)
2. Does any security subsystem planned? XEN have XSM/FLASK, but IMHO is
should be supplemented by some security layer in control/stub domains as
well. So far only known implementation is OpenXT, but it is.... very
specific. Probably some generalized security layer needed in Unicore to
supplement FLASK/XSM... Correct me please, if I misunderstanding :)
Regards,
Alexander
On Fri, Sep 8, 2017 at 3:31 PM, Felipe Huici <Felipe.Huici@neclab.eu> wrote:
> Hi Wei, Stefano,
>
> Thank you so much for agreeing to be sponsors! I’ll update the document.
>
> — Felipe
>
> ============================================================
> Dr. Felipe Huici
> Chief Researcher, Networked Systems and Data
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49
> (0)6221 4342-241
> Fax: +49
> (0)6221 4342-155
>
> e-mail:
> felipe.huici@neclab.eu
> ============================================================
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
>
>
>
>
> On 9/8/17, 1:00 PM, "Lars Kurth" <lars.kurth@citrix.com> wrote:
>
> >@Wei, @Stefano,
> >
> >On 07/09/2017, 22:16, "Stefano Stabellini" <sstabellini@kernel.org>
> wrote:
> >
> > Hi all,
> >
> > I would be glad to sponsor this proposal. I think it will be of great
> > benefit to the ecosystem. Let me know if I need to do anything
> >specific.
> >
> >Basically, all which is needed is an agreement. Which we have from you
> >both. Felipe, can then add your names to the proposal.
> >
> >Looking out for the evolving project and helping (e.g. through advice) is
> >not strictly necessary, but always welcome.
> >
> >Lars
> >
>
>
--
Regards,
Alexander Dubinin
[-- Attachment #1.2: Type: text/html, Size: 3240 bytes --]
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
2017-09-10 20:48 ` Alexander Dubinin
@ 2017-09-11 12:08 ` Simon Kuenzer
2017-09-13 8:36 ` Alexander Dubinin
` (4 more replies)
0 siblings, 5 replies; 25+ messages in thread
From: Simon Kuenzer @ 2017-09-11 12:08 UTC (permalink / raw)
To: Alexander Dubinin, Felipe Huici
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, julian.chesterfield,
rshaposhnik, xen-api, Saverio Niccolini, minios-devel,
julien.grall, committers, mirageos-devel, stefano, xen-devel,
volodymyr_babchuk
Hi Alexander,
thanks a lot for your review.
On 10.09.2017 22:48, Alexander Dubinin wrote:
> Hi Felipe, all,
>
> Great that it's going to start :) Looking forward to join :)
I am looking forward to your contributions. ;)
>
> Just my 2 cents:
>
> 1. Is this academic project, or it have specific goals and areas of
> application? Would be good to have some practical use-cases and well
> formulated list of problems (we all feel these by guts, but...), it
> aiming to solve. IMHO that will help to prioritize functionality and get
> usable result faster :)
It is kind of both, however we aim a strong focus on real world
problems: IoT, Mobile Edge Computing (MEC), Automotive, Virtual Network
Functions (VNFs), and others.
We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and
others) and tried to apply them in the several areas. While doing this,
we noticed that each area benefits differently from the properties that
Unikernels give - which is great (e.g., instant boot times for MEC, high
performance for NFV, resource efficiency for IoT). However, building and
maintaining new Unikernels (as we did with ClickOS, MiniCache, and
Minipython) is currently painful.
Because of different focuses on properties and ported/implemented
applications, most Unikernel today are bound to their own OS layers
(e.g., ClickOS uses a different Mini-OS than Mirage). Each application
requires a different subset of OS layers but also enables different
optimizations of them.
In order to solve this, we came up with the Unicore proposal. But I
agree with your suggestion at this point: It helps for the project start
to focus on some initial areas. For now, I hope this is driven by the
first contributors, and I have personally IoT in mind. Since the project
goal is so ambitious, we should keep the long-term goal in mind from the
beginning.
>
> 2. Does any security subsystem planned? XEN have XSM/FLASK, but IMHO is
> should be supplemented by some security layer in control/stub domains as
> well. So far only known implementation is OpenXT, but it is.... very
> specific. Probably some generalized security layer needed in Unicore to
> supplement FLASK/XSM... Correct me please, if I misunderstanding :)
I agree that many projects (especially embedded, stubdomains, driver
domains, NFV) have a vested interest in security and isolation. In my
view, XSM/FLASK further restricts what a VM can do and sounds kind of
orthogonal to the functionality of a VM (am I right?). The fact that
Unikernels should only pick components that are actually required to do
the job reduces the attack surface compared to general purpose OSes.
Do you see further value with FLASK/XSM which requires early
implementation and design decisions for Unicore? As far as I can tell
something like Flask is implemented mostly in the hypervisor and
toolstack, not in the guests themselves, is this right?
Thanks,
Simon
>
> Regards,
> Alexander
>
> On Fri, Sep 8, 2017 at 3:31 PM, Felipe Huici <Felipe.Huici@neclab.eu
> <mailto:Felipe.Huici@neclab.eu>> wrote:
>
> Hi Wei, Stefano,
>
> Thank you so much for agreeing to be sponsors! I’ll update the document.
>
> — Felipe
>
> ============================================================
> Dr. Felipe Huici
> Chief Researcher, Networked Systems and Data
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49
> (0)6221 4342-241
> Fax: +49
> (0)6221 4342-155
>
> e-mail:
> felipe.huici@neclab.eu <mailto:felipe.huici@neclab.eu>
> ============================================================
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
>
>
>
>
> On 9/8/17, 1:00 PM, "Lars Kurth" <lars.kurth@citrix.com
> <mailto:lars.kurth@citrix.com>> wrote:
>
> >@Wei, @Stefano,
> >
> >On 07/09/2017, 22:16, "Stefano Stabellini" <sstabellini@kernel.org
> <mailto:sstabellini@kernel.org>> wrote:
> >
> > Hi all,
> >
> > I would be glad to sponsor this proposal. I think it will be
> of great
> > benefit to the ecosystem. Let me know if I need to do anything
> >specific.
> >
> >Basically, all which is needed is an agreement. Which we have from you
> >both. Felipe, can then add your names to the proposal.
> >
> >Looking out for the evolving project and helping (e.g. through
> advice) is
> >not strictly necessary, but always welcome.
> >
> >Lars
> >
>
>
>
>
> --
> Regards,
> Alexander Dubinin
--
============================================================
Simon Kuenzer
シモン クゥンツァー
Research Scientist,
Networked Systems and Data Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49 (0)6221 4342-264
Fax: +49 (0)6221 4342-5264
e-mail: simon.kuenzer@neclab.eu
============================================================
NEC Europe Ltd | Registered Office: Athene, Odyssey
Business Park, West End Road, London, HA4 6QE, GB
Registered in England 2832014
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
2017-09-11 12:08 ` Simon Kuenzer
@ 2017-09-13 8:36 ` Alexander Dubinin
2017-09-15 13:01 ` Simon Kuenzer
2017-09-13 10:11 ` [Xen-API] " Anil Madhavapeddy
` (3 subsequent siblings)
4 siblings, 1 reply; 25+ messages in thread
From: Alexander Dubinin @ 2017-09-13 8:36 UTC (permalink / raw)
To: Simon Kuenzer
Cc: Felipe Huici, Lars Kurth, Stefano Stabellini, Wei Liu,
julian.chesterfield, rshaposhnik, xen-api, Saverio Niccolini,
minios-devel, julien.grall, committers, mirageos-devel, stefano,
xen-devel, volodymyr_babchuk
[-- Attachment #1.1: Type: text/plain, Size: 7475 bytes --]
Hello Simon, all,
1. Is this academic project, or it have specific goals and areas of
>> application? Would be good to have some practical use-cases and well
>> formulated list of problems (we all feel these by guts, but...), it aiming
>> to solve. IMHO that will help to prioritize functionality and get usable
>> result faster :)
>>
>
> It is kind of both, however we aim a strong focus on real world problems:
> IoT, Mobile Edge Computing (MEC), Automotive, Virtual Network Functions
> (VNFs), and others.
> We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and
> others) and tried to apply them in the several areas. While doing this, we
> noticed that each area benefits differently from the properties that
> Unikernels give - which is great (e.g., instant boot times for MEC, high
> performance for NFV, resource efficiency for IoT). However, building and
> maintaining new Unikernels (as we did with ClickOS, MiniCache, and
> Minipython) is currently painful.
> Because of different focuses on properties and ported/implemented
> applications, most Unikernel today are bound to their own OS layers (e.g.,
> ClickOS uses a different Mini-OS than Mirage). Each application requires a
> different subset of OS layers but also enables different optimizations of
> them.
>
> In order to solve this, we came up with the Unicore proposal. But I agree
> with your suggestion at this point: It helps for the project start to focus
> on some initial areas. For now, I hope this is driven by the first
> contributors, and I have personally IoT in mind. Since the project goal is
> so ambitious, we should keep the long-term goal in mind from the beginning.
Yes, that's exactly what I meant :)
And IMHO it would be good to not have very abstract goals - and you named
first real one - IoT.
Do you have real-life use-case with real-life hardware to implement within
this IoT?
For example, popular IoT devkit, people can use and join this efforts? And
real, useful application for it?
My interest here is mostly disaggregation and security - to have minimal,
but still functional service domains, built strictly for specific
functionality.
So far we (team, I working with) are using BuildRoot to create
Dom0/stubdoms/driverdoms/etc. based on Linux (yet).
In our case (at least, right now) guest systems are heavy
VMs(Windows/Linux/*BSD/QNX) with GPU passthrough (And not only GPU, but
other controllers, test boards, etc.).
Hardware domains most likely to be based on the OS-es, which have proven
and stable hardware support base (Linux, *BSD), but there are still need in
service domains - like TUI(Text UI) domain, where users are interacting
with host, stub-domain, dom0, which starting hardware domains, etc.
So, that could be one more goal - minimalistic service domains for x86/64
platform.
Another goal would be virtualization in Automated Control Systems area -
but it's too early (for me, at least) to talk about it yet.
Does anybody have other _specific_ targets for Unicore in mind?
>> 2. Does any security subsystem planned? XEN have XSM/FLASK, but IMHO is
>> should be supplemented by some security layer in control/stub domains as
>> well. So far only known implementation is OpenXT, but it is.... very
>> specific. Probably some generalized security layer needed in Unicore to
>> supplement FLASK/XSM... Correct me please, if I misunderstanding :)
>>
>
> I agree that many projects (especially embedded, stubdomains, driver
> domains, NFV) have a vested interest in security and isolation. In my view,
> XSM/FLASK further restricts what a VM can do and sounds kind of orthogonal
> to the functionality of a VM (am I right?). The fact that Unikernels should
> only pick components that are actually required to do the job reduces the
> attack surface compared to general purpose OSes.
> Do you see further value with FLASK/XSM which requires early
> implementation and design decisions for Unicore? As far as I can tell
> something like Flask is implemented mostly in the hypervisor and toolstack,
> not in the guests themselves, is this right?
>
> Yes, if Unicore is not supposed to be used as Dom0, and if we are
considering Unicore domains only as a guests, running in the single
security context, that's fine :)
But if, eventually, it will be used as a control domain in multi-tenant
system, which should manage XSM/FLASK and fill the gap between real
users(and their data) and VMs, restricted by FLASK - it's something to
think about. Maybe just not now :) Or it's not one of Unicore goals even :)
Just dreaming to have absolutely minimal service domains, where every byte
is known and needed :)
Regards,
Alexander
>
> Thanks,
>
> Simon
>
>
>> Regards,
>> Alexander
>>
>> On Fri, Sep 8, 2017 at 3:31 PM, Felipe Huici <Felipe.Huici@neclab.eu
>> <mailto:Felipe.Huici@neclab.eu>> wrote:
>>
>> Hi Wei, Stefano,
>>
>> Thank you so much for agreeing to be sponsors! I’ll update the
>> document.
>>
>> — Felipe
>>
>> ============================================================
>> Dr. Felipe Huici
>> Chief Researcher, Networked Systems and Da
>> <https://maps.google.com/?q=orked+Systems+and+Da&entry=gmail&source=g>ta
>> Analytics Group
>> NEC Laboratories Europe, Network Research Division
>> Kurfuerstenanlage 36, D-69115 Heidelberg
>> Tel. +49
>> (0)6221 4342-241
>> Fax: +49
>> (0)6221 4342-155
>>
>> e-mail:
>> felipe.huici@neclab.eu <mailto:felipe.huici@neclab.eu>
>> ============================================================
>> NEC Europe Limited Registered Office: NEC House, 1
>> Victoria Road, London W3 6BL Registered in England 2832014
>>
>>
>>
>>
>> On 9/8/17, 1:00 PM, "Lars Kurth" <lars.kurth@citrix.com
>> <mailto:lars.kurth@citrix.com>> wrote:
>>
>> >@Wei, @Stefano,
>> >
>> >On 07/09/2017, 22:16, "Stefano Stabellini" <sstabellini@kernel.org
>> <mailto:sstabellini@kernel.org>> wrote:
>> >
>> > Hi all,
>> >
>> > I would be glad to sponsor this proposal. I think it will be
>> of great
>> > benefit to the ecosystem. Let me know if I need to do anything
>> >specific.
>> >
>> >Basically, all which is needed is an agreement. Which we have from
>> you
>> >both. Felipe, can then add your names to the proposal.
>> >
>> >Looking out for the evolving project and helping (e.g. through
>> advice) is
>> >not strictly necessary,
>> <https://maps.google.com/?q=strictly+necessary,&entry=gmail&source=g>but
>> always welcome.
>> >
>> >Lars
>> >
>>
>>
>>
>>
>> --
>> Regards,
>> Alexander Dubinin
>>
>
> --
> ============================================================
> Simon Kuenzer
> シモン クゥンツァー
> Research Scientist,
> Networked Systems and Data Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49 (0)6221 4342-264
> Fax: +49 (0)6221 4342-5264
> e-mail: simon.kuenzer@neclab.eu
> ============================================================
> NEC Europe Ltd | Registered Office: Athene, Odyssey
> Business Park, West End Road, London, HA4 6QE, GB
> Registered in England 2832014
>
--
Regards,
Alexander Dubinin
[-- Attachment #1.2: Type: text/html, Size: 10286 bytes --]
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Xen-API] [RFC] Unicore Subproject Proposal
2017-09-11 12:08 ` Simon Kuenzer
2017-09-13 8:36 ` Alexander Dubinin
@ 2017-09-13 10:11 ` Anil Madhavapeddy
2017-09-13 16:55 ` Lars Kurth
` (2 subsequent siblings)
4 siblings, 0 replies; 25+ messages in thread
From: Anil Madhavapeddy @ 2017-09-13 10:11 UTC (permalink / raw)
To: Simon Kuenzer
Cc: Felipe Huici, julian.chesterfield, Lars Kurth,
Stefano Stabellini, rshaposhnik, xen-api, Saverio Niccolini,
minios-devel, Alexander Dubinin, julien.grall, committers,
mirageos-devel, stefano, xen-devel, volodymyr_babchuk
On 11 Sep 2017, at 13:08, Simon Kuenzer <simon.kuenzer@neclab.eu> wrote:
>
>> Just my 2 cents:
>> 1. Is this academic project, or it have specific goals and areas of application? Would be good to have some practical use-cases and well formulated list of problems (we all feel these by guts, but...), it aiming to solve. IMHO that will help to prioritize functionality and get usable result faster :)
>
> It is kind of both, however we aim a strong focus on real world problems: IoT, Mobile Edge Computing (MEC), Automotive, Virtual Network Functions (VNFs), and others.
> We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and others) and tried to apply them in the several areas. While doing this, we noticed that each area benefits differently from the properties that Unikernels give - which is great (e.g., instant boot times for MEC, high performance for NFV, resource efficiency for IoT). However, building and maintaining new Unikernels (as we did with ClickOS, MiniCache, and Minipython) is currently painful.
> Because of different focuses on properties and ported/implemented applications, most Unikernel today are bound to their own OS layers (e.g., ClickOS uses a different Mini-OS than Mirage). Each application requires a different subset of OS layers but also enables different optimizations of them.
>
> In order to solve this, we came up with the Unicore proposal. But I agree with your suggestion at this point: It helps for the project start to focus on some initial areas. For now, I hope this is driven by the first contributors, and I have personally IoT in mind. Since the project goal is so ambitious, we should keep the long-term goal in mind from the beginning.
>
Thanks very much for kicking off this initiative. Maintaining a forked MiniOS has been a multi-year source of a maintenance burden for MirageOS, and we would love to be more aligned with an upstream version and benefit from new features such as (e.g.) HVM booting.
From a MirageOS perspective, we'd be happy to switch to something that can give us just enough MiniOS for our ocaml-freestanding [1] code to boot on Xen. One requirement from our side is that we need to strip down MiniOS to remove even the C xenstore implementation, since we have pure OCaml gnt, xenstore and device driver implementations. We'd be happy to try out an alpha Unicore and let you know what is in excess to our needs as soon as you have something to publish.
So full support from MirageOS for this initiative!
regards,
Anil
[1] https://github.com/mirage/ocaml-freestanding/
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Minios-devel] [Xen-API] [RFC] Unicore Subproject Proposal
[not found] ` <C8ACDEE5-D1EE-4AFE-B59C-5FBA25623EAA@recoil.org>
@ 2017-09-13 16:13 ` Samuel Thibault
2017-09-13 19:27 ` Felipe Huici
` (3 subsequent siblings)
4 siblings, 0 replies; 25+ messages in thread
From: Samuel Thibault @ 2017-09-13 16:13 UTC (permalink / raw)
To: Anil Madhavapeddy
Cc: Felipe Huici, julian.chesterfield, Lars Kurth,
Stefano Stabellini, rshaposhnik, xen-api, Saverio Niccolini,
minios-devel, Alexander Dubinin, julien.grall, committers,
mirageos-devel, stefano, xen-devel, Simon Kuenzer,
volodymyr_babchuk
Hello,
Anil Madhavapeddy, on mer. 13 sept. 2017 11:11:03 +0100, wrote:
> Maintaining a forked MiniOS has been a multi-year source of a maintenance burden for MirageOS,
I'm just wondering why this happened?
The mini-os repository is open for development, it's just a matter of
agreeing on how to implement features :)
> One requirement from our side is that we need to strip down MiniOS to remove even the C xenstore implementation, since we have pure OCaml gnt, xenstore and device driver implementations.
That could already be a build option in MiniOS. It seems it actually
already is, via $(CONFIG_XENBUS)=n.
If launching UniCore allows to get momentum to achieve that, I'm all for
it, I'm just wondering whether the problem is not actually about putting
a name on it, but just discussing here what is needed, and see how to
implement it all within the same repository. But again, I understand
that putting a name can trigger that discussion and be a "let's do it!"
message :)
Samuel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
2017-09-11 12:08 ` Simon Kuenzer
2017-09-13 8:36 ` Alexander Dubinin
2017-09-13 10:11 ` [Xen-API] " Anil Madhavapeddy
@ 2017-09-13 16:55 ` Lars Kurth
[not found] ` <C8ACDEE5-D1EE-4AFE-B59C-5FBA25623EAA@recoil.org>
[not found] ` <3B0A2B25-6A6A-4DE9-845C-E56812B97F92@citrix.com>
4 siblings, 0 replies; 25+ messages in thread
From: Lars Kurth @ 2017-09-13 16:55 UTC (permalink / raw)
To: Simon Kuenzer, Alexander Dubinin, Felipe Huici
Cc: stefano, Stefano Stabellini, Wei Liu, julian.chesterfield,
rshaposhnik, xen-api, Saverio Niccolini, minios-devel,
julien.grall, committers, mirageos-devel, xen-devel,
volodymyr_babchuk
Simon,
It looks to me as if there is some feedback: so it may make some sense to incorporate some of it and send out a version 2. We may also want to CC some reps from other unikernel projects but Mirage OS. Or you could point them to this thread in a separate mail through respective channels or a hub like unikernel.org. Whatever you think may work best. It’s your call.
And then have a formal vote, a week after v2 of the proposal. Does this work?
Lars
On 11/09/2017, 05:08, "Simon Kuenzer" <simon.kuenzer@neclab.eu> wrote:
Hi Alexander,
thanks a lot for your review.
On 10.09.2017 22:48, Alexander Dubinin wrote:
> Hi Felipe, all,
>
> Great that it's going to start :) Looking forward to join :)
I am looking forward to your contributions. ;)
>
> Just my 2 cents:
>
> 1. Is this academic project, or it have specific goals and areas of
> application? Would be good to have some practical use-cases and well
> formulated list of problems (we all feel these by guts, but...), it
> aiming to solve. IMHO that will help to prioritize functionality and get
> usable result faster :)
It is kind of both, however we aim a strong focus on real world
problems: IoT, Mobile Edge Computing (MEC), Automotive, Virtual Network
Functions (VNFs), and others.
We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and
others) and tried to apply them in the several areas. While doing this,
we noticed that each area benefits differently from the properties that
Unikernels give - which is great (e.g., instant boot times for MEC, high
performance for NFV, resource efficiency for IoT). However, building and
maintaining new Unikernels (as we did with ClickOS, MiniCache, and
Minipython) is currently painful.
Because of different focuses on properties and ported/implemented
applications, most Unikernel today are bound to their own OS layers
(e.g., ClickOS uses a different Mini-OS than Mirage). Each application
requires a different subset of OS layers but also enables different
optimizations of them.
In order to solve this, we came up with the Unicore proposal. But I
agree with your suggestion at this point: It helps for the project start
to focus on some initial areas. For now, I hope this is driven by the
first contributors, and I have personally IoT in mind. Since the project
goal is so ambitious, we should keep the long-term goal in mind from the
beginning.
>
> 2. Does any security subsystem planned? XEN have XSM/FLASK, but IMHO is
> should be supplemented by some security layer in control/stub domains as
> well. So far only known implementation is OpenXT, but it is.... very
> specific. Probably some generalized security layer needed in Unicore to
> supplement FLASK/XSM... Correct me please, if I misunderstanding :)
I agree that many projects (especially embedded, stubdomains, driver
domains, NFV) have a vested interest in security and isolation. In my
view, XSM/FLASK further restricts what a VM can do and sounds kind of
orthogonal to the functionality of a VM (am I right?). The fact that
Unikernels should only pick components that are actually required to do
the job reduces the attack surface compared to general purpose OSes.
Do you see further value with FLASK/XSM which requires early
implementation and design decisions for Unicore? As far as I can tell
something like Flask is implemented mostly in the hypervisor and
toolstack, not in the guests themselves, is this right?
Thanks,
Simon
>
> Regards,
> Alexander
>
> On Fri, Sep 8, 2017 at 3:31 PM, Felipe Huici <Felipe.Huici@neclab.eu
> <mailto:Felipe.Huici@neclab.eu>> wrote:
>
> Hi Wei, Stefano,
>
> Thank you so much for agreeing to be sponsors! I’ll update the document.
>
> — Felipe
>
> ============================================================
> Dr. Felipe Huici
> Chief Researcher, Networked Systems and Data
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49
> (0)6221 4342-241
> Fax: +49
> (0)6221 4342-155
>
> e-mail:
> felipe.huici@neclab.eu <mailto:felipe.huici@neclab.eu>
> ============================================================
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
>
>
>
>
> On 9/8/17, 1:00 PM, "Lars Kurth" <lars.kurth@citrix.com
> <mailto:lars.kurth@citrix.com>> wrote:
>
> >@Wei, @Stefano,
> >
> >On 07/09/2017, 22:16, "Stefano Stabellini" <sstabellini@kernel.org
> <mailto:sstabellini@kernel.org>> wrote:
> >
> > Hi all,
> >
> > I would be glad to sponsor this proposal. I think it will be
> of great
> > benefit to the ecosystem. Let me know if I need to do anything
> >specific.
> >
> >Basically, all which is needed is an agreement. Which we have from you
> >both. Felipe, can then add your names to the proposal.
> >
> >Looking out for the evolving project and helping (e.g. through
> advice) is
> >not strictly necessary, but always welcome.
> >
> >Lars
> >
>
>
>
>
> --
> Regards,
> Alexander Dubinin
--
============================================================
Simon Kuenzer
シモン クゥンツァー
Research Scientist,
Networked Systems and Data Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49 (0)6221 4342-264
Fax: +49 (0)6221 4342-5264
e-mail: simon.kuenzer@neclab.eu
============================================================
NEC Europe Ltd | Registered Office: Athene, Odyssey
Business Park, West End Road, London, HA4 6QE, GB
Registered in England 2832014
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Xen-API] [RFC] Unicore Subproject Proposal
[not found] ` <C8ACDEE5-D1EE-4AFE-B59C-5FBA25623EAA@recoil.org>
2017-09-13 16:13 ` [Minios-devel] [Xen-API] " Samuel Thibault
@ 2017-09-13 19:27 ` Felipe Huici
[not found] ` <20170913161345.fnndedfh74u5k3wh@var.youpi.perso.aquilenet.fr>
` (2 subsequent siblings)
4 siblings, 0 replies; 25+ messages in thread
From: Felipe Huici @ 2017-09-13 19:27 UTC (permalink / raw)
To: Anil Madhavapeddy, Simon Kuenzer
Cc: rshaposhnik, Lars Kurth, Stefano Stabellini, julian.chesterfield,
xen-api, Saverio Niccolini, minios-devel, Alexander Dubinin,
julien.grall, committers, mirageos-devel, stefano, xen-devel,
volodymyr_babchuk
Hi Anil,
>From a MirageOS perspective, we'd be happy to switch to something that
>can give us just enough MiniOS for our ocaml-freestanding [1] code to
>boot on Xen. One requirement from our side is that we need to strip down
>MiniOS to remove even the C xenstore implementation, since we have pure
>OCaml gnt, xenstore and device driver implementations.
This sounds like a good, reasonable target, and it wouldn’t be the first
time we create guests without xenstore code (as presented by team members
in the last Xen summit and as explained in an upcoming SOSP paper).
> We'd be happy to try out an alpha Unicore and let you know what is in
>excess to our needs as soon as you have something to publish.
That would be great.
>So full support from MirageOS for this initiative!
Thanks for the support!
— Felipe
>regards,
>Anil
>
>[1] https://github.com/mirage/ocaml-freestanding/
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xen.org
>https://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [MirageOS-devel] [Minios-devel] [Xen-API] [RFC] Unicore Subproject Proposal
[not found] ` <20170913161345.fnndedfh74u5k3wh@var.youpi.perso.aquilenet.fr>
@ 2017-09-14 14:38 ` Anil Madhavapeddy
2017-09-14 14:47 ` [Minios-devel] [MirageOS-devel] " Samuel Thibault
[not found] ` <20170914144756.jorfjgwarbmpwrhq@var.youpi.perso.aquilenet.fr>
0 siblings, 2 replies; 25+ messages in thread
From: Anil Madhavapeddy @ 2017-09-14 14:38 UTC (permalink / raw)
To: Samuel Thibault
Cc: Felipe Huici, Lars Kurth, Stefano Stabellini, rshaposhnik,
xen-api, Saverio Niccolini, minios-devel, Alexander Dubinin,
julien.grall, committers, mirageos-devel, stefano, xen-devel,
Simon Kuenzer, volodymyr_babchuk
On 13 Sep 2017, at 17:13, Samuel Thibault <samuel.thibault@ens-lyon.org> wrote:
>
> Hello,
>
> Anil Madhavapeddy, on mer. 13 sept. 2017 11:11:03 +0100, wrote:
>> Maintaining a forked MiniOS has been a multi-year source of a maintenance burden for MirageOS,
>
> I'm just wondering why this happened?
>
> The mini-os repository is open for development, it's just a matter of
> agreeing on how to implement features :)
We forked it well before mini-os spun out into a separate repository,
around 5 years ago. And then some features (such as arm32 support)
took around a year to be upstreamed due to multiple iterations of the
patchsets, and we just maintained it in our fork. None of this is
relevant any more in the brave new world of MiniOS being a separate
repository.
>
>> One requirement from our side is that we need to strip down MiniOS to remove even the C xenstore implementation, since we have pure OCaml gnt, xenstore and device driver implementations.
>
> That could already be a build option in MiniOS. It seems it actually
> already is, via $(CONFIG_XENBUS)=n.
>
> If launching UniCore allows to get momentum to achieve that, I'm all for
> it, I'm just wondering whether the problem is not actually about putting
> a name on it, but just discussing here what is needed, and see how to
> implement it all within the same repository. But again, I understand
> that putting a name can trigger that discussion and be a "let's do it!"
> message :)
It's a combination of both I think. I had a very quick look at the latest mini-os
tree and ran into build problems from the master branch
(e.g. just doing make results in:
gcc -fno-builtin -Wall -Werror -Wredundant-decls -Wno-format -Wno-redundant-decls -Wformat -fno-stack-protector -fgnu89-inline -Wstrict-prototypes -Wnested-externs -Wpointer-arith -Winline -g -D__INSIDE_MINIOS__ -m64 -mno-red-zone -fno-reorder-blocks -fno-asynchronous-unwind-tables -DCONFIG_PARAVIRT -DCONFIG_START_NETWORK -DCONFIG_SPARSE_BSS -DCONFIG_BLKFRONT -DCONFIG_NETFRONT -DCONFIG_KBDFRONT -DCONFIG_FBFRONT -DCONFIG_CONSFRONT -DCONFIG_XENBUS -D__XEN_INTERFACE_VERSION__=0x00030205 -isystem /src/include -D__MINIOS__ -isystem /src/include/x86 -isystem /src/include/x86/x86_64 -c lib/string.c -o /src/lib/string.o
lib/string.c: In function '__ffsti2':
lib/string.c:29:27: error: implicit declaration of function 'ffs' [-Werror=implicit-function-declaration]
if (num == 1) return (ffs((int) lli));
^~~
lib/string.c:29:5: error: nested extern declaration of 'ffs' [-Werror=nested-externs]
if (num == 1) return (ffs((int) lli));
^~
cc1: all warnings being treated as errors
make: *** [minios.mk:68: /src/lib/string.o] Error 1
and the various combinatorial removal of config options resulted in lots
of small problems that broke the build. I imagine Unicore's build system
could take a combination of feature flags and focus on make it easier
to maintain the many variants. Or perhaps MiniOS's could do the same by
improving the Travis file in the repository to test more combinations.
Either way, I didn't quite have the time to do any of that work myself
and I had to context switch to something else after running into the
build problem I'm afraid ;-)
regards,
Anil
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Minios-devel] [MirageOS-devel] [Xen-API] [RFC] Unicore Subproject Proposal
2017-09-14 14:38 ` [MirageOS-devel] [Minios-devel] " Anil Madhavapeddy
@ 2017-09-14 14:47 ` Samuel Thibault
[not found] ` <20170914144756.jorfjgwarbmpwrhq@var.youpi.perso.aquilenet.fr>
1 sibling, 0 replies; 25+ messages in thread
From: Samuel Thibault @ 2017-09-14 14:47 UTC (permalink / raw)
To: Anil Madhavapeddy
Cc: Felipe Huici, Lars Kurth, Stefano Stabellini, rshaposhnik,
xen-api, Saverio Niccolini, minios-devel, Alexander Dubinin,
julien.grall, committers, mirageos-devel, stefano, xen-devel,
Simon Kuenzer, volodymyr_babchuk
Hello,
Anil Madhavapeddy, on jeu. 14 sept. 2017 15:38:54 +0100, wrote:
> On 13 Sep 2017, at 17:13, Samuel Thibault <samuel.thibault@ens-lyon.org> wrote:
> > Anil Madhavapeddy, on mer. 13 sept. 2017 11:11:03 +0100, wrote:
> >> Maintaining a forked MiniOS has been a multi-year source of a maintenance burden for MirageOS,
> >
> > I'm just wondering why this happened?
> >
> > The mini-os repository is open for development, it's just a matter of
> > agreeing on how to implement features :)
>
> We forked it well before mini-os spun out into a separate repository,
> around 5 years ago.
Ok :)
> It's a combination of both I think. I had a very quick look at the latest mini-os
> tree and ran into build problems from the master branch
Ok, so it's just a matter of cleanly working on it.
Samuel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Minios-devel] [Xen-API] [RFC] Unicore Subproject Proposal
[not found] ` <C8ACDEE5-D1EE-4AFE-B59C-5FBA25623EAA@recoil.org>
` (2 preceding siblings ...)
[not found] ` <20170913161345.fnndedfh74u5k3wh@var.youpi.perso.aquilenet.fr>
@ 2017-09-15 8:36 ` Simon Kuenzer
[not found] ` <eba0ace6-6160-342d-dba4-c40324e67d3f@neclab.eu>
4 siblings, 0 replies; 25+ messages in thread
From: Simon Kuenzer @ 2017-09-15 8:36 UTC (permalink / raw)
To: Anil Madhavapeddy
Cc: Felipe Huici, julian.chesterfield, Lars Kurth,
Stefano Stabellini, rshaposhnik, xen-api, Saverio Niccolini,
minios-devel, Alexander Dubinin, julien.grall, committers,
mirageos-devel, stefano, xen-devel, volodymyr_babchuk
Hey Anil,
On 13.09.2017 12:11, Anil Madhavapeddy wrote:
> On 11 Sep 2017, at 13:08, Simon Kuenzer <simon.kuenzer@neclab.eu> wrote:
>>
>>> Just my 2 cents:
>>> 1. Is this academic project, or it have specific goals and areas of application? Would be good to have some practical use-cases and well formulated list of problems (we all feel these by guts, but...), it aiming to solve. IMHO that will help to prioritize functionality and get usable result faster :)
>>
>> It is kind of both, however we aim a strong focus on real world problems: IoT, Mobile Edge Computing (MEC), Automotive, Virtual Network Functions (VNFs), and others.
>> We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and others) and tried to apply them in the several areas. While doing this, we noticed that each area benefits differently from the properties that Unikernels give - which is great (e.g., instant boot times for MEC, high performance for NFV, resource efficiency for IoT). However, building and maintaining new Unikernels (as we did with ClickOS, MiniCache, and Minipython) is currently painful.
>> Because of different focuses on properties and ported/implemented applications, most Unikernel today are bound to their own OS layers (e.g., ClickOS uses a different Mini-OS than Mirage). Each application requires a different subset of OS layers but also enables different optimizations of them.
>>
>> In order to solve this, we came up with the Unicore proposal. But I agree with your suggestion at this point: It helps for the project start to focus on some initial areas. For now, I hope this is driven by the first contributors, and I have personally IoT in mind. Since the project goal is so ambitious, we should keep the long-term goal in mind from the beginning.
>>
>
> Thanks very much for kicking off this initiative. Maintaining a forked MiniOS has been a multi-year source of a maintenance burden for MirageOS, and we would love to be more aligned with an upstream version and benefit from new features such as (e.g.) HVM booting.
>
We have the same burden with ClickOS and all the other unikernels we
have built. Features like HVM booting or support for different
hypervisors, are always something that users ask for. Since many
Unikernel projects struggle with this, I would like to have the
maintenance effort of a common base concentrated.
But we also learned that each Unikernel has own requirements: This is
why Unicore has to provide full configuration flexibility. Only then,
all Unikernel projects could really benefit from it.
So, I think we should all focus on the Unicore base and make our
individual projects successful with it ;-).
> From a MirageOS perspective, we'd be happy to switch to something that can give us just enough MiniOS for our ocaml-freestanding [1] code to boot on Xen. One requirement from our side is that we need to strip down MiniOS to remove even the C xenstore implementation, since we have pure OCaml gnt, xenstore and device driver implementations. We'd be happy to try out an alpha Unicore and let you know what is in excess to our needs as soon as you have something to publish.
Sure, lets do this! ;-) This is perfectly inline with the goals we
target with the Unicore build system.
>
> So full support from MirageOS for this initiative!
Thank you so much!
>
> regards,
> Anil
>
Simon
> [1] https://github.com/mirage/ocaml-freestanding/
>
>
> _______________________________________________
> Minios-devel mailing list
> Minios-devel@lists.xenproject.org
> https://lists.xenproject.org/cgi-bin/mailman/listinfo/minios-devel
>
--
============================================================
Simon Kuenzer
シモン クゥンツァー
Research Scientist,
Networked Systems and Data Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49 (0)6221 4342-264
Fax: +49 (0)6221 4342-5264
e-mail: simon.kuenzer@neclab.eu
============================================================
NEC Europe Ltd | Registered Office: Athene, Odyssey
Business Park, West End Road, London, HA4 6QE, GB
Registered in England 2832014
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
[not found] ` <3B0A2B25-6A6A-4DE9-845C-E56812B97F92@citrix.com>
@ 2017-09-15 8:50 ` Simon Kuenzer
2017-09-18 9:16 ` Felipe Huici
[not found] ` <D5E553D1.39F25%felipe.huici@neclab.eu-0>
2 siblings, 0 replies; 25+ messages in thread
From: Simon Kuenzer @ 2017-09-15 8:50 UTC (permalink / raw)
To: Lars Kurth, Alexander Dubinin, Felipe Huici
Cc: stefano, Stefano Stabellini, Wei Liu, julian.chesterfield,
rshaposhnik, xen-api, Saverio Niccolini, minios-devel,
julien.grall, committers, mirageos-devel, xen-devel,
volodymyr_babchuk
Hey Lars,
On 13.09.2017 18:55, Lars Kurth wrote:
> Simon,
>
> It looks to me as if there is some feedback: so it may make some sense to incorporate some of it and send out a version 2. We may also want to CC some reps from other unikernel projects but Mirage OS. Or you could point them to this thread in a separate mail through respective channels or a hub like unikernel.org. Whatever you think may work best. It’s your call.
Sounds good. I am going try the unikernel.org hub and contact some
people individually.
>
> And then have a formal vote, a week after v2 of the proposal. Does this work?
Yes, this will work. I am updating the proposal and will send it out to
this mailing list, right?
Thanks,
Simon
>
> Lars
>
> On 11/09/2017, 05:08, "Simon Kuenzer" <simon.kuenzer@neclab.eu> wrote:
>
> Hi Alexander,
>
> thanks a lot for your review.
>
> On 10.09.2017 22:48, Alexander Dubinin wrote:
> > Hi Felipe, all,
> >
> > Great that it's going to start :) Looking forward to join :)
>
> I am looking forward to your contributions. ;)
>
> >
> > Just my 2 cents:
> >
> > 1. Is this academic project, or it have specific goals and areas of
> > application? Would be good to have some practical use-cases and well
> > formulated list of problems (we all feel these by guts, but...), it
> > aiming to solve. IMHO that will help to prioritize functionality and get
> > usable result faster :)
>
> It is kind of both, however we aim a strong focus on real world
> problems: IoT, Mobile Edge Computing (MEC), Automotive, Virtual Network
> Functions (VNFs), and others.
> We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and
> others) and tried to apply them in the several areas. While doing this,
> we noticed that each area benefits differently from the properties that
> Unikernels give - which is great (e.g., instant boot times for MEC, high
> performance for NFV, resource efficiency for IoT). However, building and
> maintaining new Unikernels (as we did with ClickOS, MiniCache, and
> Minipython) is currently painful.
> Because of different focuses on properties and ported/implemented
> applications, most Unikernel today are bound to their own OS layers
> (e.g., ClickOS uses a different Mini-OS than Mirage). Each application
> requires a different subset of OS layers but also enables different
> optimizations of them.
>
> In order to solve this, we came up with the Unicore proposal. But I
> agree with your suggestion at this point: It helps for the project start
> to focus on some initial areas. For now, I hope this is driven by the
> first contributors, and I have personally IoT in mind. Since the project
> goal is so ambitious, we should keep the long-term goal in mind from the
> beginning.
>
> >
> > 2. Does any security subsystem planned? XEN have XSM/FLASK, but IMHO is
> > should be supplemented by some security layer in control/stub domains as
> > well. So far only known implementation is OpenXT, but it is.... very
> > specific. Probably some generalized security layer needed in Unicore to
> > supplement FLASK/XSM... Correct me please, if I misunderstanding :)
>
> I agree that many projects (especially embedded, stubdomains, driver
> domains, NFV) have a vested interest in security and isolation. In my
> view, XSM/FLASK further restricts what a VM can do and sounds kind of
> orthogonal to the functionality of a VM (am I right?). The fact that
> Unikernels should only pick components that are actually required to do
> the job reduces the attack surface compared to general purpose OSes.
> Do you see further value with FLASK/XSM which requires early
> implementation and design decisions for Unicore? As far as I can tell
> something like Flask is implemented mostly in the hypervisor and
> toolstack, not in the guests themselves, is this right?
>
>
> Thanks,
>
> Simon
>
> >
> > Regards,
> > Alexander
> >
> > On Fri, Sep 8, 2017 at 3:31 PM, Felipe Huici <Felipe.Huici@neclab.eu
> > <mailto:Felipe.Huici@neclab.eu>> wrote:
> >
> > Hi Wei, Stefano,
> >
> > Thank you so much for agreeing to be sponsors! I’ll update the document.
> >
> > — Felipe
> >
> > ============================================================
> > Dr. Felipe Huici
> > Chief Researcher, Networked Systems and Data
> > Analytics Group
> > NEC Laboratories Europe, Network Research Division
> > Kurfuerstenanlage 36, D-69115 Heidelberg
> > Tel. +49
> > (0)6221 4342-241
> > Fax: +49
> > (0)6221 4342-155
> >
> > e-mail:
> > felipe.huici@neclab.eu <mailto:felipe.huici@neclab.eu>
> > ============================================================
> > NEC Europe Limited Registered Office: NEC House, 1
> > Victoria Road, London W3 6BL Registered in England 2832014
> >
> >
> >
> >
> > On 9/8/17, 1:00 PM, "Lars Kurth" <lars.kurth@citrix.com
> > <mailto:lars.kurth@citrix.com>> wrote:
> >
> > >@Wei, @Stefano,
> > >
> > >On 07/09/2017, 22:16, "Stefano Stabellini" <sstabellini@kernel.org
> > <mailto:sstabellini@kernel.org>> wrote:
> > >
> > > Hi all,
> > >
> > > I would be glad to sponsor this proposal. I think it will be
> > of great
> > > benefit to the ecosystem. Let me know if I need to do anything
> > >specific.
> > >
> > >Basically, all which is needed is an agreement. Which we have from you
> > >both. Felipe, can then add your names to the proposal.
> > >
> > >Looking out for the evolving project and helping (e.g. through
> > advice) is
> > >not strictly necessary, but always welcome.
> > >
> > >Lars
> > >
> >
> >
> >
> >
> > --
> > Regards,
> > Alexander Dubinin
>
> --
> ============================================================
> Simon Kuenzer
> シモン クゥンツァー
> Research Scientist,
> Networked Systems and Data Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49 (0)6221 4342-264
> Fax: +49 (0)6221 4342-5264
> e-mail: simon.kuenzer@neclab.eu
> ============================================================
> NEC Europe Ltd | Registered Office: Athene, Odyssey
> Business Park, West End Road, London, HA4 6QE, GB
> Registered in England 2832014
>
>
--
============================================================
Simon Kuenzer
シモン クゥンツァー
Research Scientist,
Networked Systems and Data Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49 (0)6221 4342-264
Fax: +49 (0)6221 4342-5264
e-mail: simon.kuenzer@neclab.eu
============================================================
NEC Europe Ltd | Registered Office: Athene, Odyssey
Business Park, West End Road, London, HA4 6QE, GB
Registered in England 2832014
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [MirageOS-devel] [Minios-devel] [Xen-API] [RFC] Unicore Subproject Proposal
[not found] ` <eba0ace6-6160-342d-dba4-c40324e67d3f@neclab.eu>
@ 2017-09-15 9:35 ` Anil Madhavapeddy
2017-09-15 12:37 ` Simon Kuenzer
0 siblings, 1 reply; 25+ messages in thread
From: Anil Madhavapeddy @ 2017-09-15 9:35 UTC (permalink / raw)
To: Simon Kuenzer
Cc: Felipe Huici, Lars Kurth, Stefano Stabellini, rshaposhnik,
xen-api, Saverio Niccolini, minios-devel, Alexander Dubinin,
julien.grall, committers, mirageos-devel, stefano, xen-devel,
volodymyr_babchuk
[-- Attachment #1.1: Type: text/plain, Size: 3496 bytes --]
> On 15 Sep 2017, at 09:36, Simon Kuenzer <simon.kuenzer@neclab.eu> wrote:
>
> Hey Anil,
>
> On 13.09.2017 12:11, Anil Madhavapeddy wrote:
>> On 11 Sep 2017, at 13:08, Simon Kuenzer <simon.kuenzer@neclab.eu> wrote:
>>>
>>>> Just my 2 cents:
>>>> 1. Is this academic project, or it have specific goals and areas of application? Would be good to have some practical use-cases and well formulated list of problems (we all feel these by guts, but...), it aiming to solve. IMHO that will help to prioritize functionality and get usable result faster :)
>>>
>>> It is kind of both, however we aim a strong focus on real world problems: IoT, Mobile Edge Computing (MEC), Automotive, Virtual Network Functions (VNFs), and others.
>>> We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and others) and tried to apply them in the several areas. While doing this, we noticed that each area benefits differently from the properties that Unikernels give - which is great (e.g., instant boot times for MEC, high performance for NFV, resource efficiency for IoT). However, building and maintaining new Unikernels (as we did with ClickOS, MiniCache, and Minipython) is currently painful.
>>> Because of different focuses on properties and ported/implemented applications, most Unikernel today are bound to their own OS layers (e.g., ClickOS uses a different Mini-OS than Mirage). Each application requires a different subset of OS layers but also enables different optimizations of them.
>>>
>>> In order to solve this, we came up with the Unicore proposal. But I agree with your suggestion at this point: It helps for the project start to focus on some initial areas. For now, I hope this is driven by the first contributors, and I have personally IoT in mind. Since the project goal is so ambitious, we should keep the long-term goal in mind from the beginning.
>>>
>> Thanks very much for kicking off this initiative. Maintaining a forked MiniOS has been a multi-year source of a maintenance burden for MirageOS, and we would love to be more aligned with an upstream version and benefit from new features such as (e.g.) HVM booting.
>
> We have the same burden with ClickOS and all the other unikernels we have built. Features like HVM booting or support for different hypervisors, are always something that users ask for. Since many Unikernel projects struggle with this, I would like to have the maintenance effort of a common base concentrated.
> But we also learned that each Unikernel has own requirements: This is why Unicore has to provide full configuration flexibility. Only then, all Unikernel projects could really benefit from it.
>
> So, I think we should all focus on the Unicore base and make our individual projects successful with it ;-).
This sounds good. It's worth thinking through the explicit differences in goals from MiniOS, to address Samuel's points.
It seems to me that MiniOS should remain the primary "support kernel" for Xen-related activities, for instance as a stub domain support kernel. Unicore on the other hand explicitly tries to parameterise across its configuration so that it can be library linked to language runtimes more easily.
This split may help simplify MiniOS by removing some of the pseudo libc code that may be unnecessary outside Xen support functions, and also let us package up language runtimes in Unicore more easily via simple library package management and cross compilation.
regards,
Anil
[-- Attachment #1.2: Type: text/html, Size: 8650 bytes --]
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [MirageOS-devel] [Minios-devel] [Xen-API] [RFC] Unicore Subproject Proposal
2017-09-15 9:35 ` [MirageOS-devel] " Anil Madhavapeddy
@ 2017-09-15 12:37 ` Simon Kuenzer
0 siblings, 0 replies; 25+ messages in thread
From: Simon Kuenzer @ 2017-09-15 12:37 UTC (permalink / raw)
To: Anil Madhavapeddy
Cc: Felipe Huici, Lars Kurth, Stefano Stabellini, rshaposhnik,
xen-api, Saverio Niccolini, minios-devel, Alexander Dubinin,
julien.grall, committers, mirageos-devel, stefano, xen-devel,
volodymyr_babchuk
On 15.09.2017 11:35, Anil Madhavapeddy wrote:
>
>> On 15 Sep 2017, at 09:36, Simon Kuenzer <simon.kuenzer@neclab.eu
>> <mailto:simon.kuenzer@neclab.eu>> wrote:
>>
>> Hey Anil,
>>
>> On 13.09.2017 12:11, Anil Madhavapeddy wrote:
>>> On 11 Sep 2017, at 13:08, Simon Kuenzer <simon.kuenzer@neclab.eu
>>> <mailto:simon.kuenzer@neclab.eu>> wrote:
>>>>
>>>>> Just my 2 cents:
>>>>> 1. Is this academic project, or it have specific goals and areas of
>>>>> application? Would be good to have some practical use-cases and
>>>>> well formulated list of problems (we all feel these by guts,
>>>>> but...), it aiming to solve. IMHO that will help to prioritize
>>>>> functionality and get usable result faster :)
>>>>
>>>> It is kind of both, however we aim a strong focus on real world
>>>> problems: IoT, Mobile Edge Computing (MEC), Automotive, Virtual
>>>> Network Functions (VNFs), and others.
>>>> We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and
>>>> others) and tried to apply them in the several areas. While doing
>>>> this, we noticed that each area benefits differently from the
>>>> properties that Unikernels give - which is great (e.g., instant boot
>>>> times for MEC, high performance for NFV, resource efficiency for
>>>> IoT). However, building and maintaining new Unikernels (as we did
>>>> with ClickOS, MiniCache, and Minipython) is currently painful.
>>>> Because of different focuses on properties and ported/implemented
>>>> applications, most Unikernel today are bound to their own OS layers
>>>> (e.g., ClickOS uses a different Mini-OS than Mirage). Each
>>>> application requires a different subset of OS layers but also
>>>> enables different optimizations of them.
>>>>
>>>> In order to solve this, we came up with the Unicore proposal. But I
>>>> agree with your suggestion at this point: It helps for the project
>>>> start to focus on some initial areas. For now, I hope this is driven
>>>> by the first contributors, and I have personally IoT in mind. Since
>>>> the project goal is so ambitious, we should keep the long-term goal
>>>> in mind from the beginning.
>>>>
>>> Thanks very much for kicking off this initiative. Maintaining a
>>> forked MiniOS has been a multi-year source of a maintenance burden
>>> for MirageOS, and we would love to be more aligned with an upstream
>>> version and benefit from new features such as (e.g.) HVM booting.
>>
>> We have the same burden with ClickOS and all the other unikernels we
>> have built. Features like HVM booting or support for different
>> hypervisors, are always something that users ask for. Since many
>> Unikernel projects struggle with this, I would like to have the
>> maintenance effort of a common base concentrated.
>> But we also learned that each Unikernel has own requirements: This is
>> why Unicore has to provide full configuration flexibility. Only then,
>> all Unikernel projects could really benefit from it.
>>
>> So, I think we should all focus on the Unicore base and make our
>> individual projects successful with it ;-).
>
> This sounds good. It's worth thinking through the explicit differences
> in goals from MiniOS, to address Samuel's points.
Yes, I agree. I am going to have a second round through the proposal.
>
> It seems to me that MiniOS should remain the primary "support kernel"
> for Xen-related activities, for instance as a stub domain support
> kernel. Unicore on the other hand explicitly tries to parameterise
> across its configuration so that it can be library linked to language
> runtimes more easily.
>
> This split may help simplify MiniOS by removing some of the pseudo libc
> code that may be unnecessary outside Xen support functions, and also let
> us package up language runtimes in Unicore more easily via simple
> library package management and cross compilation.
It might be handled in this way during the project start, however I see
Unicore more than just a set of base libraries to support language
runtimes. The main intention is to support applications natively written
to it and to support existing applications you may want to port.
For these cases you need more than a minimal base, you want to select a
libc, a certain scheduler, and most likely a TCP/IP stack, as well as
filesystem support. These libraries should work together and should also
work with the base libraries. Additionally, the requirements to the
libraries might differ among the projects.
Our content cache node Unikernel (called MiniCache), for instance, uses
newlibc and lwIP. On top we have developed a single-purpose filesystem
and web server code. Together we could achieve impressive performance
results by using a single CPU compared to traditional web server
applications running on a traditional OS [1].
In sum, the aim with Unicore is to have fine-granularity libraries so
that it's flexible enough to accommodate a wide range of unikernels.
Because of this, I see Unicore as having potential to eventually replace
MiniOS.
>
> regards,
> Anil
>
Thanks,
Simon
[1] http://dl.acm.org/citation.cfm?doid=3050748.3050757
--
============================================================
Simon Kuenzer
シモン クゥンツァー
Research Scientist,
Networked Systems and Data Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49 (0)6221 4342-264
Fax: +49 (0)6221 4342-5264
e-mail: simon.kuenzer@neclab.eu
============================================================
NEC Europe Ltd | Registered Office: Athene, Odyssey
Business Park, West End Road, London, HA4 6QE, GB
Registered in England 2832014
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
2017-09-13 8:36 ` Alexander Dubinin
@ 2017-09-15 13:01 ` Simon Kuenzer
0 siblings, 0 replies; 25+ messages in thread
From: Simon Kuenzer @ 2017-09-15 13:01 UTC (permalink / raw)
To: Alexander Dubinin
Cc: Felipe Huici, Lars Kurth, Stefano Stabellini, Wei Liu,
julian.chesterfield, rshaposhnik, xen-api, Saverio Niccolini,
minios-devel, julien.grall, committers, mirageos-devel, stefano,
xen-devel, volodymyr_babchuk
Hey Alexander,
On 13.09.2017 10:36, Alexander Dubinin wrote:
> Hello Simon, all,
>
> 1. Is this academic project, or it have specific goals and areas
> of application? Would be good to have some practical use-cases
> and well formulated list of problems (we all feel these by guts,
> but...), it aiming to solve. IMHO that will help to prioritize
> functionality and get usable result faster :)
>
>
> It is kind of both, however we aim a strong focus on real world
> problems: IoT, Mobile Edge Computing (MEC), Automotive, Virtual
> Network Functions (VNFs), and others.
> We have played with many Unikernels (ClickOS, Mirage, Rump, OSv, and
> others) and tried to apply them in the several areas. While doing
> this, we noticed that each area benefits differently from the
> properties that Unikernels give - which is great (e.g., instant boot
> times for MEC, high performance for NFV, resource efficiency for
> IoT). However, building and maintaining new Unikernels (as we did
> with ClickOS, MiniCache, and Minipython) is currently painful.
> Because of different focuses on properties and ported/implemented
> applications, most Unikernel today are bound to their own OS layers
> (e.g., ClickOS uses a different Mini-OS than Mirage). Each
> application requires a different subset of OS layers but also
> enables different optimizations of them.
>
> In order to solve this, we came up with the Unicore proposal. But I
> agree with your suggestion at this point: It helps for the project
> start to focus on some initial areas. For now, I hope this is driven
> by the first contributors, and I have personally IoT in mind. Since
> the project goal is so ambitious, we should keep the long-term goal
> in mind from the beginning.
>
> Yes, that's exactly what I meant :)
> And IMHO it would be good to not have very abstract goals - and you
> named first real one - IoT.
> Do you have real-life use-case with real-life hardware to implement
> within this IoT?
> For example, popular IoT devkit, people can use and join this efforts?
> And real, useful application for it?
>
This is a good question. No, we haven't settled on a particular IoT
devkit yet. Do you have something in mind?
For now, we did some research prototypes for IoT by using a Cubieboard
and Unikernels that process some sensor data.
> My interest here is mostly disaggregation and security - to have
> minimal, but still functional service domains, built strictly for
> specific functionality.
> So far we (team, I working with) are using BuildRoot to create
> Dom0/stubdoms/driverdoms/etc. based on Linux (yet).
> In our case (at least, right now) guest systems are heavy
> VMs(Windows/Linux/*BSD/QNX) with GPU passthrough (And not only GPU, but
> other controllers, test boards, etc.).
>
> Hardware domains most likely to be based on the OS-es, which have proven
> and stable hardware support base (Linux, *BSD), but there are still need
> in service domains - like TUI(Text UI) domain, where users are
> interacting with host, stub-domain, dom0, which starting hardware
> domains, etc.
>
> So, that could be one more goal - minimalistic service domains for
> x86/64 platform.
Yes, sure.
>
> Another goal would be virtualization in Automated Control Systems area -
> but it's too early (for me, at least) to talk about it yet.
>
> Does anybody have other _specific_ targets for Unicore in mind?
I am also interested in answers.
>
>
>
> 2. Does any security subsystem planned? XEN have XSM/FLASK, but
> IMHO is should be supplemented by some security layer in
> control/stub domains as well. So far only known implementation
> is OpenXT, but it is.... very specific. Probably some
> generalized security layer needed in Unicore to supplement
> FLASK/XSM... Correct me please, if I misunderstanding :)
>
>
> I agree that many projects (especially embedded, stubdomains, driver
> domains, NFV) have a vested interest in security and isolation. In
> my view, XSM/FLASK further restricts what a VM can do and sounds
> kind of orthogonal to the functionality of a VM (am I right?). The
> fact that Unikernels should only pick components that are actually
> required to do the job reduces the attack surface compared to
> general purpose OSes.
> Do you see further value with FLASK/XSM which requires early
> implementation and design decisions for Unicore? As far as I can
> tell something like Flask is implemented mostly in the hypervisor
> and toolstack, not in the guests themselves, is this right?
>
> Yes, if Unicore is not supposed to be used as Dom0, and if we are
> considering Unicore domains only as a guests, running in the single
> security context, that's fine :)
> But if, eventually, it will be used as a control domain in multi-tenant
> system, which should manage XSM/FLASK and fill the gap between real
> users(and their data) and VMs, restricted by FLASK - it's something to
> think about. Maybe just not now :) Or it's not one of Unicore goals even :)
I got it. Yes, I think this could be considered as a goal for an
extension library to the Xen platform libraries. It could be seen as
"control domain" extension. Why not? ;-)
>
> Just dreaming to have absolutely minimal service domains, where every
> byte is known and needed :)
I think there are use cases for this. I think this starts with
automotive, but it could be also important for projects, like OpenXT.
>
> Regards,
> Alexander
>
>
> Thanks,
>
> Simon
>
>
> Regards,
> Alexander
>
> On Fri, Sep 8, 2017 at 3:31 PM, Felipe Huici
> <Felipe.Huici@neclab.eu <mailto:Felipe.Huici@neclab.eu>
> <mailto:Felipe.Huici@neclab.eu <mailto:Felipe.Huici@neclab.eu>>>
> wrote:
>
> Hi Wei, Stefano,
>
> Thank you so much for agreeing to be sponsors! I’ll update
> the document.
>
> — Felipe
>
> ============================================================
> Dr. Felipe Huici
> Chief Researcher, Networked Systems and Da
> <https://maps.google.com/?q=orked+Systems+and+Da&entry=gmail&source=g>ta
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49
> (0)6221 4342-241
> Fax: +49
> (0)6221 4342-155
>
> e-mail:
> felipe.huici@neclab.eu <mailto:felipe.huici@neclab.eu>
> <mailto:felipe.huici@neclab.eu <mailto:felipe.huici@neclab.eu>>
> ============================================================
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
>
>
>
>
> On 9/8/17, 1:00 PM, "Lars Kurth" <lars.kurth@citrix.com
> <mailto:lars.kurth@citrix.com>
> <mailto:lars.kurth@citrix.com
> <mailto:lars.kurth@citrix.com>>> wrote:
>
> >@Wei, @Stefano,
> >
> >On 07/09/2017, 22:16, "Stefano Stabellini"
> <sstabellini@kernel.org <mailto:sstabellini@kernel.org>
> <mailto:sstabellini@kernel.org
> <mailto:sstabellini@kernel.org>>> wrote:
> >
> > Hi all,
> >
> > I would be glad to sponsor this proposal. I think it
> will be
> of great
> > benefit to the ecosystem. Let me know if I need to do
> anything
> >specific.
> >
> >Basically, all which is needed is an agreement. Which we
> have from you
> >both. Felipe, can then add your names to the proposal.
> >
> >Looking out for the evolving project and helping (e.g.
> through
> advice) is
> >not strictly necessary,
> <https://maps.google.com/?q=strictly+necessary,&entry=gmail&source=g>but
> always welcome.
> >
> >Lars
> >
>
>
>
>
> --
> Regards,
> Alexander Dubinin
>
>
> --
> ============================================================
> Simon Kuenzer
> シモン クゥンツァー
> Research Scientist,
> Networked Systems and Data Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49 (0)6221 4342-264
> Fax: +49 (0)6221 4342-5264
> e-mail: simon.kuenzer@neclab.eu <mailto:simon.kuenzer@neclab.eu>
> ============================================================
> NEC Europe Ltd | Registered Office: Athene, Odyssey
> Business Park, West End Road, London, HA4 6QE, GB
> Registered in England 2832014
>
>
>
>
> --
> Regards,
> Alexander Dubinin
Thanks,
Simon
--
============================================================
Simon Kuenzer
シモン クゥンツァー
Research Scientist,
Networked Systems and Data Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49 (0)6221 4342-264
Fax: +49 (0)6221 4342-5264
e-mail: simon.kuenzer@neclab.eu
============================================================
NEC Europe Ltd | Registered Office: Athene, Odyssey
Business Park, West End Road, London, HA4 6QE, GB
Registered in England 2832014
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Minios-devel] [MirageOS-devel] [Xen-API] [RFC] Unicore Subproject Proposal
[not found] ` <20170914144756.jorfjgwarbmpwrhq@var.youpi.perso.aquilenet.fr>
@ 2017-09-15 13:23 ` Simon Kuenzer
0 siblings, 0 replies; 25+ messages in thread
From: Simon Kuenzer @ 2017-09-15 13:23 UTC (permalink / raw)
To: Samuel Thibault, Anil Madhavapeddy, Felipe Huici, Lars Kurth,
Stefano Stabellini, rshaposhnik, xen-api, Saverio Niccolini,
minios-devel, Alexander Dubinin, julien.grall, committers,
mirageos-devel, stefano, xen-devel, volodymyr_babchuk
Hey Samuel,
On 14.09.2017 16:47, Samuel Thibault wrote:
> Hello,
>
> Anil Madhavapeddy, on jeu. 14 sept. 2017 15:38:54 +0100, wrote:
>> On 13 Sep 2017, at 17:13, Samuel Thibault <samuel.thibault@ens-lyon.org> wrote:
>>> Anil Madhavapeddy, on mer. 13 sept. 2017 11:11:03 +0100, wrote:
>>>> Maintaining a forked MiniOS has been a multi-year source of a maintenance burden for MirageOS,
>>>
>>> I'm just wondering why this happened?
>>>
>>> The mini-os repository is open for development, it's just a matter of
>>> agreeing on how to implement features :)
>>
>> We forked it well before mini-os spun out into a separate repository,
>> around 5 years ago.
We did the same for our ClickOS, MiniCache and Minipython Unikernel: We
used our own fork of MiniOS (4-5 years ago). However, while working on
it, we noticed that there is much more to do than just introducing some
switches and add new features.
We add several functionality to our MiniOS tree (replaced network I/O,
TSO feature, persistent grants for block, select/poll support, proper
heap memory allocation, hooks into low-level code), but support for more
hypervisor stayed forever on our wishlist. We concluded that you should
either intrusively clean-up and reorganize the Mini-OS code (which we
think is non-trivial to do) or you start with a new project and port the
pieces from MiniOS that you need (which is clearly lots of work, too).
We decided for the latter because it also forces us to rethink about
internal APIs and code organization. A proper design should also
simplify adding support for more hypervisors. And because we understand
the needs of Unikernels better today ;-), I think we should consider
this as a rare chance and should make use of it.
>
> Ok :)
>
>> It's a combination of both I think. I had a very quick look at the latest mini-os
>> tree and ran into build problems from the master branch
>
> Ok, so it's just a matter of cleanly working on it.
>
> Samuel
>
Thanks,
Simon
--
============================================================
Simon Kuenzer
シモン クゥンツァー
Research Scientist,
Networked Systems and Data Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49 (0)6221 4342-264
Fax: +49 (0)6221 4342-5264
e-mail: simon.kuenzer@neclab.eu
============================================================
NEC Europe Ltd | Registered Office: Athene, Odyssey
Business Park, West End Road, London, HA4 6QE, GB
Registered in England 2832014
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
[not found] ` <3B0A2B25-6A6A-4DE9-845C-E56812B97F92@citrix.com>
2017-09-15 8:50 ` Simon Kuenzer
@ 2017-09-18 9:16 ` Felipe Huici
2017-09-19 22:56 ` Stefano Stabellini
[not found] ` <D5E553D1.39F25%felipe.huici@neclab.eu-0>
2 siblings, 1 reply; 25+ messages in thread
From: Felipe Huici @ 2017-09-18 9:16 UTC (permalink / raw)
To: Lars Kurth, Simon Kuenzer, Alexander Dubinin
Cc: stefano, Stefano Stabellini, Wei Liu, julian.chesterfield,
rshaposhnik, xen-api, info, mato, Saverio Niccolini,
minios-devel, julien.grall, committers, mirageos-devel,
xen-devel, pooka, awick, volodymyr_babchuk@epam.com
[-- Attachment #1: Type: text/plain, Size: 9923 bytes --]
Hi Lars, all,
[cc’ing authors of Erlang on Xen, HalVM and Rump].
Thanks everyone for all of the support and useful comments. We’ve
incorporated a number of them into a new version of the document (attached
and pasted at the bottom for convenience) and for those that didn’t make
it we’re keeping track of them.
Lars, FYI, Simon also did a blog post regarding Unicore on unikernel.org
(https://devel.unikernel.org/t/unicore-a-new-unikernel-project/274).
Please let us know what the next steps are.
Thanks,
— Felipe
PROPOSAL: Unicore
=================
Roles
-----
Project Leads: Simon Kuenzer <simon.kuenzer@neclab.eu>
(co-lead) Felipe Huici <felipe.huici@neclab.eu>
(co-lead) Florian Schmidt <florian.schmidt@neclab.eu>
Project Mentor: Lars Kurth <lars.kurth@citrix.com>
Project Sponsors: Stefano Stabellini <sstabellini@kernel.org>
Wei Liu <wei.liu2@citrix.com>
Background
----------
In recent years, several papers and projects dedicated to unikernels
have shown the immense potential for performance gains that these
have. By leveraging specialization and the use of minimalistic OSes,
unikernels are able to yield impressive numbers, including fast
instantiation times (tens of milliseconds or less), tiny memory
footprints (a few MBs or even KBs), high network throughput (10-40
Gb/s), and high consolidation (e.g., being able to run thousands of
instances on a single commodity server), not to mention a reduced
attack surface and the potential for easier certification. Unikernel
projects worthy of mention include MirageOS, ClickOS, Erlang on Xen,
OSv, HALVM, and Minicache, Rump, among others.
The fundamental drawback of unikernels is that they require that
applications be manually ported to the underlying minimalistic OS (e.g.
having to port nginx, snort, mysql or memcached to MiniOS or OSv); this
requires both expert work and often considerable amount of time. In
essence, we need to pick between either high performance
with unikernels, or no porting effort but decreased performance
and decreased efficiency with standard OS/VM images.
The goal of this proposal is to change this status quo by providing
a highly configurable unikernel code base; we call this base Unicore.
This project also aims to concentrate the various efforts currently going
on in the Xen community regarding minimalistic OSes (essentially different
variants of MiniOS). We think that splitting the community across these
variants is counter-productive and hope that Unicore will provide a common
place for all or most improvements and customizations of minimalistic
OSes. The long term goal is to replace something like MiniOS with a tool
that can automatically build such a minimalistic OS.
Unicore - The "Unikernel Core"
---------------------------------
The high level goal of Unicore is to be able to build unikernels targeted
at specific applications without requiring the time-consuming, expert work
that building such a unikernel requires today. An additional goal (or
hope) of Unicore is that all developers interested in unikernel
development would contribute by supplying libraries rather than working on
independent projects with different code bases as it is done now. The main
idea behind Unicore is depicted in Figure 1 and consists of two basic
components:
[Attachment: unicore-oneslider.pdf]
Figure 1. Unicore Architecture.
Library pools would contain libraries that the user of Unicore can select
from to create the unikernel. From the bottom up, library pools are
organized into (1) the architecture library tool, containing libraries
specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the
platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no
virtualization) and user-space Linux; and (3) the main library pool,
containing a rich set of functionality to build the unikernel from. This
last library includes drivers (both virtual such as netback/netfront and
physical such as ixgbe), filesystems, memory allocators, schedulers,
network stacks, standard libs (e.g. libc, openssl, etc.), runtimes (e.g. a
Python interpreter and debugging and profiling tools. These pools of
libraries constitute a code base for creating unikernels. As shown, a
library can be relatively large (e.g libc) or quite small (a scheduler),
which should allow for a fair amount of customization for the unikernel.
The Unicore build tool is in charge of compiling the application and the
selected libraries together to create a binary for a specific platform and
architecture (e.g., Xen on x86_64). The tool is currently inspired by
Linux’s kconfig system and consists of a set of Makefiles. It allows users
to select libraries, to configure them, and to warn them when library
dependencies are not met. In addition, the tool can also simultaneously
generate binaries for multiple platforms.
As an example, imagine a user wanting to generate a network driver domain
unikernel. In this case, we would assume the “application” to be the
netback driver. To select this application, the user would first run “make
menuconfig” from within the netback application folder. The Makefile there
would set a variable to indicate what the application is, and would
include the main Unicore Makefiles so that the unikernel can be built
(Step 1 in the figure). Using the menu-based system, the user chooses the
relevant libraries; for a Xen driver domain this would include a physical
network driver, the netback driver, the libxenplat library and a library
from the architecture library pool such as libx86_64arch (Step 2 in the
figure). With this in place, the user saves the configuration and types
“make” to build the unikernel (Step 3) and “xl create” to run it (Step 4).
A note on the ABI/API exposed to the application: because Unicore allows
for customization of the unikernels, the ABI (or API since there is no
kernel) would be custom, that is, defined by the libraries the user
selected. Having said that, it would be perfectly possible, for instance,
to build POSIX-compliant unikernels with it (e.g. similar to Rump, but in
principle with much more specialized OS layers).
Finally, it is worth pointing out that we use the term application
loosely: another clear target for Unicore is the building of
runtime-specific unikernels (e.g. a unikernel able to run Python or OCaml
scripts as is the case with MirageOS).
Relevance to Xen and its Community
-----------------------------------
Unikernels are important to a number of areas relevant to the Xen
community, including IoT, automotive, stub domains, and driver domain/dom0
disaggregation. Unicore could help boost the progress in all of these
areas by quickly providing the necessary tools to create unikernels for
them. For instance, for a driver domain, the user would include the
“library” containing the relevant hardware driver and corresponding
back-end driver, and in principle Unicore would take care of the rest.
In addition, Unicore could eventually replace Mini-OS, providing a
cleaner, more stable and flexible base from which to build unikernels for
projects (the modularization of Mini-OS is in fact already taking place).
Current Status
--------------
Unicore is at an early stage. For now it includes some base libraries with
code extracted from Mini-OS as well as a build tool inspired by Linux's
KConfig system. Unicore is currently able to build "hello world"
unikernels for Xen and Linux user space on x86_64 and ARMv7.
Incubation
----------
The reason behind making Unicore a Xen sub-project project is to (1)
bring the existence of Unicore to the attention of the Xen community
and to outside world; (2) to attempt to harness interest and
potentially development cycles from people and companies interested in
unikernels; (3) to concentrate maintenance resources from people
interested in unikernels within the community; and (4) to have a legal
entity behind the project.
License
-------
The main license of the run-time components of Unicore will be a 3-clause
BSD license, unless there is a good reason not to use it (e.g. we may
import 2-clause BSD licensed code from Mini-OS, which we would *not*
anticipate to change). The Makefile system would be licensed under GPL v2
or later as we want to be able to use KConfig functionality from
Buildroot/Linux.
Required Infrastructure
-----------------------
The official repositories should be created on
[http://xenbits.xenproject.org/] under `unicore.git`. There should be a
main repository for the core unicore implementation and additional
repositories for some more advanced extension libraries (e.g., lwIP,
newlib).
### Main repository
`unicore.git`
### Repositories for extension libraries
Repositories for additional libraries that are supported by the Unicore
project should exist under a separate directory:
`unicore-libs/`
For example:
`unicore-libs/lwip.git`
`unicore-libs/newlib.git`
### Mailing list
In the beginning we would use the MiniOS mailing list
(minios-devel@lists.xenproject.org). When we get traction with Unicore we
could consider splitting that traffic onto a unicore mailing list.
============================================================
Dr. Felipe Huici
Chief Researcher, Networked Systems and Data
Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49
(0)6221 4342-241
Fax: +49
(0)6221 4342-155
e-mail:
felipe.huici@neclab.eu
============================================================
NEC Europe Limited Registered Office: NEC House, 1
Victoria Road, London W3 6BL Registered in England 2832014
[-- Attachment #2: PROPOSAL Unicore.pdf --]
[-- Type: application/pdf, Size: 170579 bytes --]
[-- Attachment #3: unicore-oneslider[1].pdf --]
[-- Type: application/pdf, Size: 64563 bytes --]
[-- Attachment #4: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
2017-09-18 9:16 ` Felipe Huici
@ 2017-09-19 22:56 ` Stefano Stabellini
0 siblings, 0 replies; 25+ messages in thread
From: Stefano Stabellini @ 2017-09-19 22:56 UTC (permalink / raw)
To: Felipe Huici
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, julian.chesterfield,
rshaposhnik, xen-api, info, mato, Saverio Niccolini,
minios-devel, Alexander Dubinin, julien.grall, committers,
mirageos-devel, pooka, stefano, xen-devel
[-- Attachment #1: Type: TEXT/PLAIN, Size: 10217 bytes --]
On Mon, 18 Sep 2017, Felipe Huici wrote:
> Hi Lars, all,
>
> [cc’ing authors of Erlang on Xen, HalVM and Rump].
>
> Thanks everyone for all of the support and useful comments. We’ve
> incorporated a number of them into a new version of the document (attached
> and pasted at the bottom for convenience) and for those that didn’t make
> it we’re keeping track of them.
>
> Lars, FYI, Simon also did a blog post regarding Unicore on unikernel.org
> (https://devel.unikernel.org/t/unicore-a-new-unikernel-project/274).
>
> Please let us know what the next steps are.
The proposal looks good to me.
> Thanks,
>
> — Felipe
>
>
> PROPOSAL: Unicore
> =================
>
> Roles
> -----
> Project Leads: Simon Kuenzer <simon.kuenzer@neclab.eu>
> (co-lead) Felipe Huici <felipe.huici@neclab.eu>
> (co-lead) Florian Schmidt <florian.schmidt@neclab.eu>
> Project Mentor: Lars Kurth <lars.kurth@citrix.com>
> Project Sponsors: Stefano Stabellini <sstabellini@kernel.org>
> Wei Liu <wei.liu2@citrix.com>
>
> Background
> ----------
> In recent years, several papers and projects dedicated to unikernels
> have shown the immense potential for performance gains that these
> have. By leveraging specialization and the use of minimalistic OSes,
> unikernels are able to yield impressive numbers, including fast
> instantiation times (tens of milliseconds or less), tiny memory
> footprints (a few MBs or even KBs), high network throughput (10-40
> Gb/s), and high consolidation (e.g., being able to run thousands of
> instances on a single commodity server), not to mention a reduced
> attack surface and the potential for easier certification. Unikernel
> projects worthy of mention include MirageOS, ClickOS, Erlang on Xen,
> OSv, HALVM, and Minicache, Rump, among others.
>
> The fundamental drawback of unikernels is that they require that
> applications be manually ported to the underlying minimalistic OS (e.g.
> having to port nginx, snort, mysql or memcached to MiniOS or OSv); this
> requires both expert work and often considerable amount of time. In
> essence, we need to pick between either high performance
> with unikernels, or no porting effort but decreased performance
> and decreased efficiency with standard OS/VM images.
> The goal of this proposal is to change this status quo by providing
> a highly configurable unikernel code base; we call this base Unicore.
>
> This project also aims to concentrate the various efforts currently going
> on in the Xen community regarding minimalistic OSes (essentially different
> variants of MiniOS). We think that splitting the community across these
> variants is counter-productive and hope that Unicore will provide a common
> place for all or most improvements and customizations of minimalistic
> OSes. The long term goal is to replace something like MiniOS with a tool
> that can automatically build such a minimalistic OS.
>
>
> Unicore - The "Unikernel Core"
> ---------------------------------
> The high level goal of Unicore is to be able to build unikernels targeted
> at specific applications without requiring the time-consuming, expert work
> that building such a unikernel requires today. An additional goal (or
> hope) of Unicore is that all developers interested in unikernel
> development would contribute by supplying libraries rather than working on
> independent projects with different code bases as it is done now. The main
> idea behind Unicore is depicted in Figure 1 and consists of two basic
> components:
>
>
> [Attachment: unicore-oneslider.pdf]
>
> Figure 1. Unicore Architecture.
>
>
> Library pools would contain libraries that the user of Unicore can select
> from to create the unikernel. From the bottom up, library pools are
> organized into (1) the architecture library tool, containing libraries
> specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the
> platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no
> virtualization) and user-space Linux; and (3) the main library pool,
> containing a rich set of functionality to build the unikernel from. This
> last library includes drivers (both virtual such as netback/netfront and
> physical such as ixgbe), filesystems, memory allocators, schedulers,
> network stacks, standard libs (e.g. libc, openssl, etc.), runtimes (e.g. a
> Python interpreter and debugging and profiling tools. These pools of
> libraries constitute a code base for creating unikernels. As shown, a
> library can be relatively large (e.g libc) or quite small (a scheduler),
> which should allow for a fair amount of customization for the unikernel.
>
> The Unicore build tool is in charge of compiling the application and the
> selected libraries together to create a binary for a specific platform and
> architecture (e.g., Xen on x86_64). The tool is currently inspired by
> Linux’s kconfig system and consists of a set of Makefiles. It allows users
> to select libraries, to configure them, and to warn them when library
> dependencies are not met. In addition, the tool can also simultaneously
> generate binaries for multiple platforms.
>
> As an example, imagine a user wanting to generate a network driver domain
> unikernel. In this case, we would assume the “application” to be the
> netback driver. To select this application, the user would first run “make
> menuconfig” from within the netback application folder. The Makefile there
> would set a variable to indicate what the application is, and would
> include the main Unicore Makefiles so that the unikernel can be built
> (Step 1 in the figure). Using the menu-based system, the user chooses the
> relevant libraries; for a Xen driver domain this would include a physical
> network driver, the netback driver, the libxenplat library and a library
> from the architecture library pool such as libx86_64arch (Step 2 in the
> figure). With this in place, the user saves the configuration and types
> “make” to build the unikernel (Step 3) and “xl create” to run it (Step 4).
>
> A note on the ABI/API exposed to the application: because Unicore allows
> for customization of the unikernels, the ABI (or API since there is no
> kernel) would be custom, that is, defined by the libraries the user
> selected. Having said that, it would be perfectly possible, for instance,
> to build POSIX-compliant unikernels with it (e.g. similar to Rump, but in
> principle with much more specialized OS layers).
>
> Finally, it is worth pointing out that we use the term application
> loosely: another clear target for Unicore is the building of
> runtime-specific unikernels (e.g. a unikernel able to run Python or OCaml
> scripts as is the case with MirageOS).
>
>
> Relevance to Xen and its Community
> -----------------------------------
> Unikernels are important to a number of areas relevant to the Xen
> community, including IoT, automotive, stub domains, and driver domain/dom0
> disaggregation. Unicore could help boost the progress in all of these
> areas by quickly providing the necessary tools to create unikernels for
> them. For instance, for a driver domain, the user would include the
> “library” containing the relevant hardware driver and corresponding
> back-end driver, and in principle Unicore would take care of the rest.
>
> In addition, Unicore could eventually replace Mini-OS, providing a
> cleaner, more stable and flexible base from which to build unikernels for
> projects (the modularization of Mini-OS is in fact already taking place).
>
>
> Current Status
> --------------
> Unicore is at an early stage. For now it includes some base libraries with
> code extracted from Mini-OS as well as a build tool inspired by Linux's
> KConfig system. Unicore is currently able to build "hello world"
> unikernels for Xen and Linux user space on x86_64 and ARMv7.
>
> Incubation
> ----------
> The reason behind making Unicore a Xen sub-project project is to (1)
> bring the existence of Unicore to the attention of the Xen community
> and to outside world; (2) to attempt to harness interest and
> potentially development cycles from people and companies interested in
> unikernels; (3) to concentrate maintenance resources from people
> interested in unikernels within the community; and (4) to have a legal
> entity behind the project.
>
> License
> -------
> The main license of the run-time components of Unicore will be a 3-clause
> BSD license, unless there is a good reason not to use it (e.g. we may
> import 2-clause BSD licensed code from Mini-OS, which we would *not*
> anticipate to change). The Makefile system would be licensed under GPL v2
> or later as we want to be able to use KConfig functionality from
> Buildroot/Linux.
>
> Required Infrastructure
> -----------------------
> The official repositories should be created on
> [http://xenbits.xenproject.org/] under `unicore.git`. There should be a
> main repository for the core unicore implementation and additional
> repositories for some more advanced extension libraries (e.g., lwIP,
> newlib).
>
> ### Main repository
>
> `unicore.git`
>
> ### Repositories for extension libraries
>
> Repositories for additional libraries that are supported by the Unicore
> project should exist under a separate directory:
>
> `unicore-libs/`
>
> For example:
>
> `unicore-libs/lwip.git`
> `unicore-libs/newlib.git`
>
> ### Mailing list
>
> In the beginning we would use the MiniOS mailing list
> (minios-devel@lists.xenproject.org). When we get traction with Unicore we
> could consider splitting that traffic onto a unicore mailing list.
>
>
>
> ============================================================
> Dr. Felipe Huici
> Chief Researcher, Networked Systems and Data
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49
> (0)6221 4342-241
> Fax: +49
> (0)6221 4342-155
>
> e-mail:
> felipe.huici@neclab.eu
> ============================================================
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
>
>
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
[not found] <D5D6EB26.39481%felipe.huici@neclab.eu-0>
2017-09-07 13:41 ` [RFC] Unicore Subproject Proposal Lars Kurth
@ 2017-09-20 0:20 ` Lars Kurth
[not found] ` <776E12BC-8D5A-4B90-AF99-BFDDEBEECCE4@citrix.com>
2 siblings, 0 replies; 25+ messages in thread
From: Lars Kurth @ 2017-09-20 0:20 UTC (permalink / raw)
To: Felipe Huici, committers, xen-devel, xen-api, mirageos-devel,
minios-devel
Cc: julian.chesterfield, rshaposhnik, Saverio Niccolini, stefano,
alexander.dubinin, julien.grall, Simon Kuenzer,
volodymyr_babchuk
Felipe, Simon,
a quick note to let you know that the Advisory Board in today’s AB meeting decided to endorse your proposal.
Let me know how you proceed: from my perspective, we can kick off a formal vote before you make modifications to the proposal, but I think it is better to post v2 first.
Lars
On 07/09/2017, 05:26, "Felipe Huici" <Felipe.Huici@neclab.eu> wrote:
Dear all,
Following up on discussions that Simon Kuenzer had with several of you at
the last Xen summit, we’re now submitting a Xen subproject proposal based
on our Unicore work. Could you please review it?
Thanks,
Felipe Huici & Simon Kuenzer - NEC Labs Heidelberg.
PROPOSAL: Unicore
=================
Roles
-----
Project Leads: Simon Kuenzer <simon.kuenzer@neclab.eu> (main lead)
Felipe Huici <felipe.huici@neclab.eu> (co-lead)
Florian Schmidt <florian.schmidt@neclab.eu> (co-lead)
Project Mentor: Lars Kurth <lars.kurth@citrix.com>
Project Sponsor: -To be found-
Background
----------
In recent years, several papers and projects dedicated to unikernels have
shown the immense potential for performance gains that these have. By
leveraging specialization and the use of minimalistic OSes, unikernels are
able to yield impressive numbers, including fast instantiation times (tens
of milliseconds or less), tiny memory footprints (a few MBs or even KBs),
high network throughput (10-40 Gb/s), and high consolidation (e.g., being
able to run thousands of instances on a single commodity server), not to
mention a reduced attack surface and the potential for easier
certification. Unikernel projects worthy of mention include MirageOS,
ClickOS, Erlang on Xen, OSv, HALVM, and Minicache, among others.
The fundamental drawback of unikernels is that they require that
applications be manually ported to the underlying minimalistic OS (e.g.
having to port nginx, snort, mysql or memcached to MiniOS or OSv); this
requires both expert work and often considerable amount of time. In
essence, we need to pick between either high performance with unikernels,
or no porting effort but decreased performance and decreased efficiency
with standard OS/VM images. The goal of this proposal is to change this
status quo by providing a highly configurable unikernel code base; we call
this base Unicore.
This project also aims to concentrate the various efforts currently going
on in the Xen community regarding minimalistic OSes (essentially different
variants of MiniOS). We think that splitting the community across these
variants is counter-productive and hope that Unicore will provide a common
place for all or most improvements and customizations of minimalistic
OSes. The long term goal is to replace something like MiniOS with a tool
that can automatically build such a minimalistic OS.
Unicore - The "Unikernel Core"
---------------------------------
The high level goal of Unicore is to be able to build unikernels targeted
at specific applications without requiring the time-consuming, expert work
that building such a unikernel requires today. An additional goal (or
hope) of Unicore is that all developers interested in unikernel
development would contribute by supplying libraries rather than working on
independent projects with different code bases as it is done now. The main
idea behind Unicore is depicted in Figure 1 and consists of two basic
components:
[Attachment: unicore-oneslider.pdf]
Figure 1. Unicore architecture.
Library pools would contain libraries that the user of Unicore can select
from to create the unikernel. From the bottom up, library pools are
organized into (1) the architecture library tool, containing libraries
specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the
platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no
virtualization) and user-space Linux; and (3) the main library pool,
containing a rich set of functionality to build the unikernel from. This
last library includes drivers (both virtual such as netback/netfront and
physical such as ixgbe), filesystems, memory allocators, schedulers,
network stacks, standard libs (e.g. libc, openssl, etc.), runtimes (e.g. a
Python interpreter and debugging and profiling tools. These pools of
libraries constitute a code base for creating unikernels. As shown, a
library can be relatively large (e.g libc) or quite small (a scheduler),
which should allow for a fair amount of customization for the unikernel.
The Unicore build tool is in charge of compiling the application and the
selected libraries together to create a binary for a specific platform and
architecture (e.g., Xen on x86_64). The tool is currently inspired by
Linux’s kconfig system and consists of a set of Makefiles. It allows users
to select libraries, to configure them, and to warn them when library
dependencies are not met. In addition, the tool can also simultaneously
generate binaries for multiple platforms.
As an example, imagine a user wanting to generate a network driver domain
unikernel. In this case, we would assume the “application” to be the
netback driver. To select this application, the user would first run “make
menuconfig” from within the netback application folder. The Makefile there
would set a variable to indicate what the application is, and would
include the main Unicore Makefiles so that the unikernel can be built
(Step 1 in the figure). Using the menu-based system, the user chooses the
relevant libraries; for a Xen driver domain this would include a physical
network driver, the netback driver, the libxenplat library and a library
from the architecture library pool such as libx86_64arch (Step 2 in the
figure). With this in place, the user saves the configuration and types
“make” to build the unikernel (Step 3) and xl create to run it (Step 4).
A note on the ABI/API: because Unicore allows for customization of the
unikernels, the ABI (or API since there is no kernel) would be custom,
that is, defined by the libraries the user selected. Having said that, it
would be perfectly possible, for instance, to build POSIX-compliant
unikernels with it.
Relevance to Xen and its Community
-----------------------------------
Unikernels are important to a number of areas relevant to the Xen
community, including IoT, automotive, stub domains and driver domain
disaggregation. Unicore could help boost the progress in all of these
areas by quickly providing the necessary tools to create unikernels for
them. For instance, for a driver domain, the user would include the
“library” containing the relevant hardware driver and corresponding
back-end driver, and in principle Unicore would take care of the rest.
In addition, Unicore could eventually replace Mini-OS, providing a
cleaner, more stable and flexible base from which to build unikernels for
projects (the modularization of Mini-OS is in fact already taking place).
Current Status
--------------
Unicore is at an early stage. For now it includes some base libraries with
code extracted from Mini-OS as well as a build tool inspired by
Linux's KConfig system. Unicore is currently able to build "hello world"
unikernels for Xen and Linux user space on x86_64 and ARMv7.
Incubation
----------
The reason behind making Unicore a Xen sub-project project is to (1) bring
the existence of Unicore to the attention of the Xen community
and to outside world; (2) to attempt to harness interest and potentially
development cycles from people and companies interested in
unikernels; and (3) to concentrate maintenance resources from people
interested in unikernels within the community.
License
-------
The main license of the run-time components of Unicore will be a 3-clause
BSD license, unless there is a good reason not to use it (e.g. we may
import 2-clause BSD licensed code from Mini-OS, which we would *not*
anticipate to change). The Makefile system would be licensed under GPL v2
or later as we want to be able to use KConfig functionality from
Buildroot/Linux.
Required Infrastructure
-----------------------
The official repositories should be created on
[http://xenbits.xenproject.org/] under `unicore.git`. There should be a
main repository for the core unicore implementation and additional
repositories for some more advanced extension libraries (e.g., lwIP,
newlib).
### Main repository
`unicore.git`
### Repositories for extension libraries
Repositories for additional libraries that are supported by the Unicore
project should exist under a separate directory:
`unicore-libs/`
For example:
`unicore-libs/lwip.git`
`unicore-libs/newlib.git`
### Mailing list
In the beginning we would use the MiniOS mailing list
(minios-devel@lists.xenproject.org). When we get traction with Unicore we
could consider splitting that traffic onto a unicore mailing list.
============================================================
Dr. Felipe Huici
Chief Researcher, Networked Systems and Data
Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49
(0)6221 4342-241
Fax: +49
(0)6221 4342-155
e-mail:
felipe.huici@neclab.eu
============================================================
NEC Europe Limited Registered Office: NEC House, 1
Victoria Road, London W3 6BL Registered in England 2832014
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
[not found] ` <776E12BC-8D5A-4B90-AF99-BFDDEBEECCE4@citrix.com>
@ 2017-09-20 0:30 ` Stefano Stabellini
2017-09-20 7:30 ` Felipe Huici
1 sibling, 0 replies; 25+ messages in thread
From: Stefano Stabellini @ 2017-09-20 0:30 UTC (permalink / raw)
To: Lars Kurth
Cc: Felipe Huici, stefano, julian.chesterfield, rshaposhnik, xen-api,
Saverio Niccolini, minios-devel, alexander.dubinin, julien.grall,
committers, mirageos-devel, xen-devel, Simon Kuenzer,
volodymyr_babchuk
[-- Attachment #1: Type: TEXT/PLAIN, Size: 10649 bytes --]
On Wed, 20 Sep 2017, Lars Kurth wrote:
> Felipe, Simon,
> a quick note to let you know that the Advisory Board in today’s AB meeting decided to endorse your proposal.
> Let me know how you proceed: from my perspective, we can kick off a formal vote before you make modifications to the proposal, but I think it is better to post v2 first.
Congratulations!
> On 07/09/2017, 05:26, "Felipe Huici" <Felipe.Huici@neclab.eu> wrote:
>
> Dear all,
>
> Following up on discussions that Simon Kuenzer had with several of you at
> the last Xen summit, we’re now submitting a Xen subproject proposal based
> on our Unicore work. Could you please review it?
>
> Thanks,
>
> Felipe Huici & Simon Kuenzer - NEC Labs Heidelberg.
>
>
> PROPOSAL: Unicore
> =================
>
> Roles
> -----
> Project Leads: Simon Kuenzer <simon.kuenzer@neclab.eu> (main lead)
> Felipe Huici <felipe.huici@neclab.eu> (co-lead)
> Florian Schmidt <florian.schmidt@neclab.eu> (co-lead)
> Project Mentor: Lars Kurth <lars.kurth@citrix.com>
> Project Sponsor: -To be found-
>
> Background
> ----------
> In recent years, several papers and projects dedicated to unikernels have
> shown the immense potential for performance gains that these have. By
> leveraging specialization and the use of minimalistic OSes, unikernels are
> able to yield impressive numbers, including fast instantiation times (tens
> of milliseconds or less), tiny memory footprints (a few MBs or even KBs),
> high network throughput (10-40 Gb/s), and high consolidation (e.g., being
> able to run thousands of instances on a single commodity server), not to
> mention a reduced attack surface and the potential for easier
> certification. Unikernel projects worthy of mention include MirageOS,
> ClickOS, Erlang on Xen, OSv, HALVM, and Minicache, among others.
>
> The fundamental drawback of unikernels is that they require that
> applications be manually ported to the underlying minimalistic OS (e.g.
> having to port nginx, snort, mysql or memcached to MiniOS or OSv); this
> requires both expert work and often considerable amount of time. In
> essence, we need to pick between either high performance with unikernels,
> or no porting effort but decreased performance and decreased efficiency
> with standard OS/VM images. The goal of this proposal is to change this
> status quo by providing a highly configurable unikernel code base; we call
> this base Unicore.
>
> This project also aims to concentrate the various efforts currently going
> on in the Xen community regarding minimalistic OSes (essentially different
> variants of MiniOS). We think that splitting the community across these
> variants is counter-productive and hope that Unicore will provide a common
> place for all or most improvements and customizations of minimalistic
> OSes. The long term goal is to replace something like MiniOS with a tool
> that can automatically build such a minimalistic OS.
>
> Unicore - The "Unikernel Core"
> ---------------------------------
> The high level goal of Unicore is to be able to build unikernels targeted
> at specific applications without requiring the time-consuming, expert work
> that building such a unikernel requires today. An additional goal (or
> hope) of Unicore is that all developers interested in unikernel
> development would contribute by supplying libraries rather than working on
> independent projects with different code bases as it is done now. The main
> idea behind Unicore is depicted in Figure 1 and consists of two basic
> components:
>
>
> [Attachment: unicore-oneslider.pdf]
>
>
> Figure 1. Unicore architecture.
>
>
> Library pools would contain libraries that the user of Unicore can select
> from to create the unikernel. From the bottom up, library pools are
> organized into (1) the architecture library tool, containing libraries
> specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the
> platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no
> virtualization) and user-space Linux; and (3) the main library pool,
> containing a rich set of functionality to build the unikernel from. This
> last library includes drivers (both virtual such as netback/netfront and
> physical such as ixgbe), filesystems, memory allocators, schedulers,
> network stacks, standard libs (e.g. libc, openssl, etc.), runtimes (e.g. a
> Python interpreter and debugging and profiling tools. These pools of
> libraries constitute a code base for creating unikernels. As shown, a
> library can be relatively large (e.g libc) or quite small (a scheduler),
> which should allow for a fair amount of customization for the unikernel.
>
>
> The Unicore build tool is in charge of compiling the application and the
> selected libraries together to create a binary for a specific platform and
> architecture (e.g., Xen on x86_64). The tool is currently inspired by
> Linux’s kconfig system and consists of a set of Makefiles. It allows users
> to select libraries, to configure them, and to warn them when library
> dependencies are not met. In addition, the tool can also simultaneously
> generate binaries for multiple platforms.
>
>
> As an example, imagine a user wanting to generate a network driver domain
> unikernel. In this case, we would assume the “application” to be the
> netback driver. To select this application, the user would first run “make
> menuconfig” from within the netback application folder. The Makefile there
> would set a variable to indicate what the application is, and would
> include the main Unicore Makefiles so that the unikernel can be built
> (Step 1 in the figure). Using the menu-based system, the user chooses the
> relevant libraries; for a Xen driver domain this would include a physical
> network driver, the netback driver, the libxenplat library and a library
> from the architecture library pool such as libx86_64arch (Step 2 in the
> figure). With this in place, the user saves the configuration and types
> “make” to build the unikernel (Step 3) and xl create to run it (Step 4).
>
>
> A note on the ABI/API: because Unicore allows for customization of the
> unikernels, the ABI (or API since there is no kernel) would be custom,
> that is, defined by the libraries the user selected. Having said that, it
> would be perfectly possible, for instance, to build POSIX-compliant
> unikernels with it.
>
>
> Relevance to Xen and its Community
> -----------------------------------
> Unikernels are important to a number of areas relevant to the Xen
> community, including IoT, automotive, stub domains and driver domain
> disaggregation. Unicore could help boost the progress in all of these
> areas by quickly providing the necessary tools to create unikernels for
> them. For instance, for a driver domain, the user would include the
> “library” containing the relevant hardware driver and corresponding
> back-end driver, and in principle Unicore would take care of the rest.
>
> In addition, Unicore could eventually replace Mini-OS, providing a
> cleaner, more stable and flexible base from which to build unikernels for
> projects (the modularization of Mini-OS is in fact already taking place).
>
>
> Current Status
> --------------
> Unicore is at an early stage. For now it includes some base libraries with
> code extracted from Mini-OS as well as a build tool inspired by
> Linux's KConfig system. Unicore is currently able to build "hello world"
> unikernels for Xen and Linux user space on x86_64 and ARMv7.
>
> Incubation
> ----------
> The reason behind making Unicore a Xen sub-project project is to (1) bring
> the existence of Unicore to the attention of the Xen community
> and to outside world; (2) to attempt to harness interest and potentially
> development cycles from people and companies interested in
> unikernels; and (3) to concentrate maintenance resources from people
> interested in unikernels within the community.
>
> License
> -------
> The main license of the run-time components of Unicore will be a 3-clause
> BSD license, unless there is a good reason not to use it (e.g. we may
> import 2-clause BSD licensed code from Mini-OS, which we would *not*
> anticipate to change). The Makefile system would be licensed under GPL v2
> or later as we want to be able to use KConfig functionality from
> Buildroot/Linux.
>
> Required Infrastructure
> -----------------------
> The official repositories should be created on
> [http://xenbits.xenproject.org/] under `unicore.git`. There should be a
> main repository for the core unicore implementation and additional
> repositories for some more advanced extension libraries (e.g., lwIP,
> newlib).
>
> ### Main repository
>
> `unicore.git`
>
> ### Repositories for extension libraries
>
> Repositories for additional libraries that are supported by the Unicore
> project should exist under a separate directory:
>
> `unicore-libs/`
>
> For example:
>
> `unicore-libs/lwip.git`
> `unicore-libs/newlib.git`
>
> ### Mailing list
>
> In the beginning we would use the MiniOS mailing list
> (minios-devel@lists.xenproject.org). When we get traction with Unicore we
> could consider splitting that traffic onto a unicore mailing list.
>
>
>
>
> ============================================================
> Dr. Felipe Huici
> Chief Researcher, Networked Systems and Data
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49
> (0)6221 4342-241
> Fax: +49
> (0)6221 4342-155
>
> e-mail:
> felipe.huici@neclab.eu
> ============================================================
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
>
>
>
>
>
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
[not found] ` <776E12BC-8D5A-4B90-AF99-BFDDEBEECCE4@citrix.com>
2017-09-20 0:30 ` Stefano Stabellini
@ 2017-09-20 7:30 ` Felipe Huici
1 sibling, 0 replies; 25+ messages in thread
From: Felipe Huici @ 2017-09-20 7:30 UTC (permalink / raw)
To: Lars Kurth, committers, xen-devel, xen-api, mirageos-devel, minios-devel
Cc: julian.chesterfield, rshaposhnik, Saverio Niccolini, stefano,
alexander.dubinin, julien.grall, Simon Kuenzer,
volodymyr_babchuk
Hi Lars,
Great news, thanks! We sent v2 to the mailing list on the 18th, 2 days
ago. When you say “post v2 first”, should we be posting it somewhere else
too?
Thanks,
— Felipe
============================================================
Dr. Felipe Huici
Chief Researcher, Networked Systems and Data
Analytics Group
NEC Laboratories Europe, Network Research Division
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel. +49
(0)6221 4342-241
Fax: +49
(0)6221 4342-155
e-mail:
felipe.huici@neclab.eu
============================================================
NEC Europe Limited Registered Office: NEC House, 1
Victoria Road, London W3 6BL Registered in England 2832014
On 9/20/17, 2:20 AM, "Lars Kurth" <lars.kurth@citrix.com> wrote:
>Felipe, Simon,
>a quick note to let you know that the Advisory Board in today’s AB
>meeting decided to endorse your proposal.
>Let me know how you proceed: from my perspective, we can kick off a
>formal vote before you make modifications to the proposal, but I think it
>is better to post v2 first.
>Lars
>
>On 07/09/2017, 05:26, "Felipe Huici" <Felipe.Huici@neclab.eu> wrote:
>
> Dear all,
>
> Following up on discussions that Simon Kuenzer had with several of
>you at
> the last Xen summit, we’re now submitting a Xen subproject proposal
>based
> on our Unicore work. Could you please review it?
>
> Thanks,
>
> Felipe Huici & Simon Kuenzer - NEC Labs Heidelberg.
>
>
> PROPOSAL: Unicore
> =================
>
> Roles
> -----
> Project Leads: Simon Kuenzer <simon.kuenzer@neclab.eu> (main lead)
> Felipe Huici <felipe.huici@neclab.eu> (co-lead)
> Florian Schmidt <florian.schmidt@neclab.eu> (co-lead)
> Project Mentor: Lars Kurth <lars.kurth@citrix.com>
> Project Sponsor: -To be found-
>
> Background
> ----------
> In recent years, several papers and projects dedicated to unikernels
>have
> shown the immense potential for performance gains that these have. By
> leveraging specialization and the use of minimalistic OSes,
>unikernels are
> able to yield impressive numbers, including fast instantiation times
>(tens
> of milliseconds or less), tiny memory footprints (a few MBs or even
>KBs),
> high network throughput (10-40 Gb/s), and high consolidation (e.g.,
>being
> able to run thousands of instances on a single commodity server), not
>to
> mention a reduced attack surface and the potential for easier
> certification. Unikernel projects worthy of mention include MirageOS,
> ClickOS, Erlang on Xen, OSv, HALVM, and Minicache, among others.
>
> The fundamental drawback of unikernels is that they require that
> applications be manually ported to the underlying minimalistic OS
>(e.g.
> having to port nginx, snort, mysql or memcached to MiniOS or OSv);
>this
> requires both expert work and often considerable amount of time. In
> essence, we need to pick between either high performance with
>unikernels,
> or no porting effort but decreased performance and decreased
>efficiency
> with standard OS/VM images. The goal of this proposal is to change
>this
> status quo by providing a highly configurable unikernel code base; we
>call
> this base Unicore.
>
> This project also aims to concentrate the various efforts currently
>going
> on in the Xen community regarding minimalistic OSes (essentially
>different
> variants of MiniOS). We think that splitting the community across
>these
> variants is counter-productive and hope that Unicore will provide a
>common
> place for all or most improvements and customizations of minimalistic
> OSes. The long term goal is to replace something like MiniOS with a
>tool
> that can automatically build such a minimalistic OS.
>
> Unicore - The "Unikernel Core"
> ---------------------------------
> The high level goal of Unicore is to be able to build unikernels
>targeted
> at specific applications without requiring the time-consuming, expert
>work
> that building such a unikernel requires today. An additional goal (or
> hope) of Unicore is that all developers interested in unikernel
> development would contribute by supplying libraries rather than
>working on
> independent projects with different code bases as it is done now. The
>main
> idea behind Unicore is depicted in Figure 1 and consists of two basic
> components:
>
>
> [Attachment: unicore-oneslider.pdf]
>
>
> Figure 1. Unicore architecture.
>
>
> Library pools would contain libraries that the user of Unicore can
>select
> from to create the unikernel. From the bottom up, library pools are
> organized into (1) the architecture library tool, containing libraries
> specific to a computer architecture (e.g., x86_64, ARM32 or MIPS);
>(2) the
> platform tool, where target platforms can be Xen, KVM, bare metal
>(i.e. no
> virtualization) and user-space Linux; and (3) the main library pool,
> containing a rich set of functionality to build the unikernel from.
>This
> last library includes drivers (both virtual such as netback/netfront
>and
> physical such as ixgbe), filesystems, memory allocators, schedulers,
> network stacks, standard libs (e.g. libc, openssl, etc.), runtimes
>(e.g. a
> Python interpreter and debugging and profiling tools. These pools of
> libraries constitute a code base for creating unikernels. As shown, a
> library can be relatively large (e.g libc) or quite small (a
>scheduler),
> which should allow for a fair amount of customization for the
>unikernel.
>
>
> The Unicore build tool is in charge of compiling the application and
>the
> selected libraries together to create a binary for a specific
>platform and
> architecture (e.g., Xen on x86_64). The tool is currently inspired by
> Linux’s kconfig system and consists of a set of Makefiles. It allows
>users
> to select libraries, to configure them, and to warn them when library
> dependencies are not met. In addition, the tool can also
>simultaneously
> generate binaries for multiple platforms.
>
>
> As an example, imagine a user wanting to generate a network driver
>domain
> unikernel. In this case, we would assume the “application” to be the
> netback driver. To select this application, the user would first run
>“make
> menuconfig” from within the netback application folder. The Makefile
>there
> would set a variable to indicate what the application is, and would
> include the main Unicore Makefiles so that the unikernel can be built
> (Step 1 in the figure). Using the menu-based system, the user chooses
>the
> relevant libraries; for a Xen driver domain this would include a
>physical
> network driver, the netback driver, the libxenplat library and a
>library
> from the architecture library pool such as libx86_64arch (Step 2 in
>the
> figure). With this in place, the user saves the configuration and
>types
> “make” to build the unikernel (Step 3) and xl create to run it (Step
>4).
>
>
> A note on the ABI/API: because Unicore allows for customization of the
> unikernels, the ABI (or API since there is no kernel) would be custom,
> that is, defined by the libraries the user selected. Having said
>that, it
> would be perfectly possible, for instance, to build POSIX-compliant
> unikernels with it.
>
>
> Relevance to Xen and its Community
> -----------------------------------
> Unikernels are important to a number of areas relevant to the Xen
> community, including IoT, automotive, stub domains and driver domain
> disaggregation. Unicore could help boost the progress in all of these
> areas by quickly providing the necessary tools to create unikernels
>for
> them. For instance, for a driver domain, the user would include the
> “library” containing the relevant hardware driver and corresponding
> back-end driver, and in principle Unicore would take care of the rest.
>
> In addition, Unicore could eventually replace Mini-OS, providing a
> cleaner, more stable and flexible base from which to build unikernels
>for
> projects (the modularization of Mini-OS is in fact already taking
>place).
>
>
> Current Status
> --------------
> Unicore is at an early stage. For now it includes some base libraries
>with
> code extracted from Mini-OS as well as a build tool inspired by
> Linux's KConfig system. Unicore is currently able to build "hello
>world"
> unikernels for Xen and Linux user space on x86_64 and ARMv7.
>
> Incubation
> ----------
> The reason behind making Unicore a Xen sub-project project is to (1)
>bring
> the existence of Unicore to the attention of the Xen community
> and to outside world; (2) to attempt to harness interest and
>potentially
> development cycles from people and companies interested in
> unikernels; and (3) to concentrate maintenance resources from people
> interested in unikernels within the community.
>
> License
> -------
> The main license of the run-time components of Unicore will be a
>3-clause
> BSD license, unless there is a good reason not to use it (e.g. we may
> import 2-clause BSD licensed code from Mini-OS, which we would *not*
> anticipate to change). The Makefile system would be licensed under
>GPL v2
> or later as we want to be able to use KConfig functionality from
> Buildroot/Linux.
>
> Required Infrastructure
> -----------------------
> The official repositories should be created on
> [http://xenbits.xenproject.org/] under `unicore.git`. There should be
>a
> main repository for the core unicore implementation and additional
> repositories for some more advanced extension libraries (e.g., lwIP,
> newlib).
>
> ### Main repository
>
> `unicore.git`
>
> ### Repositories for extension libraries
>
> Repositories for additional libraries that are supported by the
>Unicore
> project should exist under a separate directory:
>
> `unicore-libs/`
>
> For example:
>
> `unicore-libs/lwip.git`
> `unicore-libs/newlib.git`
>
> ### Mailing list
>
> In the beginning we would use the MiniOS mailing list
> (minios-devel@lists.xenproject.org). When we get traction with
>Unicore we
> could consider splitting that traffic onto a unicore mailing list.
>
>
>
>
> ============================================================
> Dr. Felipe Huici
> Chief Researcher, Networked Systems and Data
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel. +49
> (0)6221 4342-241
> Fax: +49
> (0)6221 4342-155
>
> e-mail:
> felipe.huici@neclab.eu
> ============================================================
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
>
>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC] Unicore Subproject Proposal
[not found] ` <D5E553D1.39F25%felipe.huici@neclab.eu-0>
@ 2017-09-20 21:24 ` Lars Kurth
0 siblings, 0 replies; 25+ messages in thread
From: Lars Kurth @ 2017-09-20 21:24 UTC (permalink / raw)
To: Felipe Huici, Simon Kuenzer, Alexander Dubinin
Cc: stefano, Stefano Stabellini, Wei Liu, julian.chesterfield,
rshaposhnik, xen-api, info, mato, Saverio Niccolini,
minios-devel, julien.grall, committers, mirageos-devel,
xen-devel, pooka, awick, volodymyr_babchuk@epam.com
On 18/09/2017, 05:17, "Felipe Huici" <Felipe.Huici@neclab.eu> wrote:
Hi Lars, all,
[cc’ing authors of Erlang on Xen, HalVM and Rump].
Thanks everyone for all of the support and useful comments. We’ve
incorporated a number of them into a new version of the document (attached
and pasted at the bottom for convenience) and for those that didn’t make
it we’re keeping track of them.
Lars, FYI, Simon also did a blog post regarding Unicore on unikernel.org
(https://devel.unikernel.org/t/unicore-a-new-unikernel-project/274).
Please let us know what the next steps are.
I will set up the formal vote next week! It doesn’t need the full distribution list
Lars
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2017-09-20 21:24 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <D5D6EB26.39481%felipe.huici@neclab.eu-0>
2017-09-07 13:41 ` [RFC] Unicore Subproject Proposal Lars Kurth
2017-09-07 21:16 ` Stefano Stabellini
[not found] ` <alpine.DEB.2.10.1709071414310.4600@sstabellini-ThinkPad-X260>
2017-09-08 11:00 ` Lars Kurth
2017-09-08 12:31 ` Felipe Huici
2017-09-10 20:48 ` Alexander Dubinin
2017-09-11 12:08 ` Simon Kuenzer
2017-09-13 8:36 ` Alexander Dubinin
2017-09-15 13:01 ` Simon Kuenzer
2017-09-13 10:11 ` [Xen-API] " Anil Madhavapeddy
2017-09-13 16:55 ` Lars Kurth
[not found] ` <C8ACDEE5-D1EE-4AFE-B59C-5FBA25623EAA@recoil.org>
2017-09-13 16:13 ` [Minios-devel] [Xen-API] " Samuel Thibault
2017-09-13 19:27 ` Felipe Huici
[not found] ` <20170913161345.fnndedfh74u5k3wh@var.youpi.perso.aquilenet.fr>
2017-09-14 14:38 ` [MirageOS-devel] [Minios-devel] " Anil Madhavapeddy
2017-09-14 14:47 ` [Minios-devel] [MirageOS-devel] " Samuel Thibault
[not found] ` <20170914144756.jorfjgwarbmpwrhq@var.youpi.perso.aquilenet.fr>
2017-09-15 13:23 ` Simon Kuenzer
2017-09-15 8:36 ` [Minios-devel] " Simon Kuenzer
[not found] ` <eba0ace6-6160-342d-dba4-c40324e67d3f@neclab.eu>
2017-09-15 9:35 ` [MirageOS-devel] " Anil Madhavapeddy
2017-09-15 12:37 ` Simon Kuenzer
[not found] ` <3B0A2B25-6A6A-4DE9-845C-E56812B97F92@citrix.com>
2017-09-15 8:50 ` Simon Kuenzer
2017-09-18 9:16 ` Felipe Huici
2017-09-19 22:56 ` Stefano Stabellini
[not found] ` <D5E553D1.39F25%felipe.huici@neclab.eu-0>
2017-09-20 21:24 ` Lars Kurth
2017-09-20 0:20 ` Lars Kurth
[not found] ` <776E12BC-8D5A-4B90-AF99-BFDDEBEECCE4@citrix.com>
2017-09-20 0:30 ` Stefano Stabellini
2017-09-20 7:30 ` Felipe Huici
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.