BMC (Bare-Metal Container) is relased

BMC (Bare-Metal Container) is relased

[須崎有康]

Hello,

We released BMC (Bare-Metal Container) which runs a container (Docker)
image with a suitable Linux kernel on a remote physical machine.
BMC allows to change the kernel and its settings for the container
(Docker) image. As a result, the application extracts the full
performance of the physical machine.

A pre-built BMC image using DIND (Docker in Docker) technique is
offered. Users can avid burdensome setting up (ex. Apache CGI, etc).
We strongly recommend new users to try this pre-built version.

Pre-built BMC Doceker Image https://hub.docker.com/r/baremetalcontainer/bmc/
Source Code: https://github.com/baremetalcontainer/bmc
Official HP: http://www.itri.aist.go.jp/cpc/research/bmc/

BMC is exhibited at the Super Computing 2016 (AIST booth 1709) held at
Salt Lake City next week (Nov/14-17).
  http://sc16.supercomputing.org

The detail of BMC will be presented at IEEE HPCC (High Performance
Computing and Communication) 2016 (Dec/12-14) held at Sydney.
  http://www.swinflow.org/confs/2016/hpcc/

-----------
Kuni Suzaki

Re: BMC (Bare-Metal Container) is relased

[Kuniyasu Suzaki]

I made a presentation of BMC at HPCC (IEEE Conference on High Performance Computing and Communications) 2016.
   http://www.swinflow.org/confs/2016/hpcc/
The title is "Bare-Metal Container --- Direct execution of a container image on a remote machine with an optimized kernel ---".
The slide is opened at slideshare.
   http://www.slideshare.net/suzaki/baremetal-container-presented-at-hpcc2016
   Original HP: http://www.itri.aist.go.jp/cpc/research/bmc/

BMC is resemble to VMware's "vSphere Integrated Containers" and Joyent's "Triton", but 
BMC is designed to customize Linux kernel for an application in a Docker image.
   VMware's "vSphere Integrated Containers"  http://www.vmware.com/products/vsphere/integrated-containers.html
   Jyoent's "Triton"  https://www.joyent.com/triton

BMC runs a Docker image with a customized kernel on a bare-metal remote machine using remote machine management technology (WakeupOnLan, Intel AMT, or IPMI) and network bootloader "iPXE".
BMC can solve the container's (Docker's) problem which is caused by Linux kernel.

If you are interesed in, please try.

-----------
Kuni Suzaki

From: 須崎有康 <k.suzaki-XSdjUN4cZ6fPDbFq/vQRIQ@public.gmane.org>
Subject: BMC (Bare-Metal Container) is relased
Date: Fri, 11 Nov 2016 15:54:13 +0900

> Hello,
> 
> We released BMC (Bare-Metal Container) which runs a container (Docker)
> image with a suitable Linux kernel on a remote physical machine.
> BMC allows to change the kernel and its settings for the container
> (Docker) image. As a result, the application extracts the full
> performance of the physical machine.
> 
> A pre-built BMC image using DIND (Docker in Docker) technique is
> offered. Users can avid burdensome setting up (ex. Apache CGI, etc).
> We strongly recommend new users to try this pre-built version.
> 
> Pre-built BMC Doceker Image https://hub.docker.com/r/baremetalcontainer/bmc/
> Source Code: https://github.com/baremetalcontainer/bmc
> Official HP: http://www.itri.aist.go.jp/cpc/research/bmc/
> 
> BMC is exhibited at the Super Computing 2016 (AIST booth 1709) held at
> Salt Lake City next week (Nov/14-17).
>   http://sc16.supercomputing.org
> 
> The detail of BMC will be presented at IEEE HPCC (High Performance
> Computing and Communication) 2016 (Dec/12-14) held at Sydney.
>   http://www.swinflow.org/confs/2016/hpcc/
> 
> -----------
> Kuni Suzaki

Re: BMC (Bare-Metal Container) is relased

[Eric W. Biederman]

BMC is a very unfortunately name in your context as it typically refers
to basboard management controller.


Kuniyasu Suzaki <k.suzaki-XSdjUN4cZ6fPDbFq/vQRIQ@public.gmane.org> writes:
> BMC can solve the container's (Docker's) problem which is caused by
> Linux kernel.

Which problem is that?

Eric

Re: BMC (Bare-Metal Container) is relased

[Kuniyasu Suzaki]

From: "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
Subject: Re: BMC (Bare-Metal Container) is relased
Date: Mon, 19 Dec 2016 18:46:03 +1300

> BMC is a very unfortunately name in your context as it typically refers
> to basboard management controller.

Yes, we know.
We hope Bare-Metal Container grews up more famous than Baseboard Management Controller.

> Kuniyasu Suzaki <k.suzaki-XSdjUN4cZ6fPDbFq/vQRIQ@public.gmane.org> writes:
>> BMC can solve the container's (Docker's) problem which is caused by
>> Linux kernel.
> 
> Which problem is that?

For exmaple, DPDK requires "igb_uio" and "rte_kni" kernel modules, but some kernels offered as a part of Linux Distribution do not include them.
So, the container on these kernels can not run DPDK. Forthermore the normal setting of Transparet Huge Pages caused problems on some DB applications.
BMC try to apply suitable Linux kernel for a container image.
These cases are reported on my paper published at IEEE Conference on High Performance Computing and Communications (HPCC) 2016.
The paper and slide are downloadable.
   Paper PDF: https://staff.aist.go.jp/k.suzaki/HPCC16-BareMetalContainer.pdf
   Slide PDF: http://www.slideshare.net/suzaki/baremetal-container-presented-at-hpcc2016

Thank you.

------
Kuni Suzaki

Re: BMC (Bare-Metal Container) is relased

[James Bottomley]

On Mon, 2016-12-19 at 19:04 +0900, Kuniyasu Suzaki wrote:
> From: "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
> Subject: Re: BMC (Bare-Metal Container) is relased
> Date: Mon, 19 Dec 2016 18:46:03 +1300
> 
> > BMC is a very unfortunately name in your context as it typically
> > refers
> > to basboard management controller.
> 
> Yes, we know.
> We hope Bare-Metal Container grews up more famous than Baseboard
> Management Controller.
> 
> > Kuniyasu Suzaki <k.suzaki-XSdjUN4cZ6fPDbFq/vQRIQ@public.gmane.org> writes:
> > > BMC can solve the container's (Docker's) problem which is caused
> > > by
> > > Linux kernel.
> > 
> > Which problem is that?
> 
> For exmaple, DPDK requires "igb_uio" and "rte_kni" kernel modules,
> but some kernels offered as a part of Linux Distribution do not
> include them.

That makes it a Distribution problem not a kernel problem, really. 
 Constainers is OS virtualization, so if the OS doesn't provide a
feature, it can't be virtualized.

The problem for those distributions is that the features you want
inside the container aren't available in bare metal instances either,
which means if they're really a need, the distro eventually provides
them or suffers in the market place.  This tends to force all distros
to supporting all useful features meaning the kernel configuration
problem is mostly a theoretical one.

> So, the container on these kernels can not run DPDK. Forthermore the
> normal setting of Transparet Huge Pages caused problems on some DB
> applications.

The THP problem might be more interesting, but your paper mentions it
in passing as hearsay (cloudera said this about RHEL 6.2) ... is there
any more recent data about it (RHEL 6.2 is 2.6.32 based, which is now a
seven year old kernel)?

James

> BMC try to apply suitable Linux kernel for a container image.
> These cases are reported on my paper published at IEEE Conference on
> High Performance Computing and Communications (HPCC) 2016.
> The paper and slide are downloadable.
>    Paper PDF: 
> https://staff.aist.go.jp/k.suzaki/HPCC16-BareMetalContainer.pdf
>    Slide PDF: http://www.slideshare.net/suzaki/baremetal-container-pr
> esented-at-hpcc2016
> 
> Thank you.
> 
> ------
> Kuni Suzaki
> _______________________________________________
> Containers mailing list
> Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> https://lists.linuxfoundation.org/mailman/listinfo/containers
> 

Re: BMC (Bare-Metal Container) is relased

[Kuniyasu Suzaki]

From: James Bottomley <James.Bottomley-d9PhHud1JfjCXq6kfMZ53/egYHeGw8Jk@public.gmane.org>
Subject: Re: BMC (Bare-Metal Container) is relased
Date: Mon, 19 Dec 2016 07:51:15 -0800

> On Mon, 2016-12-19 at 19:04 +0900, Kuniyasu Suzaki wrote:
>> From: "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
>> Subject: Re: BMC (Bare-Metal Container) is relased
>> Date: Mon, 19 Dec 2016 18:46:03 +1300
>> 
>> For exmaple, DPDK requires "igb_uio" and "rte_kni" kernel modules,
>> but some kernels offered as a part of Linux Distribution do not
>> include them.
> 
> That makes it a Distribution problem not a kernel problem, really. 
>  Constainers is OS virtualization, so if the OS doesn't provide a
> feature, it can't be virtualized.
> 
> The problem for those distributions is that the features you want
> inside the container aren't available in bare metal instances either,
> which means if they're really a need, the distro eventually provides
> them or suffers in the market place.  This tends to force all distros
> to supporting all useful features meaning the kernel configuration
> problem is mostly a theoretical one.

Yes. DPDK on a container is just an example which does not run on a normal Linux distributions.
It is not kernel problem, but it is caused by the mismatch between application and kernel.

THP (Transparent Huge Pages) and HTT (Hyper Threading Technology) are better examples for Bare-Metal Container.
They work system-wide and affects all applications, but some applications want to turn off.
It is system centric architecture we call, and all applications must follow the decision.

On the other hand, BMC tries to offer a suitable kernel for an application on a remote machine.
We call this application centric architecture.

I guess same problem may cause on other container technologies.
  VMware's "vSphere Integrated Containers"   https://www.vmware.com/products/vsphere/integrated-containers.html   
  Joyent's "Triton"                          https://www.joyent.com/triton                          

>> So, the container on these kernels can not run DPDK. Forthermore the
>> normal setting of Transparet Huge Pages caused problems on some DB
>> applications.
> 
> The THP problem might be more interesting, but your paper mentions it
> in passing as hearsay (cloudera said this about RHEL 6.2) ... is there
> any more recent data about it (RHEL 6.2 is 2.6.32 based, which is now a
> seven year old kernel)?

You can confirm the problem using "parallel".
   https://github.com/rust-lang/rust/issues/36705

I have made a Docker image. Please try on your machine.
--------------------------------------------------------------------
On the Host Linux
$ sudo echo never > /sys/kernel/mm/transparent_hugepage/enabled
or 
$ sudo echo always > /sys/kernel/mm/transparent_hugepage/enabled

Run parallel on Docker.
$ docker run -it suzaki/rust-parallel bash
# export PATH=$PATH:/root/.cargo/bin
# time seq 1 10000 | parallel echo > /dev/null
--------------------------------------------------------------------
Thank you.

-----------
Kuni Suzaki

> James
> 
>> BMC try to apply suitable Linux kernel for a container image.
>> These cases are reported on my paper published at IEEE Conference on
>> High Performance Computing and Communications (HPCC) 2016.
>> The paper and slide are downloadable.
>>    Paper PDF: 
>> https://staff.aist.go.jp/k.suzaki/HPCC16-BareMetalContainer.pdf
>>    Slide PDF: http://www.slideshare.net/suzaki/baremetal-container-pr
>> esented-at-hpcc2016
>> 
>> Thank you.
>> 
>> ------
>> Kuni Suzaki
>> _______________________________________________
>> Containers mailing list
>> Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
>> https://lists.linuxfoundation.org/mailman/listinfo/containers
>> 
> 

Re: BMC (Bare-Metal Container) is relased

[Eric W. Biederman]

Kuniyasu Suzaki <k.suzaki-XSdjUN4cZ6fPDbFq/vQRIQ@public.gmane.org> writes:

> From: James Bottomley <James.Bottomley-d9PhHud1JfjCXq6kfMZ53/egYHeGw8Jk@public.gmane.org>
> Subject: Re: BMC (Bare-Metal Container) is relased
> Date: Mon, 19 Dec 2016 07:51:15 -0800
>
>> On Mon, 2016-12-19 at 19:04 +0900, Kuniyasu Suzaki wrote:
>>> From: "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
>>> Subject: Re: BMC (Bare-Metal Container) is relased
>>> Date: Mon, 19 Dec 2016 18:46:03 +1300
>>> 
>>> For exmaple, DPDK requires "igb_uio" and "rte_kni" kernel modules,
>>> but some kernels offered as a part of Linux Distribution do not
>>> include them.
>> 
>> That makes it a Distribution problem not a kernel problem, really. 
>>  Constainers is OS virtualization, so if the OS doesn't provide a
>> feature, it can't be virtualized.
>> 
>> The problem for those distributions is that the features you want
>> inside the container aren't available in bare metal instances either,
>> which means if they're really a need, the distro eventually provides
>> them or suffers in the market place.  This tends to force all distros
>> to supporting all useful features meaning the kernel configuration
>> problem is mostly a theoretical one.
>
> Yes. DPDK on a container is just an example which does not run on a
> normal Linux distributions.  It is not kernel problem, but it is
> caused by the mismatch between application and kernel.
>
> THP (Transparent Huge Pages) and HTT (Hyper Threading Technology) are
> better examples for Bare-Metal Container.  They work system-wide and
> affects all applications, but some applications want to turn off.  It
> is system centric architecture we call, and all applications must
> follow the decision.
>
> On the other hand, BMC tries to offer a suitable kernel for an
> application on a remote machine. We call this application centric
> architecture.

In the past I have always seen this sort of problem handled as a
property to the scheduler, and a declaration that a certain job needs
certain properties.

Eric

Re: BMC (Bare-Metal Container) is relased

[Jeremy Eder]

On Mon, Dec 19, 2016 at 10:57 PM, Eric W. Biederman <ebiederm@xmission.com>
wrote:

> In the past I have always seen this sort of problem handled as a
> property to the scheduler, and a declaration that a certain job needs
> certain properties.


Indeed, we have discussed this as a property of the scheduler in kubernetes.

- Nodes are given attributes (i.e. THP=never)
- Users declare their requirements (i.e. THP=never)
- User submits job, scheduler does matching to node
- Node executes job

This allows for shared control plane, serving disparate workloads on hybrid
infra.

​THP=never is just another node attribute, as could be things like
- Does a DPDK-capable NIC or GPU or FPGA exist, and is unused
- What kernel version is it running
- Is the node tuned correctly

...myriad other possibilities, in a rapidly evolving space.
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

Re: BMC (Bare-Metal Container) is relased

[Kuniyasu Suzaki]

From: Jeremy Eder <jeder@redhat.com>
Subject: Re: BMC (Bare-Metal Container) is relased
Date: Tue, 20 Dec 2016 10:04:12 -0500

>  Eric W. Biederman <ebiederm@xmission.com>
> wrote:
> 
>> In the past I have always seen this sort of problem handled as a
>> property to the scheduler, and a declaration that a certain job needs
>> certain properties.
> 
> Indeed, we have discussed this as a property of the scheduler in kubernetes.
> 
> - Nodes are given attributes (i.e. THP=never)
> - Users declare their requirements (i.e. THP=never)
> - User submits job, scheduler does matching to node
> - Node executes job
> 
> This allows for shared control plane, serving disparate workloads on hybrid
> infra.
> 
> ​THP=never is just another node attribute, as could be things like
> - Does a DPDK-capable NIC or GPU or FPGA exist, and is unused
> - What kernel version is it running
> - Is the node tuned correctly
> 
> ...myriad other possibilities, in a rapidly evolving space.

It sounds interesting. I did not know that kubernetes has such a function.
Is there HP or paper to described the detail? I want to compare the function with our BMC (Bare-Metal Container).

I guess the function requires nodes which turn on/off THP.
On the other hand, BMC boots a node with the Linux kernel which is suitable for an application.
BMC needs the overhead to boot a node and allows to select a node with consideration of machine architecture, power usage, etc.

--------
Kuniyasu Suzaki, National Institute of Advanced Industrial Science and Technology,
http://staff.aist.go.jp/k.suzaki

_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

Re: BMC (Bare-Metal Container) is relased

[Jeremy Eder]

On Tue, Dec 20, 2016 at 8:34 PM, Kuniyasu Suzaki <k.suzaki@aist.go.jp>
wrote:

>
> From: Jeremy Eder <jeder@redhat.com>
> Subject: Re: BMC (Bare-Metal Container) is relased
> Date: Tue, 20 Dec 2016 10:04:12 -0500
>
> >  Eric W. Biederman <ebiederm@xmission.com>
> > wrote:
> >
> >> In the past I have always seen this sort of problem handled as a
> >> property to the scheduler, and a declaration that a certain job needs
> >> certain properties.
> >
> > Indeed, we have discussed this as a property of the scheduler in
> kubernetes.
> >
> > - Nodes are given attributes (i.e. THP=never)
> > - Users declare their requirements (i.e. THP=never)
> > - User submits job, scheduler does matching to node
> > - Node executes job
> >
> > This allows for shared control plane, serving disparate workloads on
> hybrid
> > infra.
> >
> > ​THP=never is just another node attribute, as could be things like
> > - Does a DPDK-capable NIC or GPU or FPGA exist, and is unused
> > - What kernel version is it running
> > - Is the node tuned correctly
> >
> > ...myriad other possibilities, in a rapidly evolving space.
>
> It sounds interesting. I did not know that kubernetes has such a function.
> Is there HP or paper to described the detail? I want to compare the
> function with our BMC (Bare-Metal Container).
>

A lot of it is there now using taints and tolerations.  Arguably needs some
end-to-end polish/usability improvements.  Folks from Intel are working on
at least the hardware detection stuff to enable their CPUs/NICs in a
generic way.


> I guess the function requires nodes which turn on/off THP.
> On the other hand, BMC boots a node with the Linux kernel which is
> suitable for an application.
> BMC needs the overhead to boot a node and allows to select a node with
> consideration of machine architecture, power usage, etc.


Trying to avoid an orchestration debate :-)  Just figured I'd +1 Eric's
comment about how knowledge of node properties tends to eventually be
recognized at the scheduler level and use kube as an example.
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers