Xen Memory de-duplication

Xen Memory de-duplication

[Maryam Masoudian]

[-- Attachment #1.1: Type: text/plain, Size: 379 bytes --]

Hi,

I have some questions about memory de-duplication in xen.
Firstly, I want to know if this feature is available in Xen .
If it is, how can I enable it in Xen 4.6 HVM mode.

Also, if I want to do some logging or other action whenever a COW happens
on shared memory, where should I add my codes? (is the /xen/memshr folder
containing the memory de-duplication code)?

Regards,

[-- Attachment #1.2: Type: text/html, Size: 496 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen Memory de-duplication

[Wei Liu]

On Sun, Apr 17, 2016 at 03:55:00PM +0430, Maryam Masoudian wrote:
> Hi,
> 
> I have some questions about memory de-duplication in xen.
> Firstly, I want to know if this feature is available in Xen .
> If it is, how can I enable it in Xen 4.6 HVM mode.
> 
> Also, if I want to do some logging or other action whenever a COW happens
> on shared memory, where should I add my codes? (is the /xen/memshr folder
> containing the memory de-duplication code)?
> 

I don't seem to find xen/memshr, only tools/memshr -- that's the tool
for manipulating memory sharing. You might want to look into hypervsior
code arch/x86/mm/mem_sharing.c.

Note that memory sharing is not actively maintained so you might
encounter all sorts of problems and on your own to solve them.

Wei.

> Regards,

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen Memory de-duplication

[Maryam Masoudian]

[-- Attachment #1.1: Type: text/plain, Size: 1235 bytes --]

Well, yes memshr is at tools/memshr .
Then , after modifying this c file, should I compile Xen from scratch?
And is the sharing feature enabled by default in Xen 4.6 ? If not, how to
enable it?


On Sun, Apr 17, 2016 at 4:45 PM, Wei Liu <wei.liu2@citrix.com> wrote:

> On Sun, Apr 17, 2016 at 03:55:00PM +0430, Maryam Masoudian wrote:
> > Hi,
> >
> > I have some questions about memory de-duplication in xen.
> > Firstly, I want to know if this feature is available in Xen .
> > If it is, how can I enable it in Xen 4.6 HVM mode.
> >
> > Also, if I want to do some logging or other action whenever a COW happens
> > on shared memory, where should I add my codes? (is the /xen/memshr folder
> > containing the memory de-duplication code)?
> >
>
> I don't seem to find xen/memshr, only tools/memshr -- that's the tool
> for manipulating memory sharing. You might want to look into hypervsior
> code arch/x86/mm/mem_sharing.c.
>
> Note that memory sharing is not actively maintained so you might
> encounter all sorts of problems and on your own to solve them.
>
> Wei.
>
> > Regards,
>



>
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
>
>

[-- Attachment #1.2: Type: text/html, Size: 2033 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen Memory de-duplication

[Wei Liu]

Please don't top post.

On Sun, Apr 17, 2016 at 06:28:15PM +0430, Maryam Masoudian wrote:
> Well, yes memshr is at tools/memshr .
> Then , after modifying this c file, should I compile Xen from scratch?

No, you only need to compile the tools -- or, just that tool.

> And is the sharing feature enabled by default in Xen 4.6 ? If not, how to
> enable it?
> 

The hypervisor code is compiled in by default on x86_64 in 4.6.

Wei.

> 
> On Sun, Apr 17, 2016 at 4:45 PM, Wei Liu <wei.liu2@citrix.com> wrote:
> 
> > On Sun, Apr 17, 2016 at 03:55:00PM +0430, Maryam Masoudian wrote:
> > > Hi,
> > >
> > > I have some questions about memory de-duplication in xen.
> > > Firstly, I want to know if this feature is available in Xen .
> > > If it is, how can I enable it in Xen 4.6 HVM mode.
> > >
> > > Also, if I want to do some logging or other action whenever a COW happens
> > > on shared memory, where should I add my codes? (is the /xen/memshr folder
> > > containing the memory de-duplication code)?
> > >
> >
> > I don't seem to find xen/memshr, only tools/memshr -- that's the tool
> > for manipulating memory sharing. You might want to look into hypervsior
> > code arch/x86/mm/mem_sharing.c.
> >
> > Note that memory sharing is not actively maintained so you might
> > encounter all sorts of problems and on your own to solve them.
> >
> > Wei.
> >
> > > Regards,
> >
> 
> 
> 
> >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xen.org
> > > http://lists.xen.org/xen-devel
> >
> >

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen Memory de-duplication

[Wei Liu]

On Tue, Apr 19, 2016 at 07:07:01AM -0400, Maryam Masoudian wrote:
> Thanks.
> One more thing, is it possible to see the percentage of shared memory among
> VMs by xl tool?
> As there is a xl sharing and I want to know whether this provides this
> facility ...
> 

Not sure. You can check xl manpage for those commands.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen Memory de-duplication

[Tamas K Lengyel]

[-- Attachment #1.1: Type: text/plain, Size: 734 bytes --]

On Wed, Apr 20, 2016 at 7:24 AM, Wei Liu <wei.liu2@citrix.com> wrote:

> On Tue, Apr 19, 2016 at 07:07:01AM -0400, Maryam Masoudian wrote:
> > Thanks.
> > One more thing, is it possible to see the percentage of shared memory
> among
> > VMs by xl tool?
> > As there is a xl sharing and I want to know whether this provides this
> > facility ...
> >
>
> Not sure. You can check xl manpage for those commands.


xl doesn't have any command that shows you a breakdown of sharing among
VMs, but if you run xl info it will tell you the total amount of memory
shared and amount saved with sharing. If you need the information for a
particular VM, you will need to query Xen using the xc_domain_getinfo
function in the libxc library.

Tamas

[-- Attachment #1.2: Type: text/html, Size: 1121 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen Memory de-duplication

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1230 bytes --]

On Wed, 2016-04-20 at 10:25 -0600, Tamas K Lengyel wrote:
> 
> 
> On Wed, Apr 20, 2016 at 7:24 AM, Wei Liu <wei.liu2@citrix.com> wrote:
> > On Tue, Apr 19, 2016 at 07:07:01AM -0400, Maryam Masoudian wrote:
> > > Thanks.
> > > One more thing, is it possible to see the percentage of shared
> > memory among
> > > VMs by xl tool?
> > > As there is a xl sharing and I want to know whether this provides
> > this
> > > facility ...
> > >
> > 
> > Not sure. You can check xl manpage for those commands.
> xl doesn't have any command that shows you a breakdown of sharing
> among VMs, 
>
Can we add it? (as soon as practical, of course)

There are other per-vm memory related info that I'd like to be able to
query from xl --like how much memory is on each NUMA node-- so a new
sub-command like `xl meminfo' (name subject to improvement! :-D), with
various parameters selecting what to actually show, could be an useful
addition.

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen Memory de-duplication

[Tamas K Lengyel]

[-- Attachment #1.1: Type: text/plain, Size: 838 bytes --]

On Apr 20, 2016 12:34, "Dario Faggioli" <dario.faggioli@citrix.com> wrote:
>
> On Wed, 2016-04-20 at 10:25 -0600, Tamas K Lengyel wrote:
> >
> >
> > On Wed, Apr 20, 2016 at 7:24 AM, Wei Liu <wei.liu2@citrix.com> wrote:
> > > On Tue, Apr 19, 2016 at 07:07:01AM -0400, Maryam Masoudian wrote:
> > > > Thanks.
> > > > One more thing, is it possible to see the percentage of shared
> > > memory among
> > > > VMs by xl tool?
> > > > As there is a xl sharing and I want to know whether this provides
> > > this
> > > > facility ...
> > > >
> > >
> > > Not sure. You can check xl manpage for those commands.
> > xl doesn't have any command that shows you a breakdown of sharing
> > among VMs,
> >
> Can we add it? (as soon as practical, of course)

It should be quite trivial as the information is already returned by
xc_domain_getinfo.

Tamas

[-- Attachment #1.2: Type: text/html, Size: 1253 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen Memory de-duplication

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1062 bytes --]

On Wed, 2016-04-20 at 13:01 -0600, Tamas K Lengyel wrote:
> On Apr 20, 2016 12:34, "Dario Faggioli" <dario.faggioli@citrix.com>
> wrote:
> > On Wed, 2016-04-20 at 10:25 -0600, Tamas K Lengyel wrote:
> > > On Wed, Apr 20, 2016 at 7:24 AM, Wei Liu <wei.liu2@citrix.com>
> wrote:
> > > > Not sure. You can check xl manpage for those commands.
> > > xl doesn't have any command that shows you a breakdown of sharing
> > > among VMs,
> > >
> > Can we add it? (as soon as practical, of course)
> It should be quite trivial as the information is already returned by
> xc_domain_getinfo.
>
Great to see. I'll keep it in mind (or ping you), if I get round to do
it. If anyone else get to do it before, he/she should keep in mind the
NUMA angle (or ping me :-D).

Thanks and Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen Memory de-duplication

[Tamas K Lengyel]

[-- Attachment #1.1: Type: text/plain, Size: 1484 bytes --]

On Thu, Apr 21, 2016 at 9:13 AM, Maryam Masoudian <maryammasoudian@gmail.com
> wrote:

>
>
> On Thu, Apr 21, 2016 at 5:17 PM, Dario Faggioli <dario.faggioli@citrix.com
> > wrote:
>
>> On Wed, 2016-04-20 at 13:01 -0600, Tamas K Lengyel wrote:
>> > On Apr 20, 2016 12:34, "Dario Faggioli" <dario.faggioli@citrix.com>
>> > wrote:
>> > > On Wed, 2016-04-20 at 10:25 -0600, Tamas K Lengyel wrote:
>> > > > On Wed, Apr 20, 2016 at 7:24 AM, Wei Liu <wei.liu2@citrix.com>
>> > wrote:
>> > > > > Not sure. You can check xl manpage for those commands.
>> > > > xl doesn't have any command that shows you a breakdown of sharing
>> > > > among VMs,
>>
>
> Well, I didn't find any useful description for xl sharing : it just shows
> the shared memory.
> What I really want is seeing the deduplicated memory among VMs.
> I ran two windows VMs and thought of the kernel memory to be shared among
> them if the memory de-duplication is enabled in Xen.
> But the result of xl sharing for these VMs was 0 .
>

Xen won't magically share memory across VMs for you, you need to decide
what pages want to share and among what VMs. Take a look at
https://github.com/tklengyel/drakvuf/blob/master/src/xen_helper/xen_helper.c#L197
for an example on how you can dedup the entire memory of two identical VMs.
And as I already mentioned, you can see memory shared across VMs by using
the libxc library, as you can see at
https://github.com/tklengyel/drakvuf/blob/master/src/xen_helper/xen_helper.c#L245
.

Tamas

[-- Attachment #1.2: Type: text/html, Size: 2677 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen Memory De-duplication

[Tim Deegan]

At 11:20 +0100 on 12 Oct (1286882408), Thomas Goirand wrote:
> What would be nicer, would be a more general approach, and
> maybe have the possibility to use a filesystem that is already
> mounted on the dom0.

Do you want something more than NFS/CIFS mounts already offer?

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

Re: Xen Memory De-duplication

[Thomas Goirand]

Aditya Gadre wrote:
> This kind of implementation will require the disk blocks from
> different DomUs to be mapped to same physical disk block.
> For example,
> 1) Shared read only filesystem
> 2) Union based filesystem
> 3) Virtual machine images deployed on a host filesystem which has
> deduplication enabled
>
> What kind of  arrangement of filesystem is used in production
> environments for DomUs which host large number of VMs as in cloud
> enviorment?
>
I don't know for others, but for us (eg: at GPLHost), none of
what you described above is doable. Each VM has its own
LVM partition, and we wont have shared filesystem among
many VMs. Never ever. We don't use virtual machine *images*
either.

What would be nicer, would be a more general approach, and
maybe have the possibility to use a filesystem that is already
mounted on the dom0. Why? Because most of the time, what
is wasted, is the free space in each LVM, in what I described
above.

Thomas

Re: Xen Memory De-duplication

[Tim Deegan]

At 18:56 +0100 on 09 Oct (1286650583), Aditya Gadre wrote:
> Is such a approach implemented?  We intend to implement this as a
> project. What are the suspected challanges?

Yes, this was implemented last year; the patches are in the xen-unstable
tree.  They hook the read path in blocktap to detect duplicate reads of
the same block and turn them into copy-on-write mappings in the
hypervisor.  

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

Re: Xen Memory De-duplication

[Shriram Rajagopalan]

[-- Attachment #1.1: Type: text/plain, Size: 4706 bytes --]

Not sure about the DMA part, but I suggest you also take a look at satori
project code (memshr modules) in xen.
http://www.usenix.org/events/usenix09/tech/slides/milos.pdf

On Sun, Oct 10, 2010 at 5:34 AM, Pasi Kärkkäinen <pasik@iki.fi> wrote:

> On Sun, Oct 10, 2010 at 10:54:58AM +0530, Aditya Gadre wrote:
> >    This kind of implementation will require the disk blocks from
> different
> >    DomUs to be mapped to same physical disk block.
> >    For example,
> >    1) Shared read only filesystem
> >    2) Union based filesystem
> >    3) Virtual machine images deployed on a host filesystem which has
> >    deduplication enabled
> >
>
> I guess Xen blktap qcow* images should do? And maybe blktap2 VHD?
>
> -- Pasi
>
> >    What kind of  arrangement of filesystem is used in production
> environments
> >    for DomUs which host large number of VMs as in cloud enviorment?
> >
> >    On Sun, Oct 10, 2010 at 5:10 AM, Dan Magenheimer
> >    <[1]dan.magenheimer@oracle.com> wrote:
> >
> >      I*m not an expert on it but I believe this sounds very similar to
> the
> >      page sharing implementation that already exists in Xen 4.0.  The
> >      implementation in Xen only works on HVM guests and only on machines
> that
> >      have EPT though.  The patches (which were accepted into Xen) were
> posted
> >      here:
> >
> >
> >
> >      [2]
> http://lists.xensource.com/archives/html/xen-devel/2009-12/msg00797.html
> >
> >
> >
> >      From: Aditya Gadre [mailto:[3]adivb2003@gmail.com]
> >      Sent: Saturday, October 09, 2010 11:56 AM
> >      To: [4]Xen-devel@lists.xensource.com
> >      Subject: [Xen-devel] Xen Memory De-duplication
> >
> >
> >
> >      Aim is to implement Xen Memory Deduplication with minimum overhead.
> >
> >      Our approach to de-duplication is as follows
> >
> >      In most cases, Domain-U uses a small set of well-known operating
> systems
> >      such as Linux, FreeBSD and Microsoft Windows. In such environment
> many
> >      domains share read-only filesystems that contain operating system
> and
> >      frequently usedprogram files and libraries.Each domain has their own
> >      writable filesystems for storing data and temporary files. In this
> >      configuration, multiple pages scattered in different domains mostly
> >      happen to contain same disk block. So, in our approach to perform
> >      deduplication we intend to add a data structure in dom 0 which store
> >      disk block number and the machine frame number(MFN) when a read
> request
> >      for the read only code(and data) is made. Now when another domain U
> >      places the request for the block of code and Dom 0 recieves a
> request
> >      for I/O (DMA), it will first check into the data structure for the
> entry
> >      for the block. If it finds the block it will return the MFN of the
> >      already read page and map it to the requesting domain's PFN
> resulting in
> >      zero I/O processing time of blocks which are already read. This in
> turn
> >      results in de-duplication of the read only pages accessed by
> multiple
> >      domains without any overhead of hashing the page.
> >
> >      Test case scenario:
> >
> >      Consider a Dom0 linux kernel using a filesystem with deduplication
> >      enabled. Then we install a DomU kernel with the virtual disk as a
> image
> >      file on the disk(.img). Then we make multiple copies of the image to
> >      deploy multiple DomUs running same kernel. Now, as deduplication is
> >      enabled in the file system initially all the blocks of the domains
> will
> >      be pointing to the same disk blocks. Now when the kernel's are
> booted,
> >      they all will consume memory only once for the programs(code
> segment)
> >      loaded in the memory. Now as these OSs start to write to their own
> >      virtual filesystems the blocks of the image will be COW'ed by the
> >      filesystem resulting in different block number.
> >      Is such a approach implemented?  We intend to implement this as a
> >      project. What are the suspected challanges?
> >
> >      Regards,
> >      Aditya Gadre
> >
> > References
> >
> >    Visible links
> >    1. mailto:dan.magenheimer@oracle.com
> >    2.
> http://lists.xensource.com/archives/html/xen-devel/2009-12/msg00797.html
> >    3. mailto:adivb2003@gmail.com
> >    4. mailto:Xen-devel@lists.xensource.com
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>



-- 
perception is but an offspring of its own self

[-- Attachment #1.2: Type: text/html, Size: 6231 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Xen Memory De-duplication

[Pasi Kärkkäinen]

On Sun, Oct 10, 2010 at 10:54:58AM +0530, Aditya Gadre wrote:
>    This kind of implementation will require the disk blocks from different
>    DomUs to be mapped to same physical disk block.
>    For example,
>    1) Shared read only filesystem
>    2) Union based filesystem
>    3) Virtual machine images deployed on a host filesystem which has
>    deduplication enabled
> 

I guess Xen blktap qcow* images should do? And maybe blktap2 VHD? 

-- Pasi

>    What kind of  arrangement of filesystem is used in production environments
>    for DomUs which host large number of VMs as in cloud enviorment?
> 
>    On Sun, Oct 10, 2010 at 5:10 AM, Dan Magenheimer
>    <[1]dan.magenheimer@oracle.com> wrote:
> 
>      I*m not an expert on it but I believe this sounds very similar to the
>      page sharing implementation that already exists in Xen 4.0.  The
>      implementation in Xen only works on HVM guests and only on machines that
>      have EPT though.  The patches (which were accepted into Xen) were posted
>      here:
> 
> 
> 
>      [2]http://lists.xensource.com/archives/html/xen-devel/2009-12/msg00797.html
> 
> 
> 
>      From: Aditya Gadre [mailto:[3]adivb2003@gmail.com]
>      Sent: Saturday, October 09, 2010 11:56 AM
>      To: [4]Xen-devel@lists.xensource.com
>      Subject: [Xen-devel] Xen Memory De-duplication
> 
> 
> 
>      Aim is to implement Xen Memory Deduplication with minimum overhead.
> 
>      Our approach to de-duplication is as follows
> 
>      In most cases, Domain-U uses a small set of well-known operating systems
>      such as Linux, FreeBSD and Microsoft Windows. In such environment many
>      domains share read-only filesystems that contain operating system and
>      frequently usedprogram files and libraries.Each domain has their own
>      writable filesystems for storing data and temporary files. In this
>      configuration, multiple pages scattered in different domains mostly
>      happen to contain same disk block. So, in our approach to perform
>      deduplication we intend to add a data structure in dom 0 which store
>      disk block number and the machine frame number(MFN) when a read request
>      for the read only code(and data) is made. Now when another domain U
>      places the request for the block of code and Dom 0 recieves a request
>      for I/O (DMA), it will first check into the data structure for the entry
>      for the block. If it finds the block it will return the MFN of the
>      already read page and map it to the requesting domain's PFN resulting in
>      zero I/O processing time of blocks which are already read. This in turn
>      results in de-duplication of the read only pages accessed by multiple
>      domains without any overhead of hashing the page.
> 
>      Test case scenario:
> 
>      Consider a Dom0 linux kernel using a filesystem with deduplication
>      enabled. Then we install a DomU kernel with the virtual disk as a image
>      file on the disk(.img). Then we make multiple copies of the image to
>      deploy multiple DomUs running same kernel. Now, as deduplication is
>      enabled in the file system initially all the blocks of the domains will
>      be pointing to the same disk blocks. Now when the kernel's are booted,
>      they all will consume memory only once for the programs(code segment)
>      loaded in the memory. Now as these OSs start to write to their own
>      virtual filesystems the blocks of the image will be COW'ed by the
>      filesystem resulting in different block number.
>      Is such a approach implemented?  We intend to implement this as a
>      project. What are the suspected challanges?
> 
>      Regards,
>      Aditya Gadre
> 
> References
> 
>    Visible links
>    1. mailto:dan.magenheimer@oracle.com
>    2. http://lists.xensource.com/archives/html/xen-devel/2009-12/msg00797.html
>    3. mailto:adivb2003@gmail.com
>    4. mailto:Xen-devel@lists.xensource.com

Re: Xen Memory De-duplication

[Aditya Gadre]

[-- Attachment #1.1: Type: text/plain, Size: 3340 bytes --]

This kind of implementation will require the disk blocks from different
DomUs to be mapped to same physical disk block.
For example,
1) Shared read only filesystem
2) Union based filesystem
3) Virtual machine images deployed on a host filesystem which has
deduplication enabled

What kind of  arrangement of filesystem is used in production environments
for DomUs which host large number of VMs as in cloud enviorment?

On Sun, Oct 10, 2010 at 5:10 AM, Dan Magenheimer <dan.magenheimer@oracle.com
> wrote:

>  I’m not an expert on it but I believe this sounds very similar to the
> page sharing implementation that already exists in Xen 4.0.  The
> implementation in Xen only works on HVM guests and only on machines that
> have EPT though.  The patches (which were accepted into Xen) were posted
> here:
>
>
>
> http://lists.xensource.com/archives/html/xen-devel/2009-12/msg00797.html
>
>
>
> *From:* Aditya Gadre [mailto:adivb2003@gmail.com]
> *Sent:* Saturday, October 09, 2010 11:56 AM
>
> *To:* Xen-devel@lists.xensource.com
> *Subject:* [Xen-devel] Xen Memory De-duplication
>
>
>
> Aim is to implement Xen Memory Deduplication with minimum overhead.
>
> Our approach to de-duplication is as follows
>
> In most cases, Domain-U uses a small set of well-known operating systems
> such as Linux, FreeBSD and Microsoft Windows. In such environment many
> domains share read-only filesystems that contain operating system and
> frequently usedprogram files and libraries.Each domain has their own
> writable filesystems for storing data and temporary files. In this
> configuration, multiple pages scattered in different domains mostly happen
> to contain same disk block. So, in our approach to perform deduplication we
> intend to add a data structure in dom 0 which store disk block number and
> the machine frame number(MFN) when a read request for the read only code(and
> data) is made. Now when another domain U places the request for the block of
> code and Dom 0 recieves a request for I/O (DMA), it will first check into
> the data structure for the entry for the block. If it finds the block it
> will return the MFN of the already read page and map it to the requesting
> domain's PFN resulting in zero I/O processing time of blocks which are
> already read. This in turn results in de-duplication of the read only pages
> accessed by multiple domains without any overhead of hashing the page.
>
> Test case scenario:
>
> Consider a Dom0 linux kernel using a filesystem with deduplication enabled.
> Then we install a DomU kernel with the virtual disk as a image file on the
> disk(.img). Then we make multiple copies of the image to deploy multiple
> DomUs running same kernel. Now, as deduplication is enabled in the file
> system initially all the blocks of the domains will be pointing to the same
> disk blocks. Now when the kernel's are booted, they all will consume memory
> only once for the programs(code segment) loaded in the memory. Now as these
> OSs start to write to their own virtual filesystems the blocks of the image
> will be COW'ed by the filesystem resulting in different block number.
> Is such a approach implemented?  We intend to implement this as a project.
> What are the suspected challanges?
>
>   Regards,
> Aditya Gadre
>

[-- Attachment #1.2: Type: text/html, Size: 5064 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: Xen Memory De-duplication

[Dan Magenheimer]

[-- Attachment #1.1: Type: text/plain, Size: 2686 bytes --]

I'm not an expert on it but I believe this sounds very similar to the page sharing implementation that already exists in Xen 4.0.  The implementation in Xen only works on HVM guests and only on machines that have EPT though.  The patches (which were accepted into Xen) were posted here:

 

http://lists.xensource.com/archives/html/xen-devel/2009-12/msg00797.html

 

From: Aditya Gadre [mailto:adivb2003@gmail.com] 
Sent: Saturday, October 09, 2010 11:56 AM
To: Xen-devel@lists.xensource.com
Subject: [Xen-devel] Xen Memory De-duplication

 

Aim is to implement Xen Memory Deduplication with minimum overhead.

Our approach to de-duplication is as follows

In most cases, Domain-U uses a small set of well-known operating systems such as Linux, FreeBSD and Microsoft Windows. In such environment many domains share read-only filesystems that contain operating system and frequently usedprogram files and libraries.Each domain has their own writable filesystems for storing data and temporary files. In this configuration, multiple pages scattered in different domains mostly happen to contain same disk block. So, in our approach to perform deduplication we intend to add a data structure in dom 0 which store disk block number and the machine frame number(MFN) when a read request for the read only code(and data) is made. Now when another domain U places the request for the block of code and Dom 0 recieves a request for I/O (DMA), it will first check into the data structure for the entry for the block. If it finds the block it will return the MFN of the already read page and map it to the requesting domain's PFN resulting in zero I/O processing time of blocks which are already read. This in turn results in de-duplication of the read only pages accessed by multiple domains without any overhead of hashing the page.

Test case scenario: 

Consider a Dom0 linux kernel using a filesystem with deduplication enabled. Then we install a DomU kernel with the virtual disk as a image file on the disk(.img). Then we make multiple copies of the image to deploy multiple DomUs running same kernel. Now, as deduplication is enabled in the file system initially all the blocks of the domains will be pointing to the same disk blocks. Now when the kernel's are booted, they all will consume memory only once for the programs(code segment) loaded in the memory. Now as these OSs start to write to their own virtual filesystems the blocks of the image will be COW'ed by the filesystem resulting in different block number.
Is such a approach implemented?  We intend to implement this as a project. What are the suspected challanges?



Regards,
Aditya Gadre 

[-- Attachment #1.2: Type: text/html, Size: 5668 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Xen Memory De-duplication

[Pasi Kärkkäinen]

On Sat, Oct 09, 2010 at 11:26:23PM +0530, Aditya Gadre wrote:
>    Aim is to implement Xen Memory Deduplication with minimum overhead.
> 
>    Our approach to de-duplication is as follows
> 
>    In most cases, Domain-U uses a small set of well-known operating systems
>    such as Linux, FreeBSD and Microsoft Windows. In such environment many
>    domains share read-only filesystems that contain operating system and
>    frequently usedprogram files and libraries.Each domain has their own
>    writable filesystems for storing data and temporary files. In this
>    configuration, multiple pages scattered in different domains mostly happen
>    to contain same disk block. So, in our approach to perform deduplication
>    we intend to add a data structure in dom 0 which store disk block number
>    and the machine frame number(MFN) when a read request for the read only
>    code(and data) is made. Now when another domain U places the request for
>    the block of code and Dom 0 recieves a request for I/O (DMA), it will
>    first check into the data structure for the entry for the block. If it
>    finds the block it will return the MFN of the already read page and map it
>    to the requesting domain's PFN resulting in zero I/O processing time of
>    blocks which are already read. This in turn results in de-duplication of
>    the read only pages accessed by multiple domains without any overhead of
>    hashing the page.
> 
>    Test case scenario:
> 
>    Consider a Dom0 linux kernel using a filesystem with deduplication
>    enabled. Then we install a DomU kernel with the virtual disk as a image
>    file on the disk(.img). Then we make multiple copies of the image to
>    deploy multiple DomUs running same kernel. Now, as deduplication is
>    enabled in the file system initially all the blocks of the domains will be
>    pointing to the same disk blocks. Now when the kernel's are booted, they
>    all will consume memory only once for the programs(code segment) loaded in
>    the memory. Now as these OSs start to write to their own virtual
>    filesystems the blocks of the image will be COW'ed by the filesystem
>    resulting in different block number.
>    Is such a approach implemented?  We intend to implement this as a project.
>    What are the suspected challanges?
> 

Yeah, I think the image COW is possible using the Xen blktap2 vhd support,
and also maybe Xen qcow* stuff.

Also check Xen4.0 wiki page for more info about the memory sharing etc:
http://wiki.xensource.com/xenwiki/Xen4.0

-- Pasi

Xen Memory De-duplication

[Aditya Gadre]

[-- Attachment #1.1: Type: text/plain, Size: 2109 bytes --]

Aim is to implement Xen Memory Deduplication with minimum overhead.

Our approach to de-duplication is as follows

In most cases, Domain-U uses a small set of well-known operating systems
such as Linux, FreeBSD and Microsoft Windows. In such environment many
domains share read-only filesystems that contain operating system and
frequently usedprogram files and libraries.Each domain has their own
writable filesystems for storing data and temporary files. In this
configuration, multiple pages scattered in different domains mostly happen
to contain same disk block. So, in our approach to perform deduplication we
intend to add a data structure in dom 0 which store disk block number and
the machine frame number(MFN) when a read request for the read only code(and
data) is made. Now when another domain U places the request for the block of
code and Dom 0 recieves a request for I/O (DMA), it will first check into
the data structure for the entry for the block. If it finds the block it
will return the MFN of the already read page and map it to the requesting
domain's PFN resulting in zero I/O processing time of blocks which are
already read. This in turn results in de-duplication of the read only pages
accessed by multiple domains without any overhead of hashing the page.

Test case scenario:

Consider a Dom0 linux kernel using a filesystem with deduplication enabled.
Then we install a DomU kernel with the virtual disk as a image file on the
disk(.img). Then we make multiple copies of the image to deploy multiple
DomUs running same kernel. Now, as deduplication is enabled in the file
system initially all the blocks of the domains will be pointing to the same
disk blocks. Now when the kernel's are booted, they all will consume memory
only once for the programs(code segment) loaded in the memory. Now as these
OSs start to write to their own virtual filesystems the blocks of the image
will be COW'ed by the filesystem resulting in different block number.
Is such a approach implemented?  We intend to implement this as a project.
What are the suspected challanges?


Regards,
Aditya Gadre

[-- Attachment #1.2: Type: text/html, Size: 2306 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen Memory De-duplication

[Aditya Gadre]

[-- Attachment #1.1: Type: text/plain, Size: 2105 bytes --]

Aim is to implement Xen Memory Deduplication with minimum overhead.

Our approach to de-duplication is as follows?

In most cases, Domain-U uses a small set of well-known operating systems
such as Linux, FreeBSD and Microsoft Windows. In such environment many
domains share read-only filesystems that contain operating system and
frequently usedprogram files and libraries.Each domain has their own
writable filesystems for storing data and temporary files. In this
configuration, multiple pages scattered in different domains mostly happen
to contain same disk block. So, in our approach to perform deduplication we
intend to add a data structure in dom 0 which store disk block number and
the machine frame number(MFN) when a read request for the read only code(and
data) is made. Now when another domain U places the request for the block of
code and Dom 0 recieves a request for I/O (DMA), it will first check into
the data structure for the entry for the block. If it finds the block it
will return the MFN of the already read page and map it to the requesting
domain's PFN resulting in zero I/O processing time of blocks which are
already read. This in turn results in de-duplication of the read only pages
accessed by multiple domains without any overhead of hashing the page.

Test case scenario:

Consider a Dom0 linux kernel using a filesystem with deduplication enabled.
Then we install a DomU kernel with the virtual disk as a image file on the
disk(.img). Then we make multiple copies of the image to deploy multiple
DomUs running same kernel. Now, as deduplication is enabled in the file
system initially all the blocks of the domains will be pointing to the same
disk blocks. Now when the kernel's are booted, they all will consume memory
only once for the programs(code segment) loaded in the memory. Now as these
OSs start to write to their own virtual filesystems the blocks of the image
will be COW'ed by the filesystem resulting in different block number.
Is such a approach implemented?  We intend to implement this project. What
are the suspected challanges?


Regards,
Aditya Gadre

[-- Attachment #1.2: Type: text/html, Size: 2203 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen Memory De-duplication

[Aditya Gadre]

[-- Attachment #1.1: Type: text/plain, Size: 2081 bytes --]

Aim is to implement Xen Memory Deduplication with minimum overhead.

Our approach to de-duplication is as follows?

In most cases, Domain-U uses a small set of well-known operating systems
such as Linux, FreeBSD and Microsoft Windows. In such environment many
domains share read-only filesystems that contain operating system and
frequently usedprogram files and libraries.Each domain has their own
writable filesystems for storing data and temporary files. In this
configuration, multiple pages scattered in different domains mostly happen
to contain same disk block. So, in our approach to perform deduplication we
intend to add a data structure in dom 0 which store disk block number and
the machine frame number(MFN) when a read request for the read only code(and
data) is made. Now when another domain U places the request for the block of
code and Dom 0 recieves a request for I/O (DMA), it will first check into
the data structure for the entry for the block. If it finds the block it
will return the MFN of the already read page and map it to the requesting
domain's PFN resulting in zero I/O processing time of blocks which are
already read. This in turn results in de-duplication of the read only pages
accessed by multiple domains without any overhead of hashing the page.

Test case scenario:

Consider a Dom0 linux kernel using a filesystem with deduplication enabled.
Then we install a DomU kernel with the virtual disk as a image file on the
disk(.img). Then we make multiple copies of the image to deploy multiple
DomUs running same kernel. Now, as deduplication is enabled in the file
system initially all the blocks of the domains will be pointing to the same
disk blocks. Now when the kernel's are booted, they all will consume memory
only once for the programs(code segment) loaded in the memory. Now as these
OSs start to write to their own virtual filesystems the blocks of the image
will be COW'ed by the filesystem resulting in different block number.
Is such a approach implemented?  We intend to implement this project. What
are the suspected challanges?

[-- Attachment #1.2: Type: text/html, Size: 2138 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel