All of lore.kernel.org
 help / color / mirror / Atom feed
* Xen Memory de-duplication
@ 2016-04-17 11:25 Maryam Masoudian
  2016-04-17 12:15 ` Wei Liu
  0 siblings, 1 reply; 21+ messages in thread
From: Maryam Masoudian @ 2016-04-17 11:25 UTC (permalink / raw)
  To: xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 379 bytes --]

Hi,

I have some questions about memory de-duplication in xen.
Firstly, I want to know if this feature is available in Xen .
If it is, how can I enable it in Xen 4.6 HVM mode.

Also, if I want to do some logging or other action whenever a COW happens
on shared memory, where should I add my codes? (is the /xen/memshr folder
containing the memory de-duplication code)?

Regards,

[-- Attachment #1.2: Type: text/html, Size: 496 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 21+ messages in thread
* Xen Memory De-duplication
@ 2010-10-09 17:56 Aditya Gadre
  2010-10-09 19:09 ` Pasi Kärkkäinen
                   ` (2 more replies)
  0 siblings, 3 replies; 21+ messages in thread
From: Aditya Gadre @ 2010-10-09 17:56 UTC (permalink / raw)
  To: Xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 2109 bytes --]

Aim is to implement Xen Memory Deduplication with minimum overhead.

Our approach to de-duplication is as follows

In most cases, Domain-U uses a small set of well-known operating systems
such as Linux, FreeBSD and Microsoft Windows. In such environment many
domains share read-only filesystems that contain operating system and
frequently usedprogram files and libraries.Each domain has their own
writable filesystems for storing data and temporary files. In this
configuration, multiple pages scattered in different domains mostly happen
to contain same disk block. So, in our approach to perform deduplication we
intend to add a data structure in dom 0 which store disk block number and
the machine frame number(MFN) when a read request for the read only code(and
data) is made. Now when another domain U places the request for the block of
code and Dom 0 recieves a request for I/O (DMA), it will first check into
the data structure for the entry for the block. If it finds the block it
will return the MFN of the already read page and map it to the requesting
domain's PFN resulting in zero I/O processing time of blocks which are
already read. This in turn results in de-duplication of the read only pages
accessed by multiple domains without any overhead of hashing the page.

Test case scenario:

Consider a Dom0 linux kernel using a filesystem with deduplication enabled.
Then we install a DomU kernel with the virtual disk as a image file on the
disk(.img). Then we make multiple copies of the image to deploy multiple
DomUs running same kernel. Now, as deduplication is enabled in the file
system initially all the blocks of the domains will be pointing to the same
disk blocks. Now when the kernel's are booted, they all will consume memory
only once for the programs(code segment) loaded in the memory. Now as these
OSs start to write to their own virtual filesystems the blocks of the image
will be COW'ed by the filesystem resulting in different block number.
Is such a approach implemented?  We intend to implement this as a project.
What are the suspected challanges?


Regards,
Aditya Gadre

[-- Attachment #1.2: Type: text/html, Size: 2306 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 21+ messages in thread
* Xen Memory De-duplication
@ 2010-10-08 19:01 Aditya Gadre
  2010-10-08 19:02 ` Aditya Gadre
  0 siblings, 1 reply; 21+ messages in thread
From: Aditya Gadre @ 2010-10-08 19:01 UTC (permalink / raw)
  To: Xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 2081 bytes --]

Aim is to implement Xen Memory Deduplication with minimum overhead.

Our approach to de-duplication is as follows?

In most cases, Domain-U uses a small set of well-known operating systems
such as Linux, FreeBSD and Microsoft Windows. In such environment many
domains share read-only filesystems that contain operating system and
frequently usedprogram files and libraries.Each domain has their own
writable filesystems for storing data and temporary files. In this
configuration, multiple pages scattered in different domains mostly happen
to contain same disk block. So, in our approach to perform deduplication we
intend to add a data structure in dom 0 which store disk block number and
the machine frame number(MFN) when a read request for the read only code(and
data) is made. Now when another domain U places the request for the block of
code and Dom 0 recieves a request for I/O (DMA), it will first check into
the data structure for the entry for the block. If it finds the block it
will return the MFN of the already read page and map it to the requesting
domain's PFN resulting in zero I/O processing time of blocks which are
already read. This in turn results in de-duplication of the read only pages
accessed by multiple domains without any overhead of hashing the page.

Test case scenario:

Consider a Dom0 linux kernel using a filesystem with deduplication enabled.
Then we install a DomU kernel with the virtual disk as a image file on the
disk(.img). Then we make multiple copies of the image to deploy multiple
DomUs running same kernel. Now, as deduplication is enabled in the file
system initially all the blocks of the domains will be pointing to the same
disk blocks. Now when the kernel's are booted, they all will consume memory
only once for the programs(code segment) loaded in the memory. Now as these
OSs start to write to their own virtual filesystems the blocks of the image
will be COW'ed by the filesystem resulting in different block number.
Is such a approach implemented?  We intend to implement this project. What
are the suspected challanges?

[-- Attachment #1.2: Type: text/html, Size: 2138 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 21+ messages in thread
end of thread, other threads:[~2016-04-21 15:24 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-17 11:25 Xen Memory de-duplication Maryam Masoudian
2016-04-17 12:15 ` Wei Liu
2016-04-17 13:58   ` Maryam Masoudian
2016-04-17 14:20     ` Wei Liu
     [not found]       ` <CAOJrJTc5dynTwiMrtE4Logpz1_ct3oM9vz3HWh1s1sDMY+hSxw@mail.gmail.com>
2016-04-20 13:24         ` Wei Liu
2016-04-20 16:25           ` Tamas K Lengyel
2016-04-20 18:34             ` Dario Faggioli
2016-04-20 19:01               ` Tamas K Lengyel
2016-04-21 12:47                 ` Dario Faggioli
     [not found]                   ` <CAOJrJTeK3ntGGwyAjY25xL0g8AXtDcoUOcipF4jmZDCfbdA9Kw@mail.gmail.com>
2016-04-21 15:24                     ` Tamas K Lengyel
  -- strict thread matches above, loose matches on Subject: below --
2010-10-09 17:56 Xen Memory De-duplication Aditya Gadre
2010-10-09 19:09 ` Pasi Kärkkäinen
2010-10-09 23:40 ` Dan Magenheimer
2010-10-10  5:24   ` Aditya Gadre
2010-10-10 12:34     ` Pasi Kärkkäinen
2010-10-11  7:58       ` Shriram Rajagopalan
2010-10-12 10:20     ` Thomas Goirand
2010-10-12 10:33       ` Tim Deegan
2010-10-11 12:59 ` Tim Deegan
2010-10-08 19:01 Aditya Gadre
2010-10-08 19:02 ` Aditya Gadre

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.