From mboxrd@z Thu Jan 1 00:00:00 1970 From: Oren Laadan Subject: Re: How much of a mess does OpenVZ make? ;) Was: What can OpenVZ do? Date: Fri, 13 Mar 2009 18:15:01 -0400 Message-ID: <49BADAE5.8070900__23652.9621681713$1236982798$gmane$org@cs.columbia.edu> References: <1234479845.30155.220.camel@nimitz> <20090226155755.GA1456@x200.localdomain> <20090310215305.GA2078@x200.localdomain> <49B775B4.1040800@free.fr> <20090312145311.GC12390@us.ibm.com> <1236891719.32630.14.camel@bahia> <20090312212124.GA25019@us.ibm.com> <604427e00903122129y37ad791aq5fe7ef2552415da9@mail.gmail.com> <20090313053458.GA28833@us.ibm.com> <20090313193500.GA2285@x200.localdomain> <1236981097.30142.251.camel@nimitz> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1236981097.30142.251.camel@nimitz> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: containers-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: containers-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Dave Hansen Cc: linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, hpa-YMNOUZJC4hwAvxtiuMwx3w@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Alexey Dobriyan , linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, viro-RmSDqhL/yNMiFSDQTTA3OLVCufUGDwFn@public.gmane.org, mingo-X9Un+BFzKDI@public.gmane.org, mpm-VDJrAJ4Gl5ZBDgjK7y7TUQ@public.gmane.org, Andrew Morton , Sukadev Bhattiprolu , Linus Torvalds , tglx-hfZtesqFncYOwBW4kG4KsQ@public.gmane.org, xemul-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org List-Id: containers.vger.kernel.org Dave Hansen wrote: > On Fri, 2009-03-13 at 14:01 -0700, Linus Torvalds wrote: >> On Fri, 13 Mar 2009, Alexey Dobriyan wrote: >>>> Let's face it, we're not going to _ever_ checkpoint any kind of general >>>> case process. Just TCP makes that fundamentally impossible in the general >>>> case, and there are lots and lots of other cases too (just something as >>>> totally _trivial_ as all the files in the filesystem that don't get rolled >>>> back). >>> What do you mean here? Unlinked files? >> Or modified files, or anything else. "External state" is a pretty damn >> wide net. It's not just TCP sequence numbers and another machine. > > This is precisely the reason that we've focused so hard on containers, > and *didn't* just jump right into checkpoint/restart; we're trying > really hard to constrain the _truly_ external things that a process can > interact with. > > The approach so far has largely been to make things are external to a > process at least *internal* to a container. Network, pid, ipc, and uts > namespaces, for example. An ipc/sem.c semaphore may be external to a > process, so we'll just pick the whole namespace up and checkpoint it > along with the process. > > In the OpenVZ case, they've at least demonstrated that the filesystem > can be moved largely with rsync. Unlinked files need some in-kernel TLC > (or /proc mangling) but it isn't *that* bad. And in the Zap we have successfully used a log-based filesystem (specifically NILFS) to continuously snapshot the file-system atomically with taking a checkpoint, so it can easily branch off past checkpoints, including the file system. And unlinked files can be (inefficiently) handled by saving their full contents with the checkpoint image - it's not a big toll on many apps (if you exclude Wine and UML...). At least that's a start. > > We can also make the fs problem much easier by using things like dm or > btrfs snapshotting of the block device, or restricting to where on a fs > a container is allowed to write with stuff like r/o bind mounts. (or NILFS) So we argue that the FS snapshotting is related, but orthogonal in terms of implementation to c/r. Oren.