From mboxrd@z Thu Jan 1 00:00:00 1970 From: KOSAKI Motohiro Subject: Re: [ofa-general] Re: [GIT PULL] please pull ummunotify Date: Tue, 15 Sep 2009 16:03:04 +0900 (JST) Message-ID: <20090915155231.DB86.A69D9226@jp.fujitsu.com> References: <20090911064019.GZ4973@obsidianresearch.com> Mime-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Roland Dreier Cc: kosaki.motohiro-+CUm20s59erQFUHtdCDX3A@public.gmane.org, Jason Gunthorpe , Brice Goglin , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, general-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org, akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org, torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org List-Id: linux-rdma@vger.kernel.org > > > So.. What is the problem with fork? The semantics of what should > > happen seem natural enough to me, the PD doesn't get copied to the > > child, so the MR stays with the parent. COW events on the pinned > > region must be resolved so that the physical page stays with the > > process that has pinned it - the pin is logically released in the > > child because the MR doesn't exist because the PD doesn't exist. > > This is getting away from the problem that ummunotify is solving, but > handling a COW fault generated by the parent by doing the copy in the > child seems like a pretty major, tricky change to make. The child may > have forked 100 more times in the meantime, meaning we now have to > change 101 memory maps ... the cost of page faults goes through the roof > probably... Ummm... Perhaps my first question was wrong. I'm not intent to NAK your patch. I merely want to know your patch detail... ok, I ask you again as another word. - I guess you have your MPI implementaion w/ ummunotify, right? - I guess you have test sevaral pattern, right? if so, can we see your test result? - I think you can explain your MPI advantage/disadvantage against current OpenMPI (or mpich et al). - I guess your patch dramatically improve MPI implementaion, but it's not free. it request some limitation to MPI application, right? - I imagine multi thread and fork. Is there another linmitaion? - In past discuttion, you said ummunotify user should not use multi threading. you also think user should not fork? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758144AbZIOHDJ (ORCPT ); Tue, 15 Sep 2009 03:03:09 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1758118AbZIOHDH (ORCPT ); Tue, 15 Sep 2009 03:03:07 -0400 Received: from fgwmail6.fujitsu.co.jp ([192.51.44.36]:59470 "EHLO fgwmail6.fujitsu.co.jp" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757991AbZIOHDF (ORCPT ); Tue, 15 Sep 2009 03:03:05 -0400 X-SecurityPolicyCheck-FJ: OK by FujitsuOutboundMailChecker v1.3.1 From: KOSAKI Motohiro To: Roland Dreier Subject: Re: [ofa-general] Re: [GIT PULL] please pull ummunotify Cc: kosaki.motohiro@jp.fujitsu.com, Jason Gunthorpe , Brice Goglin , linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, general@lists.openfabrics.org, akpm@linux-foundation.org, torvalds@linux-foundation.org In-Reply-To: References: <20090911064019.GZ4973@obsidianresearch.com> Message-Id: <20090915155231.DB86.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Becky! ver. 2.50.07 [ja] Date: Tue, 15 Sep 2009 16:03:04 +0900 (JST) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > > > So.. What is the problem with fork? The semantics of what should > > happen seem natural enough to me, the PD doesn't get copied to the > > child, so the MR stays with the parent. COW events on the pinned > > region must be resolved so that the physical page stays with the > > process that has pinned it - the pin is logically released in the > > child because the MR doesn't exist because the PD doesn't exist. > > This is getting away from the problem that ummunotify is solving, but > handling a COW fault generated by the parent by doing the copy in the > child seems like a pretty major, tricky change to make. The child may > have forked 100 more times in the meantime, meaning we now have to > change 101 memory maps ... the cost of page faults goes through the roof > probably... Ummm... Perhaps my first question was wrong. I'm not intent to NAK your patch. I merely want to know your patch detail... ok, I ask you again as another word. - I guess you have your MPI implementaion w/ ummunotify, right? - I guess you have test sevaral pattern, right? if so, can we see your test result? - I think you can explain your MPI advantage/disadvantage against current OpenMPI (or mpich et al). - I guess your patch dramatically improve MPI implementaion, but it's not free. it request some limitation to MPI application, right? - I imagine multi thread and fork. Is there another linmitaion? - In past discuttion, you said ummunotify user should not use multi threading. you also think user should not fork?