From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933639AbcFMOic (ORCPT ); Mon, 13 Jun 2016 10:38:32 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:26019 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S933393AbcFMOib (ORCPT ); Mon, 13 Jun 2016 10:38:31 -0400 X-IBM-Helo: d06dlp02.portsmouth.uk.ibm.com X-IBM-MailFrom: schwidefsky@de.ibm.com X-IBM-RcptTo: linux-kernel@vger.kernel.org Date: Mon, 13 Jun 2016 16:37:40 +0200 From: Martin Schwidefsky To: Peter Zijlstra Cc: Heiko Carstens , Greg Kroah-Hartman , linux-kernel@vger.kernel.org Subject: Re: [PATCH 2/2] s390/topology: add drawer scheduling domain level In-Reply-To: <20160613135302.GA30909@twins.programming.kicks-ass.net> References: <1465376956-100711-1-git-send-email-heiko.carstens@de.ibm.com> <1465376956-100711-3-git-send-email-heiko.carstens@de.ibm.com> <20160613110621.GV30909@twins.programming.kicks-ass.net> <20160613112230.GA3808@osiris> <20160613130647.GZ30909@twins.programming.kicks-ass.net> <20160613151942.02028e15@mschwide> <20160613135302.GA30909@twins.programming.kicks-ass.net> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.23; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16061314-0032-0000-0000-000001D71AA5 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16061314-0033-0000-0000-00001B455225 Message-Id: <20160613163740.0750f535@mschwide> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-06-13_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1604210000 definitions=main-1606130164 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 13 Jun 2016 15:53:02 +0200 Peter Zijlstra wrote: > On Mon, Jun 13, 2016 at 03:19:42PM +0200, Martin Schwidefsky wrote: > > On Mon, 13 Jun 2016 15:06:47 +0200 > > Peter Zijlstra wrote: > > > > > On Mon, Jun 13, 2016 at 01:22:30PM +0200, Heiko Carstens wrote: > > > > Yes, and actually we are all virt/LPAR always, so this is unfortunately not > > > > very easy to do. And yes, I do agree that for the 1:1 case it most likely > > > > would make sense, however we don't have any run-time guarantee to stay 1:1. > > > > > > One option would be to make it a boot option; such that the > > > administrator has to set it. At that point, if the admin creates > > > multiple LPARs its on him. > > > > Unfortunately not good enough. The LPAR code tries to optimize the layout > > at the time a partition is activated. The landscape of already running > > partitions can change at this point. > > Would not the admin _know_ this? It would be him activating partitions > after all, no? This is all fine and good in a static environment where you can afford to stop all partitions to do a reconfiguration. There you could get away with a kernel option that enables "real" NUMA. But as a general solution this fails. Consider this scenario: you have several partitions already running with a workload that you do *not* want to interrupt right now, think stock exchange. And now another partition urgently needs more memory. To do this you have to shut it down, deactivate it, update the profile with more memory, re-activate it and restart the OS. End result: memory landscape could have changed. > > To get around this you would have to activate *all* partitions first and > > then start the operating systems in a second step. > > Arguably, you only care about the single partition covering the entire > machine case, so I don't see that being a problem. > > Again, admin _knows_ this. The single partitions case is boring, several large partitions to big for a single node is the hard part. > > And then there is concurrent repair which will move things around if a > > piece of memory goes bad. This happens rarely though. > > That would be magic disturbance indeed, nothing much to do about that. -- blue skies, Martin. "Reality continues to ruin my life." - Calvin.