From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934962Ab1ETBV4 (ORCPT ); Thu, 19 May 2011 21:21:56 -0400 Received: from mail-fx0-f46.google.com ([209.85.161.46]:43272 "EHLO mail-fx0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933379Ab1ETBVz (ORCPT ); Thu, 19 May 2011 21:21:55 -0400 DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=BGCnmpuy9JpHwQUPP+yO1U6Zqpx1vfl6mh1i/zTVcSlF9aCmH8T+FL8toK2SSWTsw+ R4xgTU7JdbUv7arsemBlITQsTY/M2MichCBtSHsXKaV2NTcWAvTDYK+XWIsrQBpapcRB ZGfcG6A/dyG+iKUcfq4trSvQLzjiKSM7zbSp0= MIME-Version: 1.0 In-Reply-To: <1305851941.7481.92.camel@pasglop> References: <1305753895-24845-1-git-send-email-ericvh@gmail.com> <1305753895-24845-6-git-send-email-ericvh@gmail.com> <1305851941.7481.92.camel@pasglop> Date: Thu, 19 May 2011 20:21:54 -0500 Message-ID: Subject: Re: [PATCH 6/7] [RFC] enable early TLBs for BG/P From: Eric Van Hensbergen To: Benjamin Herrenschmidt Cc: linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, bg-linux@lists.anl-external.org Content-Type: text/plain; charset=ISO-8859-1 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, May 19, 2011 at 7:39 PM, Benjamin Herrenschmidt wrote: > On Wed, 2011-05-18 at 16:24 -0500, Eric Van Hensbergen wrote: >> BG/P maps firmware with an early TLB > > That's a bit gross. How often do you call that firmware in practice ? > Aren't you better off instead inserting a TLB entry for it when you call > it instead ? A simple tlbsx. + tlbwe sequence would do. That would free > up a TLB entry for normal use. > Well, it depends on who you talk to. The production software BG/P guys use the firmware constantly, its the primary interface to the networks, the console, and the management software which runs the machine. As such the IO Node guys, the Compute Node Kernel guys and the ZeptoOS guys use it quite a bit. The kittyhawk guys on the other hand barely use it at all, in fact I believe they do all the interaction with it during uboot and then shut it off. IIRC, the sticky question is RAS support, there are certain things it wants to jump to firmware to deal with and expects things to be mapped an pinned into memory. Furthermore, I think it may make assumptions about where in the TLB the mappings are. Since the kittyhawk guys obviously ignore this by shutting it down, its not clear just how important this is. I'm game to try the dynamic mapping as you suggest if you would prefer it. Its worth mentioning that I believe with BG/Q, the plan is to rely on the firmware even more extensively, but I haven't looked at any of the code yet to verify whether or not this is true. -eric From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-fx0-f51.google.com (mail-fx0-f51.google.com [209.85.161.51]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by ozlabs.org (Postfix) with ESMTPS id 8735CB71C1 for ; Fri, 20 May 2011 11:21:58 +1000 (EST) Received: by fxm5 with SMTP id 5so2707631fxm.38 for ; Thu, 19 May 2011 18:21:54 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <1305851941.7481.92.camel@pasglop> References: <1305753895-24845-1-git-send-email-ericvh@gmail.com> <1305753895-24845-6-git-send-email-ericvh@gmail.com> <1305851941.7481.92.camel@pasglop> Date: Thu, 19 May 2011 20:21:54 -0500 Message-ID: Subject: Re: [PATCH 6/7] [RFC] enable early TLBs for BG/P From: Eric Van Hensbergen To: Benjamin Herrenschmidt Content-Type: text/plain; charset=ISO-8859-1 Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, bg-linux@lists.anl-external.org List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Thu, May 19, 2011 at 7:39 PM, Benjamin Herrenschmidt wrote: > On Wed, 2011-05-18 at 16:24 -0500, Eric Van Hensbergen wrote: >> BG/P maps firmware with an early TLB > > That's a bit gross. How often do you call that firmware in practice ? > Aren't you better off instead inserting a TLB entry for it when you call > it instead ? A simple tlbsx. + tlbwe sequence would do. That would free > up a TLB entry for normal use. > Well, it depends on who you talk to. The production software BG/P guys use the firmware constantly, its the primary interface to the networks, the console, and the management software which runs the machine. As such the IO Node guys, the Compute Node Kernel guys and the ZeptoOS guys use it quite a bit. The kittyhawk guys on the other hand barely use it at all, in fact I believe they do all the interaction with it during uboot and then shut it off. IIRC, the sticky question is RAS support, there are certain things it wants to jump to firmware to deal with and expects things to be mapped an pinned into memory. Furthermore, I think it may make assumptions about where in the TLB the mappings are. Since the kittyhawk guys obviously ignore this by shutting it down, its not clear just how important this is. I'm game to try the dynamic mapping as you suggest if you would prefer it. Its worth mentioning that I believe with BG/Q, the plan is to rely on the firmware even more extensively, but I haven't looked at any of the code yet to verify whether or not this is true. -eric