From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752331AbbCWIjM (ORCPT ); Mon, 23 Mar 2015 04:39:12 -0400 Received: from mail-qc0-f179.google.com ([209.85.216.179]:36103 "EHLO mail-qc0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752058AbbCWIjI (ORCPT ); Mon, 23 Mar 2015 04:39:08 -0400 MIME-Version: 1.0 In-Reply-To: <20150303133659.GD10502@8bytes.org> References: <1423480761-33453-1-git-send-email-tfiga@chromium.org> <20150303133659.GD10502@8bytes.org> From: Tomasz Figa Date: Mon, 23 Mar 2015 17:38:45 +0900 Message-ID: Subject: Re: [PATCH] CHROMIUM: iommu: rockchip: Make sure that page table state is coherent To: Joerg Roedel Cc: iommu@lists.linux-foundation.org, "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "open list:ARM/Rockchip SoC..." , Heiko Stuebner , Daniel Kurtz Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Sorry, I had to dig my way out through my backlog. On Tue, Mar 3, 2015 at 10:36 PM, Joerg Roedel wrote: > On Mon, Feb 09, 2015 at 08:19:21PM +0900, Tomasz Figa wrote: >> Even though the code uses the dt_lock spin lock to serialize mapping >> operation from different threads, it does not protect from IOMMU >> accesses that might be already taking place and thus altering state >> of the IOTLB. This means that current mapping code which first zaps >> the page table and only then updates it with new mapping which is >> prone to mentioned race. > > Could you elabortate a bit on the race and why it is sufficient to zap > only the first and the last iova? From the description and the comments > in the patch this is not clear to me. Let's start with why it's sufficient to zap only first and last iova. While unmapping, the driver zaps all iovas belonging to the mapping, so the page tables not used by any mapping won't be cached. Now when the driver creates a mapping it might end up occupying several page tables. However, since the mapping area is virtually contiguous, only the first and last page table can be shared with different mappings. This means that only first and last iovas can be already cached. In fact, we could detect if first and last page tables are shared and do not zap at all, but this wouldn't really optimize too much. Why invalidating one iova is enough to invalidate the whole page table is unclear to me as well, but it seems to be the correct way on this hardware. As for the race, it's also kind of explained by the above. The already running hardware can trigger page table look-ups in the IOMMU and so caching of the page table between our zapping and updating its contents. With this patch zapping is performed after updating the page table so the race is gone. Best regards, Tomasz From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tomasz Figa Subject: Re: [PATCH] CHROMIUM: iommu: rockchip: Make sure that page table state is coherent Date: Mon, 23 Mar 2015 17:38:45 +0900 Message-ID: References: <1423480761-33453-1-git-send-email-tfiga@chromium.org> <20150303133659.GD10502@8bytes.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20150303133659.GD10502-zLv9SwRftAIdnm+yROfE0A@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Joerg Roedel Cc: Heiko Stuebner , "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Daniel Kurtz , "open list:ARM/Rockchip SoC..." , iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" List-Id: linux-rockchip.vger.kernel.org Sorry, I had to dig my way out through my backlog. On Tue, Mar 3, 2015 at 10:36 PM, Joerg Roedel wrote: > On Mon, Feb 09, 2015 at 08:19:21PM +0900, Tomasz Figa wrote: >> Even though the code uses the dt_lock spin lock to serialize mapping >> operation from different threads, it does not protect from IOMMU >> accesses that might be already taking place and thus altering state >> of the IOTLB. This means that current mapping code which first zaps >> the page table and only then updates it with new mapping which is >> prone to mentioned race. > > Could you elabortate a bit on the race and why it is sufficient to zap > only the first and the last iova? From the description and the comments > in the patch this is not clear to me. Let's start with why it's sufficient to zap only first and last iova. While unmapping, the driver zaps all iovas belonging to the mapping, so the page tables not used by any mapping won't be cached. Now when the driver creates a mapping it might end up occupying several page tables. However, since the mapping area is virtually contiguous, only the first and last page table can be shared with different mappings. This means that only first and last iovas can be already cached. In fact, we could detect if first and last page tables are shared and do not zap at all, but this wouldn't really optimize too much. Why invalidating one iova is enough to invalidate the whole page table is unclear to me as well, but it seems to be the correct way on this hardware. As for the race, it's also kind of explained by the above. The already running hardware can trigger page table look-ups in the IOMMU and so caching of the page table between our zapping and updating its contents. With this patch zapping is performed after updating the page table so the race is gone. Best regards, Tomasz From mboxrd@z Thu Jan 1 00:00:00 1970 From: tfiga@chromium.org (Tomasz Figa) Date: Mon, 23 Mar 2015 17:38:45 +0900 Subject: [PATCH] CHROMIUM: iommu: rockchip: Make sure that page table state is coherent In-Reply-To: <20150303133659.GD10502@8bytes.org> References: <1423480761-33453-1-git-send-email-tfiga@chromium.org> <20150303133659.GD10502@8bytes.org> Message-ID: To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org Sorry, I had to dig my way out through my backlog. On Tue, Mar 3, 2015 at 10:36 PM, Joerg Roedel wrote: > On Mon, Feb 09, 2015 at 08:19:21PM +0900, Tomasz Figa wrote: >> Even though the code uses the dt_lock spin lock to serialize mapping >> operation from different threads, it does not protect from IOMMU >> accesses that might be already taking place and thus altering state >> of the IOTLB. This means that current mapping code which first zaps >> the page table and only then updates it with new mapping which is >> prone to mentioned race. > > Could you elabortate a bit on the race and why it is sufficient to zap > only the first and the last iova? From the description and the comments > in the patch this is not clear to me. Let's start with why it's sufficient to zap only first and last iova. While unmapping, the driver zaps all iovas belonging to the mapping, so the page tables not used by any mapping won't be cached. Now when the driver creates a mapping it might end up occupying several page tables. However, since the mapping area is virtually contiguous, only the first and last page table can be shared with different mappings. This means that only first and last iovas can be already cached. In fact, we could detect if first and last page tables are shared and do not zap at all, but this wouldn't really optimize too much. Why invalidating one iova is enough to invalidate the whole page table is unclear to me as well, but it seems to be the correct way on this hardware. As for the race, it's also kind of explained by the above. The already running hardware can trigger page table look-ups in the IOMMU and so caching of the page table between our zapping and updating its contents. With this patch zapping is performed after updating the page table so the race is gone. Best regards, Tomasz