Re: [Xen-devel] [PATCH V2 6/6] iommu/arm: Add Renesas IPMMU-VMSA support

[Oleksandr <olekstysh@gmail.com>]

Hi, Shimoda-san.

Thank you for the review.


>
>> From: Oleksandr Tyshchenko, Sent: Saturday, August 3, 2019 1:40 AM
>>
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU)
>> which provides address translation and access protection functionalities
>> to processing units and interconnect networks.
>>
>> Please note, current driver is supposed to work only with newest
>> Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation
> This should be "R-Car Gen3 SoCs", instead of "Gen3 SoCs".

Will update.


>
>> table format and is able to use CPU's P2M table as is if one is
>> 3-level page table (up to 40 bit IPA).
>>
>> The major differences compare to the Linux driver are:
>>
>> 1. Stage 1/Stage 2 translation. Linux driver supports Stage 1
>> translation only (with Stage 1 translation table format). It manages
>> page table by itself. But Xen driver supports Stage 2 translation
>> (with Stage 2 translation table format) to be able to share the P2M
>> with the CPU. Stage 1 translation is always bypassed in Xen driver.
>>
>> So, Xen driver is supposed to be used with newest Gen3 SoC revisions only
> Same here.

ok


>
>> (H3 ES3.0, M3 ES3.0, etc.) which IPMMU H/W supports stage 2 translation
> According to the latest manual, M3 ES3.0 is named as "M3-W+".

Will update.


>> diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
>> new file mode 100644
>> index 0000000..a34a8f8
>> --- /dev/null
>> +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
>> @@ -0,0 +1,1342 @@
>> +/*
>> + * xen/drivers/passthrough/arm/ipmmu-vmsa.c
>> + *
>> + * Driver for the Renesas IPMMU-VMSA found in R-Car Gen3 SoCs.
>> + *
>> + * The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU)
>> + * which provides address translation and access protection functionalities
>> + * to processing units and interconnect networks.
>> + *
>> + * Please note, current driver is supposed to work only with newest Gen3 SoCs
>> + * revisions which IPMMU hardware supports stage 2 translation table format and
>> + * is able to use CPU's P2M table as is.
>> + *
>> + * Based on Linux's IPMMU-VMSA driver from Renesas BSP:
>> + *    drivers/iommu/ipmmu-vmsa.c
> So, I think the Linux's Copyrights should be described here.

Yes, will add.


>
>> + * you can found at:
>> + *    url: git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git
>> + *    branch: v4.14.75-ltsi/rcar-3.9.6
>> + *    commit: e206eb5b81a60e64c35fbc3a999b1a0db2b98044
>> + * and Xen's SMMU driver:
>> + *    xen/drivers/passthrough/arm/smmu.c
>> + *
>> + * Copyright (C) 2016-2019 EPAM Systems Inc.
>> + *
>> + * This program is free software; you can redistribute it and/or
>> + * modify it under the terms and conditions of the GNU General Public
>> + * License, version 2, as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public
>> + * License along with this program; If not, see <http://www.gnu.org/licenses/>.
> I don't know that Xen license description rule, but since a few source files have
> SPDX-License-Identifier, can we also use it on the driver?

I am afraid, I don't know a correct answer for this question. I would 
leave this to maintainers.

I just followed sample copyright notice for GPL v2 License according to 
the document:

http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=CONTRIBUTING


>> + */
>> +
>> +#include <xen/delay.h>
>> +#include <xen/err.h>
>> +#include <xen/irq.h>
>> +#include <xen/lib.h>
>> +#include <xen/list.h>
> I don't know that Xen passthrough driver rule though, doesn't here need
> #include <xen/iommu.h>? (The xen/sched.h seems to have it so that
> no compile error happens though.)

Probably, yes, I should have included that header.


>> +
>> +#define IMTTBCR                        0x0008
>> +#define IMTTBCR_EAE                    (1 << 31)
>> +#define IMTTBCR_PMB                    (1 << 30)
>> +#define IMTTBCR_SH1_NON_SHAREABLE      (0 << 28)
>> +#define IMTTBCR_SH1_OUTER_SHAREABLE    (2 << 28)
>> +#define IMTTBCR_SH1_INNER_SHAREABLE    (3 << 28)
>> +#define IMTTBCR_SH1_MASK               (3 << 28)
>> +#define IMTTBCR_ORGN1_NC               (0 << 26)
>> +#define IMTTBCR_ORGN1_WB_WA            (1 << 26)
>> +#define IMTTBCR_ORGN1_WT               (2 << 26)
>> +#define IMTTBCR_ORGN1_WB               (3 << 26)
>> +#define IMTTBCR_ORGN1_MASK             (3 << 26)
>> +#define IMTTBCR_IRGN1_NC               (0 << 24)
>> +#define IMTTBCR_IRGN1_WB_WA            (1 << 24)
>> +#define IMTTBCR_IRGN1_WT               (2 << 24)
>> +#define IMTTBCR_IRGN1_WB               (3 << 24)
>> +#define IMTTBCR_IRGN1_MASK             (3 << 24)
>> +#define IMTTBCR_TSZ1_MASK              (1f << 16)
> At the moment, no one uses it though, this should be (0x1f << 16).

Will correct.


>
> <snip>
> +/* Xen IOMMU ops */
>> +static int __must_check ipmmu_iotlb_flush_all(struct domain *d)
>> +{
>> +    struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
>> +
>> +    if ( !xen_domain || !xen_domain->root_domain )
>> +        return 0;
>> +
>> +    spin_lock(&xen_domain->lock);
> Is local irq is already disabled here?
> If no, you should use spin_lock_irqsave() because the ipmmu_irq() also
> gets the lock.


No, it is not disabled. But, ipmmu_irq() uses another mmu->lock. So, I 
think, there won't be a deadlock.

Or I really missed something?

If we worry about ipmmu_tlb_invalidate() which is called here (to 
perform a flush by request from P2M code, which manages a page table) 
and from the irq handler (to perform a flush to resume address 
translation), I could use a tasklet to schedule ipmmu_tlb_invalidate() 
from the irq handler then. This way we would get this serialized. What 
do you think?


> # To be honest, in normal case, any irq on the current implementation
> # should not happen though.

Agree here.


>> +    /*
>> +     * Destroy Root IPMMU domain which context is mapped to this Xen domain
>> +     * if exits.
>> +     */
>> +    if ( xen_domain->root_domain )
>> +        ipmmu_free_root_domain(xen_domain->root_domain);
>> +
>> +    spin_unlock(&xen_domain->lock);
>> +
>> +    /*
>> +     * We assume that all master devices have already been detached from
>> +     * this Xen domain and there must be no associated Cache IPMMU domains
>> +     * in use.
>> +     */
>> +    ASSERT(list_empty(&xen_domain->cache_domains));
> I think this should be in the spin lock held by &xen_domain->lock.

OK. Will put spin_unlock after it.


>
>> +    xfree(xen_domain);
>> +    dom_iommu(d)->arch.priv = NULL;
>> +}
>> +
>> +static const struct iommu_ops ipmmu_iommu_ops =
>> +{
>> +    .init            = ipmmu_iommu_domain_init,
>> +    .hwdom_init      = ipmmu_iommu_hwdom_init,
>> +    .teardown        = ipmmu_iommu_domain_teardown,
>> +    .iotlb_flush     = ipmmu_iotlb_flush,
>> +    .iotlb_flush_all = ipmmu_iotlb_flush_all,
>> +    .assign_device   = ipmmu_assign_device,
>> +    .reassign_device = ipmmu_reassign_device,
>> +    .map_page        = arm_iommu_map_page,
>> +    .unmap_page      = arm_iommu_unmap_page,
>> +    .add_device      = ipmmu_add_device,
>> +};
>> +
>> +/* RCAR GEN3 product and cut information. */
> "R-Car Gen3 SoCs" is better than "RCAR GEN3".

Will update.


>
>> +#define RCAR_PRODUCT_MASK    0x00007F00
>> +#define RCAR_PRODUCT_H3      0x00004F00
>> +#define RCAR_PRODUCT_M3      0x00005200
> At least, I think we should be M3W, instead of M3.
> # FYI, M3-W and M3-W+ are the same value.

Will update.


-- 
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel