From: Alexander Duyck <alexander.duyck@gmail.com>
To: ethan zhao <ethan.zhao@oracle.com>, Bjorn Helgaas <helgaas@kernel.org>
Cc: bhelgaas@google.com, linux-pci@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] iov: restore NumVFs register to 0 before return from virtfn_max_buses()
Date: Tue, 27 Oct 2015 08:15:30 -0700 [thread overview]
Message-ID: <562F9512.3000907@gmail.com> (raw)
In-Reply-To: <562F43C6.7000009@oracle.com>
On 10/27/2015 02:28 AM, ethan zhao wrote:
> Alexander,
> On 2015/10/27 13:48, Alexander Duyck wrote:
>> On 10/15/2015 10:16 AM, Bjorn Helgaas wrote:
>>> Hi Ethan,
>>>
>>> On Wed, Sep 16, 2015 at 12:19:53PM +0900, Ethan Zhao wrote:
>>>> After commit 4449f079722c ("PCI: Calculate maximum number of buses
>>>> required for VFs"),the initial value of NumVFs register was left to
>>>> non-zero after sriov_init() and no VFs was enabled in device driver.
>>>> this changed the behaviour of kernel exported by lspci and sysfs etc.
>>>> so this patch restore the NumVFs register to zero after the
>>>> calculation of max_VF_buses was done and before return from
>>>> virtfn_max_buses().
>>>>
>>>> Tested on stable 4.1 and passed building on stable 4.3-rc1
>>>>
>>>> Signed-off-by: Ethan Zhao <ethan.zhao@oracle.com>
>>>> Tested-by: Sriharsha Yadagudde <sriharsha.devdas@oracle.com>
>>> Can you test the patch below? I'm trying to avoid touching
>>> PCI_SRIOV_NUM_VF in more than one place, and I think it's OK to set it
>>> and test offset/stride at the end, instead of setting NUM_VF to zero,
>>> testing offset/stride, computing max_bus, then setting NUM_VF to zero
>>> again.
>>>
>>> Bjorn
>>>
>>>
>>> commit 8e20e89658f23b8d16b1e21810e9f63c8625129c
>>> Author: Bjorn Helgaas <bhelgaas@google.com>
>>> Date: Thu Oct 15 11:31:21 2015 -0500
>>>
>>> PCI: Set SR-IOV NumVFs to zero after enumeration
>>> The enumeration path should leave NumVFs set to zero. But
>>> after
>>> 4449f079722c ("PCI: Calculate maximum number of buses required
>>> for VFs"),
>>> we call virtfn_max_buses() in the enumeration path, which
>>> changes NumVFs.
>>> This NumVFs change is visible via lspci and sysfs until a
>>> driver enables
>>> SR-IOV.
>>> Set NumVFs to zero after virtfn_max_buses() computes the
>>> maximum number of
>>> buses.
>>> Fixes: 4449f079722c ("PCI: Calculate maximum number of
>>> buses required for VFs")
>>> Based-on-patch-by: Ethan Zhao <ethan.zhao@oracle.com>
>>> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>>>
>>> diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
>>> index ee0ebff..0202ab0 100644
>>> --- a/drivers/pci/iov.c
>>> +++ b/drivers/pci/iov.c
>>> @@ -384,7 +384,7 @@ static int sriov_init(struct pci_dev *dev, int pos)
>>> int rc;
>>> int nres;
>>> u32 pgsz;
>>> - u16 ctrl, total, offset, stride;
>>> + u16 ctrl, total;
>>> struct pci_sriov *iov;
>>> struct resource *res;
>>> struct pci_dev *pdev;
>>> @@ -414,11 +414,6 @@ static int sriov_init(struct pci_dev *dev, int
>>> pos)
>>> found:
>>> pci_write_config_word(dev, pos + PCI_SRIOV_CTRL, ctrl);
>>> - pci_write_config_word(dev, pos + PCI_SRIOV_NUM_VF, 0);
>>> - pci_read_config_word(dev, pos + PCI_SRIOV_VF_OFFSET, &offset);
>>> - pci_read_config_word(dev, pos + PCI_SRIOV_VF_STRIDE, &stride);
>>> - if (!offset || (total > 1 && !stride))
>>> - return -EIO;
>>> pci_read_config_dword(dev, pos + PCI_SRIOV_SUP_PGSIZE, &pgsz);
>>> i = PAGE_SHIFT > 12 ? PAGE_SHIFT - 12 : 0;
>>> @@ -456,8 +451,6 @@ found:
>>> iov->nres = nres;
>>> iov->ctrl = ctrl;
>>> iov->total_VFs = total;
>>> - iov->offset = offset;
>>> - iov->stride = stride;
>>> iov->pgsz = pgsz;
>>> iov->self = dev;
>>> pci_read_config_dword(dev, pos + PCI_SRIOV_CAP, &iov->cap);
>>> @@ -475,6 +468,11 @@ found:
>>> dev->sriov = iov;
>>> dev->is_physfn = 1;
>>> iov->max_VF_buses = virtfn_max_buses(dev);
>>> + pci_iov_set_numvfs(dev, 0);
>>> + if (!iov->offset || (total > 1 && !iov->stride)) {
>>> + rc = -EIO;
>>> + goto failed;
>>> + }
>>> return 0;
>>
>> You might want to reorder this a bit. The problem is offset and
>> stride can be 0 if numvfs is 0. So you should probably test offset
>> and stride
> Yes, the spec says "Note: First VF Offset is unused if NumVFs is 0.
> If NumVFs is greater than 0, First VF Offset must
> 25 not be zero. "
>> first, and then reset numvfs to 0.
> Why test it before reset numvfs ?
Because pci_iov_set_numvfs will end up resetting offset and stride based
on the values when it writes the 0 into numvfs. As such testing it
after the reset could give you invalid values since offset and stride
can be 0 when numvfs is 0.
Actually maybe I will go over this and submit a couple patches myself.
Looking over things it seems like the code has gotten a bit silly since
virtfn_max_busses looks like it is iterating over VFs and that doesn't
make much sense when offset and stride are both positive values so it
should just be a matter of computing the bus of the last VF.
- Alex
next prev parent reply other threads:[~2015-10-27 15:15 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-16 3:19 [PATCH v2] iov: restore NumVFs register to 0 before return from virtfn_max_buses() Ethan Zhao
2015-10-15 17:16 ` Bjorn Helgaas
2015-10-21 20:54 ` Bjorn Helgaas
2015-10-22 1:29 ` ethan zhao
2015-10-27 1:13 ` ethan zhao
2015-10-27 5:48 ` Alexander Duyck
2015-10-27 9:28 ` ethan zhao
2015-10-27 15:15 ` Alexander Duyck [this message]
2015-10-29 3:29 ` ethan zhao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=562F9512.3000907@gmail.com \
--to=alexander.duyck@gmail.com \
--cc=bhelgaas@google.com \
--cc=ethan.zhao@oracle.com \
--cc=helgaas@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).