From: Abhishek Sahu <abhsahu@nvidia.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Alex Williamson <alex.williamson@redhat.com>,
Cornelia Huck <cohuck@redhat.com>,
Yishai Hadas <yishaih@nvidia.com>,
Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>,
Kevin Tian <kevin.tian@intel.com>,
"Rafael J . Wysocki" <rafael@kernel.org>,
Max Gurtovoy <mgurtovoy@nvidia.com>,
Bjorn Helgaas <bhelgaas@google.com>,
linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
linux-pm@vger.kernel.org, linux-pci@vger.kernel.org
Subject: Re: [PATCH v3 8/8] vfio/pci: Add the support for PCI D3cold state
Date: Tue, 31 May 2022 17:44:11 +0530 [thread overview]
Message-ID: <c73d537b-a653-bf79-68cd-ddc8f0f62a25@nvidia.com> (raw)
In-Reply-To: <20220530122546.GZ1343366@nvidia.com>
On 5/30/2022 5:55 PM, Jason Gunthorpe wrote:
> On Mon, May 30, 2022 at 04:45:59PM +0530, Abhishek Sahu wrote:
>
>> 1. In real use case, config or any other ioctl should not come along
>> with VFIO_DEVICE_FEATURE_POWER_MANAGEMENT ioctl request.
>>
>> 2. Maintain some 'access_count' which will be incremented when we
>> do any config space access or ioctl.
>
> Please don't open code locks - if you need a lock then write a proper
> lock. You can use the 'try' variants to bail out in cases where that
> is appropriate.
>
> Jason
Thanks Jason for providing your inputs.
In that case, should I introduce new rw_semaphore (For example
power_lock) and move ‘platform_pm_engaged’ under ‘power_lock’ ?
I was mainly concerned about locking rules w.r.t. existing
‘memory_lock’ and the code present in
vfio_pci_zap_and_down_write_memory_lock() which is internally taking
‘mmap_lock’ and ‘vma_lock’. But from the initial analysis, it seems
this should not cause any issue since we should not need ‘power_lock’
in the mmap fault handler or any read/write functions. We can
maintain following locking order
power_lock => memory_lock
1. At the beginning of config space access or ioctl, we can take the
lock
down_read(&vdev->power_lock);
if (vdev->platform_pm_engaged) {
up_read(&vdev->power_lock);
return -EIO;
}
And before returning from config or ioctl, we can release the lock.
2. Now ‘platform_pm_engaged’ is not protected with memory_lock and we
need to support the case where VFIO_DEVICE_FEATURE_POWER_MANAGEMENT
can be called without putting the device into D3hot explicitly.
So, I need to introduce a second variable which tracks the memory
disablement (like power_state_d3 in this patch) and will be
protected with 'memory_lock'. It will be set for both the cases,
where users change the power state to D3hot by config
write or user makes this ioctl. Inside vfio_pci_core_feature_pm(), now
the code will become
down_write(&vdev->power_lock);
...
switch (vfio_pm.low_power_state) {
case VFIO_DEVICE_LOW_POWER_STATE_ENTER:
...
vfio_pci_zap_and_down_write_memory_lock(vdev);
vdev->power_state_d3 = true;
up_write(&vdev->memory_lock);
...
up_write(&vdev->power_lock);
3. Inside __vfio_pci_memory_enabled(), we can check
vdev->power_state_d3 instead of current_state.
4. For ioctl access, as mentioned previously I need to add two
callbacks functions (one for start and one for end) in the struct
vfio_device_ops and call the same at start and end of ioctl from
vfio_device_fops_unl_ioctl().
Thanks,
Abhishek
next prev parent reply other threads:[~2022-05-31 12:14 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-25 9:26 [PATCH v3 0/8] vfio/pci: power management changes Abhishek Sahu
2022-04-25 9:26 ` [PATCH v3 1/8] vfio/pci: Invalidate mmaps and block the access in D3hot power state Abhishek Sahu
2022-04-26 1:42 ` kernel test robot
2022-04-26 14:14 ` Bjorn Helgaas
2022-04-25 9:26 ` [PATCH v3 2/8] vfio/pci: Change the PF power state to D0 before enabling VFs Abhishek Sahu
2022-04-25 9:26 ` [PATCH v3 3/8] vfio/pci: Virtualize PME related registers bits and initialize to zero Abhishek Sahu
2022-04-25 9:26 ` [PATCH v3 4/8] vfio/pci: Add support for setting driver data inside core layer Abhishek Sahu
2022-05-03 17:11 ` Alex Williamson
2022-05-04 0:20 ` Jason Gunthorpe
2022-05-04 10:32 ` Abhishek Sahu
2022-04-25 9:26 ` [PATCH v3 5/8] vfio/pci: Enable runtime PM for vfio_pci_core based drivers Abhishek Sahu
2022-05-04 19:42 ` Alex Williamson
2022-05-05 9:07 ` Abhishek Sahu
2022-04-25 9:26 ` [PATCH v3 6/8] vfio: Invoke runtime PM API for IOCTL request Abhishek Sahu
2022-05-04 19:42 ` Alex Williamson
2022-05-05 9:40 ` Abhishek Sahu
2022-05-09 22:30 ` Alex Williamson
2022-04-25 9:26 ` [PATCH v3 7/8] vfio/pci: Mask INTx during runtime suspend Abhishek Sahu
2022-04-25 9:26 ` [PATCH v3 8/8] vfio/pci: Add the support for PCI D3cold state Abhishek Sahu
2022-05-04 19:45 ` Alex Williamson
2022-05-05 12:16 ` Abhishek Sahu
2022-05-09 21:48 ` Alex Williamson
2022-05-10 13:26 ` Abhishek Sahu
2022-05-10 13:30 ` Jason Gunthorpe
2022-05-12 12:27 ` Abhishek Sahu
2022-05-12 12:47 ` Jason Gunthorpe
2022-05-30 11:15 ` Abhishek Sahu
2022-05-30 12:25 ` Jason Gunthorpe
2022-05-31 12:14 ` Abhishek Sahu [this message]
2022-05-31 19:43 ` Jason Gunthorpe
2022-05-31 22:52 ` Alex Williamson
2022-06-01 9:49 ` Abhishek Sahu
2022-06-01 16:21 ` Alex Williamson
2022-06-01 17:30 ` Jason Gunthorpe
2022-06-01 18:15 ` Alex Williamson
2022-06-01 23:17 ` Jason Gunthorpe
2022-06-02 11:52 ` Abhishek Sahu
2022-06-02 17:44 ` Alex Williamson
2022-06-03 10:19 ` Abhishek Sahu
2022-06-07 21:50 ` Alex Williamson
2022-06-08 10:12 ` Abhishek Sahu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c73d537b-a653-bf79-68cd-ddc8f0f62a25@nvidia.com \
--to=abhsahu@nvidia.com \
--cc=alex.williamson@redhat.com \
--cc=bhelgaas@google.com \
--cc=cohuck@redhat.com \
--cc=jgg@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=mgurtovoy@nvidia.com \
--cc=rafael@kernel.org \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).