LKML Archive on lore.kernel.org
 help / color / Atom feed
From: "Liu, Yi L" <yi.l.liu@intel.com>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: "kwankhede@nvidia.com" <kwankhede@nvidia.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"peterx@redhat.com" <peterx@redhat.com>,
	"baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>
Subject: RE: [PATCH v4 01/12] vfio_pci: refine user config reference in vfio-pci module
Date: Thu, 16 Jan 2020 12:19:09 +0000
Message-ID: <A2975661238FB949B60364EF0F2C25743A183E28@SHSMSX104.ccr.corp.intel.com> (raw)
In-Reply-To: <20200109154821.047f700a@w520.home>

> From: Alex Williamson [mailto:alex.williamson@redhat.com]
> Sent: Friday, January 10, 2020 6:48 AM
> To: Liu, Yi L <yi.l.liu@intel.com>
> Subject: Re: [PATCH v4 01/12] vfio_pci: refine user config reference in vfio-pci
> module
> 
> On Tue,  7 Jan 2020 20:01:38 +0800
> Liu Yi L <yi.l.liu@intel.com> wrote:
> 
> > This patch adds three fields in struct vfio_pci_device to pass the user
> > configurations of vfio-pci.ko module to some functions which could be
> > common in future usage. The values stored in struct vfio_pci_device will
> > be initiated in probe and refreshed in device open phase to allow runtime
> > modifications to parameters. e.g. disable_idle_d3 and nointxmask.
> >
> > Cc: Kevin Tian <kevin.tian@intel.com>
> > Cc: Lu Baolu <baolu.lu@linux.intel.com>
> > Signed-off-by: Liu Yi L <yi.l.liu@intel.com>
> > ---
> >  drivers/vfio/pci/vfio_pci.c         | 37 ++++++++++++++++++++++++++-----------
> >  drivers/vfio/pci/vfio_pci_private.h |  8 ++++++++
> >  2 files changed, 34 insertions(+), 11 deletions(-)
> >
> > diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
> > index 379a02c..af507c2 100644
> > --- a/drivers/vfio/pci/vfio_pci.c
> > +++ b/drivers/vfio/pci/vfio_pci.c
> > @@ -54,10 +54,10 @@ module_param(disable_idle_d3, bool, S_IRUGO |
> S_IWUSR);
> >  MODULE_PARM_DESC(disable_idle_d3,
> >  		 "Disable using the PCI D3 low power state for idle, unused devices");
> >
> > -static inline bool vfio_vga_disabled(void)
> > +static inline bool vfio_vga_disabled(struct vfio_pci_device *vdev)
> >  {
> >  #ifdef CONFIG_VFIO_PCI_VGA
> > -	return disable_vga;
> > +	return vdev->disable_vga;
> >  #else
> >  	return true;
> >  #endif
> > @@ -78,7 +78,8 @@ static unsigned int vfio_pci_set_vga_decode(void *opaque,
> bool single_vga)
> >  	unsigned char max_busnr;
> >  	unsigned int decodes;
> >
> > -	if (single_vga || !vfio_vga_disabled() || pci_is_root_bus(pdev->bus))
> > +	if (single_vga || !vfio_vga_disabled(vdev) ||
> > +		pci_is_root_bus(pdev->bus))
> >  		return VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM |
> >  		       VGA_RSRC_LEGACY_IO | VGA_RSRC_LEGACY_MEM;
> >
> > @@ -289,7 +290,7 @@ static int vfio_pci_enable(struct vfio_pci_device *vdev)
> >  	if (!vdev->pci_saved_state)
> >  		pci_dbg(pdev, "%s: Couldn't store saved state\n", __func__);
> >
> > -	if (likely(!nointxmask)) {
> > +	if (likely(!vdev->nointxmask)) {
> >  		if (vfio_pci_nointx(pdev)) {
> >  			pci_info(pdev, "Masking broken INTx support\n");
> >  			vdev->nointx = true;
> > @@ -326,7 +327,7 @@ static int vfio_pci_enable(struct vfio_pci_device *vdev)
> >  	} else
> >  		vdev->msix_bar = 0xFF;
> >
> > -	if (!vfio_vga_disabled() && vfio_pci_is_vga(pdev))
> > +	if (!vfio_vga_disabled(vdev) && vfio_pci_is_vga(pdev))
> >  		vdev->has_vga = true;
> >
> >
> > @@ -462,10 +463,17 @@ static void vfio_pci_disable(struct vfio_pci_device *vdev)
> >
> >  	vfio_pci_try_bus_reset(vdev);
> >
> > -	if (!disable_idle_d3)
> > +	if (!vdev->disable_idle_d3)
> >  		vfio_pci_set_power_state(vdev, PCI_D3hot);
> >  }
> >
> > +void vfio_pci_refresh_config(struct vfio_pci_device *vdev,
> > +			bool nointxmask, bool disable_idle_d3)
> > +{
> > +	vdev->nointxmask = nointxmask;
> > +	vdev->disable_idle_d3 = disable_idle_d3;
> 
> These two are selected (not disable_vga) because they're the only
> writable module options, correct?

yep. These were selected per previous review comments from
you. I also checked in the code, the existing 4 module options
are clarified as below, and I can see only nointxmask and disable_idle_d3
are writable. I guess this should be the evidence for selecting the
modules options to be refreshed in vfio_pci_refresh_config().

static char ids[1024] __initdata;
module_param_string(ids, ids, sizeof(ids), 0);

static bool nointxmask;
module_param_named(nointxmask, nointxmask, bool, S_IRUGO | S_IWUSR);

static bool disable_vga;
module_param(disable_vga, bool, S_IRUGO);

static bool disable_idle_d3;
module_param(disable_idle_d3, bool, S_IRUGO | S_IWUSR);

> 
> > +}
> > +
> >  static void vfio_pci_release(void *device_data)
> >  {
> >  	struct vfio_pci_device *vdev = device_data;
> > @@ -490,6 +498,8 @@ static int vfio_pci_open(void *device_data)
> >  	if (!try_module_get(THIS_MODULE))
> >  		return -ENODEV;
> >
> > +	vfio_pci_refresh_config(vdev, nointxmask, disable_idle_d3);
> > +
> >  	mutex_lock(&vdev->reflck->lock);
> >
> >  	if (!vdev->refcnt) {
> > @@ -1330,6 +1340,11 @@ static int vfio_pci_probe(struct pci_dev *pdev, const
> struct pci_device_id *id)
> >  	spin_lock_init(&vdev->irqlock);
> >  	mutex_init(&vdev->ioeventfds_lock);
> >  	INIT_LIST_HEAD(&vdev->ioeventfds_list);
> > +	vdev->nointxmask = nointxmask;
> > +#ifdef CONFIG_VFIO_PCI_VGA
> > +	vdev->disable_vga = disable_vga;
> > +#endif
> > +	vdev->disable_idle_d3 = disable_idle_d3;
> 
> But this could still use vfio_pci_refresh_config() for those writable
> options and set disable_vga separately, couldn't it? 

Right, would modify it. thanks.

> Also, since
> disable_idle_d3 is related to power handling of the device while it is
> not opened by the user, shouldn't the config also be refreshed when the
> device is released by the user?

Oh, yes. You told me to do it. But I assumed that we only care
about the config used during an open() and a release() circle.
I missed it will affect the power management. Let me add the
config refresh at release() all the same. Thanks.

> 
> >
> >  	ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev);
> >  	if (ret) {
> > @@ -1354,7 +1369,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const
> struct pci_device_id *id)
> >
> >  	vfio_pci_probe_power_state(vdev);
> >
> > -	if (!disable_idle_d3) {
> > +	if (!vdev->disable_idle_d3) {
> >  		/*
> >  		 * pci-core sets the device power state to an unknown value at
> >  		 * bootup and after being removed from a driver.  The only
> > @@ -1385,7 +1400,7 @@ static void vfio_pci_remove(struct pci_dev *pdev)
> >  	kfree(vdev->region);
> >  	mutex_destroy(&vdev->ioeventfds_lock);
> >
> > -	if (!disable_idle_d3)
> > +	if (!vdev->disable_idle_d3)
> >  		vfio_pci_set_power_state(vdev, PCI_D0);
> >
> >  	kfree(vdev->pm_save);
> > @@ -1620,7 +1635,7 @@ static void vfio_pci_try_bus_reset(struct vfio_pci_device
> *vdev)
> >  		if (!ret) {
> >  			tmp->needs_reset = false;
> >
> > -			if (tmp != vdev && !disable_idle_d3)
> > +			if (tmp != vdev && !tmp->disable_idle_d3)
> >  				vfio_pci_set_power_state(tmp, PCI_D3hot);
> >  		}
> >
> > @@ -1636,7 +1651,7 @@ static void __exit vfio_pci_cleanup(void)
> >  	vfio_pci_uninit_perm_bits();
> >  }
> >
> > -static void __init vfio_pci_fill_ids(void)
> > +static void __init vfio_pci_fill_ids(char *ids)
> 
> This might be more clear if the global was also renamed vfio_pci_ids.

Yep. Let me rename it later.

> >  {
> >  	char *p, *id;
> >  	int rc;
> > @@ -1691,7 +1706,7 @@ static int __init vfio_pci_init(void)
> >  	if (ret)
> >  		goto out_driver;
> >
> > -	vfio_pci_fill_ids();
> > +	vfio_pci_fill_ids(ids);
> >
> >  	return 0;
> >
> > diff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pci_private.h
> > index 8a2c760..0398608 100644
> > --- a/drivers/vfio/pci/vfio_pci_private.h
> > +++ b/drivers/vfio/pci/vfio_pci_private.h
> > @@ -122,6 +122,11 @@ struct vfio_pci_device {
> >  	struct list_head	dummy_resources_list;
> >  	struct mutex		ioeventfds_lock;
> >  	struct list_head	ioeventfds_list;
> > +	bool			nointxmask;
> > +#ifdef CONFIG_VFIO_PCI_VGA
> > +	bool			disable_vga;
> > +#endif
> > +	bool			disable_idle_d3;
> 
> It seems like there are more relevant places these could be within this
> structure, ex. nointxmask next to nointx, disable_vga near has_vga,
> disable_idle_d3 maybe near needs_pm_restore (even though those aren't
> conceptually related).  Not necessarily related to this series, it
> might be time to convert the existing bools to bit fields, but even
> before that the alignment of adding these as bools grouped with the
> existing bools is probably better.  Thanks,

Agreed. Will place the new bools at better place (with proper neighbors:-))

> Alex

Thanks,
Yi Liu

> 
> >  };
> >
> >  #define is_intx(vdev) (vdev->irq_type == VFIO_PCI_INTX_IRQ_INDEX)
> > @@ -130,6 +135,9 @@ struct vfio_pci_device {
> >  #define is_irq_none(vdev) (!(is_intx(vdev) || is_msi(vdev) || is_msix(vdev)))
> >  #define irq_is(vdev, type) (vdev->irq_type == type)
> >
> > +extern void vfio_pci_refresh_config(struct vfio_pci_device *vdev,
> > +				bool nointxmask, bool disable_idle_d3);
> > +
> >  extern void vfio_pci_intx_mask(struct vfio_pci_device *vdev);
> >  extern void vfio_pci_intx_unmask(struct vfio_pci_device *vdev);
> >


  reply index

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-07 12:01 [PATCH v4 00/12] vfio_pci: wrap pci device as a mediated device Liu Yi L
2020-01-07 12:01 ` [PATCH v4 01/12] vfio_pci: refine user config reference in vfio-pci module Liu Yi L
2020-01-09 22:48   ` Alex Williamson
2020-01-16 12:19     ` Liu, Yi L [this message]
2020-01-07 12:01 ` [PATCH v4 02/12] vfio_pci: move vfio_pci_is_vga/vfio_vga_disabled to header file Liu Yi L
2020-01-15 10:43   ` Cornelia Huck
2020-01-16 12:46     ` Liu, Yi L
2020-01-07 12:01 ` [PATCH v4 03/12] vfio_pci: refine vfio_pci_driver reference in vfio_pci.c Liu Yi L
2020-01-09 22:48   ` Alex Williamson
2020-01-10  7:35     ` Liu, Yi L
2020-01-07 12:01 ` [PATCH v4 04/12] vfio_pci: make common functions be extern Liu Yi L
2020-01-15 10:56   ` Cornelia Huck
2020-01-16 12:48     ` Liu, Yi L
2020-01-07 12:01 ` [PATCH v4 05/12] vfio_pci: duplicate vfio_pci.c Liu Yi L
2020-01-15 11:03   ` Cornelia Huck
2020-01-15 15:12     ` Alex Williamson
2020-01-07 12:01 ` [PATCH v4 06/12] vfio_pci: shrink vfio_pci_common.c Liu Yi L
2020-01-07 12:01 ` [PATCH v4 07/12] vfio_pci: shrink vfio_pci.c Liu Yi L
2020-01-08 11:24   ` kbuild test robot
2020-01-09 22:48   ` Alex Williamson
2020-01-16 12:42     ` Liu, Yi L
2020-01-07 12:01 ` [PATCH v4 08/12] vfio_pci: duplicate vfio_pci_private.h to include/linux Liu Yi L
2020-01-07 12:01 ` [PATCH v4 09/12] vfio: split vfio_pci_private.h into two files Liu Yi L
2020-01-09 22:48   ` Alex Williamson
2020-01-16 11:59     ` Liu, Yi L
2020-01-07 12:01 ` [PATCH v4 10/12] vfio: build vfio_pci_common.c into a kernel module Liu Yi L
2020-01-07 12:01 ` [PATCH v4 11/12] samples: add vfio-mdev-pci driver Liu Yi L
2020-01-09 22:48   ` Alex Williamson
2020-01-16 12:33     ` Liu, Yi L
2020-01-16 21:24       ` Alex Williamson
2020-01-18 14:25         ` Liu, Yi L
2020-01-20 21:07           ` Alex Williamson
2020-01-21  7:43             ` Tian, Kevin
2020-01-21  8:43               ` Yan Zhao
2020-01-21 20:04                 ` Alex Williamson
2020-01-21 21:54                   ` Yan Zhao
2020-01-23 23:33                     ` Alex Williamson
2020-01-31  2:26                       ` Yan Zhao
2020-01-15 12:30   ` Cornelia Huck
2020-01-16 13:23     ` Liu, Yi L
2020-01-16 17:40       ` Cornelia Huck
2020-01-18 14:23         ` Liu, Yi L
2020-01-20  8:55           ` Cornelia Huck
2020-01-07 12:01 ` [PATCH v4 12/12] samples: refine " Liu Yi L

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=A2975661238FB949B60364EF0F2C25743A183E28@SHSMSX104.ccr.corp.intel.com \
    --to=yi.l.liu@intel.com \
    --cc=alex.williamson@redhat.com \
    --cc=baolu.lu@linux.intel.com \
    --cc=joro@8bytes.org \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=kwankhede@nvidia.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=peterx@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git