Re: [PATCH 4/4] drm/i915: Move ioremap_wc tracking onto VMA

From: Chris Wilson <chris@chris-wilson.co.uk>
To: "Luis R. Rodriguez" <mcgrof@kernel.org>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>,
	netdev@vger.kernel.org, intel-gfx@lists.freedesktop.org,
	linux-kernel@vger.kernel.org, Ingo Molnar <mingo@kernel.org>,
	"Peter Zijlstra (Intel)" <peterz@infradead.org>,
	dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org,
	Daniel Vetter <daniel.vetter@intel.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Yishai Hadas <yishaih@mellanox.com>,
	David Hildenbrand <dahi@linux.vnet.ibm.com>
Subject: Re: [PATCH 4/4] drm/i915: Move ioremap_wc tracking onto VMA
Date: Wed, 20 Apr 2016 10:38:05 +0100	[thread overview]
Message-ID: <20160420093805.GE14602@nuc-i3427.alporthouse.com> (raw)
In-Reply-To: <20160420091054.GL1990@wotan.suse.de>

On Wed, Apr 20, 2016 at 11:10:54AM +0200, Luis R. Rodriguez wrote:
> On Tue, Apr 19, 2016 at 01:33:58PM +0100, Chris Wilson wrote:
> > diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> > index 6ce2c31b9a81..9ef47329e8ae 100644
> > --- a/drivers/gpu/drm/i915/i915_gem.c
> > +++ b/drivers/gpu/drm/i915/i915_gem.c
> > @@ -3346,6 +3346,15 @@ static void i915_gem_object_finish_gtt(struct drm_i915_gem_object *obj)
> >  					    old_write_domain);
> >  }
> >  
> > +static void __i915_vma_iounmap(struct i915_vma *vma)
> > +{
> > +	if (vma->iomap == NULL)
> > +		return;
> > +
> > +	io_mapping_unmap(vma->iomap);
> 
> The NULL check could just be done by io_mapping_unmap() then you
> can avoid this in other drivers too.
> 
> > +	vma->iomap = NULL;
> 
> You added accounting here, by simple int and inc / dec'ing it.
> I cannot confirm if it is correctly avoiding races, can you
> confirm?

Yes, the vma->pin_count is guarded by the struct_mutex atm. (The
struct_mutex is our own BKL :(

> Also you added accounting for the custom vma pinning thing and do
> GEM_BUG_ON(vma->pin_count == 0); when you unpin one instance but *you do not*
> do something like GEM_BUG_ON(vma->pin_count != 0); when you do the final full
> iounmap. That seems rather sloppy.

It's placed next to the function where pin_count == 0 and only called
from it. Yes, I did think the same...

> iomapping stuff has its own custom data structure, why not just use that data
> structure instead of the struct i915_vma and generalize this ? Drivers can
> be buggy and best if we avoid custom driver accounting and just do it in a neat
> generic fashion.

Completely different tasks, as far as I am aware. The iomapping is about
providing CPU access to the IO region, dma-remapping about providing
device access to physical memory, and our own VMA is about how the
object sits in all the different views of both CPU and device address
spaces (of which there are many, and even the CPU accessible address
space is not the entirety of that particular address space). 

> Then other drivers could use this too.

drivers/gpu/drm/ttm (you didn't hear me say that...)

> > diff --git a/drivers/gpu/drm/i915/intel_fbdev.c b/drivers/gpu/drm/i915/intel_fbdev.c
> > index 79ac202f3870..93f54a10042f 100644
> > --- a/drivers/gpu/drm/i915/intel_fbdev.c
> > +++ b/drivers/gpu/drm/i915/intel_fbdev.c
> > @@ -244,22 +245,23 @@ static int intelfb_create(struct drm_fb_helper *helper,
> >  	info->flags = FBINFO_DEFAULT | FBINFO_CAN_FORCE_OUTPUT;
> >  	info->fbops = &intelfb_ops;
> >  
> > +	vma = i915_gem_obj_to_ggtt(obj);
> > +
> >  	/* setup aperture base/size for vesafb takeover */
> >  	info->apertures->ranges[0].base = dev->mode_config.fb_base;
> >  	info->apertures->ranges[0].size = ggtt->mappable_end;
> >  
> > -	info->fix.smem_start = dev->mode_config.fb_base + i915_gem_obj_ggtt_offset(obj);
> > -	info->fix.smem_len = size;
> > +	info->fix.smem_start = dev->mode_config.fb_base + vma->node.start;
> > +	info->fix.smem_len = vma->node.size;
> >  
> > -	info->screen_base =
> > -		ioremap_wc(ggtt->mappable_base + i915_gem_obj_ggtt_offset(obj),
> > -			   size);
> > -	if (!info->screen_base) {
> > +	vaddr = i915_vma_pin_iomap(vma);
> > +	if (IS_ERR(vaddr)) {
> >  		DRM_ERROR("Failed to remap framebuffer into virtual memory\n");
> > -		ret = -ENOSPC;
> > +		ret = PTR_ERR(vaddr);
> >  		goto out_destroy_fbi;
> >  	}
> > -	info->screen_size = size;
> > +	info->screen_base = vaddr;
> > +	info->screen_size = vma->node.size;
> 
> some framebuffer drivers tend to use a generic start address of
> iinfo->fix.smem_start and a length of info->fix.smem_len, this
> driver sets the smem_start above, but its different than what
> gets ioremap for a start address:
> 
> +               ptr = io_mapping_map_wc(i915_vm_to_ggtt(vma->vm)->mappable,
> +                                       vma->node.start,
> +                                       vma->node.size);
> 
> fix.smem_start is :
> 
> 
> > +	info->fix.smem_start = dev->mode_config.fb_base + vma->node.start;
> 
> The smem_len matches though. Can you clarify if its correct for
> the io_mapping_map_wc() should not be using info->fix.smem_start
> (which is dev->mode_config.fb_base + vma->node.start)?

dev->mode_config.fb_base is the base address of the mappable region. It
is an inconsistently in naming that just hasn't annoyed me enough to
fix.

> Reason I ask is since I noticed a while ago a lot of drivers
> were using info->fix.smem_start and info->fix.smem_len consistently
> for their ioremap'd areas it might make sense instead to let the
> internal framebuffer (register_framebuffer()) optionally manage the
> ioremap_wc() for drivers, given that this is pretty generic stuff.

Apart from drivers like ours we would end up with multiple mappings to
the same region. It was just a little grevience that I think was worth
fixing. It does highlight how buggy our code is currently though as we
never relinquish that mapping when the driver is unloaded.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel