Re: [PATCH v11 2/4] uacce: add uacce driver

[Dave Jiang <dave.jiang@intel.com>]

On 1/15/20 4:18 AM, zhangfei wrote:
> Hi, Greg
> 
> On 2020/1/14 下午10:59, Greg Kroah-Hartman wrote:
>> On Mon, Jan 13, 2020 at 11:34:55AM +0800, zhangfei wrote:
>>> Hi, Greg
>>>
>>> Thanks for the review.
>>>
>>> On 2020/1/12 上午3:40, Greg Kroah-Hartman wrote:
>>>> On Sat, Jan 11, 2020 at 10:48:37AM +0800, Zhangfei Gao wrote:
>>>>> +static int uacce_fops_open(struct inode *inode, struct file *filep)
>>>>> +{
>>>>> +    struct uacce_mm *uacce_mm = NULL;
>>>>> +    struct uacce_device *uacce;
>>>>> +    struct uacce_queue *q;
>>>>> +    int ret = 0;
>>>>> +
>>>>> +    uacce = xa_load(&uacce_xa, iminor(inode));
>>>>> +    if (!uacce)
>>>>> +        return -ENODEV;
>>>>> +
>>>>> +    if (!try_module_get(uacce->parent->driver->owner))
>>>>> +        return -ENODEV;
>>>> Why are you trying to grab the module reference of the parent device?
>>>> Why is that needed and what is that going to help with here?
>>>>
>>>> This shouldn't be needed as the module reference of the owner of the
>>>> fileops for this module is incremented, and the "parent" module depends
>>>> on this module, so how could it be unloaded without this code being
>>>> unloaded?
>>>>
>>>> Yes, if you build this code into the kernel and the "parent" driver 
>>>> is a
>>>> module, then you will not have a reference, but when you remove that
>>>> parent driver the device will be removed as it has to be unregistered
>>>> before that parent driver can be removed from the system, right?
>>>>
>>>> Or what am I missing here?
>>> The refcount here is preventing rmmod "parent" module after fd is 
>>> opened,
>>> since user driver has mmap kernel memory to user space, like mmio, 
>>> which may
>>> still in-use.
>>>
>>> With the refcount protection, rmmod "parent" module will fail until
>>> application free the fd.
>>> log like: rmmod: ERROR: Module hisi_zip is in use
>> But if the "parent" module is to be unloaded, it has to unregister the
>> "child" device and that will call the destructor in here and then you
>> will tear everything down and all should be good.
>>
>> There's no need to "forbid" a module from being unloaded, even if it is
>> being used.  Look at all networking drivers, they work that way, right?
> Thanks Greg for the kind suggestion.
> 
> I still have one uncertainty.
> Does uacce has to block process continue accessing the mmapped area when 
> remove "parent" module?
> Uacce can block device access the physical memory when parent module 
> call uacce_remove.
> But application is still running, and suppose it is not the kernel 
> driver's responsibility to call unmap.
> 
> I am looking for some examples in kernel,
> looks vfio does not block process continue accessing when 
> vfio_unregister_iommu_driver either.
> 
> In my test, application will keep waiting after rmmod parent, until 
> ctrl+c, when unmap is called.
> During the process, kernel does not report any error.
> 
> Do you have any advice?

Would it work to call unmap_mapping_range() on the char dev 
inode->i_mappings? I think you need to set the vma->fault function ptr 
for the vm_operations_struct in the original mmap(). After the mappings 
are unmapped, you can set a state variable to trigger the return of 
VM_FAULT_SIGBUS in the ->fault function when the user app accesses the 
mmap region again and triggers a page fault. The user app needs to be 
programmed to catch exceptions to deal with that.

> 
>>>>> +static void uacce_release(struct device *dev)
>>>>> +{
>>>>> +    struct uacce_device *uacce = to_uacce_device(dev);
>>>>> +
>>>>> +    kfree(uacce);
>>>>> +    uacce = NULL;
>>>> That line didn't do anything :)
>>> Yes, this is a mistake.
>>> It is up to caller to set to NULL to prevent release multi times.
>> Release function is called by the driver core which will not touch the
>> value again.
> Yes, I understand, it's my mistake. Will remove it.
> 
> Thanks