[PATCH v4] cxl: Force context lock during EEH flow

[PATCH v4] cxl: Force context lock during EEH flow

[Vaibhav Jain]

During an eeh event when the cxl card is fenced and card sysfs attr
perst_reloads_same_image is set following warning message is seen in the
kernel logs:

 [   60.622727] Adapter context unlocked with 0 active contexts
 [   60.622762] ------------[ cut here ]------------
 [   60.622771] WARNING: CPU: 12 PID: 627 at
 ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 [cxl]

Even though this warning is harmless, it clutters the kernel log
during an eeh event. This warning is triggered as the EEH callback
cxl_pci_error_detected doesn't obtain a context-lock before forcibly
detaching all active context and when context-lock is released during
call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
cxl_adapter_context_unlock is triggered.

To fix this warning, we acquire the adapter context-lock via
cxl_adapter_context_lock() in the eeh callback
cxl_pci_error_detected() once all the virtual AFU PHBs are notified
and their contexts detached. The context-lock is released in
cxl_pci_slot_reset() after the adapter is successfully reconfigured
and before we call slot_reset callback on slice attached device-drivers.

Cc: stable@vger.kernel.org
Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active context exists")
Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>
---
Change-Log:

v3..v4
- Moved the call to context-unlock from cxl_pci_resume to
cxl_pci_slot_reset to let cxlflash module activate its master context
during slot reset. (Fred)

v2..v3
- As discussed with Fred removed function
cxl_adapter_context_force_lock() which may potentially expose the code
to deadlock in the future.
-  Other details of changes in cxl_pci_error_detected() to fix an
earlier issue of eeh callbacks not being passed on to all slices, is
being reworked as a separate patch.

v2..v1
- Moved the call to cxl_adapter_context_force_lock() from
cxl_pci_error_detected() to cxl_remove. (Fred)
---
 drivers/misc/cxl/pci.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/drivers/misc/cxl/pci.c b/drivers/misc/cxl/pci.c
index b27ea98..dd9a128 100644
--- a/drivers/misc/cxl/pci.c
+++ b/drivers/misc/cxl/pci.c
@@ -1496,8 +1496,6 @@ static int cxl_configure_adapter(struct cxl *adapter, struct pci_dev *dev)
 	if ((rc = cxl_native_register_psl_err_irq(adapter)))
 		goto err;
 
-	/* Release the context lock as adapter is configured */
-	cxl_adapter_context_unlock(adapter);
 	return 0;
 
 err:
@@ -1596,6 +1594,9 @@ static struct cxl *cxl_pci_init_adapter(struct pci_dev *dev)
 	if ((rc = cxl_sysfs_adapter_add(adapter)))
 		goto err_put1;
 
+	/* Release the context lock as adapter is configured */
+	cxl_adapter_context_unlock(adapter);
+
 	return adapter;
 
 err_put1:
@@ -1895,6 +1896,13 @@ static pci_ers_result_t cxl_pci_error_detected(struct pci_dev *pdev,
 		cxl_ops->afu_deactivate_mode(afu, afu->current_mode);
 		pci_deconfigure_afu(afu);
 	}
+
+	/* should take the context lock here */
+	if (cxl_adapter_context_lock(adapter) != 0)
+		dev_warn(&adapter->dev,
+			 "Couldn't take context lock with %d active-contexts\n",
+			 atomic_read(&adapter->contexts_num));
+
 	cxl_deconfigure_adapter(adapter);
 
 	return result;
@@ -1913,6 +1921,13 @@ static pci_ers_result_t cxl_pci_slot_reset(struct pci_dev *pdev)
 	if (cxl_configure_adapter(adapter, pdev))
 		goto err;
 
+	/*
+	 * Unlock context activation for the adapter. Ideally this should be
+	 * done in cxl_pci_resume but cxlflash module tries to activate the
+	 * master context as part of slot_reset callback.
+	 */
+	cxl_adapter_context_unlock(adapter);
+
 	for (i = 0; i < adapter->slices; i++) {
 		afu = adapter->afu[i];
 
-- 
2.9.3

Re: [PATCH v4] cxl: Force context lock during EEH flow

[Frederic Barrat]

Le 05/04/2017 � 13:35, Vaibhav Jain a �crit :
> During an eeh event when the cxl card is fenced and card sysfs attr
> perst_reloads_same_image is set following warning message is seen in the
> kernel logs:
>
>  [   60.622727] Adapter context unlocked with 0 active contexts
>  [   60.622762] ------------[ cut here ]------------
>  [   60.622771] WARNING: CPU: 12 PID: 627 at
>  ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 [cxl]
>
> Even though this warning is harmless, it clutters the kernel log
> during an eeh event. This warning is triggered as the EEH callback
> cxl_pci_error_detected doesn't obtain a context-lock before forcibly
> detaching all active context and when context-lock is released during
> call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
> cxl_adapter_context_unlock is triggered.
>
> To fix this warning, we acquire the adapter context-lock via
> cxl_adapter_context_lock() in the eeh callback
> cxl_pci_error_detected() once all the virtual AFU PHBs are notified
> and their contexts detached. The context-lock is released in
> cxl_pci_slot_reset() after the adapter is successfully reconfigured
> and before we call slot_reset callback on slice attached device-drivers.
>
> Cc: stable@vger.kernel.org
> Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active context exists")
> Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>
> ---

Pending test result from cxl-flash:
Acked-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>



> Change-Log:
>
> v3..v4
> - Moved the call to context-unlock from cxl_pci_resume to
> cxl_pci_slot_reset to let cxlflash module activate its master context
> during slot reset. (Fred)
>
> v2..v3
> - As discussed with Fred removed function
> cxl_adapter_context_force_lock() which may potentially expose the code
> to deadlock in the future.
> -  Other details of changes in cxl_pci_error_detected() to fix an
> earlier issue of eeh callbacks not being passed on to all slices, is
> being reworked as a separate patch.
>
> v2..v1
> - Moved the call to cxl_adapter_context_force_lock() from
> cxl_pci_error_detected() to cxl_remove. (Fred)
> ---
>  drivers/misc/cxl/pci.c | 19 +++++++++++++++++--
>  1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/misc/cxl/pci.c b/drivers/misc/cxl/pci.c
> index b27ea98..dd9a128 100644
> --- a/drivers/misc/cxl/pci.c
> +++ b/drivers/misc/cxl/pci.c
> @@ -1496,8 +1496,6 @@ static int cxl_configure_adapter(struct cxl *adapter, struct pci_dev *dev)
>  	if ((rc = cxl_native_register_psl_err_irq(adapter)))
>  		goto err;
>
> -	/* Release the context lock as adapter is configured */
> -	cxl_adapter_context_unlock(adapter);
>  	return 0;
>
>  err:
> @@ -1596,6 +1594,9 @@ static struct cxl *cxl_pci_init_adapter(struct pci_dev *dev)
>  	if ((rc = cxl_sysfs_adapter_add(adapter)))
>  		goto err_put1;
>
> +	/* Release the context lock as adapter is configured */
> +	cxl_adapter_context_unlock(adapter);
> +
>  	return adapter;
>
>  err_put1:
> @@ -1895,6 +1896,13 @@ static pci_ers_result_t cxl_pci_error_detected(struct pci_dev *pdev,
>  		cxl_ops->afu_deactivate_mode(afu, afu->current_mode);
>  		pci_deconfigure_afu(afu);
>  	}
> +
> +	/* should take the context lock here */
> +	if (cxl_adapter_context_lock(adapter) != 0)
> +		dev_warn(&adapter->dev,
> +			 "Couldn't take context lock with %d active-contexts\n",
> +			 atomic_read(&adapter->contexts_num));
> +
>  	cxl_deconfigure_adapter(adapter);
>
>  	return result;
> @@ -1913,6 +1921,13 @@ static pci_ers_result_t cxl_pci_slot_reset(struct pci_dev *pdev)
>  	if (cxl_configure_adapter(adapter, pdev))
>  		goto err;
>
> +	/*
> +	 * Unlock context activation for the adapter. Ideally this should be
> +	 * done in cxl_pci_resume but cxlflash module tries to activate the
> +	 * master context as part of slot_reset callback.
> +	 */
> +	cxl_adapter_context_unlock(adapter);
> +
>  	for (i = 0; i < adapter->slices; i++) {
>  		afu = adapter->afu[i];
>

Re: [PATCH v4] cxl: Force context lock during EEH flow

[Frederic Barrat]

Le 05/04/2017 à 13:35, Vaibhav Jain a écrit :
> During an eeh event when the cxl card is fenced and card sysfs attr
> perst_reloads_same_image is set following warning message is seen in the
> kernel logs:
>
>  [   60.622727] Adapter context unlocked with 0 active contexts
>  [   60.622762] ------------[ cut here ]------------
>  [   60.622771] WARNING: CPU: 12 PID: 627 at
>  ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 [cxl]
>
> Even though this warning is harmless, it clutters the kernel log
> during an eeh event. This warning is triggered as the EEH callback
> cxl_pci_error_detected doesn't obtain a context-lock before forcibly
> detaching all active context and when context-lock is released during
> call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
> cxl_adapter_context_unlock is triggered.
>
> To fix this warning, we acquire the adapter context-lock via
> cxl_adapter_context_lock() in the eeh callback
> cxl_pci_error_detected() once all the virtual AFU PHBs are notified
> and their contexts detached. The context-lock is released in
> cxl_pci_slot_reset() after the adapter is successfully reconfigured
> and before we call slot_reset callback on slice attached device-drivers.
>
> Cc: stable@vger.kernel.org
> Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active context exists")
> Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>
> ---

Pending test result from cxl-flash:
Acked-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>



> Change-Log:
>
> v3..v4
> - Moved the call to context-unlock from cxl_pci_resume to
> cxl_pci_slot_reset to let cxlflash module activate its master context
> during slot reset. (Fred)
>
> v2..v3
> - As discussed with Fred removed function
> cxl_adapter_context_force_lock() which may potentially expose the code
> to deadlock in the future.
> -  Other details of changes in cxl_pci_error_detected() to fix an
> earlier issue of eeh callbacks not being passed on to all slices, is
> being reworked as a separate patch.
>
> v2..v1
> - Moved the call to cxl_adapter_context_force_lock() from
> cxl_pci_error_detected() to cxl_remove. (Fred)
> ---
>  drivers/misc/cxl/pci.c | 19 +++++++++++++++++--
>  1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/misc/cxl/pci.c b/drivers/misc/cxl/pci.c
> index b27ea98..dd9a128 100644
> --- a/drivers/misc/cxl/pci.c
> +++ b/drivers/misc/cxl/pci.c
> @@ -1496,8 +1496,6 @@ static int cxl_configure_adapter(struct cxl *adapter, struct pci_dev *dev)
>  	if ((rc = cxl_native_register_psl_err_irq(adapter)))
>  		goto err;
>
> -	/* Release the context lock as adapter is configured */
> -	cxl_adapter_context_unlock(adapter);
>  	return 0;
>
>  err:
> @@ -1596,6 +1594,9 @@ static struct cxl *cxl_pci_init_adapter(struct pci_dev *dev)
>  	if ((rc = cxl_sysfs_adapter_add(adapter)))
>  		goto err_put1;
>
> +	/* Release the context lock as adapter is configured */
> +	cxl_adapter_context_unlock(adapter);
> +
>  	return adapter;
>
>  err_put1:
> @@ -1895,6 +1896,13 @@ static pci_ers_result_t cxl_pci_error_detected(struct pci_dev *pdev,
>  		cxl_ops->afu_deactivate_mode(afu, afu->current_mode);
>  		pci_deconfigure_afu(afu);
>  	}
> +
> +	/* should take the context lock here */
> +	if (cxl_adapter_context_lock(adapter) != 0)
> +		dev_warn(&adapter->dev,
> +			 "Couldn't take context lock with %d active-contexts\n",
> +			 atomic_read(&adapter->contexts_num));
> +
>  	cxl_deconfigure_adapter(adapter);
>
>  	return result;
> @@ -1913,6 +1921,13 @@ static pci_ers_result_t cxl_pci_slot_reset(struct pci_dev *pdev)
>  	if (cxl_configure_adapter(adapter, pdev))
>  		goto err;
>
> +	/*
> +	 * Unlock context activation for the adapter. Ideally this should be
> +	 * done in cxl_pci_resume but cxlflash module tries to activate the
> +	 * master context as part of slot_reset callback.
> +	 */
> +	cxl_adapter_context_unlock(adapter);
> +
>  	for (i = 0; i < adapter->slices; i++) {
>  		afu = adapter->afu[i];
>

Re: [PATCH v4] cxl: Force context lock during EEH flow

[Michael Ellerman]

Frederic Barrat <fbarrat@linux.vnet.ibm.com> writes:

> Le 05/04/2017 à 13:35, Vaibhav Jain a écrit :
>> During an eeh event when the cxl card is fenced and card sysfs attr
>> perst_reloads_same_image is set following warning message is seen in the
>> kernel logs:
>>
>>  [   60.622727] Adapter context unlocked with 0 active contexts
>>  [   60.622762] ------------[ cut here ]------------
>>  [   60.622771] WARNING: CPU: 12 PID: 627 at
>>  ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 [cxl]
>>
>> Even though this warning is harmless, it clutters the kernel log
>> during an eeh event. This warning is triggered as the EEH callback
>> cxl_pci_error_detected doesn't obtain a context-lock before forcibly
>> detaching all active context and when context-lock is released during
>> call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
>> cxl_adapter_context_unlock is triggered.
>>
>> To fix this warning, we acquire the adapter context-lock via
>> cxl_adapter_context_lock() in the eeh callback
>> cxl_pci_error_detected() once all the virtual AFU PHBs are notified
>> and their contexts detached. The context-lock is released in
>> cxl_pci_slot_reset() after the adapter is successfully reconfigured
>> and before we call slot_reset callback on slice attached device-drivers.
>>
>> Cc: stable@vger.kernel.org
>> Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active context exists")
>> Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
>> Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>
>> ---
>
> Pending test result from cxl-flash:
> Acked-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>

Still pending ... ?

cheers

Re: [PATCH v4] cxl: Force context lock during EEH flow

[Michael Ellerman]

Frederic Barrat <fbarrat@linux.vnet.ibm.com> writes:

> Le 05/04/2017 =C3=A0 13:35, Vaibhav Jain a =C3=A9crit :
>> During an eeh event when the cxl card is fenced and card sysfs attr
>> perst_reloads_same_image is set following warning message is seen in the
>> kernel logs:
>>
>>  [   60.622727] Adapter context unlocked with 0 active contexts
>>  [   60.622762] ------------[ cut here ]------------
>>  [   60.622771] WARNING: CPU: 12 PID: 627 at
>>  ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 [cx=
l]
>>
>> Even though this warning is harmless, it clutters the kernel log
>> during an eeh event. This warning is triggered as the EEH callback
>> cxl_pci_error_detected doesn't obtain a context-lock before forcibly
>> detaching all active context and when context-lock is released during
>> call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
>> cxl_adapter_context_unlock is triggered.
>>
>> To fix this warning, we acquire the adapter context-lock via
>> cxl_adapter_context_lock() in the eeh callback
>> cxl_pci_error_detected() once all the virtual AFU PHBs are notified
>> and their contexts detached. The context-lock is released in
>> cxl_pci_slot_reset() after the adapter is successfully reconfigured
>> and before we call slot_reset callback on slice attached device-drivers.
>>
>> Cc: stable@vger.kernel.org
>> Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active context exi=
sts")
>> Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
>> Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>
>> ---
>
> Pending test result from cxl-flash:
> Acked-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>

Still pending ... ?

cheers

Re: [PATCH v4] cxl: Force context lock during EEH flow

[Frederic Barrat]

Le 11/04/2017 à 12:40, Michael Ellerman a écrit :
> Frederic Barrat <fbarrat@linux.vnet.ibm.com> writes:
>
>> Le 05/04/2017 à 13:35, Vaibhav Jain a écrit :
>>> During an eeh event when the cxl card is fenced and card sysfs attr
>>> perst_reloads_same_image is set following warning message is seen in the
>>> kernel logs:
>>>
>>>  [   60.622727] Adapter context unlocked with 0 active contexts
>>>  [   60.622762] ------------[ cut here ]------------
>>>  [   60.622771] WARNING: CPU: 12 PID: 627 at
>>>  ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 [cxl]
>>>
>>> Even though this warning is harmless, it clutters the kernel log
>>> during an eeh event. This warning is triggered as the EEH callback
>>> cxl_pci_error_detected doesn't obtain a context-lock before forcibly
>>> detaching all active context and when context-lock is released during
>>> call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
>>> cxl_adapter_context_unlock is triggered.
>>>
>>> To fix this warning, we acquire the adapter context-lock via
>>> cxl_adapter_context_lock() in the eeh callback
>>> cxl_pci_error_detected() once all the virtual AFU PHBs are notified
>>> and their contexts detached. The context-lock is released in
>>> cxl_pci_slot_reset() after the adapter is successfully reconfigured
>>> and before we call slot_reset callback on slice attached device-drivers.
>>>
>>> Cc: stable@vger.kernel.org
>>> Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active context exists")
>>> Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
>>> Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>
>>> ---
>>
>> Pending test result from cxl-flash:
>> Acked-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
>
> Still pending ... ?

Yes, still waiting. It was mentioned in a call with the cxlflash team 
yesterday.

   Fred



>
> cheers
>

Re: [PATCH v4] cxl: Force context lock during EEH flow

[Matthew R. Ochs]

> On Apr 11, 2017, at 6:13 AM, Frederic Barrat <fbarrat@linux.vnet.ibm.com> wrote:
> Le 11/04/2017 à 12:40, Michael Ellerman a écrit :
>> Frederic Barrat <fbarrat@linux.vnet.ibm.com> writes:
>> 
>>> Le 05/04/2017 à 13:35, Vaibhav Jain a écrit :
>>>> During an eeh event when the cxl card is fenced and card sysfs attr
>>>> perst_reloads_same_image is set following warning message is seen in the
>>>> kernel logs:
>>>> 
>>>> [   60.622727] Adapter context unlocked with 0 active contexts
>>>> [   60.622762] ------------[ cut here ]------------
>>>> [   60.622771] WARNING: CPU: 12 PID: 627 at
>>>> ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 [cxl]
>>>> 
>>>> Even though this warning is harmless, it clutters the kernel log
>>>> during an eeh event. This warning is triggered as the EEH callback
>>>> cxl_pci_error_detected doesn't obtain a context-lock before forcibly
>>>> detaching all active context and when context-lock is released during
>>>> call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
>>>> cxl_adapter_context_unlock is triggered.
>>>> 
>>>> To fix this warning, we acquire the adapter context-lock via
>>>> cxl_adapter_context_lock() in the eeh callback
>>>> cxl_pci_error_detected() once all the virtual AFU PHBs are notified
>>>> and their contexts detached. The context-lock is released in
>>>> cxl_pci_slot_reset() after the adapter is successfully reconfigured
>>>> and before we call slot_reset callback on slice attached device-drivers.
>>>> 
>>>> Cc: stable@vger.kernel.org
>>>> Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active context exists")
>>>> Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
>>>> Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>
>>>> ---
>>> 
>>> Pending test result from cxl-flash:
>>> Acked-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
>> 
>> Still pending ... ?
> 
> Yes, still waiting. It was mentioned in a call with the cxlflash team yesterday.

I have it queued up for us to do on Monday.

-matt

Re: [PATCH v4] cxl: Force context lock during EEH flow

[Matthew R. Ochs]

> On Apr 11, 2017, at 6:13 AM, Frederic Barrat =
<fbarrat@linux.vnet.ibm.com> wrote:
> Le 11/04/2017 =C3=A0 12:40, Michael Ellerman a =C3=A9crit :
>> Frederic Barrat <fbarrat@linux.vnet.ibm.com> writes:
>>=20
>>> Le 05/04/2017 =C3=A0 13:35, Vaibhav Jain a =C3=A9crit :
>>>> During an eeh event when the cxl card is fenced and card sysfs attr
>>>> perst_reloads_same_image is set following warning message is seen =
in the
>>>> kernel logs:
>>>>=20
>>>> [   60.622727] Adapter context unlocked with 0 active contexts
>>>> [   60.622762] ------------[ cut here ]------------
>>>> [   60.622771] WARNING: CPU: 12 PID: 627 at
>>>> ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 =
[cxl]
>>>>=20
>>>> Even though this warning is harmless, it clutters the kernel log
>>>> during an eeh event. This warning is triggered as the EEH callback
>>>> cxl_pci_error_detected doesn't obtain a context-lock before =
forcibly
>>>> detaching all active context and when context-lock is released =
during
>>>> call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
>>>> cxl_adapter_context_unlock is triggered.
>>>>=20
>>>> To fix this warning, we acquire the adapter context-lock via
>>>> cxl_adapter_context_lock() in the eeh callback
>>>> cxl_pci_error_detected() once all the virtual AFU PHBs are notified
>>>> and their contexts detached. The context-lock is released in
>>>> cxl_pci_slot_reset() after the adapter is successfully reconfigured
>>>> and before we call slot_reset callback on slice attached =
device-drivers.
>>>>=20
>>>> Cc: stable@vger.kernel.org
>>>> Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active =
context exists")
>>>> Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
>>>> Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>
>>>> ---
>>>=20
>>> Pending test result from cxl-flash:
>>> Acked-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
>>=20
>> Still pending ... ?
>=20
> Yes, still waiting. It was mentioned in a call with the cxlflash team =
yesterday.

I have it queued up for us to do on Monday.

-matt

Re: [PATCH v4] cxl: Force context lock during EEH flow

[Matthew R. Ochs]

> On Apr 5, 2017, at 6:35 AM, Vaibhav Jain <vaibhav@linux.vnet.ibm.com> wrote:
> 
> During an eeh event when the cxl card is fenced and card sysfs attr
> perst_reloads_same_image is set following warning message is seen in the
> kernel logs:
> 
> [   60.622727] Adapter context unlocked with 0 active contexts
> [   60.622762] ------------[ cut here ]------------
> [   60.622771] WARNING: CPU: 12 PID: 627 at
> ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 [cxl]
> 
> Even though this warning is harmless, it clutters the kernel log
> during an eeh event. This warning is triggered as the EEH callback
> cxl_pci_error_detected doesn't obtain a context-lock before forcibly
> detaching all active context and when context-lock is released during
> call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
> cxl_adapter_context_unlock is triggered.
> 
> To fix this warning, we acquire the adapter context-lock via
> cxl_adapter_context_lock() in the eeh callback
> cxl_pci_error_detected() once all the virtual AFU PHBs are notified
> and their contexts detached. The context-lock is released in
> cxl_pci_slot_reset() after the adapter is successfully reconfigured
> and before we call slot_reset callback on slice attached device-drivers.
> 
> Cc: stable@vger.kernel.org
> Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active context exists")
> Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>

Looks good. Uma should be following up later today after testing completes.

Reviewed-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>

Re: [PATCH v4] cxl: Force context lock during EEH flow

[Matthew R. Ochs]

> On Apr 5, 2017, at 6:35 AM, Vaibhav Jain <vaibhav@linux.vnet.ibm.com> =
wrote:
>=20
> During an eeh event when the cxl card is fenced and card sysfs attr
> perst_reloads_same_image is set following warning message is seen in =
the
> kernel logs:
>=20
> [   60.622727] Adapter context unlocked with 0 active contexts
> [   60.622762] ------------[ cut here ]------------
> [   60.622771] WARNING: CPU: 12 PID: 627 at
> ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 =
[cxl]
>=20
> Even though this warning is harmless, it clutters the kernel log
> during an eeh event. This warning is triggered as the EEH callback
> cxl_pci_error_detected doesn't obtain a context-lock before forcibly
> detaching all active context and when context-lock is released during
> call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
> cxl_adapter_context_unlock is triggered.
>=20
> To fix this warning, we acquire the adapter context-lock via
> cxl_adapter_context_lock() in the eeh callback
> cxl_pci_error_detected() once all the virtual AFU PHBs are notified
> and their contexts detached. The context-lock is released in
> cxl_pci_slot_reset() after the adapter is successfully reconfigured
> and before we call slot_reset callback on slice attached =
device-drivers.
>=20
> Cc: stable@vger.kernel.org
> Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active context =
exists")
> Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>

Looks good. Uma should be following up later today after testing =
completes.

Reviewed-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>

Re: [PATCH v4] cxl: Force context lock during EEH flow

[Uma Krishnan]

On 4/5/2017 6:35 AM, Vaibhav Jain wrote:
> During an eeh event when the cxl card is fenced and card sysfs attr
> perst_reloads_same_image is set following warning message is seen in the
> kernel logs:
>
>  [   60.622727] Adapter context unlocked with 0 active contexts
>  [   60.622762] ------------[ cut here ]------------
>  [   60.622771] WARNING: CPU: 12 PID: 627 at
>  ../drivers/misc/cxl/main.c:325 cxl_adapter_context_unlock+0x60/0x80 [cxl]
>
> Even though this warning is harmless, it clutters the kernel log
> during an eeh event. This warning is triggered as the EEH callback
> cxl_pci_error_detected doesn't obtain a context-lock before forcibly
> detaching all active context and when context-lock is released during
> call to cxl_configure_adapter from cxl_pci_slot_reset, a warning in
> cxl_adapter_context_unlock is triggered.
>
> To fix this warning, we acquire the adapter context-lock via
> cxl_adapter_context_lock() in the eeh callback
> cxl_pci_error_detected() once all the virtual AFU PHBs are notified
> and their contexts detached. The context-lock is released in
> cxl_pci_slot_reset() after the adapter is successfully reconfigured
> and before we call slot_reset callback on slice attached device-drivers.
>
> Cc: stable@vger.kernel.org
> Fixes: 70b565bbdb91("cxl: Prevent adapter reset if an active context exists")
> Reported-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>
> ---
> Change-Log:
>
> v3..v4
> - Moved the call to context-unlock from cxl_pci_resume to
> cxl_pci_slot_reset to let cxlflash module activate its master context
> during slot reset. (Fred)
>
> v2..v3
> - As discussed with Fred removed function
> cxl_adapter_context_force_lock() which may potentially expose the code
> to deadlock in the future.
> -  Other details of changes in cxl_pci_error_detected() to fix an
> earlier issue of eeh callbacks not being passed on to all slices, is
> being reworked as a separate patch.
>
> v2..v1
> - Moved the call to cxl_adapter_context_force_lock() from
> cxl_pci_error_detected() to cxl_remove. (Fred)
> ---
>  drivers/misc/cxl/pci.c | 19 +++++++++++++++++--
>  1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/misc/cxl/pci.c b/drivers/misc/cxl/pci.c
> index b27ea98..dd9a128 100644
> --- a/drivers/misc/cxl/pci.c
> +++ b/drivers/misc/cxl/pci.c
> @@ -1496,8 +1496,6 @@ static int cxl_configure_adapter(struct cxl *adapter, struct pci_dev *dev)
>  	if ((rc = cxl_native_register_psl_err_irq(adapter)))
>  		goto err;
>
> -	/* Release the context lock as adapter is configured */
> -	cxl_adapter_context_unlock(adapter);
>  	return 0;
>
>  err:
> @@ -1596,6 +1594,9 @@ static struct cxl *cxl_pci_init_adapter(struct pci_dev *dev)
>  	if ((rc = cxl_sysfs_adapter_add(adapter)))
>  		goto err_put1;
>
> +	/* Release the context lock as adapter is configured */
> +	cxl_adapter_context_unlock(adapter);
> +
>  	return adapter;
>
>  err_put1:
> @@ -1895,6 +1896,13 @@ static pci_ers_result_t cxl_pci_error_detected(struct pci_dev *pdev,
>  		cxl_ops->afu_deactivate_mode(afu, afu->current_mode);
>  		pci_deconfigure_afu(afu);
>  	}
> +
> +	/* should take the context lock here */
> +	if (cxl_adapter_context_lock(adapter) != 0)
> +		dev_warn(&adapter->dev,
> +			 "Couldn't take context lock with %d active-contexts\n",
> +			 atomic_read(&adapter->contexts_num));
> +
>  	cxl_deconfigure_adapter(adapter);
>
>  	return result;
> @@ -1913,6 +1921,13 @@ static pci_ers_result_t cxl_pci_slot_reset(struct pci_dev *pdev)
>  	if (cxl_configure_adapter(adapter, pdev))
>  		goto err;
>
> +	/*
> +	 * Unlock context activation for the adapter. Ideally this should be
> +	 * done in cxl_pci_resume but cxlflash module tries to activate the
> +	 * master context as part of slot_reset callback.
> +	 */
> +	cxl_adapter_context_unlock(adapter);
> +
>  	for (i = 0; i < adapter->slices; i++) {
>  		afu = adapter->afu[i];
>

Looks good. FV regressions and EEH recoveries (in parallel to I/O) were 
successful.

Tested-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>