[5/5] switchtec: MRPC DMA mode implementation
diff mbox series

Message ID 1544433144-7563-6-git-send-email-wesley.sheng@microchip.com
State In Next
Commit 9e49d901514632de74bab8dfce69d637c350fd98
Headers show
Series
  • Switchtec MRPC DMA mode support
Related show

Commit Message

Wesley Sheng Dec. 10, 2018, 9:12 a.m. UTC
MRPC normal mode requires the host to read the MRPC command status and
output data from BAR. This results in high latency responses from the
Memory Read TLP and potential Completion Timeout (CTO).

MRPC DMA mode implementation includes:
Macro definitions for registers and data structures corresponding to
MRPC DMA mode.

Add module parameter use_dma_mrpc to select between MRPC DMA mode and
MRPC normal mode.

Add MRPC mode functionality to:
* Retrieve MRPC DMA mode version
* Allocate DMA buffer, ISR registration, and enable DMA function during
  initialization
* Check MRPC execution status and collect execution results from DMA buffer
* Release DMA buffer and disable DMA function when unloading module

MRPC DMA mode is a new feature of firmware and the driver will fall back
to MRPC normal mode if there is no support in the legacy firmware.

Include <linux/io-64-nonatomic-lo-hi.h> so that readq/writeq is replaced
by two readl/writel on systems that do not support it.

Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
---
 drivers/pci/switch/switchtec.c | 108 +++++++++++++++++++++++++++++++++++++----
 include/linux/switchtec.h      |  16 ++++++
 2 files changed, 114 insertions(+), 10 deletions(-)

Comments

Bjorn Helgaas Dec. 12, 2018, 10:52 p.m. UTC | #1
On Mon, Dec 10, 2018 at 05:12:24PM +0800, Wesley Sheng wrote:
> MRPC normal mode requires the host to read the MRPC command status and
> output data from BAR. This results in high latency responses from the
> Memory Read TLP and potential Completion Timeout (CTO).
> 
> MRPC DMA mode implementation includes:
> Macro definitions for registers and data structures corresponding to
> MRPC DMA mode.
> 
> Add module parameter use_dma_mrpc to select between MRPC DMA mode and
> MRPC normal mode.
> 
> Add MRPC mode functionality to:
> * Retrieve MRPC DMA mode version
> * Allocate DMA buffer, ISR registration, and enable DMA function during
>   initialization
> * Check MRPC execution status and collect execution results from DMA buffer
> * Release DMA buffer and disable DMA function when unloading module
> 
> MRPC DMA mode is a new feature of firmware and the driver will fall back
> to MRPC normal mode if there is no support in the legacy firmware.
> 
> Include <linux/io-64-nonatomic-lo-hi.h> so that readq/writeq is replaced
> by two readl/writel on systems that do not support it.
> 
> Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
> ---
>  drivers/pci/switch/switchtec.c | 108 +++++++++++++++++++++++++++++++++++++----
>  include/linux/switchtec.h      |  16 ++++++
>  2 files changed, 114 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c
> index 0b8862b..6b726cb 100644
> --- a/drivers/pci/switch/switchtec.c
> +++ b/drivers/pci/switch/switchtec.c
> @@ -13,7 +13,7 @@
>  #include <linux/uaccess.h>
>  #include <linux/poll.h>
>  #include <linux/wait.h>
> -
> +#include <linux/io-64-nonatomic-lo-hi.h>
>  #include <linux/nospec.h>
>  
>  MODULE_DESCRIPTION("Microsemi Switchtec(tm) PCIe Management Driver");
> @@ -25,6 +25,11 @@ static int max_devices = 16;
>  module_param(max_devices, int, 0644);
>  MODULE_PARM_DESC(max_devices, "max number of switchtec device instances");
>  
> +static bool use_dma_mrpc = 1;
> +module_param(use_dma_mrpc, bool, 0644);
> +MODULE_PARM_DESC(use_dma_mrpc,
> +		 "Enable the use of the DMA MRPC feature");

What's the purpose of the module parameter?  Can't you automatically
figure out whether the firmware supports DMA mode?

Module parameters make life difficult for users and lead to code
that's rarely used and poorly tested, so I think we should avoid them
whenever we can.

If you *can't* automatically figure out when to use DMA, please make
it clear in the changelog that the "use_dma_mrpc" parameter is
required with legacy firmware.  And in that case, it seems like it
should be named "disable_dma" or similar, since it defaults to being
enabled.

Bjorn
Logan Gunthorpe Dec. 12, 2018, 10:58 p.m. UTC | #2
On 2018-12-12 3:52 p.m., Bjorn Helgaas wrote:
> On Mon, Dec 10, 2018 at 05:12:24PM +0800, Wesley Sheng wrote:
>> MRPC normal mode requires the host to read the MRPC command status and
>> output data from BAR. This results in high latency responses from the
>> Memory Read TLP and potential Completion Timeout (CTO).
>>
>> MRPC DMA mode implementation includes:
>> Macro definitions for registers and data structures corresponding to
>> MRPC DMA mode.
>>
>> Add module parameter use_dma_mrpc to select between MRPC DMA mode and
>> MRPC normal mode.
>>
>> Add MRPC mode functionality to:
>> * Retrieve MRPC DMA mode version
>> * Allocate DMA buffer, ISR registration, and enable DMA function during
>>   initialization
>> * Check MRPC execution status and collect execution results from DMA buffer
>> * Release DMA buffer and disable DMA function when unloading module
>>
>> MRPC DMA mode is a new feature of firmware and the driver will fall back
>> to MRPC normal mode if there is no support in the legacy firmware.
>>
>> Include <linux/io-64-nonatomic-lo-hi.h> so that readq/writeq is replaced
>> by two readl/writel on systems that do not support it.
>>
>> Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
>> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
>> ---
>>  drivers/pci/switch/switchtec.c | 108 +++++++++++++++++++++++++++++++++++++----
>>  include/linux/switchtec.h      |  16 ++++++
>>  2 files changed, 114 insertions(+), 10 deletions(-)
>>
>> diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c
>> index 0b8862b..6b726cb 100644
>> --- a/drivers/pci/switch/switchtec.c
>> +++ b/drivers/pci/switch/switchtec.c
>> @@ -13,7 +13,7 @@
>>  #include <linux/uaccess.h>
>>  #include <linux/poll.h>
>>  #include <linux/wait.h>
>> -
>> +#include <linux/io-64-nonatomic-lo-hi.h>
>>  #include <linux/nospec.h>
>>  
>>  MODULE_DESCRIPTION("Microsemi Switchtec(tm) PCIe Management Driver");
>> @@ -25,6 +25,11 @@ static int max_devices = 16;
>>  module_param(max_devices, int, 0644);
>>  MODULE_PARM_DESC(max_devices, "max number of switchtec device instances");
>>  
>> +static bool use_dma_mrpc = 1;
>> +module_param(use_dma_mrpc, bool, 0644);
>> +MODULE_PARM_DESC(use_dma_mrpc,
>> +		 "Enable the use of the DMA MRPC feature");
> 
> What's the purpose of the module parameter?  Can't you automatically
> figure out whether the firmware supports DMA mode?
> 
> Module parameters make life difficult for users and lead to code
> that's rarely used and poorly tested, so I think we should avoid them
> whenever we can.
> 
> If you *can't* automatically figure out when to use DMA, please make
> it clear in the changelog that the "use_dma_mrpc" parameter is
> required with legacy firmware.  And in that case, it seems like it
> should be named "disable_dma" or similar, since it defaults to being
> enabled.

The code should already automatically determine whether the firmware
supports DMA or not. This parameter is just to aid debugging/testing so
we can run the module without DMA even if the firmware indicates it has
support.

It's not that critical so I'm sure we can remove it at this point if you
don't think that's a good enough justification.

Logan
Bjorn Helgaas Dec. 13, 2018, 3:16 p.m. UTC | #3
On Wed, Dec 12, 2018 at 03:58:22PM -0700, Logan Gunthorpe wrote:
> On 2018-12-12 3:52 p.m., Bjorn Helgaas wrote:
> > On Mon, Dec 10, 2018 at 05:12:24PM +0800, Wesley Sheng wrote:
> >> MRPC normal mode requires the host to read the MRPC command status and
> >> output data from BAR. This results in high latency responses from the
> >> Memory Read TLP and potential Completion Timeout (CTO).
> >>
> >> MRPC DMA mode implementation includes:
> >> Macro definitions for registers and data structures corresponding to
> >> MRPC DMA mode.
> >>
> >> Add module parameter use_dma_mrpc to select between MRPC DMA mode and
> >> MRPC normal mode.
> >>
> >> Add MRPC mode functionality to:
> >> * Retrieve MRPC DMA mode version
> >> * Allocate DMA buffer, ISR registration, and enable DMA function during
> >>   initialization
> >> * Check MRPC execution status and collect execution results from DMA buffer
> >> * Release DMA buffer and disable DMA function when unloading module
> >>
> >> MRPC DMA mode is a new feature of firmware and the driver will fall back
> >> to MRPC normal mode if there is no support in the legacy firmware.
> >>
> >> Include <linux/io-64-nonatomic-lo-hi.h> so that readq/writeq is replaced
> >> by two readl/writel on systems that do not support it.
> >>
> >> Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
> >> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
> >> ---
> >>  drivers/pci/switch/switchtec.c | 108 +++++++++++++++++++++++++++++++++++++----
> >>  include/linux/switchtec.h      |  16 ++++++
> >>  2 files changed, 114 insertions(+), 10 deletions(-)
> >>
> >> diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c
> >> index 0b8862b..6b726cb 100644
> >> --- a/drivers/pci/switch/switchtec.c
> >> +++ b/drivers/pci/switch/switchtec.c
> >> @@ -13,7 +13,7 @@
> >>  #include <linux/uaccess.h>
> >>  #include <linux/poll.h>
> >>  #include <linux/wait.h>
> >> -
> >> +#include <linux/io-64-nonatomic-lo-hi.h>
> >>  #include <linux/nospec.h>
> >>  
> >>  MODULE_DESCRIPTION("Microsemi Switchtec(tm) PCIe Management Driver");
> >> @@ -25,6 +25,11 @@ static int max_devices = 16;
> >>  module_param(max_devices, int, 0644);
> >>  MODULE_PARM_DESC(max_devices, "max number of switchtec device instances");
> >>  
> >> +static bool use_dma_mrpc = 1;
> >> +module_param(use_dma_mrpc, bool, 0644);
> >> +MODULE_PARM_DESC(use_dma_mrpc,
> >> +		 "Enable the use of the DMA MRPC feature");
> > 
> > What's the purpose of the module parameter?  Can't you automatically
> > figure out whether the firmware supports DMA mode?
> > 
> > Module parameters make life difficult for users and lead to code
> > that's rarely used and poorly tested, so I think we should avoid them
> > whenever we can.
> > 
> > If you *can't* automatically figure out when to use DMA, please make
> > it clear in the changelog that the "use_dma_mrpc" parameter is
> > required with legacy firmware.  And in that case, it seems like it
> > should be named "disable_dma" or similar, since it defaults to being
> > enabled.
> 
> The code should already automatically determine whether the firmware
> supports DMA or not. This parameter is just to aid debugging/testing so
> we can run the module without DMA even if the firmware indicates it has
> support.
> 
> It's not that critical so I'm sure we can remove it at this point if you
> don't think that's a good enough justification.

You and Kurt are the maintainers of this, so it's up to you.  I
personally probably would not add a parameter that's only for
debugging, but at the end of the day I only want to double-check
that this is all as you intend.

I reworked the changelog as follows.


commit a5fa833d8008b9f8e211e2fbc7bc1f45f6457133
Author: Wesley Sheng <wesley.sheng@microchip.com>
Date:   Mon Dec 10 17:12:24 2018 +0800

    switchtec: Add MRPC DMA mode support
    
    MRPC normal mode requires the host to read the MRPC command status and
    output data from BAR.  This results in high latency responses from the
    Memory Read TLP and potential Completion Timeout (CTO).
    
    Add support for MRPC DMA mode, including related macro definitions and data
    structures and code to:
    
      * Retrieve MRPC DMA mode version from adapter firmware
      * Allocate DMA buffer, register ISR, and enable DMA during init
      * Check MRPC execution status and collect execution results from DMA buffer
      * Release DMA buffer and disable DMA function when unloading module
    
    MRPC DMA mode is a new feature of firmware, and the driver will fall back
    to MRPC normal mode if there is no support in the legacy firmware.
    
    Add a module parameter, "use_dma_mrpc", to select between MRPC DMA mode and
    MRPC normal mode.  Since the driver automatically detects DMA support in
    the firmware, this parameter is just for debugging and testing.
    
    Include <linux/io-64-nonatomic-lo-hi.h> so that readq/writeq is replaced by
    two readl/writel on systems that do not support it.
    
    Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
    [bhelgaas: changelog]
    Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
    Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Bjorn Helgaas Dec. 13, 2018, 3:17 p.m. UTC | #4
On Mon, Dec 10, 2018 at 05:12:24PM +0800, Wesley Sheng wrote:
> MRPC normal mode requires the host to read the MRPC command status and
> output data from BAR. This results in high latency responses from the
> Memory Read TLP and potential Completion Timeout (CTO).
> 
> MRPC DMA mode implementation includes:
> Macro definitions for registers and data structures corresponding to
> MRPC DMA mode.
> 
> Add module parameter use_dma_mrpc to select between MRPC DMA mode and
> MRPC normal mode.
> 
> Add MRPC mode functionality to:
> * Retrieve MRPC DMA mode version
> * Allocate DMA buffer, ISR registration, and enable DMA function during
>   initialization
> * Check MRPC execution status and collect execution results from DMA buffer
> * Release DMA buffer and disable DMA function when unloading module
> 
> MRPC DMA mode is a new feature of firmware and the driver will fall back
> to MRPC normal mode if there is no support in the legacy firmware.
> 
> Include <linux/io-64-nonatomic-lo-hi.h> so that readq/writeq is replaced
> by two readl/writel on systems that do not support it.
> 
> Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>

>  static void init_pff(struct switchtec_dev *stdev)
> @@ -1294,6 +1367,19 @@ static int switchtec_init_pci(struct switchtec_dev *stdev,
>  
>  	pci_set_drvdata(pdev, stdev);
>  
> +	if (!use_dma_mrpc)
> +		return 0;
> +
> +	if (!(ioread32(&stdev->mmio_mrpc->dma_ver) ? true : false))
> +		return 0;

This is ... harder to decode than necessary.  It's obvious that the
intent is to return if the adapter firmware doesn't support DMA, but I
lost interest before I could verify that it works as intended.

Bjorn
Logan Gunthorpe Dec. 13, 2018, 4:46 p.m. UTC | #5
On 2018-12-13 8:17 a.m., Bjorn Helgaas wrote:
>>  static void init_pff(struct switchtec_dev *stdev)
>> @@ -1294,6 +1367,19 @@ static int switchtec_init_pci(struct switchtec_dev *stdev,
>>  
>>  	pci_set_drvdata(pdev, stdev);
>>  
>> +	if (!use_dma_mrpc)
>> +		return 0;
>> +
>> +	if (!(ioread32(&stdev->mmio_mrpc->dma_ver) ? true : false))
>> +		return 0;
> 
> This is ... harder to decode than necessary.  It's obvious that the
> intent is to return if the adapter firmware doesn't support DMA, but I
> lost interest before I could verify that it works as intended.

Oh, ick, yes. I must have missed that in my review.

@Wesley, that would definitely be something worth cleaning up. That
ternary operator doesn't make any sense.

Thanks, Bjorn, for the review and picking up the series.

Logan

Patch
diff mbox series

diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c
index 0b8862b..6b726cb 100644
--- a/drivers/pci/switch/switchtec.c
+++ b/drivers/pci/switch/switchtec.c
@@ -13,7 +13,7 @@ 
 #include <linux/uaccess.h>
 #include <linux/poll.h>
 #include <linux/wait.h>
-
+#include <linux/io-64-nonatomic-lo-hi.h>
 #include <linux/nospec.h>
 
 MODULE_DESCRIPTION("Microsemi Switchtec(tm) PCIe Management Driver");
@@ -25,6 +25,11 @@  static int max_devices = 16;
 module_param(max_devices, int, 0644);
 MODULE_PARM_DESC(max_devices, "max number of switchtec device instances");
 
+static bool use_dma_mrpc = 1;
+module_param(use_dma_mrpc, bool, 0644);
+MODULE_PARM_DESC(use_dma_mrpc,
+		 "Enable the use of the DMA MRPC feature");
+
 static dev_t switchtec_devt;
 static DEFINE_IDA(switchtec_minor_ida);
 
@@ -141,6 +146,11 @@  static void mrpc_cmd_submit(struct switchtec_dev *stdev)
 	stuser = list_entry(stdev->mrpc_queue.next, struct switchtec_user,
 			    list);
 
+	if (stdev->dma_mrpc) {
+		stdev->dma_mrpc->status = SWITCHTEC_MRPC_STATUS_INPROGRESS;
+		memset(stdev->dma_mrpc->data, 0xFF, SWITCHTEC_MRPC_PAYLOAD_SIZE);
+	}
+
 	stuser_set_state(stuser, MRPC_RUNNING);
 	stdev->mrpc_busy = 1;
 	memcpy_toio(&stdev->mmio_mrpc->input_data,
@@ -180,7 +190,11 @@  static void mrpc_complete_cmd(struct switchtec_dev *stdev)
 	stuser = list_entry(stdev->mrpc_queue.next, struct switchtec_user,
 			    list);
 
-	stuser->status = ioread32(&stdev->mmio_mrpc->status);
+	if (stdev->dma_mrpc)
+		stuser->status = stdev->dma_mrpc->status;
+	else
+		stuser->status = ioread32(&stdev->mmio_mrpc->status);
+
 	if (stuser->status == SWITCHTEC_MRPC_STATUS_INPROGRESS)
 		return;
 
@@ -190,13 +204,19 @@  static void mrpc_complete_cmd(struct switchtec_dev *stdev)
 	if (stuser->status != SWITCHTEC_MRPC_STATUS_DONE)
 		goto out;
 
-	stuser->return_code = ioread32(&stdev->mmio_mrpc->ret_value);
+	if (stdev->dma_mrpc)
+		stuser->return_code = stdev->dma_mrpc->rtn_code;
+	else
+		stuser->return_code = ioread32(&stdev->mmio_mrpc->ret_value);
 	if (stuser->return_code != 0)
 		goto out;
 
-	memcpy_fromio(stuser->data, &stdev->mmio_mrpc->output_data,
-		      stuser->read_len);
-
+	if (stdev->dma_mrpc)
+		memcpy(stuser->data, &stdev->dma_mrpc->data,
+			      stuser->read_len);
+	else
+		memcpy_fromio(stuser->data, &stdev->mmio_mrpc->output_data,
+			      stuser->read_len);
 out:
 	complete_all(&stuser->comp);
 	list_del_init(&stuser->list);
@@ -231,7 +251,10 @@  static void mrpc_timeout_work(struct work_struct *work)
 
 	mutex_lock(&stdev->mrpc_mutex);
 
-	status = ioread32(&stdev->mmio_mrpc->status);
+	if (stdev->dma_mrpc)
+		status = stdev->dma_mrpc->status;
+	else
+		status = ioread32(&stdev->mmio_mrpc->status);
 	if (status == SWITCHTEC_MRPC_STATUS_INPROGRESS) {
 		schedule_delayed_work(&stdev->mrpc_timeout,
 				      msecs_to_jiffies(500));
@@ -239,7 +262,6 @@  static void mrpc_timeout_work(struct work_struct *work)
 	}
 
 	mrpc_complete_cmd(stdev);
-
 out:
 	mutex_unlock(&stdev->mrpc_mutex);
 }
@@ -1030,10 +1052,24 @@  static void enable_link_state_events(struct switchtec_dev *stdev)
 	}
 }
 
+static void enable_dma_mrpc(struct switchtec_dev *stdev)
+{
+	writeq(stdev->dma_mrpc_dma_addr, &stdev->mmio_mrpc->dma_addr);
+	flush_wc_buf(stdev);
+	iowrite32(SWITCHTEC_DMA_MRPC_EN, &stdev->mmio_mrpc->dma_en);
+}
+
 static void stdev_release(struct device *dev)
 {
 	struct switchtec_dev *stdev = to_stdev(dev);
 
+	if (stdev->dma_mrpc) {
+		iowrite32(0, &stdev->mmio_mrpc->dma_en);
+		flush_wc_buf(stdev);
+		writeq(0, &stdev->mmio_mrpc->dma_addr);
+		dma_free_coherent(&stdev->pdev->dev, sizeof(*stdev->dma_mrpc),
+				stdev->dma_mrpc, stdev->dma_mrpc_dma_addr);
+	}
 	kfree(stdev);
 }
 
@@ -1189,10 +1225,27 @@  static irqreturn_t switchtec_event_isr(int irq, void *dev)
 	return ret;
 }
 
+
+static irqreturn_t switchtec_dma_mrpc_isr(int irq, void *dev)
+{
+	struct switchtec_dev *stdev = dev;
+	irqreturn_t ret = IRQ_NONE;
+
+	iowrite32(SWITCHTEC_EVENT_CLEAR |
+		  SWITCHTEC_EVENT_EN_IRQ,
+		  &stdev->mmio_part_cfg->mrpc_comp_hdr);
+	schedule_work(&stdev->mrpc_work);
+
+	ret = IRQ_HANDLED;
+	return ret;
+}
+
 static int switchtec_init_isr(struct switchtec_dev *stdev)
 {
 	int nvecs;
 	int event_irq;
+	int dma_mrpc_irq;
+	int rc;
 
 	nvecs = pci_alloc_irq_vectors(stdev->pdev, 1, 4,
 				      PCI_IRQ_MSIX | PCI_IRQ_MSI);
@@ -1207,9 +1260,29 @@  static int switchtec_init_isr(struct switchtec_dev *stdev)
 	if (event_irq < 0)
 		return event_irq;
 
-	return devm_request_irq(&stdev->pdev->dev, event_irq,
+	rc = devm_request_irq(&stdev->pdev->dev, event_irq,
 				switchtec_event_isr, 0,
 				KBUILD_MODNAME, stdev);
+
+	if (rc)
+		return rc;
+
+	if (!stdev->dma_mrpc)
+		return rc;
+
+	dma_mrpc_irq = ioread32(&stdev->mmio_mrpc->dma_vector);
+	if (dma_mrpc_irq < 0 || dma_mrpc_irq >= nvecs)
+		return -EFAULT;
+
+	dma_mrpc_irq  = pci_irq_vector(stdev->pdev, dma_mrpc_irq);
+	if (dma_mrpc_irq < 0)
+		return dma_mrpc_irq;
+
+	rc = devm_request_irq(&stdev->pdev->dev, dma_mrpc_irq,
+				switchtec_dma_mrpc_isr, 0,
+				KBUILD_MODNAME, stdev);
+
+	return rc;
 }
 
 static void init_pff(struct switchtec_dev *stdev)
@@ -1294,6 +1367,19 @@  static int switchtec_init_pci(struct switchtec_dev *stdev,
 
 	pci_set_drvdata(pdev, stdev);
 
+	if (!use_dma_mrpc)
+		return 0;
+
+	if (!(ioread32(&stdev->mmio_mrpc->dma_ver) ? true : false))
+		return 0;
+
+	stdev->dma_mrpc = dma_zalloc_coherent(&stdev->pdev->dev,
+					      sizeof(*stdev->dma_mrpc),
+					      &stdev->dma_mrpc_dma_addr,
+					      GFP_KERNEL);
+	if (stdev->dma_mrpc == NULL)
+		return -ENOMEM;
+
 	return 0;
 }
 
@@ -1325,6 +1411,9 @@  static int switchtec_pci_probe(struct pci_dev *pdev,
 		  &stdev->mmio_part_cfg->mrpc_comp_hdr);
 	enable_link_state_events(stdev);
 
+	if (stdev->dma_mrpc)
+		enable_dma_mrpc(stdev);
+
 	rc = cdev_device_add(&stdev->cdev, &stdev->dev);
 	if (rc)
 		goto err_devadd;
@@ -1350,7 +1439,6 @@  static void switchtec_pci_remove(struct pci_dev *pdev)
 	cdev_device_del(&stdev->cdev, &stdev->dev);
 	ida_simple_remove(&switchtec_minor_ida, MINOR(stdev->dev.devt));
 	dev_info(&stdev->dev, "unregistered.\n");
-
 	stdev_kill(stdev);
 	put_device(&stdev->dev);
 }
diff --git a/include/linux/switchtec.h b/include/linux/switchtec.h
index ab400af..eee0412 100644
--- a/include/linux/switchtec.h
+++ b/include/linux/switchtec.h
@@ -29,6 +29,7 @@ 
 #define SWITCHTEC_EVENT_EN_IRQ   BIT(3)
 #define SWITCHTEC_EVENT_FATAL    BIT(4)
 
+#define SWITCHTEC_DMA_MRPC_EN	BIT(0)
 enum {
 	SWITCHTEC_GAS_MRPC_OFFSET       = 0x0000,
 	SWITCHTEC_GAS_TOP_CFG_OFFSET    = 0x1000,
@@ -46,6 +47,10 @@  struct mrpc_regs {
 	u32 cmd;
 	u32 status;
 	u32 ret_value;
+	u32 dma_en;
+	u64 dma_addr;
+	u32 dma_vector;
+	u32 dma_ver;
 } __packed;
 
 enum mrpc_status {
@@ -342,6 +347,14 @@  struct pff_csr_regs {
 
 struct switchtec_ntb;
 
+struct dma_mrpc_output {
+	u32 status;
+	u32 cmd_id;
+	u32 rtn_code;
+	u32 output_size;
+	u8 data[SWITCHTEC_MRPC_PAYLOAD_SIZE];
+};
+
 struct switchtec_dev {
 	struct pci_dev *pdev;
 	struct device dev;
@@ -381,6 +394,9 @@  struct switchtec_dev {
 	u8 link_event_count[SWITCHTEC_MAX_PFF_CSR];
 
 	struct switchtec_ntb *sndev;
+
+	struct dma_mrpc_output *dma_mrpc;
+	dma_addr_t dma_mrpc_dma_addr;
 };
 
 static inline struct switchtec_dev *to_stdev(struct device *dev)