[PATCH v2 0/2] HWRNG/PCI: Add driver for Cavium Thunder RNG

[PATCH v2 0/2] HWRNG/PCI: Add driver for Cavium Thunder RNG

[Omer Khaliq]

There is a hardware error rendering the FDL field incorrect for the some
Thunder RNG devices. The first patch adds a PCI quirk to fix the problem.

The second patch adds the driver.

Changes from v1:
	Use PCI quirks as advised.
	Removed unecessary headers
	Format changes as advised

Ananth Jasty (1):
  PCI: quirk fixup for cavium invalid sriov link value.

Omer Khaliq (1):
  HWRNG: thunderx: Add Cavium HWRNG driver for ThunderX SoC.

 drivers/char/hw_random/Kconfig         | 13 +++++
 drivers/char/hw_random/Makefile        |  1 +
 drivers/char/hw_random/cavium-rng-vf.c | 99 ++++++++++++++++++++++++++++++++++
 drivers/char/hw_random/cavium-rng.c    | 94 ++++++++++++++++++++++++++++++++
 drivers/pci/quirks.c                   | 11 ++++
 5 files changed, 218 insertions(+)
 create mode 100644 drivers/char/hw_random/cavium-rng-vf.c
 create mode 100644 drivers/char/hw_random/cavium-rng.c

-- 
1.9.1

[PATCH v2 1/2] PCI: quirk fixup for cavium invalid sriov link value.

[Omer Khaliq]

From: Ananth Jasty <Ananth.Jasty@cavium.com>

Cavium cn88xx hardware presents an incorrect SR-IOV Function
Dependency Link, add a fixup quirk for the affected devices.

Acked-by: David Daney <david.daney@cavium.com>
Signed-off-by: Ananth Jasty <Ananth.Jasty@cavium.com>
Signed-off-by: Omer Khaliq <okhaliq@caviumnetworks.com>
---
 drivers/pci/quirks.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 37ff015..5980aae 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -834,6 +834,17 @@ static void quirk_amd_ioapic(struct pci_dev *dev)
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD,	PCI_DEVICE_ID_AMD_VIPER_7410,	quirk_amd_ioapic);
 #endif /* CONFIG_X86_IO_APIC */
 
+#ifdef CONFIG_ARM64
+
+static void quirk_cavium_sriov_rnm_link(struct pci_dev *dev)
+{
+	/* Fix for improper SRIOV configuration on Cavium cn88xx  RNM device */
+	if (dev->subsystem_device == 0xa118)
+		dev->sriov->link = dev->devfn;
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_CAVIUM, 0xa018, quirk_cavium_sriov_rnm_link);
+#endif
+
 /*
  * Some settings of MMRBC can lead to data corruption so block changes.
  * See AMD 8131 HyperTransport PCI-X Tunnel Revision Guide
-- 
1.9.1

[PATCH v2 2/2] HWRNG: thunderx: Add Cavium HWRNG driver for ThunderX SoC.

[Omer Khaliq]

The Cavium ThunderX SoC has a hardware random number generator.
This driver provides support using the HWRNG framework.

Signed-off-by: Omer Khaliq <okhaliq@caviumnetworks.com>
Signed-off-by: Ananth Jasty <Ananth.Jasty@cavium.com>
Acked-by: David Daney <david.daney@cavium.com>
---
 drivers/char/hw_random/Kconfig         | 13 +++++
 drivers/char/hw_random/Makefile        |  1 +
 drivers/char/hw_random/cavium-rng-vf.c | 99 ++++++++++++++++++++++++++++++++++
 drivers/char/hw_random/cavium-rng.c    | 94 ++++++++++++++++++++++++++++++++
 4 files changed, 207 insertions(+)
 create mode 100644 drivers/char/hw_random/cavium-rng-vf.c
 create mode 100644 drivers/char/hw_random/cavium-rng.c

diff --git a/drivers/char/hw_random/Kconfig b/drivers/char/hw_random/Kconfig
index 56ad5a5..fb9c7ad 100644
--- a/drivers/char/hw_random/Kconfig
+++ b/drivers/char/hw_random/Kconfig
@@ -410,6 +410,19 @@ config HW_RANDOM_MESON
 
 	  If unsure, say Y.
 
+config HW_RANDOM_CAVIUM
+       tristate "Cavium ThunderX Random Number Generator support"
+       depends on HW_RANDOM && PCI && (ARM64 || (COMPILE_TEST && 64BIT))
+       default HW_RANDOM
+       ---help---
+         This driver provides kernel-side support for the Random Number
+         Generator hardware found on Cavium SoCs.
+
+         To compile this driver as a module, choose M here: the
+         module will be called cavium_rng.
+
+         If unsure, say Y.
+
 endif # HW_RANDOM
 
 config UML_RANDOM
diff --git a/drivers/char/hw_random/Makefile b/drivers/char/hw_random/Makefile
index 04bb0b0..5f52b1e 100644
--- a/drivers/char/hw_random/Makefile
+++ b/drivers/char/hw_random/Makefile
@@ -35,3 +35,4 @@ obj-$(CONFIG_HW_RANDOM_XGENE) += xgene-rng.o
 obj-$(CONFIG_HW_RANDOM_STM32) += stm32-rng.o
 obj-$(CONFIG_HW_RANDOM_PIC32) += pic32-rng.o
 obj-$(CONFIG_HW_RANDOM_MESON) += meson-rng.o
+obj-$(CONFIG_HW_RANDOM_CAVIUM) += cavium-rng.o cavium-rng-vf.o
diff --git a/drivers/char/hw_random/cavium-rng-vf.c b/drivers/char/hw_random/cavium-rng-vf.c
new file mode 100644
index 0000000..066ae0e
--- /dev/null
+++ b/drivers/char/hw_random/cavium-rng-vf.c
@@ -0,0 +1,99 @@
+/*
+ * Hardware Random Number Generator support for Cavium, Inc.
+ * Thunder processor family.
+ *
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License.  See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2016 Cavium, Inc.
+ */
+
+#include <linux/hw_random.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/pci_ids.h>
+
+struct cavium_rng {
+	struct hwrng ops;
+	void __iomem *result;
+};
+
+/* Read data from the RNG unit */
+static int cavium_rng_read(struct hwrng *rng, void *dat, size_t max, bool wait)
+{
+	struct cavium_rng *p = container_of(rng, struct cavium_rng, ops);
+	unsigned int size = max;
+
+	while (size >= 8) {
+		*((u64 *)dat) = readq(p->result);
+		size -= 8;
+		dat += 8;
+	}
+	while (size > 0) {
+		*((u8 *)dat) = readb(p->result);
+		size--;
+		dat++;
+	}
+	return max;
+}
+
+/* Map Cavium RNG to an HWRNG object */
+static int cavium_rng_probe_vf(struct	pci_dev		*pdev,
+			 const struct	pci_device_id	*id)
+{
+	struct	cavium_rng *rng;
+	int	ret;
+
+	rng = devm_kzalloc(&pdev->dev, sizeof(*rng), GFP_KERNEL);
+	if (!rng)
+		return -ENOMEM;
+
+	/* Map the RNG result */
+	rng->result = pcim_iomap(pdev, 0, 0);
+	if (!rng->result) {
+		dev_err(&pdev->dev, "Error iomap failed retrieving result.\n");
+		return -ENOMEM;
+	}
+
+	rng->ops.name    = "cavium rng";
+	rng->ops.read    = cavium_rng_read;
+	rng->ops.quality = 1000;
+
+	pci_set_drvdata(pdev, rng);
+
+	ret = hwrng_register(&rng->ops);
+	if (ret) {
+		dev_err(&pdev->dev, "Error registering device as HWRNG.\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+/* Remove the VF */
+void  cavium_rng_remove_vf(struct pci_dev *pdev)
+{
+	struct cavium_rng *rng;
+
+	rng = pci_get_drvdata(pdev);
+	hwrng_unregister(&rng->ops);
+}
+
+static const struct pci_device_id cavium_rng_vf_id_table[] = {
+	{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, 0xa033), 0, 0, 0},
+	{0,},
+};
+MODULE_DEVICE_TABLE(pci, cavium_rng_vf_id_table);
+
+static struct pci_driver cavium_rng_vf_driver = {
+	.name		= "cavium_rng_vf",
+	.id_table	= cavium_rng_vf_id_table,
+	.probe		= cavium_rng_probe_vf,
+	.remove		= cavium_rng_remove_vf,
+};
+module_pci_driver(cavium_rng_vf_driver);
+
+MODULE_AUTHOR("Omer Khaliq <okhaliq@caviumnetworks.com>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/char/hw_random/cavium-rng.c b/drivers/char/hw_random/cavium-rng.c
new file mode 100644
index 0000000..a944e0a
--- /dev/null
+++ b/drivers/char/hw_random/cavium-rng.c
@@ -0,0 +1,94 @@
+/*
+ * Hardware Random Number Generator support for Cavium Inc.
+ * Thunder processor family.
+ *
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License.  See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2016 Cavium, Inc.
+ */
+
+#include <linux/hw_random.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/pci_ids.h>
+
+#define THUNDERX_RNM_ENT_EN     0x1
+#define THUNDERX_RNM_RNG_EN     0x2
+
+struct cavium_rng_pf {
+	void __iomem *control_status;
+};
+
+/* Enable the RNG hardware and activate the VF */
+static int cavium_rng_probe(struct pci_dev *pdev,
+			const struct pci_device_id *id)
+{
+	struct	cavium_rng_pf *rng;
+	int	iov_err;
+
+	rng = devm_kzalloc(&pdev->dev, sizeof(*rng), GFP_KERNEL);
+	if (!rng)
+		return -ENOMEM;
+
+	/*Map the RNG control */
+	rng->control_status = pcim_iomap(pdev, 0, 0);
+	if (!rng->control_status) {
+		dev_err(&pdev->dev,
+			"Error iomap failed retrieving control_status.\n");
+		return -ENOMEM;
+	}
+
+	/* Enable the RNG hardware and entropy source */
+	writeq(THUNDERX_RNM_RNG_EN | THUNDERX_RNM_ENT_EN,
+		rng->control_status);
+
+	pci_set_drvdata(pdev, rng);
+
+	/* Enable the Cavium RNG as a VF */
+	iov_err = pci_enable_sriov(pdev, 1);
+	if (iov_err != 0) {
+		/* Disable the RNG hardware and entropy source */
+		writeq(0, rng->control_status);
+		dev_err(&pdev->dev,
+			"Error initializing RNG virtual function,(%i).\n",
+			iov_err);
+		return iov_err;
+	}
+
+	return 0;
+}
+
+/* Disable VF and RNG Hardware */
+void  cavium_rng_remove(struct pci_dev *pdev)
+{
+	struct cavium_rng_pf *rng;
+
+	rng = pci_get_drvdata(pdev);
+
+	/* Remove the VF */
+	pci_disable_sriov(pdev);
+
+	/* Disable the RNG hardware and entropy source */
+	writeq(0, rng->control_status);
+}
+
+static const struct pci_device_id cavium_rng_pf_id_table[] = {
+	{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, 0xa018), 0, 0, 0}, /* Thunder RNM */
+	{0,},
+};
+
+MODULE_DEVICE_TABLE(pci, cavium_rng_pf_id_table);
+
+static struct pci_driver cavium_rng_pf_driver = {
+	.name		= "cavium_rng_pf",
+	.id_table	= cavium_rng_pf_id_table,
+	.probe		= cavium_rng_probe,
+	.remove		= cavium_rng_remove,
+};
+
+module_pci_driver(cavium_rng_pf_driver);
+MODULE_AUTHOR("Omer Khaliq <okhaliq@caviumnetworks.com>");
+MODULE_LICENSE("GPL");
-- 
1.9.1

Re: [PATCH v2 2/2] HWRNG: thunderx: Add Cavium HWRNG driver for ThunderX SoC.

[Corentin LABBE]

Hello

> +/* Read data from the RNG unit */
> +static int cavium_rng_read(struct hwrng *rng, void *dat, size_t max, bool wait)
> +{
> +	struct cavium_rng *p = container_of(rng, struct cavium_rng, ops);
> +	unsigned int size = max;
> +
> +	while (size >= 8) {
> +		*((u64 *)dat) = readq(p->result);
> +		size -= 8;
> +		dat += 8;
> +	}

I think you could use readsq()
This will increase throughput

Regards

LABBE Corentin

Re: [PATCH v2 1/2] PCI: quirk fixup for cavium invalid sriov link value.

[Bjorn Helgaas]

On Tue, Aug 23, 2016 at 04:27:14PM -0700, Omer Khaliq wrote:
> From: Ananth Jasty <Ananth.Jasty@cavium.com>
> 
> Cavium cn88xx hardware presents an incorrect SR-IOV Function
> Dependency Link, add a fixup quirk for the affected devices.
> 
> Acked-by: David Daney <david.daney@cavium.com>
> Signed-off-by: Ananth Jasty <Ananth.Jasty@cavium.com>
> Signed-off-by: Omer Khaliq <okhaliq@caviumnetworks.com>

Acked-by: Bjorn Helgaas <bhelgaas@google.com>

If you update this patch for any reason, please update the subject
like this:

  -PCI: quirk fixup for cavium invalid sriov link value.
  +PCI: Fix incorrect Cavium cn8xx SR-IOV Function Dependency Link

This will make "git log --oneline drivers/pci/quirks.c" show more
useful information and look more consistent.

I assume somebody else will merge this patch along with your HWRNG
driver.

> ---
>  drivers/pci/quirks.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
> index 37ff015..5980aae 100644
> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -834,6 +834,17 @@ static void quirk_amd_ioapic(struct pci_dev *dev)
>  DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD,	PCI_DEVICE_ID_AMD_VIPER_7410,	quirk_amd_ioapic);
>  #endif /* CONFIG_X86_IO_APIC */
>  
> +#ifdef CONFIG_ARM64
> +
> +static void quirk_cavium_sriov_rnm_link(struct pci_dev *dev)
> +{
> +	/* Fix for improper SRIOV configuration on Cavium cn88xx  RNM device */
> +	if (dev->subsystem_device == 0xa118)
> +		dev->sriov->link = dev->devfn;
> +}
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_CAVIUM, 0xa018, quirk_cavium_sriov_rnm_link);
> +#endif
> +
>  /*
>   * Some settings of MMRBC can lead to data corruption so block changes.
>   * See AMD 8131 HyperTransport PCI-X Tunnel Revision Guide
> -- 
> 1.9.1
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 2/2] HWRNG: thunderx: Add Cavium HWRNG driver for ThunderX SoC.

[David Daney]

On 08/23/2016 10:46 PM, Corentin LABBE wrote:
> Hello
>
>> +/* Read data from the RNG unit */
>> +static int cavium_rng_read(struct hwrng *rng, void *dat, size_t max, bool wait)
>> +{
>> +	struct cavium_rng *p = container_of(rng, struct cavium_rng, ops);
>> +	unsigned int size = max;
>> +
>> +	while (size >= 8) {
>> +		*((u64 *)dat) = readq(p->result);
>> +		size -= 8;
>> +		dat += 8;
>> +	}
>
> I think you could use readsq()
> This will increase throughput

If you look at the implementation of readsq(), you will see that it is a 
similar loop.  Since the overhead is primarily I/O latency from the RNG 
hardware, the throughput cannot really be changed with micro 
optimizations to this simple loop.

Also, on big-endian kernels, it appears that a loop of readq() and 
readsq() will give different results as readq will byte swap the result 
and readsq does not.  Since this is a RNG, the byte swapping is not 
important, but it is a difference.

Because of this, I think it should be acceptable to stick with the loop 
we currently have.

If the hwrng maintainers want to change the loop, to a readsq(), we 
might investigate this more.

Thanks,
David Daney

Re: [PATCH v2 0/2] HWRNG/PCI: Add driver for Cavium Thunder RNG

[Herbert Xu]

On Tue, Aug 23, 2016 at 04:27:13PM -0700, Omer Khaliq wrote:
> There is a hardware error rendering the FDL field incorrect for the some
> Thunder RNG devices. The first patch adds a PCI quirk to fix the problem.
> 
> The second patch adds the driver.
> 
> Changes from v1:
> 	Use PCI quirks as advised.
> 	Removed unecessary headers
> 	Format changes as advised

All applied.  Thanks.
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt