* [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC
@ 2019-02-22 15:30 Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 1/7] mic: vop: Cast pointers to unsigned long Vincent Whitchurch
` (9 more replies)
0 siblings, 10 replies; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
This allows the drivers/misc/mic/vop/ code to be used on other architectures
and adds a loopback driver to be able to use/test/evaluate it without special
hardware.
See v1 for more info:
https://lore.kernel.org/lkml/20190116163253.23780-1-vincent.whitchurch@axis.com/
v2:
- Rebase on top of the patches which have merged
- Use unsigned long instead of uintptr_t
- Drop "Use consistent DMA" for now since it's not needed for loopback
- Add race fix
- Build on all architectures, not just ARM and x86
- Add sample userspace code
Vincent Whitchurch (7):
mic: vop: Cast pointers to unsigned long
mic: Rename ioremap pointer to remap
mic: vop: Allow building on more systems
mic: vop: Add loopback driver
mic: vop: Fix init race with shared interrupts
samples: mic: Split out vop code from mpssd
samples: mic: Add sample VOP userspace
drivers/misc/mic/Kconfig | 13 +-
drivers/misc/mic/bus/scif_bus.h | 8 +-
drivers/misc/mic/bus/vop_bus.h | 8 +-
drivers/misc/mic/card/mic_device.c | 8 +-
drivers/misc/mic/host/mic_boot.c | 8 +-
drivers/misc/mic/scif/scif_map.h | 4 +-
drivers/misc/mic/vop/Makefile | 2 +
drivers/misc/mic/vop/vop_loopback.c | 382 ++++++++
drivers/misc/mic/vop/vop_main.c | 62 +-
drivers/misc/mic/vop/vop_vringh.c | 15 +-
samples/mic/mpssd/.gitignore | 1 +
samples/mic/mpssd/Makefile | 9 +-
samples/mic/mpssd/mpssd.c | 1317 +-------------------------
samples/mic/mpssd/mpssd.h | 1 +
samples/mic/mpssd/vop.c | 1357 +++++++++++++++++++++++++++
samples/mic/mpssd/vopd.c | 25 +
16 files changed, 1866 insertions(+), 1354 deletions(-)
create mode 100644 drivers/misc/mic/vop/vop_loopback.c
create mode 100644 samples/mic/mpssd/vop.c
create mode 100644 samples/mic/mpssd/vopd.c
--
2.20.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v2 char-misc-next 1/7] mic: vop: Cast pointers to unsigned long
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
@ 2019-02-22 15:30 ` Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 1/7] " Vincent Whitchurch
` (8 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
Fix these on 32-bit:
vop_vringh.c:711:13: error: cast from pointer to integer of different
size [-Werror=pointer-to-int-cast]
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
drivers/misc/mic/vop/vop_main.c | 13 +++++++++----
drivers/misc/mic/vop/vop_vringh.c | 5 +++--
2 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/drivers/misc/mic/vop/vop_main.c b/drivers/misc/mic/vop/vop_main.c
index 3f96e19e57eb..263e0c3fa7c6 100644
--- a/drivers/misc/mic/vop/vop_main.c
+++ b/drivers/misc/mic/vop/vop_main.c
@@ -514,7 +514,7 @@ static int _vop_add_device(struct mic_device_desc __iomem *d,
vdev->desc = d;
vdev->dc = (void __iomem *)d + _vop_aligned_desc_size(d);
vdev->dnode = dnode;
- vdev->vdev.priv = (void *)(u64)dnode;
+ vdev->vdev.priv = (void *)(unsigned long)dnode;
init_completion(&vdev->reset_done);
vdev->h2c_vdev_db = vpdev->hw_ops->next_db(vpdev);
@@ -536,7 +536,7 @@ static int _vop_add_device(struct mic_device_desc __iomem *d,
offset, type);
goto free_irq;
}
- writeq((u64)vdev, &vdev->dc->vdev);
+ writeq((unsigned long)vdev, &vdev->dc->vdev);
dev_dbg(_vop_dev(vdev), "%s: registered vop device %u type %u vdev %p\n",
__func__, offset, type, vdev);
@@ -563,13 +563,18 @@ static int vop_match_desc(struct device *dev, void *data)
return vdev->desc == (void __iomem *)data;
}
+static struct _vop_vdev *vop_dc_to_vdev(struct mic_device_ctrl *dc)
+{
+ return (struct _vop_vdev *)(unsigned long)readq(&dc->vdev);
+}
+
static void _vop_handle_config_change(struct mic_device_desc __iomem *d,
unsigned int offset,
struct vop_device *vpdev)
{
struct mic_device_ctrl __iomem *dc
= (void __iomem *)d + _vop_aligned_desc_size(d);
- struct _vop_vdev *vdev = (struct _vop_vdev *)readq(&dc->vdev);
+ struct _vop_vdev *vdev = vop_dc_to_vdev(dc);
if (ioread8(&dc->config_change) != MIC_VIRTIO_PARAM_CONFIG_CHANGED)
return;
@@ -588,7 +593,7 @@ static int _vop_remove_device(struct mic_device_desc __iomem *d,
{
struct mic_device_ctrl __iomem *dc
= (void __iomem *)d + _vop_aligned_desc_size(d);
- struct _vop_vdev *vdev = (struct _vop_vdev *)readq(&dc->vdev);
+ struct _vop_vdev *vdev = vop_dc_to_vdev(dc);
u8 status;
int ret = -1;
diff --git a/drivers/misc/mic/vop/vop_vringh.c b/drivers/misc/mic/vop/vop_vringh.c
index 0bac8bce933c..73d9fe6bc18d 100644
--- a/drivers/misc/mic/vop/vop_vringh.c
+++ b/drivers/misc/mic/vop/vop_vringh.c
@@ -712,16 +712,17 @@ static int vop_vringh_copy(struct vop_vdev *vdev, struct vringh_kiov *iov,
while (len && iov->i < iov->used) {
struct kvec *kiov = &iov->iov[iov->i];
+ unsigned long daddr = (unsigned long)kiov->iov_base;
partlen = min(kiov->iov_len, len);
if (read)
ret = vop_virtio_copy_to_user(vdev, ubuf, partlen,
- (u64)kiov->iov_base,
+ daddr,
kiov->iov_len,
vr_idx);
else
ret = vop_virtio_copy_from_user(vdev, ubuf, partlen,
- (u64)kiov->iov_base,
+ daddr,
kiov->iov_len,
vr_idx);
if (ret) {
--
2.20.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v2 char-misc-next 1/7] vop: Cast pointers to unsigned long
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 1/7] mic: vop: Cast pointers to unsigned long Vincent Whitchurch
@ 2019-02-22 15:30 ` Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 2/7] mic: Rename ioremap pointer to remap Vincent Whitchurch
` (7 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
Fix these on 32-bit:
vop_vringh.c:711:13: error: cast from pointer to integer of different
size [-Werror=pointer-to-int-cast]
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
drivers/misc/mic/vop/vop_main.c | 13 +++++++++----
drivers/misc/mic/vop/vop_vringh.c | 5 +++--
2 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/drivers/misc/mic/vop/vop_main.c b/drivers/misc/mic/vop/vop_main.c
index 3f96e19e57eb..263e0c3fa7c6 100644
--- a/drivers/misc/mic/vop/vop_main.c
+++ b/drivers/misc/mic/vop/vop_main.c
@@ -514,7 +514,7 @@ static int _vop_add_device(struct mic_device_desc __iomem *d,
vdev->desc = d;
vdev->dc = (void __iomem *)d + _vop_aligned_desc_size(d);
vdev->dnode = dnode;
- vdev->vdev.priv = (void *)(u64)dnode;
+ vdev->vdev.priv = (void *)(unsigned long)dnode;
init_completion(&vdev->reset_done);
vdev->h2c_vdev_db = vpdev->hw_ops->next_db(vpdev);
@@ -536,7 +536,7 @@ static int _vop_add_device(struct mic_device_desc __iomem *d,
offset, type);
goto free_irq;
}
- writeq((u64)vdev, &vdev->dc->vdev);
+ writeq((unsigned long)vdev, &vdev->dc->vdev);
dev_dbg(_vop_dev(vdev), "%s: registered vop device %u type %u vdev %p\n",
__func__, offset, type, vdev);
@@ -563,13 +563,18 @@ static int vop_match_desc(struct device *dev, void *data)
return vdev->desc == (void __iomem *)data;
}
+static struct _vop_vdev *vop_dc_to_vdev(struct mic_device_ctrl *dc)
+{
+ return (struct _vop_vdev *)(unsigned long)readq(&dc->vdev);
+}
+
static void _vop_handle_config_change(struct mic_device_desc __iomem *d,
unsigned int offset,
struct vop_device *vpdev)
{
struct mic_device_ctrl __iomem *dc
= (void __iomem *)d + _vop_aligned_desc_size(d);
- struct _vop_vdev *vdev = (struct _vop_vdev *)readq(&dc->vdev);
+ struct _vop_vdev *vdev = vop_dc_to_vdev(dc);
if (ioread8(&dc->config_change) != MIC_VIRTIO_PARAM_CONFIG_CHANGED)
return;
@@ -588,7 +593,7 @@ static int _vop_remove_device(struct mic_device_desc __iomem *d,
{
struct mic_device_ctrl __iomem *dc
= (void __iomem *)d + _vop_aligned_desc_size(d);
- struct _vop_vdev *vdev = (struct _vop_vdev *)readq(&dc->vdev);
+ struct _vop_vdev *vdev = vop_dc_to_vdev(dc);
u8 status;
int ret = -1;
diff --git a/drivers/misc/mic/vop/vop_vringh.c b/drivers/misc/mic/vop/vop_vringh.c
index 0bac8bce933c..73d9fe6bc18d 100644
--- a/drivers/misc/mic/vop/vop_vringh.c
+++ b/drivers/misc/mic/vop/vop_vringh.c
@@ -712,16 +712,17 @@ static int vop_vringh_copy(struct vop_vdev *vdev, struct vringh_kiov *iov,
while (len && iov->i < iov->used) {
struct kvec *kiov = &iov->iov[iov->i];
+ unsigned long daddr = (unsigned long)kiov->iov_base;
partlen = min(kiov->iov_len, len);
if (read)
ret = vop_virtio_copy_to_user(vdev, ubuf, partlen,
- (u64)kiov->iov_base,
+ daddr,
kiov->iov_len,
vr_idx);
else
ret = vop_virtio_copy_from_user(vdev, ubuf, partlen,
- (u64)kiov->iov_base,
+ daddr,
kiov->iov_len,
vr_idx);
if (ret) {
--
2.20.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v2 char-misc-next 2/7] mic: Rename ioremap pointer to remap
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 1/7] mic: vop: Cast pointers to unsigned long Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 1/7] " Vincent Whitchurch
@ 2019-02-22 15:30 ` Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 3/7] mic: vop: Allow building on more systems Vincent Whitchurch
` (6 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
Some architectures (like MIPS) implement ioremap as a macro, and this
leads to conflicts with the ioremap function pointer in various mic
structures.
drivers/misc/mic/vop/vop_vringh.c:
In function 'vop_virtio_init_post':
drivers/misc/mic/vop/vop_vringh.c:86:13:
error: macro "ioremap" passed 3 arguments, but takes just 2
Rename ioremap to remap to fix this. Likewise for iounmap.
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
drivers/misc/mic/bus/scif_bus.h | 8 ++++----
drivers/misc/mic/bus/vop_bus.h | 8 ++++----
drivers/misc/mic/card/mic_device.c | 8 ++++----
drivers/misc/mic/host/mic_boot.c | 8 ++++----
drivers/misc/mic/scif/scif_map.h | 4 ++--
drivers/misc/mic/vop/vop_main.c | 7 +++----
drivers/misc/mic/vop/vop_vringh.c | 10 +++++-----
7 files changed, 26 insertions(+), 27 deletions(-)
diff --git a/drivers/misc/mic/bus/scif_bus.h b/drivers/misc/mic/bus/scif_bus.h
index ff59568219ad..377a4f38cd7e 100644
--- a/drivers/misc/mic/bus/scif_bus.h
+++ b/drivers/misc/mic/bus/scif_bus.h
@@ -88,8 +88,8 @@ struct scif_driver {
* @send_intr: Send an interrupt to the remote node on a specified doorbell.
* @send_p2p_intr: Send an interrupt to the peer node on a specified doorbell
* which is specifically targeted for a peer to peer node.
- * @ioremap: Map a buffer with the specified physical address and length.
- * @iounmap: Unmap a buffer previously mapped.
+ * @remap: Map a buffer with the specified physical address and length.
+ * @unmap: Unmap a buffer previously mapped.
*/
struct scif_hw_ops {
int (*next_db)(struct scif_hw_dev *sdev);
@@ -104,9 +104,9 @@ struct scif_hw_ops {
void (*send_intr)(struct scif_hw_dev *sdev, int db);
void (*send_p2p_intr)(struct scif_hw_dev *sdev, int db,
struct mic_mw *mw);
- void __iomem * (*ioremap)(struct scif_hw_dev *sdev,
+ void __iomem * (*remap)(struct scif_hw_dev *sdev,
phys_addr_t pa, size_t len);
- void (*iounmap)(struct scif_hw_dev *sdev, void __iomem *va);
+ void (*unmap)(struct scif_hw_dev *sdev, void __iomem *va);
};
int scif_register_driver(struct scif_driver *driver);
diff --git a/drivers/misc/mic/bus/vop_bus.h b/drivers/misc/mic/bus/vop_bus.h
index fff7a865d721..cf5f3fae573c 100644
--- a/drivers/misc/mic/bus/vop_bus.h
+++ b/drivers/misc/mic/bus/vop_bus.h
@@ -87,8 +87,8 @@ struct vop_driver {
* @get_dp: Get access to the virtio device page used by the self
* node to add/remove/configure virtio devices.
* @send_intr: Send an interrupt to the peer node on a specified doorbell.
- * @ioremap: Map a buffer with the specified DMA address and length.
- * @iounmap: Unmap a buffer previously mapped.
+ * @remap: Map a buffer with the specified DMA address and length.
+ * @unmap: Unmap a buffer previously mapped.
* @dma_filter: The DMA filter function to use for obtaining access to
* a DMA channel on the peer node.
*/
@@ -104,9 +104,9 @@ struct vop_hw_ops {
void __iomem * (*get_remote_dp)(struct vop_device *vpdev);
void * (*get_dp)(struct vop_device *vpdev);
void (*send_intr)(struct vop_device *vpdev, int db);
- void __iomem * (*ioremap)(struct vop_device *vpdev,
+ void __iomem * (*remap)(struct vop_device *vpdev,
dma_addr_t pa, size_t len);
- void (*iounmap)(struct vop_device *vpdev, void __iomem *va);
+ void (*unmap)(struct vop_device *vpdev, void __iomem *va);
};
struct vop_device *
diff --git a/drivers/misc/mic/card/mic_device.c b/drivers/misc/mic/card/mic_device.c
index e749af48f736..dcd07ef29801 100644
--- a/drivers/misc/mic/card/mic_device.c
+++ b/drivers/misc/mic/card/mic_device.c
@@ -245,8 +245,8 @@ static struct scif_hw_ops scif_hw_ops = {
.next_db = ___mic_next_db,
.send_intr = ___mic_send_intr,
.send_p2p_intr = ___mic_send_p2p_intr,
- .ioremap = ___mic_ioremap,
- .iounmap = ___mic_iounmap,
+ .remap = ___mic_ioremap,
+ .unmap = ___mic_iounmap,
};
static inline struct mic_driver *vpdev_to_mdrv(struct vop_device *vpdev)
@@ -316,8 +316,8 @@ static struct vop_hw_ops vop_hw_ops = {
.next_db = __mic_next_db,
.get_remote_dp = __mic_get_remote_dp,
.send_intr = __mic_send_intr,
- .ioremap = __mic_ioremap,
- .iounmap = __mic_iounmap,
+ .remap = __mic_ioremap,
+ .unmap = __mic_iounmap,
};
static int mic_request_dma_chans(struct mic_driver *mdrv)
diff --git a/drivers/misc/mic/host/mic_boot.c b/drivers/misc/mic/host/mic_boot.c
index 6479435ac96b..079c36f0ce6e 100644
--- a/drivers/misc/mic/host/mic_boot.c
+++ b/drivers/misc/mic/host/mic_boot.c
@@ -133,8 +133,8 @@ static struct vop_hw_ops vop_hw_ops = {
.get_dp = __mic_get_dp,
.get_remote_dp = __mic_get_remote_dp,
.send_intr = __mic_send_intr,
- .ioremap = __mic_ioremap,
- .iounmap = __mic_iounmap,
+ .remap = __mic_ioremap,
+ .unmap = __mic_iounmap,
};
static inline struct mic_device *scdev_to_mdev(struct scif_hw_dev *scdev)
@@ -315,8 +315,8 @@ static struct scif_hw_ops scif_hw_ops = {
.ack_interrupt = ___mic_ack_interrupt,
.next_db = ___mic_next_db,
.send_intr = ___mic_send_intr,
- .ioremap = ___mic_ioremap,
- .iounmap = ___mic_iounmap,
+ .remap = ___mic_ioremap,
+ .unmap = ___mic_iounmap,
};
static inline struct mic_device *mbdev_to_mdev(struct mbus_device *mbdev)
diff --git a/drivers/misc/mic/scif/scif_map.h b/drivers/misc/mic/scif/scif_map.h
index 3e86360ba5a6..7b380534eba1 100644
--- a/drivers/misc/mic/scif/scif_map.h
+++ b/drivers/misc/mic/scif/scif_map.h
@@ -97,7 +97,7 @@ scif_ioremap(dma_addr_t phys, size_t size, struct scif_dev *scifdev)
out_virt = phys_to_virt(phys);
else
out_virt = (void __force *)
- sdev->hw_ops->ioremap(sdev, phys, size);
+ sdev->hw_ops->remap(sdev, phys, size);
return out_virt;
}
@@ -107,7 +107,7 @@ scif_iounmap(void *virt, size_t len, struct scif_dev *scifdev)
if (!scifdev_self(scifdev)) {
struct scif_hw_dev *sdev = scifdev->sdev;
- sdev->hw_ops->iounmap(sdev, (void __force __iomem *)virt);
+ sdev->hw_ops->unmap(sdev, (void __force __iomem *)virt);
}
}
diff --git a/drivers/misc/mic/vop/vop_main.c b/drivers/misc/mic/vop/vop_main.c
index 263e0c3fa7c6..e37b2c2152a2 100644
--- a/drivers/misc/mic/vop/vop_main.c
+++ b/drivers/misc/mic/vop/vop_main.c
@@ -270,7 +270,7 @@ static void vop_del_vq(struct virtqueue *vq, int n)
free_pages((unsigned long)vdev->used_virt[n],
get_order(vdev->used_size[n]));
vring_del_virtqueue(vq);
- vpdev->hw_ops->iounmap(vpdev, vdev->vr[n]);
+ vpdev->hw_ops->unmap(vpdev, vdev->vr[n]);
vdev->vr[n] = NULL;
}
@@ -338,8 +338,7 @@ static struct virtqueue *vop_find_vq(struct virtio_device *dev,
memcpy_fromio(&config, vqconfig, sizeof(config));
_vr_size = vring_size(le16_to_cpu(config.num), MIC_VIRTIO_RING_ALIGN);
vr_size = PAGE_ALIGN(_vr_size + sizeof(struct _mic_vring_info));
- va = vpdev->hw_ops->ioremap(vpdev, le64_to_cpu(config.address),
- vr_size);
+ va = vpdev->hw_ops->remap(vpdev, le64_to_cpu(config.address), vr_size);
if (!va)
return ERR_PTR(-ENOMEM);
vdev->vr[index] = va;
@@ -393,7 +392,7 @@ static struct virtqueue *vop_find_vq(struct virtio_device *dev,
free_pages((unsigned long)used,
get_order(vdev->used_size[index]));
unmap:
- vpdev->hw_ops->iounmap(vpdev, vdev->vr[index]);
+ vpdev->hw_ops->unmap(vpdev, vdev->vr[index]);
return ERR_PTR(err);
}
diff --git a/drivers/misc/mic/vop/vop_vringh.c b/drivers/misc/mic/vop/vop_vringh.c
index 73d9fe6bc18d..3632fce40590 100644
--- a/drivers/misc/mic/vop/vop_vringh.c
+++ b/drivers/misc/mic/vop/vop_vringh.c
@@ -80,7 +80,7 @@ static void vop_virtio_init_post(struct vop_vdev *vdev)
continue;
}
vdev->vvr[i].vrh.vring.used =
- (void __force *)vpdev->hw_ops->ioremap(
+ (void __force *)vpdev->hw_ops->remap(
vpdev,
le64_to_cpu(vqconfig[i].used_address),
used_size);
@@ -528,7 +528,7 @@ static int vop_virtio_copy_to_user(struct vop_vdev *vdev, void __user *ubuf,
int vr_idx)
{
struct vop_device *vpdev = vdev->vpdev;
- void __iomem *dbuf = vpdev->hw_ops->ioremap(vpdev, daddr, len);
+ void __iomem *dbuf = vpdev->hw_ops->remap(vpdev, daddr, len);
struct vop_vringh *vvr = &vdev->vvr[vr_idx];
struct vop_info *vi = dev_get_drvdata(&vpdev->dev);
size_t dma_alignment;
@@ -588,7 +588,7 @@ static int vop_virtio_copy_to_user(struct vop_vdev *vdev, void __user *ubuf,
}
err = 0;
err:
- vpdev->hw_ops->iounmap(vpdev, dbuf);
+ vpdev->hw_ops->unmap(vpdev, dbuf);
dev_dbg(vop_dev(vdev),
"%s: ubuf %p dbuf %p len 0x%zx vr_idx 0x%x\n",
__func__, ubuf, dbuf, len, vr_idx);
@@ -606,7 +606,7 @@ static int vop_virtio_copy_from_user(struct vop_vdev *vdev, void __user *ubuf,
int vr_idx)
{
struct vop_device *vpdev = vdev->vpdev;
- void __iomem *dbuf = vpdev->hw_ops->ioremap(vpdev, daddr, len);
+ void __iomem *dbuf = vpdev->hw_ops->remap(vpdev, daddr, len);
struct vop_vringh *vvr = &vdev->vvr[vr_idx];
struct vop_info *vi = dev_get_drvdata(&vdev->vpdev->dev);
size_t dma_alignment;
@@ -676,7 +676,7 @@ static int vop_virtio_copy_from_user(struct vop_vdev *vdev, void __user *ubuf,
vdev->out_bytes += len;
err = 0;
err:
- vpdev->hw_ops->iounmap(vpdev, dbuf);
+ vpdev->hw_ops->unmap(vpdev, dbuf);
dev_dbg(vop_dev(vdev),
"%s: ubuf %p dbuf %p len 0x%zx vr_idx 0x%x\n",
__func__, ubuf, dbuf, len, vr_idx);
--
2.20.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v2 char-misc-next 3/7] mic: vop: Allow building on more systems
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
` (2 preceding siblings ...)
2019-02-22 15:30 ` [PATCH v2 char-misc-next 2/7] mic: Rename ioremap pointer to remap Vincent Whitchurch
@ 2019-02-22 15:30 ` Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 3/7] " Vincent Whitchurch
` (5 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
VOP_BUS does not actually depend on x86-64 or PCI or X86_DEV_DMA_OPS.
The dependency on X86_DEV_DMA_OPS has been unnecessary since commit
5657933dbb6e25fe ("treewide: Move dma_ops from struct dev_archdata into
struct device").
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
drivers/misc/mic/Kconfig | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/misc/mic/Kconfig b/drivers/misc/mic/Kconfig
index 227cc7443671..242dcee14689 100644
--- a/drivers/misc/mic/Kconfig
+++ b/drivers/misc/mic/Kconfig
@@ -38,7 +38,6 @@ comment "VOP Bus Driver"
config VOP_BUS
tristate "VOP Bus Driver"
- depends on 64BIT && PCI && X86 && X86_DEV_DMA_OPS
help
This option is selected by any driver which registers a
device or driver on the VOP Bus, such as CONFIG_INTEL_MIC_HOST
@@ -132,7 +131,7 @@ comment "VOP Driver"
config VOP
tristate "VOP Driver"
- depends on 64BIT && PCI && X86 && VOP_BUS
+ depends on VOP_BUS
select VHOST_RING
select VIRTIO
help
--
2.20.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v2 char-misc-next 3/7] vop: Allow building on more systems
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
` (3 preceding siblings ...)
2019-02-22 15:30 ` [PATCH v2 char-misc-next 3/7] mic: vop: Allow building on more systems Vincent Whitchurch
@ 2019-02-22 15:30 ` Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 4/7] mic: vop: Add loopback driver Vincent Whitchurch
` (4 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
VOP_BUS does not actually depend on x86-64 or PCI or X86_DEV_DMA_OPS.
The dependency on X86_DEV_DMA_OPS has been unnecessary since commit
5657933dbb6e25fe ("treewide: Move dma_ops from struct dev_archdata into
struct device").
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
drivers/misc/mic/Kconfig | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/misc/mic/Kconfig b/drivers/misc/mic/Kconfig
index 227cc7443671..242dcee14689 100644
--- a/drivers/misc/mic/Kconfig
+++ b/drivers/misc/mic/Kconfig
@@ -38,7 +38,6 @@ comment "VOP Bus Driver"
config VOP_BUS
tristate "VOP Bus Driver"
- depends on 64BIT && PCI && X86 && X86_DEV_DMA_OPS
help
This option is selected by any driver which registers a
device or driver on the VOP Bus, such as CONFIG_INTEL_MIC_HOST
@@ -132,7 +131,7 @@ comment "VOP Driver"
config VOP
tristate "VOP Driver"
- depends on 64BIT && PCI && X86 && VOP_BUS
+ depends on VOP_BUS
select VHOST_RING
select VIRTIO
help
--
2.20.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v2 char-misc-next 4/7] mic: vop: Add loopback driver
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
` (4 preceding siblings ...)
2019-02-22 15:30 ` [PATCH v2 char-misc-next 3/7] " Vincent Whitchurch
@ 2019-02-22 15:30 ` Vincent Whitchurch
2019-02-26 11:56 ` Greg KH
2019-02-22 15:30 ` [PATCH v2 char-misc-next 4/7] " Vincent Whitchurch
` (3 subsequent siblings)
9 siblings, 1 reply; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
Add a loopback driver to allow testing and evaluation of the VOP
framework without special hardware. The host and the guest will run
under the same kernel.
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
drivers/misc/mic/Kconfig | 10 +
drivers/misc/mic/vop/Makefile | 2 +
drivers/misc/mic/vop/vop_loopback.c | 382 ++++++++++++++++++++++++++++
3 files changed, 394 insertions(+)
create mode 100644 drivers/misc/mic/vop/vop_loopback.c
diff --git a/drivers/misc/mic/Kconfig b/drivers/misc/mic/Kconfig
index 242dcee14689..2e2f745afb3a 100644
--- a/drivers/misc/mic/Kconfig
+++ b/drivers/misc/mic/Kconfig
@@ -148,6 +148,16 @@ config VOP
OS and tools for MIC to use with this driver are available from
<http://software.intel.com/en-us/mic-developer>.
+config VOP_LOOPBACK
+ tristate "VOP loopback driver"
+ depends on VOP
+ help
+ This enables a loopback driver to test and evaluate the VOP
+ infrastructure without actual PCIe hardware. The host and the guest
+ sides run under the same kernel.
+
+ If unsure, say N.
+
if VOP
source "drivers/vhost/Kconfig.vringh"
endif
diff --git a/drivers/misc/mic/vop/Makefile b/drivers/misc/mic/vop/Makefile
index 78819c8999f1..a6ead25c4418 100644
--- a/drivers/misc/mic/vop/Makefile
+++ b/drivers/misc/mic/vop/Makefile
@@ -7,3 +7,5 @@ obj-m := vop.o
vop-objs += vop_main.o
vop-objs += vop_debugfs.o
vop-objs += vop_vringh.o
+
+obj-$(CONFIG_VOP_LOOPBACK) += vop_loopback.o
diff --git a/drivers/misc/mic/vop/vop_loopback.c b/drivers/misc/mic/vop/vop_loopback.c
new file mode 100644
index 000000000000..76d787db3d3e
--- /dev/null
+++ b/drivers/misc/mic/vop/vop_loopback.c
@@ -0,0 +1,382 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2019 Axis Communications AB
+ */
+
+#include <linux/platform_device.h>
+#include <linux/dma-direction.h>
+#include <linux/dma-mapping.h>
+#include <linux/mic_common.h>
+#include <linux/of_device.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+#include <linux/io.h>
+
+#include "../bus/vop_bus.h"
+
+struct mic_irq {
+ struct list_head list;
+ int irq;
+ irqreturn_t (*func)(int irq, void *data);
+ void *data;
+};
+
+struct vop_loopback_end {
+ struct vop_loopback *loopback;
+ const char *name;
+ struct vop_device *vop;
+ struct list_head irqs;
+ struct mutex mutex;
+ struct work_struct work;
+};
+
+struct vop_loopback {
+ struct device *dev;
+ void *dp;
+ struct vop_loopback_end host;
+ struct vop_loopback_end guest;
+};
+
+static inline struct vop_loopback *vop_to_loopback(struct device *dev)
+{
+ return dev_get_drvdata(dev->parent);
+}
+
+static dma_addr_t
+vop_loopback_dma_map_page(struct device *dev, struct page *page,
+ unsigned long offset, size_t size,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ return page_to_phys(page) + offset;
+}
+
+static void vop_loopback_dma_unmap_page(struct device *dev, dma_addr_t dma_addr,
+ size_t size, enum dma_data_direction dir,
+ unsigned long attrs)
+{
+}
+
+static int vop_loopback_dma_mmap(struct device *dev, struct vm_area_struct *vma,
+ void *cpu_addr, dma_addr_t dma_addr, size_t size,
+ unsigned long attrs)
+{
+ return remap_pfn_range(vma, vma->vm_start,
+ PHYS_PFN(dma_addr), size,
+ vma->vm_page_prot);
+}
+
+static void *vop_loopback_dma_alloc(struct device *dev, size_t size,
+ dma_addr_t *handle, gfp_t gfp,
+ unsigned long attrs)
+{
+ void *p = (void *) __get_free_pages(gfp, get_order(size));
+
+ if (p)
+ *handle = virt_to_phys(p);
+
+ return p;
+}
+
+static void vop_loopback_dma_free(struct device *dev, size_t size,
+ void *cpu_addr, dma_addr_t handle,
+ unsigned long attrs)
+{
+ free_pages((unsigned long) (uintptr_t) cpu_addr, get_order(size));
+}
+
+static const struct dma_map_ops vop_loopback_dma_ops = {
+ .map_page = vop_loopback_dma_map_page,
+ .unmap_page = vop_loopback_dma_unmap_page,
+ .mmap = vop_loopback_dma_mmap,
+ .alloc = vop_loopback_dma_alloc,
+ .free = vop_loopback_dma_free,
+};
+
+
+static void vop_loopback_ack_interrupt(struct vop_device *vop, int num)
+{
+}
+
+static int vop_loopback_next_db(struct vop_device *vop)
+{
+ return 0;
+}
+
+static void *vop_loopback_get_dp(struct vop_device *vop)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ return loopback->dp;
+}
+
+static void __iomem *vop_loopback_get_remote_dp(struct vop_device *vop)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ return (void __iomem *) loopback->dp;
+}
+
+static struct mic_irq *
+vop_loopback_request_irq(struct vop_loopback_end *end,
+ irqreturn_t (*func)(int irq, void *data),
+ void *data, int intr_src)
+{
+ struct mic_irq *mic_irq;
+
+ mic_irq = kzalloc(sizeof(*mic_irq), GFP_KERNEL);
+ if (!mic_irq)
+ return ERR_PTR(-ENOMEM);
+
+ mic_irq->irq = intr_src;
+ mic_irq->func = func;
+ mic_irq->data = data;
+
+ mutex_lock(&end->mutex);
+ list_add(&mic_irq->list, &end->irqs);
+ mutex_unlock(&end->mutex);
+
+ return mic_irq;
+}
+
+static struct mic_irq *
+vop_loopback_request_irq_host(struct vop_device *vop,
+ irqreturn_t (*func)(int irq, void *data),
+ const char *name, void *data, int intr_src)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ return vop_loopback_request_irq(&loopback->host, func, data, intr_src);
+}
+
+static struct mic_irq *
+vop_loopback_request_irq_guest(struct vop_device *vop,
+ irqreturn_t (*func)(int irq, void *data),
+ const char *name, void *data, int intr_src)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ return vop_loopback_request_irq(&loopback->guest, func, data, intr_src);
+}
+
+static void vop_loopback_free_irq(struct vop_loopback_end *end,
+ struct mic_irq *cookie)
+{
+ mutex_lock(&end->mutex);
+ list_del(&cookie->list);
+ mutex_unlock(&end->mutex);
+
+ kfree(cookie);
+}
+
+static void vop_loopback_free_irq_host(struct vop_device *vop,
+ struct mic_irq *cookie, void *data)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ vop_loopback_free_irq(&loopback->host, cookie);
+}
+
+static void vop_loopback_free_irq_guest(struct vop_device *vop,
+ struct mic_irq *cookie, void *data)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ vop_loopback_free_irq(&loopback->guest, cookie);
+}
+
+static void vop_loopback_send_intr_host(struct vop_device *vop, int db)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ schedule_work(&loopback->guest.work);
+}
+
+static void vop_loopback_send_intr_guest(struct vop_device *vop, int db)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ schedule_work(&loopback->host.work);
+}
+
+static void __iomem *vop_loopback_ioremap(struct vop_device *vop,
+ dma_addr_t pa, size_t len)
+{
+ return (void __iomem *) memremap(pa, len, MEMREMAP_WB);
+}
+
+static void vop_loopback_iounmap(struct vop_device *vop, void __iomem *va)
+{
+ memunmap((void __force *) va);
+}
+
+static struct vop_hw_ops vop_loopback_host_ops = {
+ .request_irq = vop_loopback_request_irq_host,
+ .free_irq = vop_loopback_free_irq_host,
+ .ack_interrupt = vop_loopback_ack_interrupt,
+ .next_db = vop_loopback_next_db,
+ .get_dp = vop_loopback_get_dp,
+ .get_remote_dp = vop_loopback_get_remote_dp,
+ .send_intr = vop_loopback_send_intr_host,
+ .remap = vop_loopback_ioremap,
+ .unmap = vop_loopback_iounmap,
+};
+
+static struct vop_hw_ops vop_loopback_guest_ops = {
+ .request_irq = vop_loopback_request_irq_guest,
+ .free_irq = vop_loopback_free_irq_guest,
+ .ack_interrupt = vop_loopback_ack_interrupt,
+ .next_db = vop_loopback_next_db,
+ .get_dp = vop_loopback_get_dp,
+ .get_remote_dp = vop_loopback_get_remote_dp,
+ .send_intr = vop_loopback_send_intr_guest,
+ .remap = vop_loopback_ioremap,
+ .unmap = vop_loopback_iounmap,
+};
+
+static void vop_loopback_irq(struct work_struct *work)
+{
+ struct vop_loopback_end *end = container_of(work, struct vop_loopback_end, work);
+ struct vop_loopback *loopback = end->loopback;
+ struct mic_irq *mic_irq;
+
+ dev_dbg(loopback->dev, "%s irq\n", end->name);
+
+ mutex_lock(&end->mutex);
+ list_for_each_entry(mic_irq, &end->irqs, list) {
+ irqreturn_t ret;
+
+ dev_dbg(loopback->dev, "calling %pS\n", mic_irq->func);
+ ret = mic_irq->func(mic_irq->irq, mic_irq->data);
+ dev_dbg(loopback->dev, "%pS ret %d\n", mic_irq->func, ret);
+ }
+ mutex_unlock(&end->mutex);
+}
+
+static void vop_loopback_bootparam_init(struct vop_loopback *loopback)
+{
+ struct mic_bootparam *bootparam = loopback->dp;
+
+ bootparam->magic = cpu_to_le32(MIC_MAGIC);
+ bootparam->h2c_config_db = -1;
+ bootparam->node_id = 1;
+ bootparam->scif_host_dma_addr = 0x0;
+ bootparam->scif_card_dma_addr = 0x0;
+ bootparam->c2h_scif_db = -1;
+ bootparam->h2c_scif_db = -1;
+}
+
+static void vop_loopback_end_init(struct vop_loopback *loopback,
+ struct vop_loopback_end *end,
+ const char *name)
+{
+ end->loopback = loopback;
+ end->name = name;
+
+ INIT_WORK(&end->work, vop_loopback_irq);
+
+ INIT_LIST_HEAD(&end->irqs);
+ mutex_init(&end->mutex);
+}
+
+static int vop_loopback_probe(struct platform_device *pdev)
+{
+ struct vop_loopback *loopback;
+ int ret;
+
+ loopback = devm_kzalloc(&pdev->dev, sizeof(*loopback), GFP_KERNEL);
+ if (!loopback)
+ return -ENOMEM;
+
+ loopback->dp = (void *) devm_get_free_pages(&pdev->dev,
+ GFP_KERNEL | __GFP_ZERO,
+ get_order(MIC_DP_SIZE));
+ if (!loopback->dp)
+ return -ENOMEM;
+
+ loopback->dev = &pdev->dev;
+
+ vop_loopback_end_init(loopback, &loopback->host, "host");
+ vop_loopback_end_init(loopback, &loopback->guest, "guest");
+ vop_loopback_bootparam_init(loopback);
+
+ platform_set_drvdata(pdev, loopback);
+
+ loopback->host.vop = vop_register_device(&pdev->dev, VOP_DEV_TRNSP,
+ &vop_loopback_dma_ops,
+ &vop_loopback_host_ops, 1,
+ NULL, NULL);
+ if (IS_ERR(loopback->host.vop))
+ return PTR_ERR(loopback->host.vop);
+
+ loopback->guest.vop = vop_register_device(&pdev->dev, VOP_DEV_TRNSP,
+ &vop_loopback_dma_ops,
+ &vop_loopback_guest_ops,
+ 0, NULL, NULL);
+ if (IS_ERR(loopback->guest.vop)) {
+ ret = PTR_ERR(loopback->guest.vop);
+ goto err_unregister_host;
+ }
+
+ schedule_work(&loopback->guest.work);
+
+ return 0;
+
+err_unregister_host:
+ vop_unregister_device(loopback->host.vop);
+ return ret;
+}
+
+static int vop_loopback_remove(struct platform_device *pdev)
+{
+ struct vop_loopback *loopback = platform_get_drvdata(pdev);
+
+ vop_unregister_device(loopback->guest.vop);
+ vop_unregister_device(loopback->host.vop);
+
+ return 0;
+}
+
+static struct platform_driver vop_loopback = {
+ .probe = vop_loopback_probe,
+ .remove = vop_loopback_remove,
+ .driver = {
+ .name = "vop-loopback",
+ },
+};
+
+static struct platform_device *loopback_dev;
+
+static int __init vop_loopback_init(void)
+{
+ int ret;
+
+ loopback_dev = platform_device_register_simple("vop-loopback", 0,
+ NULL, 0);
+ if (IS_ERR(loopback_dev))
+ return PTR_ERR(loopback_dev);
+
+ ret = platform_driver_register(&vop_loopback);
+ if (ret)
+ goto err_remove_dev;
+
+ return 0;
+
+err_remove_dev:
+ platform_device_unregister(loopback_dev);
+ return ret;
+}
+
+static void __exit vop_loopback_exit(void)
+{
+ platform_driver_unregister(&vop_loopback);
+ platform_device_unregister(loopback_dev);
+}
+
+module_init(vop_loopback_init);
+module_exit(vop_loopback_exit);
+
+MODULE_LICENSE("GPL v2");
--
2.20.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v2 char-misc-next 4/7] vop: Add loopback driver
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
` (5 preceding siblings ...)
2019-02-22 15:30 ` [PATCH v2 char-misc-next 4/7] mic: vop: Add loopback driver Vincent Whitchurch
@ 2019-02-22 15:30 ` Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 5/7] mic: vop: Fix init race with shared interrupts Vincent Whitchurch
` (2 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
Add a loopback driver to allow testing and evaluation of the VOP
framework without special hardware. The host and the guest will run
under the same kernel.
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
drivers/misc/mic/Kconfig | 10 +
drivers/misc/mic/vop/Makefile | 2 +
drivers/misc/mic/vop/vop_loopback.c | 382 ++++++++++++++++++++++++++++
3 files changed, 394 insertions(+)
create mode 100644 drivers/misc/mic/vop/vop_loopback.c
diff --git a/drivers/misc/mic/Kconfig b/drivers/misc/mic/Kconfig
index 242dcee14689..2e2f745afb3a 100644
--- a/drivers/misc/mic/Kconfig
+++ b/drivers/misc/mic/Kconfig
@@ -148,6 +148,16 @@ config VOP
OS and tools for MIC to use with this driver are available from
<http://software.intel.com/en-us/mic-developer>.
+config VOP_LOOPBACK
+ tristate "VOP loopback driver"
+ depends on VOP
+ help
+ This enables a loopback driver to test and evaluate the VOP
+ infrastructure without actual PCIe hardware. The host and the guest
+ sides run under the same kernel.
+
+ If unsure, say N.
+
if VOP
source "drivers/vhost/Kconfig.vringh"
endif
diff --git a/drivers/misc/mic/vop/Makefile b/drivers/misc/mic/vop/Makefile
index 78819c8999f1..a6ead25c4418 100644
--- a/drivers/misc/mic/vop/Makefile
+++ b/drivers/misc/mic/vop/Makefile
@@ -7,3 +7,5 @@ obj-m := vop.o
vop-objs += vop_main.o
vop-objs += vop_debugfs.o
vop-objs += vop_vringh.o
+
+obj-$(CONFIG_VOP_LOOPBACK) += vop_loopback.o
diff --git a/drivers/misc/mic/vop/vop_loopback.c b/drivers/misc/mic/vop/vop_loopback.c
new file mode 100644
index 000000000000..76d787db3d3e
--- /dev/null
+++ b/drivers/misc/mic/vop/vop_loopback.c
@@ -0,0 +1,382 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2019 Axis Communications AB
+ */
+
+#include <linux/platform_device.h>
+#include <linux/dma-direction.h>
+#include <linux/dma-mapping.h>
+#include <linux/mic_common.h>
+#include <linux/of_device.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+#include <linux/io.h>
+
+#include "../bus/vop_bus.h"
+
+struct mic_irq {
+ struct list_head list;
+ int irq;
+ irqreturn_t (*func)(int irq, void *data);
+ void *data;
+};
+
+struct vop_loopback_end {
+ struct vop_loopback *loopback;
+ const char *name;
+ struct vop_device *vop;
+ struct list_head irqs;
+ struct mutex mutex;
+ struct work_struct work;
+};
+
+struct vop_loopback {
+ struct device *dev;
+ void *dp;
+ struct vop_loopback_end host;
+ struct vop_loopback_end guest;
+};
+
+static inline struct vop_loopback *vop_to_loopback(struct device *dev)
+{
+ return dev_get_drvdata(dev->parent);
+}
+
+static dma_addr_t
+vop_loopback_dma_map_page(struct device *dev, struct page *page,
+ unsigned long offset, size_t size,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ return page_to_phys(page) + offset;
+}
+
+static void vop_loopback_dma_unmap_page(struct device *dev, dma_addr_t dma_addr,
+ size_t size, enum dma_data_direction dir,
+ unsigned long attrs)
+{
+}
+
+static int vop_loopback_dma_mmap(struct device *dev, struct vm_area_struct *vma,
+ void *cpu_addr, dma_addr_t dma_addr, size_t size,
+ unsigned long attrs)
+{
+ return remap_pfn_range(vma, vma->vm_start,
+ PHYS_PFN(dma_addr), size,
+ vma->vm_page_prot);
+}
+
+static void *vop_loopback_dma_alloc(struct device *dev, size_t size,
+ dma_addr_t *handle, gfp_t gfp,
+ unsigned long attrs)
+{
+ void *p = (void *) __get_free_pages(gfp, get_order(size));
+
+ if (p)
+ *handle = virt_to_phys(p);
+
+ return p;
+}
+
+static void vop_loopback_dma_free(struct device *dev, size_t size,
+ void *cpu_addr, dma_addr_t handle,
+ unsigned long attrs)
+{
+ free_pages((unsigned long) (uintptr_t) cpu_addr, get_order(size));
+}
+
+static const struct dma_map_ops vop_loopback_dma_ops = {
+ .map_page = vop_loopback_dma_map_page,
+ .unmap_page = vop_loopback_dma_unmap_page,
+ .mmap = vop_loopback_dma_mmap,
+ .alloc = vop_loopback_dma_alloc,
+ .free = vop_loopback_dma_free,
+};
+
+
+static void vop_loopback_ack_interrupt(struct vop_device *vop, int num)
+{
+}
+
+static int vop_loopback_next_db(struct vop_device *vop)
+{
+ return 0;
+}
+
+static void *vop_loopback_get_dp(struct vop_device *vop)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ return loopback->dp;
+}
+
+static void __iomem *vop_loopback_get_remote_dp(struct vop_device *vop)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ return (void __iomem *) loopback->dp;
+}
+
+static struct mic_irq *
+vop_loopback_request_irq(struct vop_loopback_end *end,
+ irqreturn_t (*func)(int irq, void *data),
+ void *data, int intr_src)
+{
+ struct mic_irq *mic_irq;
+
+ mic_irq = kzalloc(sizeof(*mic_irq), GFP_KERNEL);
+ if (!mic_irq)
+ return ERR_PTR(-ENOMEM);
+
+ mic_irq->irq = intr_src;
+ mic_irq->func = func;
+ mic_irq->data = data;
+
+ mutex_lock(&end->mutex);
+ list_add(&mic_irq->list, &end->irqs);
+ mutex_unlock(&end->mutex);
+
+ return mic_irq;
+}
+
+static struct mic_irq *
+vop_loopback_request_irq_host(struct vop_device *vop,
+ irqreturn_t (*func)(int irq, void *data),
+ const char *name, void *data, int intr_src)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ return vop_loopback_request_irq(&loopback->host, func, data, intr_src);
+}
+
+static struct mic_irq *
+vop_loopback_request_irq_guest(struct vop_device *vop,
+ irqreturn_t (*func)(int irq, void *data),
+ const char *name, void *data, int intr_src)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ return vop_loopback_request_irq(&loopback->guest, func, data, intr_src);
+}
+
+static void vop_loopback_free_irq(struct vop_loopback_end *end,
+ struct mic_irq *cookie)
+{
+ mutex_lock(&end->mutex);
+ list_del(&cookie->list);
+ mutex_unlock(&end->mutex);
+
+ kfree(cookie);
+}
+
+static void vop_loopback_free_irq_host(struct vop_device *vop,
+ struct mic_irq *cookie, void *data)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ vop_loopback_free_irq(&loopback->host, cookie);
+}
+
+static void vop_loopback_free_irq_guest(struct vop_device *vop,
+ struct mic_irq *cookie, void *data)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ vop_loopback_free_irq(&loopback->guest, cookie);
+}
+
+static void vop_loopback_send_intr_host(struct vop_device *vop, int db)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ schedule_work(&loopback->guest.work);
+}
+
+static void vop_loopback_send_intr_guest(struct vop_device *vop, int db)
+{
+ struct vop_loopback *loopback = vop_to_loopback(&vop->dev);
+
+ schedule_work(&loopback->host.work);
+}
+
+static void __iomem *vop_loopback_ioremap(struct vop_device *vop,
+ dma_addr_t pa, size_t len)
+{
+ return (void __iomem *) memremap(pa, len, MEMREMAP_WB);
+}
+
+static void vop_loopback_iounmap(struct vop_device *vop, void __iomem *va)
+{
+ memunmap((void __force *) va);
+}
+
+static struct vop_hw_ops vop_loopback_host_ops = {
+ .request_irq = vop_loopback_request_irq_host,
+ .free_irq = vop_loopback_free_irq_host,
+ .ack_interrupt = vop_loopback_ack_interrupt,
+ .next_db = vop_loopback_next_db,
+ .get_dp = vop_loopback_get_dp,
+ .get_remote_dp = vop_loopback_get_remote_dp,
+ .send_intr = vop_loopback_send_intr_host,
+ .remap = vop_loopback_ioremap,
+ .unmap = vop_loopback_iounmap,
+};
+
+static struct vop_hw_ops vop_loopback_guest_ops = {
+ .request_irq = vop_loopback_request_irq_guest,
+ .free_irq = vop_loopback_free_irq_guest,
+ .ack_interrupt = vop_loopback_ack_interrupt,
+ .next_db = vop_loopback_next_db,
+ .get_dp = vop_loopback_get_dp,
+ .get_remote_dp = vop_loopback_get_remote_dp,
+ .send_intr = vop_loopback_send_intr_guest,
+ .remap = vop_loopback_ioremap,
+ .unmap = vop_loopback_iounmap,
+};
+
+static void vop_loopback_irq(struct work_struct *work)
+{
+ struct vop_loopback_end *end = container_of(work, struct vop_loopback_end, work);
+ struct vop_loopback *loopback = end->loopback;
+ struct mic_irq *mic_irq;
+
+ dev_dbg(loopback->dev, "%s irq\n", end->name);
+
+ mutex_lock(&end->mutex);
+ list_for_each_entry(mic_irq, &end->irqs, list) {
+ irqreturn_t ret;
+
+ dev_dbg(loopback->dev, "calling %pS\n", mic_irq->func);
+ ret = mic_irq->func(mic_irq->irq, mic_irq->data);
+ dev_dbg(loopback->dev, "%pS ret %d\n", mic_irq->func, ret);
+ }
+ mutex_unlock(&end->mutex);
+}
+
+static void vop_loopback_bootparam_init(struct vop_loopback *loopback)
+{
+ struct mic_bootparam *bootparam = loopback->dp;
+
+ bootparam->magic = cpu_to_le32(MIC_MAGIC);
+ bootparam->h2c_config_db = -1;
+ bootparam->node_id = 1;
+ bootparam->scif_host_dma_addr = 0x0;
+ bootparam->scif_card_dma_addr = 0x0;
+ bootparam->c2h_scif_db = -1;
+ bootparam->h2c_scif_db = -1;
+}
+
+static void vop_loopback_end_init(struct vop_loopback *loopback,
+ struct vop_loopback_end *end,
+ const char *name)
+{
+ end->loopback = loopback;
+ end->name = name;
+
+ INIT_WORK(&end->work, vop_loopback_irq);
+
+ INIT_LIST_HEAD(&end->irqs);
+ mutex_init(&end->mutex);
+}
+
+static int vop_loopback_probe(struct platform_device *pdev)
+{
+ struct vop_loopback *loopback;
+ int ret;
+
+ loopback = devm_kzalloc(&pdev->dev, sizeof(*loopback), GFP_KERNEL);
+ if (!loopback)
+ return -ENOMEM;
+
+ loopback->dp = (void *) devm_get_free_pages(&pdev->dev,
+ GFP_KERNEL | __GFP_ZERO,
+ get_order(MIC_DP_SIZE));
+ if (!loopback->dp)
+ return -ENOMEM;
+
+ loopback->dev = &pdev->dev;
+
+ vop_loopback_end_init(loopback, &loopback->host, "host");
+ vop_loopback_end_init(loopback, &loopback->guest, "guest");
+ vop_loopback_bootparam_init(loopback);
+
+ platform_set_drvdata(pdev, loopback);
+
+ loopback->host.vop = vop_register_device(&pdev->dev, VOP_DEV_TRNSP,
+ &vop_loopback_dma_ops,
+ &vop_loopback_host_ops, 1,
+ NULL, NULL);
+ if (IS_ERR(loopback->host.vop))
+ return PTR_ERR(loopback->host.vop);
+
+ loopback->guest.vop = vop_register_device(&pdev->dev, VOP_DEV_TRNSP,
+ &vop_loopback_dma_ops,
+ &vop_loopback_guest_ops,
+ 0, NULL, NULL);
+ if (IS_ERR(loopback->guest.vop)) {
+ ret = PTR_ERR(loopback->guest.vop);
+ goto err_unregister_host;
+ }
+
+ schedule_work(&loopback->guest.work);
+
+ return 0;
+
+err_unregister_host:
+ vop_unregister_device(loopback->host.vop);
+ return ret;
+}
+
+static int vop_loopback_remove(struct platform_device *pdev)
+{
+ struct vop_loopback *loopback = platform_get_drvdata(pdev);
+
+ vop_unregister_device(loopback->guest.vop);
+ vop_unregister_device(loopback->host.vop);
+
+ return 0;
+}
+
+static struct platform_driver vop_loopback = {
+ .probe = vop_loopback_probe,
+ .remove = vop_loopback_remove,
+ .driver = {
+ .name = "vop-loopback",
+ },
+};
+
+static struct platform_device *loopback_dev;
+
+static int __init vop_loopback_init(void)
+{
+ int ret;
+
+ loopback_dev = platform_device_register_simple("vop-loopback", 0,
+ NULL, 0);
+ if (IS_ERR(loopback_dev))
+ return PTR_ERR(loopback_dev);
+
+ ret = platform_driver_register(&vop_loopback);
+ if (ret)
+ goto err_remove_dev;
+
+ return 0;
+
+err_remove_dev:
+ platform_device_unregister(loopback_dev);
+ return ret;
+}
+
+static void __exit vop_loopback_exit(void)
+{
+ platform_driver_unregister(&vop_loopback);
+ platform_device_unregister(loopback_dev);
+}
+
+module_init(vop_loopback_init);
+module_exit(vop_loopback_exit);
+
+MODULE_LICENSE("GPL v2");
--
2.20.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v2 char-misc-next 5/7] mic: vop: Fix init race with shared interrupts
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
` (6 preceding siblings ...)
2019-02-22 15:30 ` [PATCH v2 char-misc-next 4/7] " Vincent Whitchurch
@ 2019-02-22 15:30 ` Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 6/7] samples: mic: Split out vop code from mpssd Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 7/7] samples: mic: Add sample VOP userspace Vincent Whitchurch
9 siblings, 0 replies; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
If a vop virtio device's interrupt is shared with others, there is a
window where the interrupt can hit and the ->vqs list can be attempted
to be traversed before register_virtio_device() has initialized it,
leading to a NULL pointer dereference in vop_virtio_intr_handler().
Fix this by keeping a local list of virtqueues in this driver and using
that instead of the list inside the struct virtio_device, similar to how
virtio-pci handles this.
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
drivers/misc/mic/vop/vop_main.c | 42 ++++++++++++++++++++++++++++++---
1 file changed, 39 insertions(+), 3 deletions(-)
diff --git a/drivers/misc/mic/vop/vop_main.c b/drivers/misc/mic/vop/vop_main.c
index e37b2c2152a2..6764feea6f55 100644
--- a/drivers/misc/mic/vop/vop_main.c
+++ b/drivers/misc/mic/vop/vop_main.c
@@ -40,6 +40,11 @@
#define VOP_MAX_VRINGS 4
+struct vop_vq {
+ struct virtqueue *vq;
+ struct list_head list;
+};
+
/*
* _vop_vdev - Allocated per virtio device instance injected by the peer.
*
@@ -68,6 +73,9 @@ struct _vop_vdev {
int used_size[VOP_MAX_VRINGS];
struct completion reset_done;
struct mic_irq *virtio_cookie;
+ struct vop_vq *vqs;
+ struct list_head virtqueues;
+ spinlock_t virtqueues_lock;
int c2h_vdev_db;
int h2c_vdev_db;
int dnode;
@@ -264,6 +272,12 @@ static void vop_del_vq(struct virtqueue *vq, int n)
{
struct _vop_vdev *vdev = to_vopvdev(vq->vdev);
struct vop_device *vpdev = vdev->vpdev;
+ struct vop_vq *vopvq = &vdev->vqs[n];
+ unsigned long flags;
+
+ spin_lock_irqsave(&vdev->virtqueues_lock, flags);
+ list_del(&vopvq->list);
+ spin_unlock_irqrestore(&vdev->virtqueues_lock, flags);
dma_unmap_single(&vpdev->dev, vdev->used[n],
vdev->used_size[n], DMA_BIDIRECTIONAL);
@@ -284,6 +298,9 @@ static void vop_del_vqs(struct virtio_device *dev)
list_for_each_entry_safe(vq, n, &dev->vqs, list)
vop_del_vq(vq, idx++);
+
+ kfree(vdev->vqs);
+ vdev->vqs = NULL;
}
static struct virtqueue *vop_new_virtqueue(unsigned int index,
@@ -411,7 +428,13 @@ static int vop_find_vqs(struct virtio_device *dev, unsigned nvqs,
if (nvqs > ioread8(&vdev->desc->num_vq))
return -ENOENT;
+ vdev->vqs = kcalloc(nvqs, sizeof(*vdev->vqs), GFP_KERNEL);
+ if (!vdev->vqs)
+ return -ENOMEM;
+
for (i = 0; i < nvqs; ++i) {
+ unsigned long flags;
+
if (!names[i]) {
vqs[i] = NULL;
continue;
@@ -425,6 +448,12 @@ static int vop_find_vqs(struct virtio_device *dev, unsigned nvqs,
err = PTR_ERR(vqs[i]);
goto error;
}
+
+ vdev->vqs[i].vq = vqs[i];
+
+ spin_lock_irqsave(&vdev->virtqueues_lock, flags);
+ list_add(&vdev->vqs[i].list, &vdev->virtqueues);
+ spin_unlock_irqrestore(&vdev->virtqueues_lock, flags);
}
iowrite8(1, &dc->used_address_updated);
@@ -468,13 +497,17 @@ static struct virtio_config_ops vop_vq_config_ops = {
static irqreturn_t vop_virtio_intr_handler(int irq, void *data)
{
+ unsigned long flags;
struct _vop_vdev *vdev = data;
struct vop_device *vpdev = vdev->vpdev;
- struct virtqueue *vq;
+ struct vop_vq *vopvq;
vpdev->hw_ops->ack_interrupt(vpdev, vdev->h2c_vdev_db);
- list_for_each_entry(vq, &vdev->vdev.vqs, list)
- vring_interrupt(0, vq);
+
+ spin_lock_irqsave(&vdev->virtqueues_lock, flags);
+ list_for_each_entry(vopvq, &vdev->virtqueues, list)
+ vring_interrupt(0, vopvq->vq);
+ spin_unlock_irqrestore(&vdev->virtqueues_lock, flags);
return IRQ_HANDLED;
}
@@ -516,6 +549,9 @@ static int _vop_add_device(struct mic_device_desc __iomem *d,
vdev->vdev.priv = (void *)(unsigned long)dnode;
init_completion(&vdev->reset_done);
+ INIT_LIST_HEAD(&vdev->virtqueues);
+ spin_lock_init(&vdev->virtqueues_lock);
+
vdev->h2c_vdev_db = vpdev->hw_ops->next_db(vpdev);
vdev->virtio_cookie = vpdev->hw_ops->request_irq(vpdev,
vop_virtio_intr_handler, "virtio intr",
--
2.20.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v2 char-misc-next 6/7] samples: mic: Split out vop code from mpssd
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
` (7 preceding siblings ...)
2019-02-22 15:30 ` [PATCH v2 char-misc-next 5/7] mic: vop: Fix init race with shared interrupts Vincent Whitchurch
@ 2019-02-22 15:30 ` Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 7/7] samples: mic: Add sample VOP userspace Vincent Whitchurch
9 siblings, 0 replies; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
mpssd.c contains both code to boot the MIC hardware and code to serve
virtio devices over vop. Split the vop-related code to a separate file
so that it can be used by other hardware and vop-loopback.
Except for changes to change_virtblk_backend() and the introduction of
serve_virtio(), the rest is just movement of unmodified code between
files.
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
samples/mic/mpssd/Makefile | 4 +-
samples/mic/mpssd/mpssd.c | 1317 +---------------------------------
samples/mic/mpssd/mpssd.h | 1 +
samples/mic/mpssd/vop.c | 1357 ++++++++++++++++++++++++++++++++++++
4 files changed, 1364 insertions(+), 1315 deletions(-)
create mode 100644 samples/mic/mpssd/vop.c
diff --git a/samples/mic/mpssd/Makefile b/samples/mic/mpssd/Makefile
index a7a6e0c70424..bc94054dbe2b 100644
--- a/samples/mic/mpssd/Makefile
+++ b/samples/mic/mpssd/Makefile
@@ -14,8 +14,8 @@ CFLAGS += -DDEBUG=$(DEBUG)
endif
all: $(PROGS)
-mpssd: mpssd.c sysfs.c
- $(CC) $(CFLAGS) mpssd.c sysfs.c -o mpssd -lpthread
+mpssd: mpssd.c sysfs.c vop.o
+ $(CC) $(CFLAGS) $^ -o $@ -lpthread
install:
install mpssd /usr/sbin/mpssd
diff --git a/samples/mic/mpssd/mpssd.c b/samples/mic/mpssd/mpssd.c
index f42ce551bb48..5e4233c1ff93 100644
--- a/samples/mic/mpssd/mpssd.c
+++ b/samples/mic/mpssd/mpssd.c
@@ -31,1292 +31,18 @@
#include <features.h>
#include <sys/types.h>
#include <sys/stat.h>
-#include <sys/mman.h>
-#include <sys/socket.h>
-#include <linux/virtio_ring.h>
-#include <linux/virtio_net.h>
-#include <linux/virtio_console.h>
-#include <linux/virtio_blk.h>
-#include <linux/version.h>
#include "mpssd.h"
#include <linux/mic_ioctl.h>
#include <linux/mic_common.h>
#include <tools/endian.h>
-static void *init_mic(void *arg);
-
static FILE *logfp;
-static struct mic_info mic_list;
-
-#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
-
-#define min_t(type, x, y) ({ \
- type __min1 = (x); \
- type __min2 = (y); \
- __min1 < __min2 ? __min1 : __min2; })
-
-/* align addr on a size boundary - adjust address up/down if needed */
-#define _ALIGN_DOWN(addr, size) ((addr)&(~((size)-1)))
-#define _ALIGN_UP(addr, size) _ALIGN_DOWN(addr + size - 1, size)
-
-/* align addr on a size boundary - adjust address up if needed */
-#define _ALIGN(addr, size) _ALIGN_UP(addr, size)
-
-/* to align the pointer to the (next) page boundary */
-#define PAGE_ALIGN(addr) _ALIGN(addr, PAGE_SIZE)
-
-#define READ_ONCE(x) (*(volatile typeof(x) *)&(x))
-
-#define GSO_ENABLED 1
-#define MAX_GSO_SIZE (64 * 1024)
-#define ETH_H_LEN 14
-#define MAX_NET_PKT_SIZE (_ALIGN_UP(MAX_GSO_SIZE + ETH_H_LEN, 64))
-#define MIC_DEVICE_PAGE_END 0x1000
-
-#ifndef VIRTIO_NET_HDR_F_DATA_VALID
-#define VIRTIO_NET_HDR_F_DATA_VALID 2 /* Csum is valid */
-#endif
-static struct {
- struct mic_device_desc dd;
- struct mic_vqconfig vqconfig[2];
- __u32 host_features, guest_acknowledgements;
- struct virtio_console_config cons_config;
-} virtcons_dev_page = {
- .dd = {
- .type = VIRTIO_ID_CONSOLE,
- .num_vq = ARRAY_SIZE(virtcons_dev_page.vqconfig),
- .feature_len = sizeof(virtcons_dev_page.host_features),
- .config_len = sizeof(virtcons_dev_page.cons_config),
- },
- .vqconfig[0] = {
- .num = htole16(MIC_VRING_ENTRIES),
- },
- .vqconfig[1] = {
- .num = htole16(MIC_VRING_ENTRIES),
- },
-};
-
-static struct {
- struct mic_device_desc dd;
- struct mic_vqconfig vqconfig[2];
- __u32 host_features, guest_acknowledgements;
- struct virtio_net_config net_config;
-} virtnet_dev_page = {
- .dd = {
- .type = VIRTIO_ID_NET,
- .num_vq = ARRAY_SIZE(virtnet_dev_page.vqconfig),
- .feature_len = sizeof(virtnet_dev_page.host_features),
- .config_len = sizeof(virtnet_dev_page.net_config),
- },
- .vqconfig[0] = {
- .num = htole16(MIC_VRING_ENTRIES),
- },
- .vqconfig[1] = {
- .num = htole16(MIC_VRING_ENTRIES),
- },
-#if GSO_ENABLED
- .host_features = htole32(
- 1 << VIRTIO_NET_F_CSUM |
- 1 << VIRTIO_NET_F_GSO |
- 1 << VIRTIO_NET_F_GUEST_TSO4 |
- 1 << VIRTIO_NET_F_GUEST_TSO6 |
- 1 << VIRTIO_NET_F_GUEST_ECN),
-#else
- .host_features = 0,
-#endif
-};
-
-static const char *mic_config_dir = "/etc/mpss";
-static const char *virtblk_backend = "VIRTBLK_BACKEND";
-static struct {
- struct mic_device_desc dd;
- struct mic_vqconfig vqconfig[1];
- __u32 host_features, guest_acknowledgements;
- struct virtio_blk_config blk_config;
-} virtblk_dev_page = {
- .dd = {
- .type = VIRTIO_ID_BLOCK,
- .num_vq = ARRAY_SIZE(virtblk_dev_page.vqconfig),
- .feature_len = sizeof(virtblk_dev_page.host_features),
- .config_len = sizeof(virtblk_dev_page.blk_config),
- },
- .vqconfig[0] = {
- .num = htole16(MIC_VRING_ENTRIES),
- },
- .host_features =
- htole32(1<<VIRTIO_BLK_F_SEG_MAX),
- .blk_config = {
- .seg_max = htole32(MIC_VRING_ENTRIES - 2),
- .capacity = htole64(0),
- }
-};
+static void *init_mic(void *arg);
+static struct mic_info mic_list;
static char *myname;
-static int
-tap_configure(struct mic_info *mic, char *dev)
-{
- pid_t pid;
- char *ifargv[7];
- char ipaddr[IFNAMSIZ];
- int ret = 0;
-
- pid = fork();
- if (pid == 0) {
- ifargv[0] = "ip";
- ifargv[1] = "link";
- ifargv[2] = "set";
- ifargv[3] = dev;
- ifargv[4] = "up";
- ifargv[5] = NULL;
- mpsslog("Configuring %s\n", dev);
- ret = execvp("ip", ifargv);
- if (ret < 0) {
- mpsslog("%s execvp failed errno %s\n",
- mic->name, strerror(errno));
- return ret;
- }
- }
- if (pid < 0) {
- mpsslog("%s fork failed errno %s\n",
- mic->name, strerror(errno));
- return ret;
- }
-
- ret = waitpid(pid, NULL, 0);
- if (ret < 0) {
- mpsslog("%s waitpid failed errno %s\n",
- mic->name, strerror(errno));
- return ret;
- }
-
- snprintf(ipaddr, IFNAMSIZ, "172.31.%d.254/24", mic->id + 1);
-
- pid = fork();
- if (pid == 0) {
- ifargv[0] = "ip";
- ifargv[1] = "addr";
- ifargv[2] = "add";
- ifargv[3] = ipaddr;
- ifargv[4] = "dev";
- ifargv[5] = dev;
- ifargv[6] = NULL;
- mpsslog("Configuring %s ipaddr %s\n", dev, ipaddr);
- ret = execvp("ip", ifargv);
- if (ret < 0) {
- mpsslog("%s execvp failed errno %s\n",
- mic->name, strerror(errno));
- return ret;
- }
- }
- if (pid < 0) {
- mpsslog("%s fork failed errno %s\n",
- mic->name, strerror(errno));
- return ret;
- }
-
- ret = waitpid(pid, NULL, 0);
- if (ret < 0) {
- mpsslog("%s waitpid failed errno %s\n",
- mic->name, strerror(errno));
- return ret;
- }
- mpsslog("MIC name %s %s %d DONE!\n",
- mic->name, __func__, __LINE__);
- return 0;
-}
-
-static int tun_alloc(struct mic_info *mic, char *dev)
-{
- struct ifreq ifr;
- int fd, err;
-#if GSO_ENABLED
- unsigned offload;
-#endif
- fd = open("/dev/net/tun", O_RDWR);
- if (fd < 0) {
- mpsslog("Could not open /dev/net/tun %s\n", strerror(errno));
- goto done;
- }
-
- memset(&ifr, 0, sizeof(ifr));
-
- ifr.ifr_flags = IFF_TAP | IFF_NO_PI | IFF_VNET_HDR;
- if (*dev)
- strncpy(ifr.ifr_name, dev, IFNAMSIZ);
-
- err = ioctl(fd, TUNSETIFF, (void *)&ifr);
- if (err < 0) {
- mpsslog("%s %s %d TUNSETIFF failed %s\n",
- mic->name, __func__, __LINE__, strerror(errno));
- close(fd);
- return err;
- }
-#if GSO_ENABLED
- offload = TUN_F_CSUM | TUN_F_TSO4 | TUN_F_TSO6 | TUN_F_TSO_ECN;
-
- err = ioctl(fd, TUNSETOFFLOAD, offload);
- if (err < 0) {
- mpsslog("%s %s %d TUNSETOFFLOAD failed %s\n",
- mic->name, __func__, __LINE__, strerror(errno));
- close(fd);
- return err;
- }
-#endif
- strcpy(dev, ifr.ifr_name);
- mpsslog("Created TAP %s\n", dev);
-done:
- return fd;
-}
-
-#define NET_FD_VIRTIO_NET 0
-#define NET_FD_TUN 1
-#define MAX_NET_FD 2
-
-static void set_dp(struct mic_info *mic, int type, void *dp)
-{
- switch (type) {
- case VIRTIO_ID_CONSOLE:
- mic->mic_console.console_dp = dp;
- return;
- case VIRTIO_ID_NET:
- mic->mic_net.net_dp = dp;
- return;
- case VIRTIO_ID_BLOCK:
- mic->mic_virtblk.block_dp = dp;
- return;
- }
- mpsslog("%s %s %d not found\n", mic->name, __func__, type);
- assert(0);
-}
-
-static void *get_dp(struct mic_info *mic, int type)
-{
- switch (type) {
- case VIRTIO_ID_CONSOLE:
- return mic->mic_console.console_dp;
- case VIRTIO_ID_NET:
- return mic->mic_net.net_dp;
- case VIRTIO_ID_BLOCK:
- return mic->mic_virtblk.block_dp;
- }
- mpsslog("%s %s %d not found\n", mic->name, __func__, type);
- assert(0);
- return NULL;
-}
-
-static struct mic_device_desc *get_device_desc(struct mic_info *mic, int type)
-{
- struct mic_device_desc *d;
- int i;
- void *dp = get_dp(mic, type);
-
- for (i = sizeof(struct mic_bootparam); i < PAGE_SIZE;
- i += mic_total_desc_size(d)) {
- d = dp + i;
-
- /* End of list */
- if (d->type == 0)
- break;
-
- if (d->type == -1)
- continue;
-
- mpsslog("%s %s d-> type %d d %p\n",
- mic->name, __func__, d->type, d);
-
- if (d->type == (__u8)type)
- return d;
- }
- mpsslog("%s %s %d not found\n", mic->name, __func__, type);
- return NULL;
-}
-
-/* See comments in vhost.c for explanation of next_desc() */
-static unsigned next_desc(struct vring_desc *desc)
-{
- unsigned int next;
-
- if (!(le16toh(desc->flags) & VRING_DESC_F_NEXT))
- return -1U;
- next = le16toh(desc->next);
- return next;
-}
-
-/* Sum up all the IOVEC length */
-static ssize_t
-sum_iovec_len(struct mic_copy_desc *copy)
-{
- ssize_t sum = 0;
- unsigned int i;
-
- for (i = 0; i < copy->iovcnt; i++)
- sum += copy->iov[i].iov_len;
- return sum;
-}
-
-static inline void verify_out_len(struct mic_info *mic,
- struct mic_copy_desc *copy)
-{
- if (copy->out_len != sum_iovec_len(copy)) {
- mpsslog("%s %s %d BUG copy->out_len 0x%x len 0x%zx\n",
- mic->name, __func__, __LINE__,
- copy->out_len, sum_iovec_len(copy));
- assert(copy->out_len == sum_iovec_len(copy));
- }
-}
-
-/* Display an iovec */
-static void
-disp_iovec(struct mic_info *mic, struct mic_copy_desc *copy,
- const char *s, int line)
-{
- unsigned int i;
-
- for (i = 0; i < copy->iovcnt; i++)
- mpsslog("%s %s %d copy->iov[%d] addr %p len 0x%zx\n",
- mic->name, s, line, i,
- copy->iov[i].iov_base, copy->iov[i].iov_len);
-}
-
-static inline __u16 read_avail_idx(struct mic_vring *vr)
-{
- return READ_ONCE(vr->info->avail_idx);
-}
-
-static inline void txrx_prepare(int type, bool tx, struct mic_vring *vr,
- struct mic_copy_desc *copy, ssize_t len)
-{
- copy->vr_idx = tx ? 0 : 1;
- copy->update_used = true;
- if (type == VIRTIO_ID_NET)
- copy->iov[1].iov_len = len - sizeof(struct virtio_net_hdr);
- else
- copy->iov[0].iov_len = len;
-}
-
-/* Central API which triggers the copies */
-static int
-mic_virtio_copy(struct mic_info *mic, int fd,
- struct mic_vring *vr, struct mic_copy_desc *copy)
-{
- int ret;
-
- ret = ioctl(fd, MIC_VIRTIO_COPY_DESC, copy);
- if (ret) {
- mpsslog("%s %s %d errno %s ret %d\n",
- mic->name, __func__, __LINE__,
- strerror(errno), ret);
- }
- return ret;
-}
-
-static inline unsigned _vring_size(unsigned int num, unsigned long align)
-{
- return ((sizeof(struct vring_desc) * num + sizeof(__u16) * (3 + num)
- + align - 1) & ~(align - 1))
- + sizeof(__u16) * 3 + sizeof(struct vring_used_elem) * num;
-}
-
-/*
- * This initialization routine requires at least one
- * vring i.e. vr0. vr1 is optional.
- */
-static void *
-init_vr(struct mic_info *mic, int fd, int type,
- struct mic_vring *vr0, struct mic_vring *vr1, int num_vq)
-{
- int vr_size;
- char *va;
-
- vr_size = PAGE_ALIGN(_vring_size(MIC_VRING_ENTRIES,
- MIC_VIRTIO_RING_ALIGN) +
- sizeof(struct _mic_vring_info));
- va = mmap(NULL, MIC_DEVICE_PAGE_END + vr_size * num_vq,
- PROT_READ, MAP_SHARED, fd, 0);
- if (MAP_FAILED == va) {
- mpsslog("%s %s %d mmap failed errno %s\n",
- mic->name, __func__, __LINE__,
- strerror(errno));
- goto done;
- }
- set_dp(mic, type, va);
- vr0->va = (struct mic_vring *)&va[MIC_DEVICE_PAGE_END];
- vr0->info = vr0->va +
- _vring_size(MIC_VRING_ENTRIES, MIC_VIRTIO_RING_ALIGN);
- vring_init(&vr0->vr,
- MIC_VRING_ENTRIES, vr0->va, MIC_VIRTIO_RING_ALIGN);
- mpsslog("%s %s vr0 %p vr0->info %p vr_size 0x%x vring 0x%x ",
- __func__, mic->name, vr0->va, vr0->info, vr_size,
- _vring_size(MIC_VRING_ENTRIES, MIC_VIRTIO_RING_ALIGN));
- mpsslog("magic 0x%x expected 0x%x\n",
- le32toh(vr0->info->magic), MIC_MAGIC + type);
- assert(le32toh(vr0->info->magic) == MIC_MAGIC + type);
- if (vr1) {
- vr1->va = (struct mic_vring *)
- &va[MIC_DEVICE_PAGE_END + vr_size];
- vr1->info = vr1->va + _vring_size(MIC_VRING_ENTRIES,
- MIC_VIRTIO_RING_ALIGN);
- vring_init(&vr1->vr,
- MIC_VRING_ENTRIES, vr1->va, MIC_VIRTIO_RING_ALIGN);
- mpsslog("%s %s vr1 %p vr1->info %p vr_size 0x%x vring 0x%x ",
- __func__, mic->name, vr1->va, vr1->info, vr_size,
- _vring_size(MIC_VRING_ENTRIES, MIC_VIRTIO_RING_ALIGN));
- mpsslog("magic 0x%x expected 0x%x\n",
- le32toh(vr1->info->magic), MIC_MAGIC + type + 1);
- assert(le32toh(vr1->info->magic) == MIC_MAGIC + type + 1);
- }
-done:
- return va;
-}
-
-static int
-wait_for_card_driver(struct mic_info *mic, int fd, int type)
-{
- struct pollfd pollfd;
- int err;
- struct mic_device_desc *desc = get_device_desc(mic, type);
- __u8 prev_status;
-
- if (!desc)
- return -ENODEV;
- prev_status = desc->status;
- pollfd.fd = fd;
- mpsslog("%s %s Waiting .... desc-> type %d status 0x%x\n",
- mic->name, __func__, type, desc->status);
-
- while (1) {
- pollfd.events = POLLIN;
- pollfd.revents = 0;
- err = poll(&pollfd, 1, -1);
- if (err < 0) {
- mpsslog("%s %s poll failed %s\n",
- mic->name, __func__, strerror(errno));
- continue;
- }
-
- if (pollfd.revents) {
- if (desc->status != prev_status) {
- mpsslog("%s %s Waiting... desc-> type %d "
- "status 0x%x\n",
- mic->name, __func__, type,
- desc->status);
- prev_status = desc->status;
- }
- if (desc->status & VIRTIO_CONFIG_S_DRIVER_OK) {
- mpsslog("%s %s poll.revents %d\n",
- mic->name, __func__, pollfd.revents);
- mpsslog("%s %s desc-> type %d status 0x%x\n",
- mic->name, __func__, type,
- desc->status);
- break;
- }
- }
- }
- return 0;
-}
-
-/* Spin till we have some descriptors */
-static void
-spin_for_descriptors(struct mic_info *mic, struct mic_vring *vr)
-{
- __u16 avail_idx = read_avail_idx(vr);
-
- while (avail_idx == le16toh(READ_ONCE(vr->vr.avail->idx))) {
-#ifdef DEBUG
- mpsslog("%s %s waiting for desc avail %d info_avail %d\n",
- mic->name, __func__,
- le16toh(vr->vr.avail->idx), vr->info->avail_idx);
-#endif
- sched_yield();
- }
-}
-
-static void *
-virtio_net(void *arg)
-{
- static __u8 vnet_hdr[2][sizeof(struct virtio_net_hdr)];
- static __u8 vnet_buf[2][MAX_NET_PKT_SIZE] __attribute__ ((aligned(64)));
- struct iovec vnet_iov[2][2] = {
- { { .iov_base = vnet_hdr[0], .iov_len = sizeof(vnet_hdr[0]) },
- { .iov_base = vnet_buf[0], .iov_len = sizeof(vnet_buf[0]) } },
- { { .iov_base = vnet_hdr[1], .iov_len = sizeof(vnet_hdr[1]) },
- { .iov_base = vnet_buf[1], .iov_len = sizeof(vnet_buf[1]) } },
- };
- struct iovec *iov0 = vnet_iov[0], *iov1 = vnet_iov[1];
- struct mic_info *mic = (struct mic_info *)arg;
- char if_name[IFNAMSIZ];
- struct pollfd net_poll[MAX_NET_FD];
- struct mic_vring tx_vr, rx_vr;
- struct mic_copy_desc copy;
- struct mic_device_desc *desc;
- int err;
-
- snprintf(if_name, IFNAMSIZ, "mic%d", mic->id);
- mic->mic_net.tap_fd = tun_alloc(mic, if_name);
- if (mic->mic_net.tap_fd < 0)
- goto done;
-
- if (tap_configure(mic, if_name))
- goto done;
- mpsslog("MIC name %s id %d\n", mic->name, mic->id);
-
- net_poll[NET_FD_VIRTIO_NET].fd = mic->mic_net.virtio_net_fd;
- net_poll[NET_FD_VIRTIO_NET].events = POLLIN;
- net_poll[NET_FD_TUN].fd = mic->mic_net.tap_fd;
- net_poll[NET_FD_TUN].events = POLLIN;
-
- if (MAP_FAILED == init_vr(mic, mic->mic_net.virtio_net_fd,
- VIRTIO_ID_NET, &tx_vr, &rx_vr,
- virtnet_dev_page.dd.num_vq)) {
- mpsslog("%s init_vr failed %s\n",
- mic->name, strerror(errno));
- goto done;
- }
-
- copy.iovcnt = 2;
- desc = get_device_desc(mic, VIRTIO_ID_NET);
-
- while (1) {
- ssize_t len;
-
- net_poll[NET_FD_VIRTIO_NET].revents = 0;
- net_poll[NET_FD_TUN].revents = 0;
-
- /* Start polling for data from tap and virtio net */
- err = poll(net_poll, 2, -1);
- if (err < 0) {
- mpsslog("%s poll failed %s\n",
- __func__, strerror(errno));
- continue;
- }
- if (!(desc->status & VIRTIO_CONFIG_S_DRIVER_OK)) {
- err = wait_for_card_driver(mic,
- mic->mic_net.virtio_net_fd,
- VIRTIO_ID_NET);
- if (err) {
- mpsslog("%s %s %d Exiting...\n",
- mic->name, __func__, __LINE__);
- break;
- }
- }
- /*
- * Check if there is data to be read from TUN and write to
- * virtio net fd if there is.
- */
- if (net_poll[NET_FD_TUN].revents & POLLIN) {
- copy.iov = iov0;
- len = readv(net_poll[NET_FD_TUN].fd,
- copy.iov, copy.iovcnt);
- if (len > 0) {
- struct virtio_net_hdr *hdr
- = (struct virtio_net_hdr *)vnet_hdr[0];
-
- /* Disable checksums on the card since we are on
- a reliable PCIe link */
- hdr->flags |= VIRTIO_NET_HDR_F_DATA_VALID;
-#ifdef DEBUG
- mpsslog("%s %s %d hdr->flags 0x%x ", mic->name,
- __func__, __LINE__, hdr->flags);
- mpsslog("copy.out_len %d hdr->gso_type 0x%x\n",
- copy.out_len, hdr->gso_type);
-#endif
-#ifdef DEBUG
- disp_iovec(mic, copy, __func__, __LINE__);
- mpsslog("%s %s %d read from tap 0x%lx\n",
- mic->name, __func__, __LINE__,
- len);
-#endif
- spin_for_descriptors(mic, &tx_vr);
- txrx_prepare(VIRTIO_ID_NET, 1, &tx_vr, ©,
- len);
-
- err = mic_virtio_copy(mic,
- mic->mic_net.virtio_net_fd, &tx_vr,
- ©);
- if (err < 0) {
- mpsslog("%s %s %d mic_virtio_copy %s\n",
- mic->name, __func__, __LINE__,
- strerror(errno));
- }
- if (!err)
- verify_out_len(mic, ©);
-#ifdef DEBUG
- disp_iovec(mic, copy, __func__, __LINE__);
- mpsslog("%s %s %d wrote to net 0x%lx\n",
- mic->name, __func__, __LINE__,
- sum_iovec_len(©));
-#endif
- /* Reinitialize IOV for next run */
- iov0[1].iov_len = MAX_NET_PKT_SIZE;
- } else if (len < 0) {
- disp_iovec(mic, ©, __func__, __LINE__);
- mpsslog("%s %s %d read failed %s ", mic->name,
- __func__, __LINE__, strerror(errno));
- mpsslog("cnt %d sum %zd\n",
- copy.iovcnt, sum_iovec_len(©));
- }
- }
-
- /*
- * Check if there is data to be read from virtio net and
- * write to TUN if there is.
- */
- if (net_poll[NET_FD_VIRTIO_NET].revents & POLLIN) {
- while (rx_vr.info->avail_idx !=
- le16toh(rx_vr.vr.avail->idx)) {
- copy.iov = iov1;
- txrx_prepare(VIRTIO_ID_NET, 0, &rx_vr, ©,
- MAX_NET_PKT_SIZE
- + sizeof(struct virtio_net_hdr));
-
- err = mic_virtio_copy(mic,
- mic->mic_net.virtio_net_fd, &rx_vr,
- ©);
- if (!err) {
-#ifdef DEBUG
- struct virtio_net_hdr *hdr
- = (struct virtio_net_hdr *)
- vnet_hdr[1];
-
- mpsslog("%s %s %d hdr->flags 0x%x, ",
- mic->name, __func__, __LINE__,
- hdr->flags);
- mpsslog("out_len %d gso_type 0x%x\n",
- copy.out_len,
- hdr->gso_type);
-#endif
- /* Set the correct output iov_len */
- iov1[1].iov_len = copy.out_len -
- sizeof(struct virtio_net_hdr);
- verify_out_len(mic, ©);
-#ifdef DEBUG
- disp_iovec(mic, copy, __func__,
- __LINE__);
- mpsslog("%s %s %d ",
- mic->name, __func__, __LINE__);
- mpsslog("read from net 0x%lx\n",
- sum_iovec_len(copy));
-#endif
- len = writev(net_poll[NET_FD_TUN].fd,
- copy.iov, copy.iovcnt);
- if (len != sum_iovec_len(©)) {
- mpsslog("Tun write failed %s ",
- strerror(errno));
- mpsslog("len 0x%zx ", len);
- mpsslog("read_len 0x%zx\n",
- sum_iovec_len(©));
- } else {
-#ifdef DEBUG
- disp_iovec(mic, ©, __func__,
- __LINE__);
- mpsslog("%s %s %d ",
- mic->name, __func__,
- __LINE__);
- mpsslog("wrote to tap 0x%lx\n",
- len);
-#endif
- }
- } else {
- mpsslog("%s %s %d mic_virtio_copy %s\n",
- mic->name, __func__, __LINE__,
- strerror(errno));
- break;
- }
- }
- }
- if (net_poll[NET_FD_VIRTIO_NET].revents & POLLERR)
- mpsslog("%s: %s: POLLERR\n", __func__, mic->name);
- }
-done:
- pthread_exit(NULL);
-}
-
-/* virtio_console */
-#define VIRTIO_CONSOLE_FD 0
-#define MONITOR_FD (VIRTIO_CONSOLE_FD + 1)
-#define MAX_CONSOLE_FD (MONITOR_FD + 1) /* must be the last one + 1 */
-#define MAX_BUFFER_SIZE PAGE_SIZE
-
-static void *
-virtio_console(void *arg)
-{
- static __u8 vcons_buf[2][PAGE_SIZE];
- struct iovec vcons_iov[2] = {
- { .iov_base = vcons_buf[0], .iov_len = sizeof(vcons_buf[0]) },
- { .iov_base = vcons_buf[1], .iov_len = sizeof(vcons_buf[1]) },
- };
- struct iovec *iov0 = &vcons_iov[0], *iov1 = &vcons_iov[1];
- struct mic_info *mic = (struct mic_info *)arg;
- int err;
- struct pollfd console_poll[MAX_CONSOLE_FD];
- int pty_fd;
- char *pts_name;
- ssize_t len;
- struct mic_vring tx_vr, rx_vr;
- struct mic_copy_desc copy;
- struct mic_device_desc *desc;
-
- pty_fd = posix_openpt(O_RDWR);
- if (pty_fd < 0) {
- mpsslog("can't open a pseudoterminal master device: %s\n",
- strerror(errno));
- goto _return;
- }
- pts_name = ptsname(pty_fd);
- if (pts_name == NULL) {
- mpsslog("can't get pts name\n");
- goto _close_pty;
- }
- printf("%s console message goes to %s\n", mic->name, pts_name);
- mpsslog("%s console message goes to %s\n", mic->name, pts_name);
- err = grantpt(pty_fd);
- if (err < 0) {
- mpsslog("can't grant access: %s %s\n",
- pts_name, strerror(errno));
- goto _close_pty;
- }
- err = unlockpt(pty_fd);
- if (err < 0) {
- mpsslog("can't unlock a pseudoterminal: %s %s\n",
- pts_name, strerror(errno));
- goto _close_pty;
- }
- console_poll[MONITOR_FD].fd = pty_fd;
- console_poll[MONITOR_FD].events = POLLIN;
-
- console_poll[VIRTIO_CONSOLE_FD].fd = mic->mic_console.virtio_console_fd;
- console_poll[VIRTIO_CONSOLE_FD].events = POLLIN;
-
- if (MAP_FAILED == init_vr(mic, mic->mic_console.virtio_console_fd,
- VIRTIO_ID_CONSOLE, &tx_vr, &rx_vr,
- virtcons_dev_page.dd.num_vq)) {
- mpsslog("%s init_vr failed %s\n",
- mic->name, strerror(errno));
- goto _close_pty;
- }
-
- copy.iovcnt = 1;
- desc = get_device_desc(mic, VIRTIO_ID_CONSOLE);
-
- for (;;) {
- console_poll[MONITOR_FD].revents = 0;
- console_poll[VIRTIO_CONSOLE_FD].revents = 0;
- err = poll(console_poll, MAX_CONSOLE_FD, -1);
- if (err < 0) {
- mpsslog("%s %d: poll failed: %s\n", __func__, __LINE__,
- strerror(errno));
- continue;
- }
- if (!(desc->status & VIRTIO_CONFIG_S_DRIVER_OK)) {
- err = wait_for_card_driver(mic,
- mic->mic_console.virtio_console_fd,
- VIRTIO_ID_CONSOLE);
- if (err) {
- mpsslog("%s %s %d Exiting...\n",
- mic->name, __func__, __LINE__);
- break;
- }
- }
-
- if (console_poll[MONITOR_FD].revents & POLLIN) {
- copy.iov = iov0;
- len = readv(pty_fd, copy.iov, copy.iovcnt);
- if (len > 0) {
-#ifdef DEBUG
- disp_iovec(mic, copy, __func__, __LINE__);
- mpsslog("%s %s %d read from tap 0x%lx\n",
- mic->name, __func__, __LINE__,
- len);
-#endif
- spin_for_descriptors(mic, &tx_vr);
- txrx_prepare(VIRTIO_ID_CONSOLE, 1, &tx_vr,
- ©, len);
-
- err = mic_virtio_copy(mic,
- mic->mic_console.virtio_console_fd,
- &tx_vr, ©);
- if (err < 0) {
- mpsslog("%s %s %d mic_virtio_copy %s\n",
- mic->name, __func__, __LINE__,
- strerror(errno));
- }
- if (!err)
- verify_out_len(mic, ©);
-#ifdef DEBUG
- disp_iovec(mic, copy, __func__, __LINE__);
- mpsslog("%s %s %d wrote to net 0x%lx\n",
- mic->name, __func__, __LINE__,
- sum_iovec_len(copy));
-#endif
- /* Reinitialize IOV for next run */
- iov0->iov_len = PAGE_SIZE;
- } else if (len < 0) {
- disp_iovec(mic, ©, __func__, __LINE__);
- mpsslog("%s %s %d read failed %s ",
- mic->name, __func__, __LINE__,
- strerror(errno));
- mpsslog("cnt %d sum %zd\n",
- copy.iovcnt, sum_iovec_len(©));
- }
- }
-
- if (console_poll[VIRTIO_CONSOLE_FD].revents & POLLIN) {
- while (rx_vr.info->avail_idx !=
- le16toh(rx_vr.vr.avail->idx)) {
- copy.iov = iov1;
- txrx_prepare(VIRTIO_ID_CONSOLE, 0, &rx_vr,
- ©, PAGE_SIZE);
-
- err = mic_virtio_copy(mic,
- mic->mic_console.virtio_console_fd,
- &rx_vr, ©);
- if (!err) {
- /* Set the correct output iov_len */
- iov1->iov_len = copy.out_len;
- verify_out_len(mic, ©);
-#ifdef DEBUG
- disp_iovec(mic, copy, __func__,
- __LINE__);
- mpsslog("%s %s %d ",
- mic->name, __func__, __LINE__);
- mpsslog("read from net 0x%lx\n",
- sum_iovec_len(copy));
-#endif
- len = writev(pty_fd,
- copy.iov, copy.iovcnt);
- if (len != sum_iovec_len(©)) {
- mpsslog("Tun write failed %s ",
- strerror(errno));
- mpsslog("len 0x%zx ", len);
- mpsslog("read_len 0x%zx\n",
- sum_iovec_len(©));
- } else {
-#ifdef DEBUG
- disp_iovec(mic, copy, __func__,
- __LINE__);
- mpsslog("%s %s %d ",
- mic->name, __func__,
- __LINE__);
- mpsslog("wrote to tap 0x%lx\n",
- len);
-#endif
- }
- } else {
- mpsslog("%s %s %d mic_virtio_copy %s\n",
- mic->name, __func__, __LINE__,
- strerror(errno));
- break;
- }
- }
- }
- if (console_poll[NET_FD_VIRTIO_NET].revents & POLLERR)
- mpsslog("%s: %s: POLLERR\n", __func__, mic->name);
- }
-_close_pty:
- close(pty_fd);
-_return:
- pthread_exit(NULL);
-}
-
-static void
-add_virtio_device(struct mic_info *mic, struct mic_device_desc *dd)
-{
- char path[PATH_MAX];
- int fd, err;
-
- snprintf(path, PATH_MAX, "/dev/vop_virtio%d", mic->id);
- fd = open(path, O_RDWR);
- if (fd < 0) {
- mpsslog("Could not open %s %s\n", path, strerror(errno));
- return;
- }
-
- err = ioctl(fd, MIC_VIRTIO_ADD_DEVICE, dd);
- if (err < 0) {
- mpsslog("Could not add %d %s\n", dd->type, strerror(errno));
- close(fd);
- return;
- }
- switch (dd->type) {
- case VIRTIO_ID_NET:
- mic->mic_net.virtio_net_fd = fd;
- mpsslog("Added VIRTIO_ID_NET for %s\n", mic->name);
- break;
- case VIRTIO_ID_CONSOLE:
- mic->mic_console.virtio_console_fd = fd;
- mpsslog("Added VIRTIO_ID_CONSOLE for %s\n", mic->name);
- break;
- case VIRTIO_ID_BLOCK:
- mic->mic_virtblk.virtio_block_fd = fd;
- mpsslog("Added VIRTIO_ID_BLOCK for %s\n", mic->name);
- break;
- }
-}
-
-static bool
-set_backend_file(struct mic_info *mic)
-{
- FILE *config;
- char buff[PATH_MAX], *line, *evv, *p;
-
- snprintf(buff, PATH_MAX, "%s/mpssd%03d.conf", mic_config_dir, mic->id);
- config = fopen(buff, "r");
- if (config == NULL)
- return false;
- do { /* look for "virtblk_backend=XXXX" */
- line = fgets(buff, PATH_MAX, config);
- if (line == NULL)
- break;
- if (*line == '#')
- continue;
- p = strchr(line, '\n');
- if (p)
- *p = '\0';
- } while (strncmp(line, virtblk_backend, strlen(virtblk_backend)) != 0);
- fclose(config);
- if (line == NULL)
- return false;
- evv = strchr(line, '=');
- if (evv == NULL)
- return false;
- mic->mic_virtblk.backend_file = malloc(strlen(evv) + 1);
- if (mic->mic_virtblk.backend_file == NULL) {
- mpsslog("%s %d can't allocate memory\n", mic->name, mic->id);
- return false;
- }
- strcpy(mic->mic_virtblk.backend_file, evv + 1);
- return true;
-}
-
-#define SECTOR_SIZE 512
-static bool
-set_backend_size(struct mic_info *mic)
-{
- mic->mic_virtblk.backend_size = lseek(mic->mic_virtblk.backend, 0,
- SEEK_END);
- if (mic->mic_virtblk.backend_size < 0) {
- mpsslog("%s: can't seek: %s\n",
- mic->name, mic->mic_virtblk.backend_file);
- return false;
- }
- virtblk_dev_page.blk_config.capacity =
- mic->mic_virtblk.backend_size / SECTOR_SIZE;
- if ((mic->mic_virtblk.backend_size % SECTOR_SIZE) != 0)
- virtblk_dev_page.blk_config.capacity++;
-
- virtblk_dev_page.blk_config.capacity =
- htole64(virtblk_dev_page.blk_config.capacity);
-
- return true;
-}
-
-static bool
-open_backend(struct mic_info *mic)
-{
- if (!set_backend_file(mic))
- goto _error_exit;
- mic->mic_virtblk.backend = open(mic->mic_virtblk.backend_file, O_RDWR);
- if (mic->mic_virtblk.backend < 0) {
- mpsslog("%s: can't open: %s\n", mic->name,
- mic->mic_virtblk.backend_file);
- goto _error_free;
- }
- if (!set_backend_size(mic))
- goto _error_close;
- mic->mic_virtblk.backend_addr = mmap(NULL,
- mic->mic_virtblk.backend_size,
- PROT_READ|PROT_WRITE, MAP_SHARED,
- mic->mic_virtblk.backend, 0L);
- if (mic->mic_virtblk.backend_addr == MAP_FAILED) {
- mpsslog("%s: can't map: %s %s\n",
- mic->name, mic->mic_virtblk.backend_file,
- strerror(errno));
- goto _error_close;
- }
- return true;
-
- _error_close:
- close(mic->mic_virtblk.backend);
- _error_free:
- free(mic->mic_virtblk.backend_file);
- _error_exit:
- return false;
-}
-
-static void
-close_backend(struct mic_info *mic)
-{
- munmap(mic->mic_virtblk.backend_addr, mic->mic_virtblk.backend_size);
- close(mic->mic_virtblk.backend);
- free(mic->mic_virtblk.backend_file);
-}
-
-static bool
-start_virtblk(struct mic_info *mic, struct mic_vring *vring)
-{
- if (((unsigned long)&virtblk_dev_page.blk_config % 8) != 0) {
- mpsslog("%s: blk_config is not 8 byte aligned.\n",
- mic->name);
- return false;
- }
- add_virtio_device(mic, &virtblk_dev_page.dd);
- if (MAP_FAILED == init_vr(mic, mic->mic_virtblk.virtio_block_fd,
- VIRTIO_ID_BLOCK, vring, NULL,
- virtblk_dev_page.dd.num_vq)) {
- mpsslog("%s init_vr failed %s\n",
- mic->name, strerror(errno));
- return false;
- }
- return true;
-}
-
-static void
-stop_virtblk(struct mic_info *mic)
-{
- int vr_size, ret;
-
- vr_size = PAGE_ALIGN(_vring_size(MIC_VRING_ENTRIES,
- MIC_VIRTIO_RING_ALIGN) +
- sizeof(struct _mic_vring_info));
- ret = munmap(mic->mic_virtblk.block_dp,
- MIC_DEVICE_PAGE_END + vr_size * virtblk_dev_page.dd.num_vq);
- if (ret < 0)
- mpsslog("%s munmap errno %d\n", mic->name, errno);
- close(mic->mic_virtblk.virtio_block_fd);
-}
-
-static __u8
-header_error_check(struct vring_desc *desc)
-{
- if (le32toh(desc->len) != sizeof(struct virtio_blk_outhdr)) {
- mpsslog("%s() %d: length is not sizeof(virtio_blk_outhd)\n",
- __func__, __LINE__);
- return -EIO;
- }
- if (!(le16toh(desc->flags) & VRING_DESC_F_NEXT)) {
- mpsslog("%s() %d: alone\n",
- __func__, __LINE__);
- return -EIO;
- }
- if (le16toh(desc->flags) & VRING_DESC_F_WRITE) {
- mpsslog("%s() %d: not read\n",
- __func__, __LINE__);
- return -EIO;
- }
- return 0;
-}
-
-static int
-read_header(int fd, struct virtio_blk_outhdr *hdr, __u32 desc_idx)
-{
- struct iovec iovec;
- struct mic_copy_desc copy;
-
- iovec.iov_len = sizeof(*hdr);
- iovec.iov_base = hdr;
- copy.iov = &iovec;
- copy.iovcnt = 1;
- copy.vr_idx = 0; /* only one vring on virtio_block */
- copy.update_used = false; /* do not update used index */
- return ioctl(fd, MIC_VIRTIO_COPY_DESC, ©);
-}
-
-static int
-transfer_blocks(int fd, struct iovec *iovec, __u32 iovcnt)
-{
- struct mic_copy_desc copy;
-
- copy.iov = iovec;
- copy.iovcnt = iovcnt;
- copy.vr_idx = 0; /* only one vring on virtio_block */
- copy.update_used = false; /* do not update used index */
- return ioctl(fd, MIC_VIRTIO_COPY_DESC, ©);
-}
-
-static __u8
-status_error_check(struct vring_desc *desc)
-{
- if (le32toh(desc->len) != sizeof(__u8)) {
- mpsslog("%s() %d: length is not sizeof(status)\n",
- __func__, __LINE__);
- return -EIO;
- }
- return 0;
-}
-
-static int
-write_status(int fd, __u8 *status)
-{
- struct iovec iovec;
- struct mic_copy_desc copy;
-
- iovec.iov_base = status;
- iovec.iov_len = sizeof(*status);
- copy.iov = &iovec;
- copy.iovcnt = 1;
- copy.vr_idx = 0; /* only one vring on virtio_block */
- copy.update_used = true; /* Update used index */
- return ioctl(fd, MIC_VIRTIO_COPY_DESC, ©);
-}
-
-#ifndef VIRTIO_BLK_T_GET_ID
-#define VIRTIO_BLK_T_GET_ID 8
-#endif
-
-static void *
-virtio_block(void *arg)
-{
- struct mic_info *mic = (struct mic_info *)arg;
- int ret;
- struct pollfd block_poll;
- struct mic_vring vring;
- __u16 avail_idx;
- __u32 desc_idx;
- struct vring_desc *desc;
- struct iovec *iovec, *piov;
- __u8 status;
- __u32 buffer_desc_idx;
- struct virtio_blk_outhdr hdr;
- void *fos;
-
- for (;;) { /* forever */
- if (!open_backend(mic)) { /* No virtblk */
- for (mic->mic_virtblk.signaled = 0;
- !mic->mic_virtblk.signaled;)
- sleep(1);
- continue;
- }
-
- /* backend file is specified. */
- if (!start_virtblk(mic, &vring))
- goto _close_backend;
- iovec = malloc(sizeof(*iovec) *
- le32toh(virtblk_dev_page.blk_config.seg_max));
- if (!iovec) {
- mpsslog("%s: can't alloc iovec: %s\n",
- mic->name, strerror(ENOMEM));
- goto _stop_virtblk;
- }
-
- block_poll.fd = mic->mic_virtblk.virtio_block_fd;
- block_poll.events = POLLIN;
- for (mic->mic_virtblk.signaled = 0;
- !mic->mic_virtblk.signaled;) {
- block_poll.revents = 0;
- /* timeout in 1 sec to see signaled */
- ret = poll(&block_poll, 1, 1000);
- if (ret < 0) {
- mpsslog("%s %d: poll failed: %s\n",
- __func__, __LINE__,
- strerror(errno));
- continue;
- }
-
- if (!(block_poll.revents & POLLIN)) {
-#ifdef DEBUG
- mpsslog("%s %d: block_poll.revents=0x%x\n",
- __func__, __LINE__, block_poll.revents);
-#endif
- continue;
- }
-
- /* POLLIN */
- while (vring.info->avail_idx !=
- le16toh(vring.vr.avail->idx)) {
- /* read header element */
- avail_idx =
- vring.info->avail_idx &
- (vring.vr.num - 1);
- desc_idx = le16toh(
- vring.vr.avail->ring[avail_idx]);
- desc = &vring.vr.desc[desc_idx];
-#ifdef DEBUG
- mpsslog("%s() %d: avail_idx=%d ",
- __func__, __LINE__,
- vring.info->avail_idx);
- mpsslog("vring.vr.num=%d desc=%p\n",
- vring.vr.num, desc);
-#endif
- status = header_error_check(desc);
- ret = read_header(
- mic->mic_virtblk.virtio_block_fd,
- &hdr, desc_idx);
- if (ret < 0) {
- mpsslog("%s() %d %s: ret=%d %s\n",
- __func__, __LINE__,
- mic->name, ret,
- strerror(errno));
- break;
- }
- /* buffer element */
- piov = iovec;
- status = 0;
- fos = mic->mic_virtblk.backend_addr +
- (hdr.sector * SECTOR_SIZE);
- buffer_desc_idx = next_desc(desc);
- desc_idx = buffer_desc_idx;
- for (desc = &vring.vr.desc[buffer_desc_idx];
- desc->flags & VRING_DESC_F_NEXT;
- desc_idx = next_desc(desc),
- desc = &vring.vr.desc[desc_idx]) {
- piov->iov_len = desc->len;
- piov->iov_base = fos;
- piov++;
- fos += desc->len;
- }
- /* Returning NULLs for VIRTIO_BLK_T_GET_ID. */
- if (hdr.type & ~(VIRTIO_BLK_T_OUT |
- VIRTIO_BLK_T_GET_ID)) {
- /*
- VIRTIO_BLK_T_IN - does not do
- anything. Probably for documenting.
- VIRTIO_BLK_T_SCSI_CMD - for
- virtio_scsi.
- VIRTIO_BLK_T_FLUSH - turned off in
- config space.
- VIRTIO_BLK_T_BARRIER - defined but not
- used in anywhere.
- */
- mpsslog("%s() %d: type %x ",
- __func__, __LINE__,
- hdr.type);
- mpsslog("is not supported\n");
- status = -ENOTSUP;
-
- } else {
- ret = transfer_blocks(
- mic->mic_virtblk.virtio_block_fd,
- iovec,
- piov - iovec);
- if (ret < 0 &&
- status != 0)
- status = ret;
- }
- /* write status and update used pointer */
- if (status != 0)
- status = status_error_check(desc);
- ret = write_status(
- mic->mic_virtblk.virtio_block_fd,
- &status);
-#ifdef DEBUG
- mpsslog("%s() %d: write status=%d on desc=%p\n",
- __func__, __LINE__,
- status, desc);
-#endif
- }
- }
- free(iovec);
-_stop_virtblk:
- stop_virtblk(mic);
-_close_backend:
- close_backend(mic);
- } /* forever */
-
- pthread_exit(NULL);
-}
-
static void
reset(struct mic_info *mic)
{
@@ -1602,15 +328,6 @@ set_log_buf_info(struct mic_info *mic)
close(fd);
}
-static void
-change_virtblk_backend(int x, siginfo_t *siginfo, void *p)
-{
- struct mic_info *mic;
-
- for (mic = mic_list.next; mic != NULL; mic = mic->next)
- mic->mic_virtblk.signaled = 1/* true */;
-}
-
static void
set_mic_boot_params(struct mic_info *mic)
{
@@ -1626,10 +343,6 @@ init_mic(void *arg)
.sa_flags = 0,
.sa_handler = SIG_IGN
};
- struct sigaction act = {
- .sa_flags = SA_SIGINFO,
- .sa_sigaction = change_virtblk_backend,
- };
char buffer[PATH_MAX];
int err, fd;
@@ -1676,30 +389,8 @@ init_mic(void *arg)
mic->pid = fork();
switch (mic->pid) {
case 0:
- add_virtio_device(mic, &virtcons_dev_page.dd);
- add_virtio_device(mic, &virtnet_dev_page.dd);
- err = pthread_create(&mic->mic_console.console_thread, NULL,
- virtio_console, mic);
- if (err)
- mpsslog("%s virtcons pthread_create failed %s\n",
- mic->name, strerror(err));
- err = pthread_create(&mic->mic_net.net_thread, NULL,
- virtio_net, mic);
- if (err)
- mpsslog("%s virtnet pthread_create failed %s\n",
- mic->name, strerror(err));
- err = pthread_create(&mic->mic_virtblk.block_thread, NULL,
- virtio_block, mic);
- if (err)
- mpsslog("%s virtblk pthread_create failed %s\n",
- mic->name, strerror(err));
- sigemptyset(&act.sa_mask);
- err = sigaction(SIGUSR1, &act, NULL);
- if (err)
- mpsslog("%s sigaction SIGUSR1 failed %s\n",
- mic->name, strerror(errno));
- while (1)
- sleep(60);
+ serve_virtio(mic, &mic_list);
+ break;
case -1:
mpsslog("fork failed MIC name %s id %d errno %d\n",
mic->name, mic->id, errno);
diff --git a/samples/mic/mpssd/mpssd.h b/samples/mic/mpssd/mpssd.h
index 8bd64944aacc..f3e8c13e3948 100644
--- a/samples/mic/mpssd/mpssd.h
+++ b/samples/mic/mpssd/mpssd.h
@@ -100,4 +100,5 @@ __attribute__((format(printf, 1, 2)))
void mpsslog(char *format, ...);
char *readsysfs(char *dir, char *entry);
int setsysfs(char *dir, char *entry, char *value);
+void serve_virtio(struct mic_info *mic, struct mic_info *list);
#endif
diff --git a/samples/mic/mpssd/vop.c b/samples/mic/mpssd/vop.c
new file mode 100644
index 000000000000..3d30c0cb8c7d
--- /dev/null
+++ b/samples/mic/mpssd/vop.c
@@ -0,0 +1,1357 @@
+/*
+ * Intel MIC Platform Software Stack (MPSS)
+ *
+ * Copyright(c) 2013 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * Intel MIC User Space Tools.
+ */
+
+#define _GNU_SOURCE
+
+#include <stdlib.h>
+#include <fcntl.h>
+#include <getopt.h>
+#include <assert.h>
+#include <unistd.h>
+#include <stdbool.h>
+#include <signal.h>
+#include <poll.h>
+#include <features.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <sys/socket.h>
+#include <linux/virtio_ring.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_console.h>
+#include <linux/virtio_blk.h>
+#include <linux/version.h>
+#include "mpssd.h"
+#include <linux/mic_ioctl.h>
+#include <linux/mic_common.h>
+#include <tools/endian.h>
+
+static struct mic_info *mic_list;
+
+#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
+
+#define min_t(type, x, y) ({ \
+ type __min1 = (x); \
+ type __min2 = (y); \
+ __min1 < __min2 ? __min1 : __min2; })
+
+/* align addr on a size boundary - adjust address up/down if needed */
+#define _ALIGN_DOWN(addr, size) ((addr)&(~((size)-1)))
+#define _ALIGN_UP(addr, size) _ALIGN_DOWN(addr + size - 1, size)
+
+/* align addr on a size boundary - adjust address up if needed */
+#define _ALIGN(addr, size) _ALIGN_UP(addr, size)
+
+/* to align the pointer to the (next) page boundary */
+#define PAGE_ALIGN(addr) _ALIGN(addr, PAGE_SIZE)
+
+#define READ_ONCE(x) (*(volatile typeof(x) *)&(x))
+
+#define GSO_ENABLED 1
+#define MAX_GSO_SIZE (64 * 1024)
+#define ETH_H_LEN 14
+#define MAX_NET_PKT_SIZE (_ALIGN_UP(MAX_GSO_SIZE + ETH_H_LEN, 64))
+#define MIC_DEVICE_PAGE_END 0x1000
+
+#ifndef VIRTIO_NET_HDR_F_DATA_VALID
+#define VIRTIO_NET_HDR_F_DATA_VALID 2 /* Csum is valid */
+#endif
+
+static struct {
+ struct mic_device_desc dd;
+ struct mic_vqconfig vqconfig[2];
+ __u32 host_features, guest_acknowledgements;
+ struct virtio_console_config cons_config;
+} virtcons_dev_page = {
+ .dd = {
+ .type = VIRTIO_ID_CONSOLE,
+ .num_vq = ARRAY_SIZE(virtcons_dev_page.vqconfig),
+ .feature_len = sizeof(virtcons_dev_page.host_features),
+ .config_len = sizeof(virtcons_dev_page.cons_config),
+ },
+ .vqconfig[0] = {
+ .num = htole16(MIC_VRING_ENTRIES),
+ },
+ .vqconfig[1] = {
+ .num = htole16(MIC_VRING_ENTRIES),
+ },
+};
+
+static struct {
+ struct mic_device_desc dd;
+ struct mic_vqconfig vqconfig[2];
+ __u32 host_features, guest_acknowledgements;
+ struct virtio_net_config net_config;
+} virtnet_dev_page = {
+ .dd = {
+ .type = VIRTIO_ID_NET,
+ .num_vq = ARRAY_SIZE(virtnet_dev_page.vqconfig),
+ .feature_len = sizeof(virtnet_dev_page.host_features),
+ .config_len = sizeof(virtnet_dev_page.net_config),
+ },
+ .vqconfig[0] = {
+ .num = htole16(MIC_VRING_ENTRIES),
+ },
+ .vqconfig[1] = {
+ .num = htole16(MIC_VRING_ENTRIES),
+ },
+#if GSO_ENABLED
+ .host_features = htole32(
+ 1 << VIRTIO_NET_F_CSUM |
+ 1 << VIRTIO_NET_F_GSO |
+ 1 << VIRTIO_NET_F_GUEST_TSO4 |
+ 1 << VIRTIO_NET_F_GUEST_TSO6 |
+ 1 << VIRTIO_NET_F_GUEST_ECN),
+#else
+ .host_features = 0,
+#endif
+};
+
+static const char *mic_config_dir = "/etc/mpss";
+static const char *virtblk_backend = "VIRTBLK_BACKEND";
+static struct {
+ struct mic_device_desc dd;
+ struct mic_vqconfig vqconfig[1];
+ __u32 host_features, guest_acknowledgements;
+ struct virtio_blk_config blk_config;
+} virtblk_dev_page = {
+ .dd = {
+ .type = VIRTIO_ID_BLOCK,
+ .num_vq = ARRAY_SIZE(virtblk_dev_page.vqconfig),
+ .feature_len = sizeof(virtblk_dev_page.host_features),
+ .config_len = sizeof(virtblk_dev_page.blk_config),
+ },
+ .vqconfig[0] = {
+ .num = htole16(MIC_VRING_ENTRIES),
+ },
+ .host_features =
+ htole32(1<<VIRTIO_BLK_F_SEG_MAX),
+ .blk_config = {
+ .seg_max = htole32(MIC_VRING_ENTRIES - 2),
+ .capacity = htole64(0),
+ }
+};
+
+static int
+tap_configure(struct mic_info *mic, char *dev)
+{
+ pid_t pid;
+ char *ifargv[7];
+ char ipaddr[IFNAMSIZ];
+ int ret = 0;
+
+ pid = fork();
+ if (pid == 0) {
+ ifargv[0] = "ip";
+ ifargv[1] = "link";
+ ifargv[2] = "set";
+ ifargv[3] = dev;
+ ifargv[4] = "up";
+ ifargv[5] = NULL;
+ mpsslog("Configuring %s\n", dev);
+ ret = execvp("ip", ifargv);
+ if (ret < 0) {
+ mpsslog("%s execvp failed errno %s\n",
+ mic->name, strerror(errno));
+ return ret;
+ }
+ }
+ if (pid < 0) {
+ mpsslog("%s fork failed errno %s\n",
+ mic->name, strerror(errno));
+ return ret;
+ }
+
+ ret = waitpid(pid, NULL, 0);
+ if (ret < 0) {
+ mpsslog("%s waitpid failed errno %s\n",
+ mic->name, strerror(errno));
+ return ret;
+ }
+
+ snprintf(ipaddr, IFNAMSIZ, "172.31.%d.254/24", mic->id + 1);
+
+ pid = fork();
+ if (pid == 0) {
+ ifargv[0] = "ip";
+ ifargv[1] = "addr";
+ ifargv[2] = "add";
+ ifargv[3] = ipaddr;
+ ifargv[4] = "dev";
+ ifargv[5] = dev;
+ ifargv[6] = NULL;
+ mpsslog("Configuring %s ipaddr %s\n", dev, ipaddr);
+ ret = execvp("ip", ifargv);
+ if (ret < 0) {
+ mpsslog("%s execvp failed errno %s\n",
+ mic->name, strerror(errno));
+ return ret;
+ }
+ }
+ if (pid < 0) {
+ mpsslog("%s fork failed errno %s\n",
+ mic->name, strerror(errno));
+ return ret;
+ }
+
+ ret = waitpid(pid, NULL, 0);
+ if (ret < 0) {
+ mpsslog("%s waitpid failed errno %s\n",
+ mic->name, strerror(errno));
+ return ret;
+ }
+ mpsslog("MIC name %s %s %d DONE!\n",
+ mic->name, __func__, __LINE__);
+ return 0;
+}
+
+static int tun_alloc(struct mic_info *mic, char *dev)
+{
+ struct ifreq ifr;
+ int fd, err;
+#if GSO_ENABLED
+ unsigned offload;
+#endif
+ fd = open("/dev/net/tun", O_RDWR);
+ if (fd < 0) {
+ mpsslog("Could not open /dev/net/tun %s\n", strerror(errno));
+ goto done;
+ }
+
+ memset(&ifr, 0, sizeof(ifr));
+
+ ifr.ifr_flags = IFF_TAP | IFF_NO_PI | IFF_VNET_HDR;
+ if (*dev)
+ strncpy(ifr.ifr_name, dev, IFNAMSIZ);
+
+ err = ioctl(fd, TUNSETIFF, (void *)&ifr);
+ if (err < 0) {
+ mpsslog("%s %s %d TUNSETIFF failed %s\n",
+ mic->name, __func__, __LINE__, strerror(errno));
+ close(fd);
+ return err;
+ }
+#if GSO_ENABLED
+ offload = TUN_F_CSUM | TUN_F_TSO4 | TUN_F_TSO6 | TUN_F_TSO_ECN;
+
+ err = ioctl(fd, TUNSETOFFLOAD, offload);
+ if (err < 0) {
+ mpsslog("%s %s %d TUNSETOFFLOAD failed %s\n",
+ mic->name, __func__, __LINE__, strerror(errno));
+ close(fd);
+ return err;
+ }
+#endif
+ strcpy(dev, ifr.ifr_name);
+ mpsslog("Created TAP %s\n", dev);
+done:
+ return fd;
+}
+
+#define NET_FD_VIRTIO_NET 0
+#define NET_FD_TUN 1
+#define MAX_NET_FD 2
+
+static void set_dp(struct mic_info *mic, int type, void *dp)
+{
+ switch (type) {
+ case VIRTIO_ID_CONSOLE:
+ mic->mic_console.console_dp = dp;
+ return;
+ case VIRTIO_ID_NET:
+ mic->mic_net.net_dp = dp;
+ return;
+ case VIRTIO_ID_BLOCK:
+ mic->mic_virtblk.block_dp = dp;
+ return;
+ }
+ mpsslog("%s %s %d not found\n", mic->name, __func__, type);
+ assert(0);
+}
+
+static void *get_dp(struct mic_info *mic, int type)
+{
+ switch (type) {
+ case VIRTIO_ID_CONSOLE:
+ return mic->mic_console.console_dp;
+ case VIRTIO_ID_NET:
+ return mic->mic_net.net_dp;
+ case VIRTIO_ID_BLOCK:
+ return mic->mic_virtblk.block_dp;
+ }
+ mpsslog("%s %s %d not found\n", mic->name, __func__, type);
+ assert(0);
+ return NULL;
+}
+
+static struct mic_device_desc *get_device_desc(struct mic_info *mic, int type)
+{
+ struct mic_device_desc *d;
+ int i;
+ void *dp = get_dp(mic, type);
+
+ for (i = sizeof(struct mic_bootparam); i < PAGE_SIZE;
+ i += mic_total_desc_size(d)) {
+ d = dp + i;
+
+ /* End of list */
+ if (d->type == 0)
+ break;
+
+ if (d->type == -1)
+ continue;
+
+ mpsslog("%s %s d-> type %d d %p\n",
+ mic->name, __func__, d->type, d);
+
+ if (d->type == (__u8)type)
+ return d;
+ }
+ mpsslog("%s %s %d not found\n", mic->name, __func__, type);
+ return NULL;
+}
+
+/* See comments in vhost.c for explanation of next_desc() */
+static unsigned next_desc(struct vring_desc *desc)
+{
+ unsigned int next;
+
+ if (!(le16toh(desc->flags) & VRING_DESC_F_NEXT))
+ return -1U;
+ next = le16toh(desc->next);
+ return next;
+}
+
+/* Sum up all the IOVEC length */
+static ssize_t
+sum_iovec_len(struct mic_copy_desc *copy)
+{
+ ssize_t sum = 0;
+ unsigned int i;
+
+ for (i = 0; i < copy->iovcnt; i++)
+ sum += copy->iov[i].iov_len;
+ return sum;
+}
+
+static inline void verify_out_len(struct mic_info *mic,
+ struct mic_copy_desc *copy)
+{
+ if (copy->out_len != sum_iovec_len(copy)) {
+ mpsslog("%s %s %d BUG copy->out_len 0x%x len 0x%zx\n",
+ mic->name, __func__, __LINE__,
+ copy->out_len, sum_iovec_len(copy));
+ assert(copy->out_len == sum_iovec_len(copy));
+ }
+}
+
+/* Display an iovec */
+static void
+disp_iovec(struct mic_info *mic, struct mic_copy_desc *copy,
+ const char *s, int line)
+{
+ unsigned int i;
+
+ for (i = 0; i < copy->iovcnt; i++)
+ mpsslog("%s %s %d copy->iov[%d] addr %p len 0x%zx\n",
+ mic->name, s, line, i,
+ copy->iov[i].iov_base, copy->iov[i].iov_len);
+}
+
+static inline __u16 read_avail_idx(struct mic_vring *vr)
+{
+ return READ_ONCE(vr->info->avail_idx);
+}
+
+static inline void txrx_prepare(int type, bool tx, struct mic_vring *vr,
+ struct mic_copy_desc *copy, ssize_t len)
+{
+ copy->vr_idx = tx ? 0 : 1;
+ copy->update_used = true;
+ if (type == VIRTIO_ID_NET)
+ copy->iov[1].iov_len = len - sizeof(struct virtio_net_hdr);
+ else
+ copy->iov[0].iov_len = len;
+}
+
+/* Central API which triggers the copies */
+static int
+mic_virtio_copy(struct mic_info *mic, int fd,
+ struct mic_vring *vr, struct mic_copy_desc *copy)
+{
+ int ret;
+
+ ret = ioctl(fd, MIC_VIRTIO_COPY_DESC, copy);
+ if (ret) {
+ mpsslog("%s %s %d errno %s ret %d\n",
+ mic->name, __func__, __LINE__,
+ strerror(errno), ret);
+ }
+ return ret;
+}
+
+static inline unsigned _vring_size(unsigned int num, unsigned long align)
+{
+ return ((sizeof(struct vring_desc) * num + sizeof(__u16) * (3 + num)
+ + align - 1) & ~(align - 1))
+ + sizeof(__u16) * 3 + sizeof(struct vring_used_elem) * num;
+}
+
+/*
+ * This initialization routine requires at least one
+ * vring i.e. vr0. vr1 is optional.
+ */
+static void *
+init_vr(struct mic_info *mic, int fd, int type,
+ struct mic_vring *vr0, struct mic_vring *vr1, int num_vq)
+{
+ int vr_size;
+ char *va;
+
+ vr_size = PAGE_ALIGN(_vring_size(MIC_VRING_ENTRIES,
+ MIC_VIRTIO_RING_ALIGN) +
+ sizeof(struct _mic_vring_info));
+ va = mmap(NULL, MIC_DEVICE_PAGE_END + vr_size * num_vq,
+ PROT_READ, MAP_SHARED, fd, 0);
+ if (MAP_FAILED == va) {
+ mpsslog("%s %s %d mmap failed errno %s\n",
+ mic->name, __func__, __LINE__,
+ strerror(errno));
+ goto done;
+ }
+ set_dp(mic, type, va);
+ vr0->va = (struct mic_vring *)&va[MIC_DEVICE_PAGE_END];
+ vr0->info = vr0->va +
+ _vring_size(MIC_VRING_ENTRIES, MIC_VIRTIO_RING_ALIGN);
+ vring_init(&vr0->vr,
+ MIC_VRING_ENTRIES, vr0->va, MIC_VIRTIO_RING_ALIGN);
+ mpsslog("%s %s vr0 %p vr0->info %p vr_size 0x%x vring 0x%x ",
+ __func__, mic->name, vr0->va, vr0->info, vr_size,
+ _vring_size(MIC_VRING_ENTRIES, MIC_VIRTIO_RING_ALIGN));
+ mpsslog("magic 0x%x expected 0x%x\n",
+ le32toh(vr0->info->magic), MIC_MAGIC + type);
+ assert(le32toh(vr0->info->magic) == MIC_MAGIC + type);
+ if (vr1) {
+ vr1->va = (struct mic_vring *)
+ &va[MIC_DEVICE_PAGE_END + vr_size];
+ vr1->info = vr1->va + _vring_size(MIC_VRING_ENTRIES,
+ MIC_VIRTIO_RING_ALIGN);
+ vring_init(&vr1->vr,
+ MIC_VRING_ENTRIES, vr1->va, MIC_VIRTIO_RING_ALIGN);
+ mpsslog("%s %s vr1 %p vr1->info %p vr_size 0x%x vring 0x%x ",
+ __func__, mic->name, vr1->va, vr1->info, vr_size,
+ _vring_size(MIC_VRING_ENTRIES, MIC_VIRTIO_RING_ALIGN));
+ mpsslog("magic 0x%x expected 0x%x\n",
+ le32toh(vr1->info->magic), MIC_MAGIC + type + 1);
+ assert(le32toh(vr1->info->magic) == MIC_MAGIC + type + 1);
+ }
+done:
+ return va;
+}
+
+static int
+wait_for_card_driver(struct mic_info *mic, int fd, int type)
+{
+ struct pollfd pollfd;
+ int err;
+ struct mic_device_desc *desc = get_device_desc(mic, type);
+ __u8 prev_status;
+
+ if (!desc)
+ return -ENODEV;
+ prev_status = desc->status;
+ pollfd.fd = fd;
+ mpsslog("%s %s Waiting .... desc-> type %d status 0x%x\n",
+ mic->name, __func__, type, desc->status);
+
+ while (1) {
+ pollfd.events = POLLIN;
+ pollfd.revents = 0;
+ err = poll(&pollfd, 1, -1);
+ if (err < 0) {
+ mpsslog("%s %s poll failed %s\n",
+ mic->name, __func__, strerror(errno));
+ continue;
+ }
+
+ if (pollfd.revents) {
+ if (desc->status != prev_status) {
+ mpsslog("%s %s Waiting... desc-> type %d "
+ "status 0x%x\n",
+ mic->name, __func__, type,
+ desc->status);
+ prev_status = desc->status;
+ }
+ if (desc->status & VIRTIO_CONFIG_S_DRIVER_OK) {
+ mpsslog("%s %s poll.revents %d\n",
+ mic->name, __func__, pollfd.revents);
+ mpsslog("%s %s desc-> type %d status 0x%x\n",
+ mic->name, __func__, type,
+ desc->status);
+ break;
+ }
+ }
+ }
+ return 0;
+}
+
+/* Spin till we have some descriptors */
+static void
+spin_for_descriptors(struct mic_info *mic, struct mic_vring *vr)
+{
+ __u16 avail_idx = read_avail_idx(vr);
+
+ while (avail_idx == le16toh(READ_ONCE(vr->vr.avail->idx))) {
+#ifdef DEBUG
+ mpsslog("%s %s waiting for desc avail %d info_avail %d\n",
+ mic->name, __func__,
+ le16toh(vr->vr.avail->idx), vr->info->avail_idx);
+#endif
+ sched_yield();
+ }
+}
+
+static void *
+virtio_net(void *arg)
+{
+ static __u8 vnet_hdr[2][sizeof(struct virtio_net_hdr)];
+ static __u8 vnet_buf[2][MAX_NET_PKT_SIZE] __attribute__ ((aligned(64)));
+ struct iovec vnet_iov[2][2] = {
+ { { .iov_base = vnet_hdr[0], .iov_len = sizeof(vnet_hdr[0]) },
+ { .iov_base = vnet_buf[0], .iov_len = sizeof(vnet_buf[0]) } },
+ { { .iov_base = vnet_hdr[1], .iov_len = sizeof(vnet_hdr[1]) },
+ { .iov_base = vnet_buf[1], .iov_len = sizeof(vnet_buf[1]) } },
+ };
+ struct iovec *iov0 = vnet_iov[0], *iov1 = vnet_iov[1];
+ struct mic_info *mic = (struct mic_info *)arg;
+ char if_name[IFNAMSIZ];
+ struct pollfd net_poll[MAX_NET_FD];
+ struct mic_vring tx_vr, rx_vr;
+ struct mic_copy_desc copy;
+ struct mic_device_desc *desc;
+ int err;
+
+ snprintf(if_name, IFNAMSIZ, "mic%d", mic->id);
+ mic->mic_net.tap_fd = tun_alloc(mic, if_name);
+ if (mic->mic_net.tap_fd < 0)
+ goto done;
+
+ if (tap_configure(mic, if_name))
+ goto done;
+ mpsslog("MIC name %s id %d\n", mic->name, mic->id);
+
+ net_poll[NET_FD_VIRTIO_NET].fd = mic->mic_net.virtio_net_fd;
+ net_poll[NET_FD_VIRTIO_NET].events = POLLIN;
+ net_poll[NET_FD_TUN].fd = mic->mic_net.tap_fd;
+ net_poll[NET_FD_TUN].events = POLLIN;
+
+ if (MAP_FAILED == init_vr(mic, mic->mic_net.virtio_net_fd,
+ VIRTIO_ID_NET, &tx_vr, &rx_vr,
+ virtnet_dev_page.dd.num_vq)) {
+ mpsslog("%s init_vr failed %s\n",
+ mic->name, strerror(errno));
+ goto done;
+ }
+
+ copy.iovcnt = 2;
+ desc = get_device_desc(mic, VIRTIO_ID_NET);
+
+ while (1) {
+ ssize_t len;
+
+ net_poll[NET_FD_VIRTIO_NET].revents = 0;
+ net_poll[NET_FD_TUN].revents = 0;
+
+ /* Start polling for data from tap and virtio net */
+ err = poll(net_poll, 2, -1);
+ if (err < 0) {
+ mpsslog("%s poll failed %s\n",
+ __func__, strerror(errno));
+ continue;
+ }
+ if (!(desc->status & VIRTIO_CONFIG_S_DRIVER_OK)) {
+ err = wait_for_card_driver(mic,
+ mic->mic_net.virtio_net_fd,
+ VIRTIO_ID_NET);
+ if (err) {
+ mpsslog("%s %s %d Exiting...\n",
+ mic->name, __func__, __LINE__);
+ break;
+ }
+ }
+ /*
+ * Check if there is data to be read from TUN and write to
+ * virtio net fd if there is.
+ */
+ if (net_poll[NET_FD_TUN].revents & POLLIN) {
+ copy.iov = iov0;
+ len = readv(net_poll[NET_FD_TUN].fd,
+ copy.iov, copy.iovcnt);
+ if (len > 0) {
+ struct virtio_net_hdr *hdr
+ = (struct virtio_net_hdr *)vnet_hdr[0];
+
+ /* Disable checksums on the card since we are on
+ a reliable PCIe link */
+ hdr->flags |= VIRTIO_NET_HDR_F_DATA_VALID;
+#ifdef DEBUG
+ mpsslog("%s %s %d hdr->flags 0x%x ", mic->name,
+ __func__, __LINE__, hdr->flags);
+ mpsslog("copy.out_len %d hdr->gso_type 0x%x\n",
+ copy.out_len, hdr->gso_type);
+#endif
+#ifdef DEBUG
+ disp_iovec(mic, copy, __func__, __LINE__);
+ mpsslog("%s %s %d read from tap 0x%lx\n",
+ mic->name, __func__, __LINE__,
+ len);
+#endif
+ spin_for_descriptors(mic, &tx_vr);
+ txrx_prepare(VIRTIO_ID_NET, 1, &tx_vr, ©,
+ len);
+
+ err = mic_virtio_copy(mic,
+ mic->mic_net.virtio_net_fd, &tx_vr,
+ ©);
+ if (err < 0) {
+ mpsslog("%s %s %d mic_virtio_copy %s\n",
+ mic->name, __func__, __LINE__,
+ strerror(errno));
+ }
+ if (!err)
+ verify_out_len(mic, ©);
+#ifdef DEBUG
+ disp_iovec(mic, copy, __func__, __LINE__);
+ mpsslog("%s %s %d wrote to net 0x%lx\n",
+ mic->name, __func__, __LINE__,
+ sum_iovec_len(©));
+#endif
+ /* Reinitialize IOV for next run */
+ iov0[1].iov_len = MAX_NET_PKT_SIZE;
+ } else if (len < 0) {
+ disp_iovec(mic, ©, __func__, __LINE__);
+ mpsslog("%s %s %d read failed %s ", mic->name,
+ __func__, __LINE__, strerror(errno));
+ mpsslog("cnt %d sum %zd\n",
+ copy.iovcnt, sum_iovec_len(©));
+ }
+ }
+
+ /*
+ * Check if there is data to be read from virtio net and
+ * write to TUN if there is.
+ */
+ if (net_poll[NET_FD_VIRTIO_NET].revents & POLLIN) {
+ while (rx_vr.info->avail_idx !=
+ le16toh(rx_vr.vr.avail->idx)) {
+ copy.iov = iov1;
+ txrx_prepare(VIRTIO_ID_NET, 0, &rx_vr, ©,
+ MAX_NET_PKT_SIZE
+ + sizeof(struct virtio_net_hdr));
+
+ err = mic_virtio_copy(mic,
+ mic->mic_net.virtio_net_fd, &rx_vr,
+ ©);
+ if (!err) {
+#ifdef DEBUG
+ struct virtio_net_hdr *hdr
+ = (struct virtio_net_hdr *)
+ vnet_hdr[1];
+
+ mpsslog("%s %s %d hdr->flags 0x%x, ",
+ mic->name, __func__, __LINE__,
+ hdr->flags);
+ mpsslog("out_len %d gso_type 0x%x\n",
+ copy.out_len,
+ hdr->gso_type);
+#endif
+ /* Set the correct output iov_len */
+ iov1[1].iov_len = copy.out_len -
+ sizeof(struct virtio_net_hdr);
+ verify_out_len(mic, ©);
+#ifdef DEBUG
+ disp_iovec(mic, copy, __func__,
+ __LINE__);
+ mpsslog("%s %s %d ",
+ mic->name, __func__, __LINE__);
+ mpsslog("read from net 0x%lx\n",
+ sum_iovec_len(copy));
+#endif
+ len = writev(net_poll[NET_FD_TUN].fd,
+ copy.iov, copy.iovcnt);
+ if (len != sum_iovec_len(©)) {
+ mpsslog("Tun write failed %s ",
+ strerror(errno));
+ mpsslog("len 0x%zx ", len);
+ mpsslog("read_len 0x%zx\n",
+ sum_iovec_len(©));
+ } else {
+#ifdef DEBUG
+ disp_iovec(mic, ©, __func__,
+ __LINE__);
+ mpsslog("%s %s %d ",
+ mic->name, __func__,
+ __LINE__);
+ mpsslog("wrote to tap 0x%lx\n",
+ len);
+#endif
+ }
+ } else {
+ mpsslog("%s %s %d mic_virtio_copy %s\n",
+ mic->name, __func__, __LINE__,
+ strerror(errno));
+ break;
+ }
+ }
+ }
+ if (net_poll[NET_FD_VIRTIO_NET].revents & POLLERR)
+ mpsslog("%s: %s: POLLERR\n", __func__, mic->name);
+ }
+done:
+ pthread_exit(NULL);
+}
+
+/* virtio_console */
+#define VIRTIO_CONSOLE_FD 0
+#define MONITOR_FD (VIRTIO_CONSOLE_FD + 1)
+#define MAX_CONSOLE_FD (MONITOR_FD + 1) /* must be the last one + 1 */
+#define MAX_BUFFER_SIZE PAGE_SIZE
+
+static void *
+virtio_console(void *arg)
+{
+ static __u8 vcons_buf[2][PAGE_SIZE];
+ struct iovec vcons_iov[2] = {
+ { .iov_base = vcons_buf[0], .iov_len = sizeof(vcons_buf[0]) },
+ { .iov_base = vcons_buf[1], .iov_len = sizeof(vcons_buf[1]) },
+ };
+ struct iovec *iov0 = &vcons_iov[0], *iov1 = &vcons_iov[1];
+ struct mic_info *mic = (struct mic_info *)arg;
+ int err;
+ struct pollfd console_poll[MAX_CONSOLE_FD];
+ int pty_fd;
+ char *pts_name;
+ ssize_t len;
+ struct mic_vring tx_vr, rx_vr;
+ struct mic_copy_desc copy;
+ struct mic_device_desc *desc;
+
+ pty_fd = posix_openpt(O_RDWR);
+ if (pty_fd < 0) {
+ mpsslog("can't open a pseudoterminal master device: %s\n",
+ strerror(errno));
+ goto _return;
+ }
+ pts_name = ptsname(pty_fd);
+ if (pts_name == NULL) {
+ mpsslog("can't get pts name\n");
+ goto _close_pty;
+ }
+ printf("%s console message goes to %s\n", mic->name, pts_name);
+ mpsslog("%s console message goes to %s\n", mic->name, pts_name);
+ err = grantpt(pty_fd);
+ if (err < 0) {
+ mpsslog("can't grant access: %s %s\n",
+ pts_name, strerror(errno));
+ goto _close_pty;
+ }
+ err = unlockpt(pty_fd);
+ if (err < 0) {
+ mpsslog("can't unlock a pseudoterminal: %s %s\n",
+ pts_name, strerror(errno));
+ goto _close_pty;
+ }
+ console_poll[MONITOR_FD].fd = pty_fd;
+ console_poll[MONITOR_FD].events = POLLIN;
+
+ console_poll[VIRTIO_CONSOLE_FD].fd = mic->mic_console.virtio_console_fd;
+ console_poll[VIRTIO_CONSOLE_FD].events = POLLIN;
+
+ if (MAP_FAILED == init_vr(mic, mic->mic_console.virtio_console_fd,
+ VIRTIO_ID_CONSOLE, &tx_vr, &rx_vr,
+ virtcons_dev_page.dd.num_vq)) {
+ mpsslog("%s init_vr failed %s\n",
+ mic->name, strerror(errno));
+ goto _close_pty;
+ }
+
+ copy.iovcnt = 1;
+ desc = get_device_desc(mic, VIRTIO_ID_CONSOLE);
+
+ for (;;) {
+ console_poll[MONITOR_FD].revents = 0;
+ console_poll[VIRTIO_CONSOLE_FD].revents = 0;
+ err = poll(console_poll, MAX_CONSOLE_FD, -1);
+ if (err < 0) {
+ mpsslog("%s %d: poll failed: %s\n", __func__, __LINE__,
+ strerror(errno));
+ continue;
+ }
+ if (!(desc->status & VIRTIO_CONFIG_S_DRIVER_OK)) {
+ err = wait_for_card_driver(mic,
+ mic->mic_console.virtio_console_fd,
+ VIRTIO_ID_CONSOLE);
+ if (err) {
+ mpsslog("%s %s %d Exiting...\n",
+ mic->name, __func__, __LINE__);
+ break;
+ }
+ }
+
+ if (console_poll[MONITOR_FD].revents & POLLIN) {
+ copy.iov = iov0;
+ len = readv(pty_fd, copy.iov, copy.iovcnt);
+ if (len > 0) {
+#ifdef DEBUG
+ disp_iovec(mic, copy, __func__, __LINE__);
+ mpsslog("%s %s %d read from tap 0x%lx\n",
+ mic->name, __func__, __LINE__,
+ len);
+#endif
+ spin_for_descriptors(mic, &tx_vr);
+ txrx_prepare(VIRTIO_ID_CONSOLE, 1, &tx_vr,
+ ©, len);
+
+ err = mic_virtio_copy(mic,
+ mic->mic_console.virtio_console_fd,
+ &tx_vr, ©);
+ if (err < 0) {
+ mpsslog("%s %s %d mic_virtio_copy %s\n",
+ mic->name, __func__, __LINE__,
+ strerror(errno));
+ }
+ if (!err)
+ verify_out_len(mic, ©);
+#ifdef DEBUG
+ disp_iovec(mic, copy, __func__, __LINE__);
+ mpsslog("%s %s %d wrote to net 0x%lx\n",
+ mic->name, __func__, __LINE__,
+ sum_iovec_len(copy));
+#endif
+ /* Reinitialize IOV for next run */
+ iov0->iov_len = PAGE_SIZE;
+ } else if (len < 0) {
+ disp_iovec(mic, ©, __func__, __LINE__);
+ mpsslog("%s %s %d read failed %s ",
+ mic->name, __func__, __LINE__,
+ strerror(errno));
+ mpsslog("cnt %d sum %zd\n",
+ copy.iovcnt, sum_iovec_len(©));
+ }
+ }
+
+ if (console_poll[VIRTIO_CONSOLE_FD].revents & POLLIN) {
+ while (rx_vr.info->avail_idx !=
+ le16toh(rx_vr.vr.avail->idx)) {
+ copy.iov = iov1;
+ txrx_prepare(VIRTIO_ID_CONSOLE, 0, &rx_vr,
+ ©, PAGE_SIZE);
+
+ err = mic_virtio_copy(mic,
+ mic->mic_console.virtio_console_fd,
+ &rx_vr, ©);
+ if (!err) {
+ /* Set the correct output iov_len */
+ iov1->iov_len = copy.out_len;
+ verify_out_len(mic, ©);
+#ifdef DEBUG
+ disp_iovec(mic, copy, __func__,
+ __LINE__);
+ mpsslog("%s %s %d ",
+ mic->name, __func__, __LINE__);
+ mpsslog("read from net 0x%lx\n",
+ sum_iovec_len(copy));
+#endif
+ len = writev(pty_fd,
+ copy.iov, copy.iovcnt);
+ if (len != sum_iovec_len(©)) {
+ mpsslog("Tun write failed %s ",
+ strerror(errno));
+ mpsslog("len 0x%zx ", len);
+ mpsslog("read_len 0x%zx\n",
+ sum_iovec_len(©));
+ } else {
+#ifdef DEBUG
+ disp_iovec(mic, copy, __func__,
+ __LINE__);
+ mpsslog("%s %s %d ",
+ mic->name, __func__,
+ __LINE__);
+ mpsslog("wrote to tap 0x%lx\n",
+ len);
+#endif
+ }
+ } else {
+ mpsslog("%s %s %d mic_virtio_copy %s\n",
+ mic->name, __func__, __LINE__,
+ strerror(errno));
+ break;
+ }
+ }
+ }
+ if (console_poll[NET_FD_VIRTIO_NET].revents & POLLERR)
+ mpsslog("%s: %s: POLLERR\n", __func__, mic->name);
+ }
+_close_pty:
+ close(pty_fd);
+_return:
+ pthread_exit(NULL);
+}
+
+static void
+add_virtio_device(struct mic_info *mic, struct mic_device_desc *dd)
+{
+ char path[PATH_MAX];
+ int fd, err;
+
+ snprintf(path, PATH_MAX, "/dev/vop_virtio%d", mic->id);
+ fd = open(path, O_RDWR);
+ if (fd < 0) {
+ mpsslog("Could not open %s %s\n", path, strerror(errno));
+ return;
+ }
+
+ err = ioctl(fd, MIC_VIRTIO_ADD_DEVICE, dd);
+ if (err < 0) {
+ mpsslog("Could not add %d %s\n", dd->type, strerror(errno));
+ close(fd);
+ return;
+ }
+ switch (dd->type) {
+ case VIRTIO_ID_NET:
+ mic->mic_net.virtio_net_fd = fd;
+ mpsslog("Added VIRTIO_ID_NET for %s\n", mic->name);
+ break;
+ case VIRTIO_ID_CONSOLE:
+ mic->mic_console.virtio_console_fd = fd;
+ mpsslog("Added VIRTIO_ID_CONSOLE for %s\n", mic->name);
+ break;
+ case VIRTIO_ID_BLOCK:
+ mic->mic_virtblk.virtio_block_fd = fd;
+ mpsslog("Added VIRTIO_ID_BLOCK for %s\n", mic->name);
+ break;
+ }
+}
+
+static bool
+set_backend_file(struct mic_info *mic)
+{
+ FILE *config;
+ char buff[PATH_MAX], *line, *evv, *p;
+
+ snprintf(buff, PATH_MAX, "%s/mpssd%03d.conf", mic_config_dir, mic->id);
+ config = fopen(buff, "r");
+ if (config == NULL)
+ return false;
+ do { /* look for "virtblk_backend=XXXX" */
+ line = fgets(buff, PATH_MAX, config);
+ if (line == NULL)
+ break;
+ if (*line == '#')
+ continue;
+ p = strchr(line, '\n');
+ if (p)
+ *p = '\0';
+ } while (strncmp(line, virtblk_backend, strlen(virtblk_backend)) != 0);
+ fclose(config);
+ if (line == NULL)
+ return false;
+ evv = strchr(line, '=');
+ if (evv == NULL)
+ return false;
+ mic->mic_virtblk.backend_file = malloc(strlen(evv) + 1);
+ if (mic->mic_virtblk.backend_file == NULL) {
+ mpsslog("%s %d can't allocate memory\n", mic->name, mic->id);
+ return false;
+ }
+ strcpy(mic->mic_virtblk.backend_file, evv + 1);
+ return true;
+}
+
+#define SECTOR_SIZE 512
+static bool
+set_backend_size(struct mic_info *mic)
+{
+ mic->mic_virtblk.backend_size = lseek(mic->mic_virtblk.backend, 0,
+ SEEK_END);
+ if (mic->mic_virtblk.backend_size < 0) {
+ mpsslog("%s: can't seek: %s\n",
+ mic->name, mic->mic_virtblk.backend_file);
+ return false;
+ }
+ virtblk_dev_page.blk_config.capacity =
+ mic->mic_virtblk.backend_size / SECTOR_SIZE;
+ if ((mic->mic_virtblk.backend_size % SECTOR_SIZE) != 0)
+ virtblk_dev_page.blk_config.capacity++;
+
+ virtblk_dev_page.blk_config.capacity =
+ htole64(virtblk_dev_page.blk_config.capacity);
+
+ return true;
+}
+
+static bool
+open_backend(struct mic_info *mic)
+{
+ if (!set_backend_file(mic))
+ goto _error_exit;
+ mic->mic_virtblk.backend = open(mic->mic_virtblk.backend_file, O_RDWR);
+ if (mic->mic_virtblk.backend < 0) {
+ mpsslog("%s: can't open: %s\n", mic->name,
+ mic->mic_virtblk.backend_file);
+ goto _error_free;
+ }
+ if (!set_backend_size(mic))
+ goto _error_close;
+ mic->mic_virtblk.backend_addr = mmap(NULL,
+ mic->mic_virtblk.backend_size,
+ PROT_READ|PROT_WRITE, MAP_SHARED,
+ mic->mic_virtblk.backend, 0L);
+ if (mic->mic_virtblk.backend_addr == MAP_FAILED) {
+ mpsslog("%s: can't map: %s %s\n",
+ mic->name, mic->mic_virtblk.backend_file,
+ strerror(errno));
+ goto _error_close;
+ }
+ return true;
+
+ _error_close:
+ close(mic->mic_virtblk.backend);
+ _error_free:
+ free(mic->mic_virtblk.backend_file);
+ _error_exit:
+ return false;
+}
+
+static void
+close_backend(struct mic_info *mic)
+{
+ munmap(mic->mic_virtblk.backend_addr, mic->mic_virtblk.backend_size);
+ close(mic->mic_virtblk.backend);
+ free(mic->mic_virtblk.backend_file);
+}
+
+static bool
+start_virtblk(struct mic_info *mic, struct mic_vring *vring)
+{
+ if (((unsigned long)&virtblk_dev_page.blk_config % 8) != 0) {
+ mpsslog("%s: blk_config is not 8 byte aligned.\n",
+ mic->name);
+ return false;
+ }
+ add_virtio_device(mic, &virtblk_dev_page.dd);
+ if (MAP_FAILED == init_vr(mic, mic->mic_virtblk.virtio_block_fd,
+ VIRTIO_ID_BLOCK, vring, NULL,
+ virtblk_dev_page.dd.num_vq)) {
+ mpsslog("%s init_vr failed %s\n",
+ mic->name, strerror(errno));
+ return false;
+ }
+ return true;
+}
+
+static void
+stop_virtblk(struct mic_info *mic)
+{
+ int vr_size, ret;
+
+ vr_size = PAGE_ALIGN(_vring_size(MIC_VRING_ENTRIES,
+ MIC_VIRTIO_RING_ALIGN) +
+ sizeof(struct _mic_vring_info));
+ ret = munmap(mic->mic_virtblk.block_dp,
+ MIC_DEVICE_PAGE_END + vr_size * virtblk_dev_page.dd.num_vq);
+ if (ret < 0)
+ mpsslog("%s munmap errno %d\n", mic->name, errno);
+ close(mic->mic_virtblk.virtio_block_fd);
+}
+
+static __u8
+header_error_check(struct vring_desc *desc)
+{
+ if (le32toh(desc->len) != sizeof(struct virtio_blk_outhdr)) {
+ mpsslog("%s() %d: length is not sizeof(virtio_blk_outhd)\n",
+ __func__, __LINE__);
+ return -EIO;
+ }
+ if (!(le16toh(desc->flags) & VRING_DESC_F_NEXT)) {
+ mpsslog("%s() %d: alone\n",
+ __func__, __LINE__);
+ return -EIO;
+ }
+ if (le16toh(desc->flags) & VRING_DESC_F_WRITE) {
+ mpsslog("%s() %d: not read\n",
+ __func__, __LINE__);
+ return -EIO;
+ }
+ return 0;
+}
+
+static int
+read_header(int fd, struct virtio_blk_outhdr *hdr, __u32 desc_idx)
+{
+ struct iovec iovec;
+ struct mic_copy_desc copy;
+
+ iovec.iov_len = sizeof(*hdr);
+ iovec.iov_base = hdr;
+ copy.iov = &iovec;
+ copy.iovcnt = 1;
+ copy.vr_idx = 0; /* only one vring on virtio_block */
+ copy.update_used = false; /* do not update used index */
+ return ioctl(fd, MIC_VIRTIO_COPY_DESC, ©);
+}
+
+static int
+transfer_blocks(int fd, struct iovec *iovec, __u32 iovcnt)
+{
+ struct mic_copy_desc copy;
+
+ copy.iov = iovec;
+ copy.iovcnt = iovcnt;
+ copy.vr_idx = 0; /* only one vring on virtio_block */
+ copy.update_used = false; /* do not update used index */
+ return ioctl(fd, MIC_VIRTIO_COPY_DESC, ©);
+}
+
+static __u8
+status_error_check(struct vring_desc *desc)
+{
+ if (le32toh(desc->len) != sizeof(__u8)) {
+ mpsslog("%s() %d: length is not sizeof(status)\n",
+ __func__, __LINE__);
+ return -EIO;
+ }
+ return 0;
+}
+
+static int
+write_status(int fd, __u8 *status)
+{
+ struct iovec iovec;
+ struct mic_copy_desc copy;
+
+ iovec.iov_base = status;
+ iovec.iov_len = sizeof(*status);
+ copy.iov = &iovec;
+ copy.iovcnt = 1;
+ copy.vr_idx = 0; /* only one vring on virtio_block */
+ copy.update_used = true; /* Update used index */
+ return ioctl(fd, MIC_VIRTIO_COPY_DESC, ©);
+}
+
+#ifndef VIRTIO_BLK_T_GET_ID
+#define VIRTIO_BLK_T_GET_ID 8
+#endif
+
+static void *
+virtio_block(void *arg)
+{
+ struct mic_info *mic = (struct mic_info *)arg;
+ int ret;
+ struct pollfd block_poll;
+ struct mic_vring vring;
+ __u16 avail_idx;
+ __u32 desc_idx;
+ struct vring_desc *desc;
+ struct iovec *iovec, *piov;
+ __u8 status;
+ __u32 buffer_desc_idx;
+ struct virtio_blk_outhdr hdr;
+ void *fos;
+
+ for (;;) { /* forever */
+ if (!open_backend(mic)) { /* No virtblk */
+ for (mic->mic_virtblk.signaled = 0;
+ !mic->mic_virtblk.signaled;)
+ sleep(1);
+ continue;
+ }
+
+ /* backend file is specified. */
+ if (!start_virtblk(mic, &vring))
+ goto _close_backend;
+ iovec = malloc(sizeof(*iovec) *
+ le32toh(virtblk_dev_page.blk_config.seg_max));
+ if (!iovec) {
+ mpsslog("%s: can't alloc iovec: %s\n",
+ mic->name, strerror(ENOMEM));
+ goto _stop_virtblk;
+ }
+
+ block_poll.fd = mic->mic_virtblk.virtio_block_fd;
+ block_poll.events = POLLIN;
+ for (mic->mic_virtblk.signaled = 0;
+ !mic->mic_virtblk.signaled;) {
+ block_poll.revents = 0;
+ /* timeout in 1 sec to see signaled */
+ ret = poll(&block_poll, 1, 1000);
+ if (ret < 0) {
+ mpsslog("%s %d: poll failed: %s\n",
+ __func__, __LINE__,
+ strerror(errno));
+ continue;
+ }
+
+ if (!(block_poll.revents & POLLIN)) {
+#ifdef DEBUG
+ mpsslog("%s %d: block_poll.revents=0x%x\n",
+ __func__, __LINE__, block_poll.revents);
+#endif
+ continue;
+ }
+
+ /* POLLIN */
+ while (vring.info->avail_idx !=
+ le16toh(vring.vr.avail->idx)) {
+ /* read header element */
+ avail_idx =
+ vring.info->avail_idx &
+ (vring.vr.num - 1);
+ desc_idx = le16toh(
+ vring.vr.avail->ring[avail_idx]);
+ desc = &vring.vr.desc[desc_idx];
+#ifdef DEBUG
+ mpsslog("%s() %d: avail_idx=%d ",
+ __func__, __LINE__,
+ vring.info->avail_idx);
+ mpsslog("vring.vr.num=%d desc=%p\n",
+ vring.vr.num, desc);
+#endif
+ status = header_error_check(desc);
+ ret = read_header(
+ mic->mic_virtblk.virtio_block_fd,
+ &hdr, desc_idx);
+ if (ret < 0) {
+ mpsslog("%s() %d %s: ret=%d %s\n",
+ __func__, __LINE__,
+ mic->name, ret,
+ strerror(errno));
+ break;
+ }
+ /* buffer element */
+ piov = iovec;
+ status = 0;
+ fos = mic->mic_virtblk.backend_addr +
+ (hdr.sector * SECTOR_SIZE);
+ buffer_desc_idx = next_desc(desc);
+ desc_idx = buffer_desc_idx;
+ for (desc = &vring.vr.desc[buffer_desc_idx];
+ desc->flags & VRING_DESC_F_NEXT;
+ desc_idx = next_desc(desc),
+ desc = &vring.vr.desc[desc_idx]) {
+ piov->iov_len = desc->len;
+ piov->iov_base = fos;
+ piov++;
+ fos += desc->len;
+ }
+ /* Returning NULLs for VIRTIO_BLK_T_GET_ID. */
+ if (hdr.type & ~(VIRTIO_BLK_T_OUT |
+ VIRTIO_BLK_T_GET_ID)) {
+ /*
+ VIRTIO_BLK_T_IN - does not do
+ anything. Probably for documenting.
+ VIRTIO_BLK_T_SCSI_CMD - for
+ virtio_scsi.
+ VIRTIO_BLK_T_FLUSH - turned off in
+ config space.
+ VIRTIO_BLK_T_BARRIER - defined but not
+ used in anywhere.
+ */
+ mpsslog("%s() %d: type %x ",
+ __func__, __LINE__,
+ hdr.type);
+ mpsslog("is not supported\n");
+ status = -ENOTSUP;
+
+ } else {
+ ret = transfer_blocks(
+ mic->mic_virtblk.virtio_block_fd,
+ iovec,
+ piov - iovec);
+ if (ret < 0 &&
+ status != 0)
+ status = ret;
+ }
+ /* write status and update used pointer */
+ if (status != 0)
+ status = status_error_check(desc);
+ ret = write_status(
+ mic->mic_virtblk.virtio_block_fd,
+ &status);
+#ifdef DEBUG
+ mpsslog("%s() %d: write status=%d on desc=%p\n",
+ __func__, __LINE__,
+ status, desc);
+#endif
+ }
+ }
+ free(iovec);
+_stop_virtblk:
+ stop_virtblk(mic);
+_close_backend:
+ close_backend(mic);
+ } /* forever */
+
+ pthread_exit(NULL);
+}
+
+static void
+change_virtblk_backend(int x, siginfo_t *siginfo, void *p)
+{
+ struct mic_info *mic;
+
+ for (mic = mic_list->next; mic != NULL; mic = mic->next)
+ mic->mic_virtblk.signaled = 1/* true */;
+}
+
+void
+serve_virtio(struct mic_info *mic, struct mic_info *list)
+{
+ struct sigaction act = {
+ .sa_flags = SA_SIGINFO,
+ .sa_sigaction = change_virtblk_backend,
+ };
+ int err;
+
+ add_virtio_device(mic, &virtcons_dev_page.dd);
+ add_virtio_device(mic, &virtnet_dev_page.dd);
+ err = pthread_create(&mic->mic_console.console_thread, NULL,
+ virtio_console, mic);
+ if (err)
+ mpsslog("%s virtcons pthread_create failed %s\n",
+ mic->name, strerror(err));
+ err = pthread_create(&mic->mic_net.net_thread, NULL,
+ virtio_net, mic);
+ if (err)
+ mpsslog("%s virtnet pthread_create failed %s\n",
+ mic->name, strerror(err));
+ err = pthread_create(&mic->mic_virtblk.block_thread, NULL,
+ virtio_block, mic);
+ if (err)
+ mpsslog("%s virtblk pthread_create failed %s\n",
+ mic->name, strerror(err));
+ sigemptyset(&act.sa_mask);
+ err = sigaction(SIGUSR1, &act, NULL);
+ if (err)
+ mpsslog("%s sigaction SIGUSR1 failed %s\n",
+ mic->name, strerror(errno));
+ while (1)
+ sleep(60);
+}
--
2.20.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v2 char-misc-next 7/7] samples: mic: Add sample VOP userspace
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
` (8 preceding siblings ...)
2019-02-22 15:30 ` [PATCH v2 char-misc-next 6/7] samples: mic: Split out vop code from mpssd Vincent Whitchurch
@ 2019-02-22 15:30 ` Vincent Whitchurch
9 siblings, 0 replies; 12+ messages in thread
From: Vincent Whitchurch @ 2019-02-22 15:30 UTC (permalink / raw)
To: sudeep.dutt, ashutosh.dixit, gregkh, arnd
Cc: linux-kernel, virtualization, Vincent Whitchurch
Add a sample userspace program which can be used with the VOP framework
with vop-loopback and non-MIC hardware.
For example, the following commands can be used to test networking over
vop-loopback. (IPv6 is used with zone indices to force traffic through
the interfaces instead of short-circuit delivery, since both interfaces
are on the same system.)
# modprobe vop_loopback
# ./vopd &
Setup host interface:
# ip a a dev mic0 scope link fe80::0
# ip link set dev mic0 up
Setup guest interface:
# ip a a dev eth1 scope link fe80::1
# ip link set dev eth1 up
Ping from host to guest:
# ping6 fe80::1%mic0
Ping from guest to host:
# ping6 fe80::0%eth1
Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
---
samples/mic/mpssd/.gitignore | 1 +
samples/mic/mpssd/Makefile | 5 ++++-
samples/mic/mpssd/vopd.c | 25 +++++++++++++++++++++++++
3 files changed, 30 insertions(+), 1 deletion(-)
create mode 100644 samples/mic/mpssd/vopd.c
diff --git a/samples/mic/mpssd/.gitignore b/samples/mic/mpssd/.gitignore
index 8b7c72f07c92..34c001135abb 100644
--- a/samples/mic/mpssd/.gitignore
+++ b/samples/mic/mpssd/.gitignore
@@ -1 +1,2 @@
mpssd
+vopd
diff --git a/samples/mic/mpssd/Makefile b/samples/mic/mpssd/Makefile
index bc94054dbe2b..b94b18de5a43 100644
--- a/samples/mic/mpssd/Makefile
+++ b/samples/mic/mpssd/Makefile
@@ -5,7 +5,7 @@ ARCH ?= $(shell echo $(uname_M) | sed -e s/i.86/x86/ -e s/x86_64/x86/)
ifeq ($(ARCH),x86)
-PROGS := mpssd
+PROGS := mpssd vopd
CC = $(CROSS_COMPILE)gcc
CFLAGS := -I../../../usr/include -I../../../tools/include
@@ -17,6 +17,9 @@ all: $(PROGS)
mpssd: mpssd.c sysfs.c vop.o
$(CC) $(CFLAGS) $^ -o $@ -lpthread
+vopd: vopd.c vop.o
+ $(CC) $(CFLAGS) $^ -o $@ -lpthread
+
install:
install mpssd /usr/sbin/mpssd
install micctrl /usr/sbin/micctrl
diff --git a/samples/mic/mpssd/vopd.c b/samples/mic/mpssd/vopd.c
new file mode 100644
index 000000000000..48c1877fed28
--- /dev/null
+++ b/samples/mic/mpssd/vopd.c
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <stdio.h>
+
+#include "mpssd.h"
+
+void mpsslog(char *format, ...)
+{
+ va_list args;
+
+ va_start(args, format);
+ vprintf(format, args);
+ va_end(args);
+}
+
+int main(int argc, char *argv[])
+{
+ struct mic_info themic = {
+ .name = "mic",
+ };
+
+ serve_virtio(&themic, &themic);
+
+ return 0;
+}
--
2.20.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH v2 char-misc-next 4/7] mic: vop: Add loopback driver
2019-02-22 15:30 ` [PATCH v2 char-misc-next 4/7] mic: vop: Add loopback driver Vincent Whitchurch
@ 2019-02-26 11:56 ` Greg KH
0 siblings, 0 replies; 12+ messages in thread
From: Greg KH @ 2019-02-26 11:56 UTC (permalink / raw)
To: Vincent Whitchurch
Cc: sudeep.dutt, ashutosh.dixit, arnd, linux-kernel, virtualization,
Vincent Whitchurch
On Fri, Feb 22, 2019 at 04:30:52PM +0100, Vincent Whitchurch wrote:
> Add a loopback driver to allow testing and evaluation of the VOP
> framework without special hardware. The host and the guest will run
> under the same kernel.
>
> Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com>
> ---
> drivers/misc/mic/Kconfig | 10 +
> drivers/misc/mic/vop/Makefile | 2 +
> drivers/misc/mic/vop/vop_loopback.c | 382 ++++++++++++++++++++++++++++
> 3 files changed, 394 insertions(+)
> create mode 100644 drivers/misc/mic/vop/vop_loopback.c
You sent 2 4/7 patches in this series, is that intentional? Are they
different?
I've applied the first 3 here, can you rebase and resend the rest of
these, without any duplicated patches?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2019-02-26 11:56 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-22 15:30 [PATCH v2 char-misc-next 0/7] Virtio-over-PCIe on non-MIC Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 1/7] mic: vop: Cast pointers to unsigned long Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 1/7] " Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 2/7] mic: Rename ioremap pointer to remap Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 3/7] mic: vop: Allow building on more systems Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 3/7] " Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 4/7] mic: vop: Add loopback driver Vincent Whitchurch
2019-02-26 11:56 ` Greg KH
2019-02-22 15:30 ` [PATCH v2 char-misc-next 4/7] " Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 5/7] mic: vop: Fix init race with shared interrupts Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 6/7] samples: mic: Split out vop code from mpssd Vincent Whitchurch
2019-02-22 15:30 ` [PATCH v2 char-misc-next 7/7] samples: mic: Add sample VOP userspace Vincent Whitchurch
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).