Re: Xen PV: Sample new PV driver for buffer sharing between domains

[Omkar Bolla <omkar.bolla@pathpartnertech.com>]

[-- Attachment #1.1: Type: text/plain, Size: 4318 bytes --]

Hi,
I tried to run script after pause the domain and unpause domain after run
script. But I ended up with same error

Below I shared  PV device log, and attached my FE and BE driver and script
that i used,
root@hikey960:/debian#
[XEN_BUF]xen_vdevb_be_probe(): 124: Probe called. We are good to go.
[  165.087419] [XEN_BUF]xen_vdevb_be_probe(): 125:  ffffffc017fb7000 1
[  165.093759] [XEN_BUF]xen_vdevb_be_probe(): 137: info->domid: 1
[  165.099641] [XEN_BUF]xen_vdevb_be_probe(): 138: devicetype: vdevb,
nodename: backend/vdevb/1/0, otherend: /local/domain/1/device/vdevb/0
[  165.112939] [XEN_BUF]xen_vdevb_be_frontend_changed(): 177: dev->state:
XenbusStateInitialising-1, frontend_state: XenbusStateInitialising-1

root@hikey960:/debian#
root@hikey960:/debian#
root@hikey960:/debian#
root@hikey960:/debian#
root@hikey960:/debian# xl console debian
[   22.243570] [XEN_BUF]xen_vdevb_fe_probe(): 24: Probe called. We are good
to go.
[   22.243606] [XEN_BUF]xen_vdevb_fe_probe(): 25:  ffffffc0160b4000 0
[   22.243620] [XEN_BUF]xen_vdevb_fe_probe(): 38: info->domid: 0
[   22.243633] [XEN_BUF]xen_vdevb_fe_probe(): 39: devicetype: vdevb,
nodename: device/vdevb/0, otherend: /local/domain/0/backend/vdevb/1/0
[   22.244669] [XEN_BUF]xen_vdevb_fe_backend_changed(): 64: dev->state:
XenbusStateInitialising-1, backend_state: XenbusStateInitWait-2
[   22.244701] [XEN_BUF]frontend_connect(): 53: Connecting the frontend now
[   22.245866] vdevb vdevb-0: 13 writing new state
[   22.246085] vdevb vdevb-0: failed to write error node for device/vdevb/0
(13 writing new state)
[   22.250005] vdevb vdevb-0: 13 writing new state
[   22.250220] vdevb vdevb-0: failed to write error node for device/vdevb/0
(13 writing new state)
root@hikey960:~#


Thanks,
Omkar  B


On Thu, Sep 27, 2018 at 4:33 PM Juergen Gross <jgross@suse.com> wrote:

> On 27/09/2018 12:35, Omkar Bolla wrote:
> > Hi,
> >
> > Sorry, I forgot, I used code from github chapter [2] from that link, and
> > I just changed name "mydevice"  to "vdevb"
>
> Okay.
>
> >
> >> Error 13 is EACCESS. I guess the access rights of the Xenstore nodes
> >> are not sufficient to write the needed entries.
> > Where I have to provide access rights, i.e from Kernel code or from from
> > command line in domain-0 or modify in xen source?
>
> I guess you have your frontend already loaded when running the
> script creating the Xenstore entries?
>
> This would explain the problem: as soon as the entries are written
> the frontend will react. At this point the access rights are not setup
> properly, this is done a little bit later in the script.
>
> Possible solutions are to either load the frontend driver only after
> setting up the Xenstore entries, or to pause the domain while doing
> so and unpause it afterwards (or start the domain paused, create the
> Xenstore entries, and unpause the domain).
>
> The really correct way to do it would be to setup Xenstore in a single
> transaction (write all the entries and set access rights).
>
> > Any thing that I have to do/change in xenbits xen-4.8 sources code, to
> > add new PV device?
>
> Only if you want to include domain config file entries for your device.
>
> >
> >> Did you modify Xen tools (xl/libxl) for adding the new device type?
> > No, is it needed to modify some thing in xl/libxl for adding new device
> > type?
>
> This was just a question to learn how Xenstore is being initialized
> in your scenario.
>
> >
> >> If not you need to setup the Xenstore nodes manually.
> > Setup manually Xenstore means, using commands?
>
> Yes, like your script does.
>
>
> Juergen
>

-- 






This
message contains confidential information and is intended only 
for the
individual(s) named. If you are not the intended
recipient, you are 
notified that disclosing, copying, distributing or taking any
action in 
reliance on the contents of this mail and attached file/s is strictly

prohibited. Please notify the
sender immediately and delete this e-mail 
from your system. E-mail transmission
cannot be guaranteed to be secured or 
error-free as information could be
intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain
viruses. The sender therefore does 
not accept liability for any errors or
omissions in the contents of this 
message, which arise as a result of e-mail
transmission.

[-- Attachment #1.2: Type: text/html, Size: 6986 bytes --]

[-- Attachment #2: xen_buf_frontend.c --]
[-- Type: text/x-csrc, Size: 3526 bytes --]

#include <linux/module.h>  /* Needed by all modules */
#include <linux/kernel.h>  /* Needed for KERN_ALERT */

#include <xen/xen.h>       /* We are doing something with Xen */
#include <xen/xenbus.h>

#include "xen_buf.h"

struct vdevbfrnt_info {
	struct xenbus_device *dev;
	unsigned int evtchn;
	unsigned int irq;

	grant_ref_t ring_ref;
	
};
// The function is called on activation of the device
static int xen_vdevb_fe_probe(struct xenbus_device *dev,
              const struct xenbus_device_id *id)
{
	struct vdevbfrnt_info *info = NULL;
	
	
	pr_log("Probe called. We are good to go.\n");
	pr_log(" %p %d\n", dev, dev->otherend_id);

	/* Allocating memory for private structure */
	info = kzalloc(sizeof(struct vdevbfrnt_info), GFP_KERNEL);
	if (!info) {
		xenbus_dev_fatal(dev, -ENOMEM,
				 "allocating frontend structure");
		return -ENOMEM;
	}

	info->dev = dev;
	dev_set_drvdata(&dev->dev, info);
//	info->domid = dev->otherend_id;
	pr_log("info->domid: %d\n", dev->otherend_id);
	pr_log("devicetype: %s, nodename: %s, otherend: %s\n", dev->devicetype, dev->nodename, dev->otherend);

//	xenbus_switch_state(dev, XenbusStateInitialised);
	return 0;
}

static int xen_vdevb_fe_remove(struct xenbus_device *dev)
{
	pr_log("\n");
	return 0;
}
// This is where we set up xenstore files and event channels
static int frontend_connect(struct xenbus_device *dev)
{
	pr_log("Connecting the frontend now\n");
	return 0;
}

// The function is called on a state change of the backend driver
static void xen_vdevb_fe_backend_changed(struct xenbus_device *dev,
			    enum xenbus_state backend_state)
{
	struct vdevbfrnt_info *info = dev_get_drvdata(&dev->dev);

//pr_log("info: %p\n", info);
pr_log("dev->state: %s-%d, backend_state: %s-%d\n", xenbus_state_array[dev->state], dev->state, xenbus_state_array[backend_state], backend_state);
	switch (backend_state)
	{
		case XenbusStateInitialising:
			xenbus_switch_state(dev, XenbusStateInitialising);
			break;
		case XenbusStateInitialised:
		case XenbusStateReconfiguring:
		case XenbusStateReconfigured:
		case XenbusStateUnknown:
			break;

		case XenbusStateInitWait:
			if (dev->state != XenbusStateInitialising)
				break;
			if (frontend_connect(dev) != 0)
				break;

			xenbus_switch_state(dev, XenbusStateConnected);

			break;

		case XenbusStateConnected:
			pr_log("Other side says it is connected as well.\n");
			break;

		case XenbusStateClosed:
			if (dev->state == XenbusStateClosed)
				break;
			/* Missed the backend's CLOSING state -- fallthrough */
		case XenbusStateClosing:
			xenbus_frontend_closed(dev);
	}
}

// This defines the name of the devices the driver reacts to
static const struct xenbus_device_id xen_vdevb_fe_ids[] = {
	{ "vdevb"  },
	{ ""  }
};

// We set up the callback functions
static struct xenbus_driver xen_vdevb_fe_driver = {
	.ids  = xen_vdevb_fe_ids,
	.probe = xen_vdevb_fe_probe,
	.remove = xen_vdevb_fe_remove,
	.otherend_changed = xen_vdevb_fe_backend_changed,
};

// On loading this kernel module, we register as a frontend driver
static int __init xen_vdevb_fe_init(void)
{
pr_log("xen_domain_type: %d, xen_domain: %d\n", xen_domain_type,xen_domain());
	pr_log("Hello World!\n");

	return xenbus_register_frontend(&xen_vdevb_fe_driver);
}
module_init(xen_vdevb_fe_init);

// ...and on unload we unregister
static void __exit xen_vdevb_fe_exit(void)
{
	xenbus_unregister_driver(&xen_vdevb_fe_driver);
	pr_log( "Goodbye world.\n");
}
module_exit(xen_vdevb_fe_exit);

MODULE_LICENSE("GPL");
MODULE_ALIAS("xen-clk: vdevb");

[-- Attachment #3: xen_buf_backend.c --]
[-- Type: text/x-csrc, Size: 6181 bytes --]

#include <linux/module.h>  /* Needed by all modules */
#include <linux/kernel.h>  /* Needed for KERN_ALERT */

#include <xen/xen.h>       /* We are doing something with Xen */
#include <xen/xenbus.h>

#include "xen_buf.h"




struct vdevbbk_info {
	struct xenbus_device *dev;

	domid_t domid;
	unsigned int irq;

//	struct vscsiif_back_ring ring;
	int ring_error;

	spinlock_t ring_lock;
	atomic_t nr_unreplied_reqs;

	spinlock_t vdevb_lock;
	struct list_head vdevb_entry_lists;

	wait_queue_head_t waiting_to_free;
};

// This is where we set up path watchers and event channels
static void backend_connect(struct xenbus_device *dev)
{
	pr_log("Connecting the backend now\n");
}
// This will destroy event channel handlers
static void backend_disconnect(struct xenbus_device *dev)
{
	pr_log("Connecting the backend now\n");
}
// We try to switch to the next state from a previous one
static void set_backend_state(struct xenbus_device *dev,
			      enum xenbus_state state)
{
//pr_log("dev->state: %s-%d, state: %s-%d\n", xenbus_state_array[dev->state], dev->state, xenbus_state_array[state], state);
	while (dev->state != state) {
		switch (dev->state) {
		case XenbusStateInitialising:
			switch (state) {
			case XenbusStateInitWait:
			case XenbusStateConnected:
			case XenbusStateClosing:
				xenbus_switch_state(dev, XenbusStateInitWait);
				break;
			case XenbusStateClosed:
				xenbus_switch_state(dev, XenbusStateClosed);
				break;
			default:
				BUG();
			}
			break;
		case XenbusStateClosed:
			switch (state) {
			case XenbusStateInitWait:
			case XenbusStateConnected:
				xenbus_switch_state(dev, XenbusStateInitWait);
				break;
			case XenbusStateClosing:
				xenbus_switch_state(dev, XenbusStateClosing);
				break;
			default:
				BUG();
			}
			break;
		case XenbusStateInitWait:
			switch (state) {
			case XenbusStateConnected:
				backend_connect(dev);
				xenbus_switch_state(dev, XenbusStateConnected);
				break;
			case XenbusStateClosing:
			case XenbusStateClosed:
				xenbus_switch_state(dev, XenbusStateClosing);
				break;
			default:
				BUG();
			}
			break;
		case XenbusStateConnected:
			switch (state) {
			case XenbusStateInitWait:
			case XenbusStateClosing:
			case XenbusStateClosed:
				backend_disconnect(dev);
				xenbus_switch_state(dev, XenbusStateClosing);
				break;
			default:
				BUG();
			}
			break;
		case XenbusStateClosing:
			switch (state) {
			case XenbusStateInitWait:
			case XenbusStateConnected:
			case XenbusStateClosed:
				xenbus_switch_state(dev, XenbusStateClosed);
				break;
			default:
				BUG();
			}
			break;
		default:
			BUG();
		}
	}
}

// The function is called on activation of the device
static int xen_vdevb_be_probe(struct xenbus_device *dev,
			const struct xenbus_device_id *id)
{
	struct vdevbbk_info *info = NULL;
	int ret = 0;

	pr_log("Probe called. We are good to go.\n");
	pr_log(" %p %d\n", dev, dev->otherend_id);

	/* Allocating memory for private structure */
	info = kzalloc(sizeof(struct vdevbbk_info), GFP_KERNEL);
	if (!info) {
		xenbus_dev_fatal(dev, -ENOMEM, 
				"allocating backend structure");
                return -ENOMEM;
        }
	info->dev = dev;
	dev_set_drvdata(&dev->dev, info);
	info->domid = dev->otherend_id;
	pr_log("info->domid: %d\n", info->domid);
	pr_log("devicetype: %s, nodename: %s, otherend: %s\n", dev->devicetype, dev->nodename, dev->otherend);

#if 0
	spin_lock_init(&info->ring_lock);
	info->ring_error = 0;
	atomic_set(&info->nr_unreplied_reqs, 0);
	init_waitqueue_head(&info->waiting_to_free);
	info->irq = 0;
	INIT_LIST_HEAD(&info->vdevb_entry_lists);
	spin_lock_init(&info->vdevb_lock);
#endif
	
	ret = xenbus_printf(XBT_NIL, dev->nodename, "vdevb-version", "%s", "0.1");
	if (ret)
                xenbus_dev_error(dev, ret, "writing vdevb-version");

	ret = xenbus_switch_state(dev, XenbusStateInitialising);
	if (ret)
		goto error;
	return 0;

 error:
	pr_warn("%s failed\n", __func__);
	kfree(info);
	dev_set_drvdata(&dev->dev, NULL);
	return ret;
}

static int xen_vdevb_be_remove(struct xenbus_device *dev)
{
	pr_log("\n");
	return 0;
}

// The function is called on a state change of the frontend driver
static void xen_vdevb_be_frontend_changed(struct xenbus_device *dev,
			enum xenbus_state frontend_state)
{
	struct vdevbbk_info *info = dev_get_drvdata(&dev->dev);
pr_log("dev->state: %s-%d, frontend_state: %s-%d\n", xenbus_state_array[dev->state], dev->state, xenbus_state_array[frontend_state], frontend_state);

	switch (frontend_state) {
		case XenbusStateInitialising:
			set_backend_state(dev, XenbusStateInitWait);
			break;

		case XenbusStateInitialised:
			break;

		case XenbusStateConnected:
			set_backend_state(dev, XenbusStateConnected);
			break;

		case XenbusStateClosing:
			set_backend_state(dev, XenbusStateClosing);
			break;

		case XenbusStateClosed:
			set_backend_state(dev, XenbusStateClosed);
			if (xenbus_dev_is_online(dev))
				break;
			/* fall through if not online */
		case XenbusStateUnknown:
			set_backend_state(dev, XenbusStateClosed);
			device_unregister(&dev->dev);
			break;

		default:
			xenbus_dev_fatal(dev, -EINVAL,
				        "saw state %s (%d) at frontend",
					xenbus_strstate(frontend_state),
					frontend_state);
			break;
	}
}

// This defines the name of the devices the driver reacts to
static const struct xenbus_device_id xen_vdevb_be_ids[] = {
	{ "vdevb" },
	{ "" }
};

// We set up the callback functions
static struct xenbus_driver xen_vdevb_be_driver = {
	.ids  = xen_vdevb_be_ids,
	.probe = xen_vdevb_be_probe,
	.remove = xen_vdevb_be_remove,
	.otherend_changed = xen_vdevb_be_frontend_changed,
};

// On loading this kernel module, we register as a frontend driver
static int __init xen_vdevb_be_init(void)
{
	pr_log("xen_domain_type: %d, xen_domain: %d\n", xen_domain_type,xen_domain());
	pr_log("Hello World!\n");

	return xenbus_register_backend(&xen_vdevb_be_driver);
}
module_init(xen_vdevb_be_init);

// ...and on unload we unregister
static void __exit xen_vdevb_be_exit(void)
{
	xenbus_unregister_driver(&xen_vdevb_be_driver);
	pr_log("Goodbye world.\n");
}
module_exit(xen_vdevb_be_exit);

MODULE_LICENSE("GPL");
MODULE_ALIAS("xen-clk-backend :vdevb");

[-- Attachment #4: xen_buf.h --]
[-- Type: text/x-chdr, Size: 1128 bytes --]

#ifndef __XEN_BUF__

#define pr_log(fmt, ...) { \
pr_info("[XEN_BUF]%s(): %d: "fmt, __func__, __LINE__, ##__VA_ARGS__); \
}

static char *xenbus_state_array[] = {
                              
	"XenbusStateUnknown",
	"XenbusStateInitialising",
	"XenbusStateInitWait",        /* Finished early
                                         initialisation, but waiting
                                         for information from the peer
                                         or hotplug scripts. */
	"XenbusStateInitialised",     /* Initialised and waiting for a
                                         connection from the peer. */
	"XenbusStateConnected",
	"XenbusStateClosing",         /* The device is being closed
                                         due to an error or an unplug
                                         event. */
	"XenbusStateClosed",
                        
        /*                      
        * Reconfiguring: The device is being reconfigured.
        */              
        "XenbusStateReconfiguring",
                        
        "XenbusStateReconfigured",
};

#endif /* __XEN_BUF__ */

[-- Attachment #5: activate.sh --]
[-- Type: application/x-shellscript, Size: 936 bytes --]

[-- Attachment #6: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel