All of lore.kernel.org
 help / color / mirror / Atom feed
* [DOCDAY PATCH] docs: initial documentation for xenstore paths
@ 2012-07-30 14:03 Ian Campbell
  2012-07-30 16:11 ` [DOCDAY PATCH] docs: update xenstore-paths.markdown with HVM paths Ian Campbell
                   ` (2 more replies)
  0 siblings, 3 replies; 17+ messages in thread
From: Ian Campbell @ 2012-07-30 14:03 UTC (permalink / raw)
  To: xen-devel; +Cc: Ian Campbell

This is based upon my inspection of a system with a single PV domain running
and is therefore very incomplete.

There are several things I'm not sure of here, mostly marked with XXX in the
text.

In particular:

 - We seem to expose various things to the guest which really it has no need to
   know (at least not via xenstore). e.g. its own domid, its device model pid,
   the size of the video ram, store port and gref.
 - Missing reference for ~/device-model/*
 - Missing protocol reference for ~/control/shutdown
 - What is the distinction between /vm/UUID and /local/domain/DOMID

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 docs/INDEX                        |    1 +
 docs/misc/xenstore-paths.markdown |  294 +++++++++++++++++++++++++++++++++++++
 2 files changed, 295 insertions(+), 0 deletions(-)

diff --git a/docs/INDEX b/docs/INDEX
index 5a0a2c2..f5ccae2 100644
--- a/docs/INDEX
+++ b/docs/INDEX
@@ -12,6 +12,7 @@ misc/kexec_and_kdump		Kexec and Kdump for Xen
 misc/tscmode			TSC Mode HOWTO
 misc/vbd-interface		Xen Guest Disk (VBD) Interface
 misc/xenstore			Xenstore protocol specification
+misc/xenstore-paths		Xenstore path documentation
 misc/xl-disk-configuration	XL Disk Configuration
 misc/xl-network-configuration	XL Network Configuration
 misc/distro_mapping		Distro Directory Layouts
diff --git a/docs/misc/xenstore-paths.markdown b/docs/misc/xenstore-paths.markdown
index e69de29..967ed7b 100644
--- a/docs/misc/xenstore-paths.markdown
+++ b/docs/misc/xenstore-paths.markdown
@@ -0,0 +1,294 @@
+# XenStore Paths
+
+This document attempts to defines all the paths which are in common
+use by either guests, front-/back-end drivers, toolstacks etc.
+
+The XenStore wire protocol itself is described in
+[xenstore.txt](xenstore.txt).
+
+## Notation
+
+This document is intended to be partially machine readable, such that
+test system etc can use it to validate whether the xenstore paths used
+by a test are allowable etc.
+
+Therefore the following notation conventions apply:
+
+A xenstore path is generically defined as:
+
+        PATH = VALUES [TAGS]
+    
+        PATH/* [TAGS]
+
+The first syntax defines a simple path with a single value. The second
+syntax defines an aggregated set of paths which are usually described
+externally to this document. The text will give a pointer to the
+appropriate external documentation.
+
+PATH can contain simple regex constructs following the POSIX regexp
+syntax described in regexp(7). In addition the following additional
+wild card names are defined and are evaluated before regexp expansion:
+
+* ~ -- expands to an arbitrary a domain's home path (described below).
+  Only valid at the begining of a path.
+* $DEVID -- a per-device type device identifier. Typically an integer.
+* $DOMID -- a domain identifier, an integer. Typically this refers to
+  the "other" domain. i.e. ~ refers to the domain providing a service
+  while $DOMID is the consumer of that service.
+* $UUID -- a UUID in the form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+
+VALUES are strings and can take the following forms:
+
+* PATH -- a XenStore path.
+* STRING -- an arbitrary string.
+* INTEGER -- the decimal representation of an integer.
+ * MEMKB -- the decimal representation of a number of kilobytes.
+ * EVTCHN -- the decimal representation of an event channel.
+ * GNTREF -- the decimal representation of a grant reference.
+* "a literal string" -- literal strings are contained within quotes.
+* (VALUE | VALUE | ... ) -- a set of alternatives. Alternatives are
+  separated by a "|" and all the alternatives are enclosed in "(" and
+  ")".
+
+Additional TAGS may follow as a comma separated set of the following
+tags enclosed in square brackets.
+
+* w -- Path is writable by the containing domain, that is the domain
+  whose home path ~ this key is under or which /vm/$UUID refers to. By
+  default paths under both of these locationsare read only for the
+  domain.
+* HVM -- Path is valid for HVM domains only
+* PV --  Path is valid for PV domains only
+* BACKEND -- Path is valid for a backend domain (AKA driver domain)
+
+Owning domain means the domain whose home path this tag is found
+under.
+
+Lack of either a __HVM__ or __PV__ tag indicates that the path is
+valid for either type of domain (including PVonHVM and similar mixed
+domain types).
+
+## Domain Home Path
+
+Every domain has a home path within the xenstore hierarchy. This is
+the path where the majority of the domain-visible information about
+each domain is stored.
+
+This path is:
+    
+      /local/domain/$DOMID
+
+All non-absolute paths are relative to this path.
+
+Although this path could be considered a "Home Directory" for the
+domain it would not usually be writable by the domain. The tools will
+create writable subdirectories as necessary.
+
+## Per Domain Paths
+
+## General Paths
+
+#### ~/vm = PATH []
+
+A pointer back to the domain's /vm/$UUID path (described below).
+
+#### ~/name = STRING []
+
+The guests name.
+
+#### ~/domid = INTEGER   []
+
+The domain's own ID.
+
+XXX why is this exposed to the guest here?
+
+#### ~/image/device-model-pid = INTEGER   [r]
+
+The process ID of the device model associated with this domain, if it
+has one.
+
+XXX why is this visible to the guest?
+
+#### ~/cpu/[0-9]+/availability = ("online"|"offline") [PV]
+
+One node for each virtual CPU up to the guest's configured
+maximum. Valid values are "online" and "offline". 
+
+#### ~/memory/static-max = MEMKB []
+
+Specifies a static maximum amount memory which this domain should
+expect to be given. In the absence of in-guest memory hotplug support
+this set on domain boot and is usually the maximum amount of RAM which
+a guest can make use of .
+
+#### ~/memory/target = MEMKB []
+
+The current balloon target for the domain. The balloon driver within the guest is expected to make every effort 
+
+#### ~/memory/videoram = MEMKB [HVM]
+
+The size of the video RAM this domain is configured with.
+
+XXX why is this exposed to the guest here instead of as a PCI BAR or
+some other property of the virtual GFX card? Perhaps should be
+non-guest visible.
+
+#### ~/device/suspend/event-channel = ""|EVTCHN [w]
+
+The domain's suspend event channel. The use of a suspend event channel
+is optional at the domain's discression. If it is not used then this
+path will be left blank.
+
+### Frontend device paths
+
+Paravirtual device frontends are generally specified by their own
+directory within the XenStore hierarchy. Usually this is under
+~/device/$TYPE/$DEVID although there are exceptions, e.g. ~/console
+for the first PV console.
+
+#### ~/device/vbd/$DEVID/* []
+
+A virtual block device frontend. Described by
+[xen/include/public/io/blkif.h][BLKIF]
+
+#### ~/device/vfb/$DEVID/* []
+
+A virtual framebuffer frontend. Described by
+[xen/include/public/io/fbif.h][FBIF]
+
+#### ~/device/vkbd/$DEVID/* []
+
+A virtual keyboard device frontend. Described by
+[xen/include/public/io/kbdif.h][KBDIF]
+
+#### ~/device/vif/$DEVID/* []
+
+A virtual network device frontend. Described by
+[xen/include/public/io/netif.h][NETIF]
+
+#### ~/console/* []
+
+The primary PV console device. Described in [console.txt](console.txt)
+
+#### ~/device/console/$DEVID/* []
+
+A secondary PV console device. Described in [console.txt](console.txt)
+
+#### ~/device/serial/$DEVID/* [HVM]
+
+An emulated serial device
+
+#### ~/store/port = EVTCHN []
+
+The event channel used by the domains connection to XenStore.
+
+XXX why is this exposed to the guest?
+
+#### ~/store/ring-ref = GNTREF []
+
+The grant reference of the domain's XenStore ring.
+
+XXX why is this exposed to the guest?
+
+### Backend Device Paths
+
+Paravirtual device backends are generally specified by their own
+directory within the XenStore hierarchy. Usually this is under
+~/backend/$TYPE/$DOMID/$DEVID.
+
+#### ~/backend/vbd/$DOMID/$DEVID/* []
+
+A virtual block device backend. Described by
+[xen/include/public/io/blkif.h][BLKIF]
+
+#### ~/backend/vfb/$DOMID/$DEVID/* []
+
+A virtual framebuffer backend. Described by
+[xen/include/public/io/fbif.h][FBIF]
+
+#### ~/backend/vkbd/$DOMID/$DEVID/* []
+
+A virtual keyboard device backend. Described by
+[xen/include/public/io/kbdif.h][KBDIF]
+
+#### ~/backend/vif/$DOMID/$DEVID/* []
+
+A virtual network device backend. Described by
+[xen/include/public/io/netif.h][NETIF]
+
+#### ~/backend/console/$DOMID/$DEVID/* []
+
+A PV console backend. Described in [console.txt](console.txt)
+
+#### ~/device-model/$DOMID/* []
+
+Information relating to device models running in the domain. $DOMID is
+the target domain of the device model.
+
+XXX where is the contents of this directory specified?
+
+#### ~/libxl/disable_udev = ("1"|"0") []
+
+Indicates whether device hotplug scripts in this domain should be run
+by udev ("0") or will be run by the toolstack directly ("1").
+
+### Platform Feature and Control Paths
+
+#### ~/control/shutdown = (""|COMMAND) [w]
+
+This is the PV shutdown control node. A toolstack can write various
+commands here to cause various guest shutdown, reboot or suspend
+activities. The guest acknowledges a request by writing the empty
+string back to the command node.
+
+XXX where is this protocol and the valid keys defined?
+
+#### ~/control/platform-feature-multiprocessor-suspend = (0|1) []
+
+Indicates to the guest that this platform supports the multiprocessor
+suspend feature.
+
+#### ~/control/platform-feature-xs\_reset\_watches = (0|1) []
+
+Indicates to the guest that this platform supports the
+XS_RESET_WATCHES xenstore message. See xen/include/public/io/xs\_wire.h
+for the XenStore wire protocol definition.
+
+### Domain Controlled Paths
+
+#### ~/data/* [w]
+
+A domain writable path. Available for arbitrary domain use.
+
+## Virtual Machine paths
+
+XXX somehow describe how /vm is different to /local/domain/
+
+### /vm/$UUID/uuid = UUID []
+
+Value is the same UUID as the path.
+
+### /vm/$UUID/name = STRING []
+
+The domains name.
+
+### /vm/$UUID/image/* []
+
+Various information relating to the domain builder.
+
+### /vm/$UUID/start_time = INTEGER "." INTEGER []
+
+The time which the guest was started in SECONDS.MICROSECONDS format
+
+## Platform-Level paths
+
+### libxl Specific Paths
+
+#### /libxl/$DOMID/dm-version ("qemu_xen"|"qemu_xen_traditional") = []
+
+The device model version for a domain.
+
+[BLKIF]: http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,blkif.h.html
+[FBIF]: http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,fbif.h.html
+[KBDIF]: http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,kbdif.h.html
+[NETIF]: http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,netif.h.html
-- 
1.7.2.5

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [DOCDAY PATCH] docs: update xenstore-paths.markdown with HVM paths.
  2012-07-30 14:03 [DOCDAY PATCH] docs: initial documentation for xenstore paths Ian Campbell
@ 2012-07-30 16:11 ` Ian Campbell
  2012-08-09 10:01 ` [DOCDAY PATCH] docs: initial documentation for xenstore paths Ian Campbell
  2012-08-09 21:02 ` Guest knowledge of own domid [was: docs: initial documentation for xenstore paths] Daniel De Graaf
  2 siblings, 0 replies; 17+ messages in thread
From: Ian Campbell @ 2012-07-30 16:11 UTC (permalink / raw)
  To: xen-devel; +Cc: Ian Campbell

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 docs/misc/xenstore-paths.markdown |   47 +++++++++++++++++++++++++++++++++++-
 1 files changed, 45 insertions(+), 2 deletions(-)

diff --git a/docs/misc/xenstore-paths.markdown b/docs/misc/xenstore-paths.markdown
index 967ed7b..193e592 100644
--- a/docs/misc/xenstore-paths.markdown
+++ b/docs/misc/xenstore-paths.markdown
@@ -41,7 +41,8 @@ VALUES are strings and can take the following forms:
 
 * PATH -- a XenStore path.
 * STRING -- an arbitrary string.
-* INTEGER -- the decimal representation of an integer.
+* INTEGER -- An integer, in decimal representation unless otherwise
+  noted.
  * MEMKB -- the decimal representation of a number of kilobytes.
  * EVTCHN -- the decimal representation of an event channel.
  * GNTREF -- the decimal representation of a grant reference.
@@ -60,6 +61,9 @@ tags enclosed in square brackets.
 * HVM -- Path is valid for HVM domains only
 * PV --  Path is valid for PV domains only
 * BACKEND -- Path is valid for a backend domain (AKA driver domain)
+* INTERNAL -- Although a path is visible to the domain its use is
+  reserved for the virtual firmware or Xen platform code. Guest
+  Operating Systems should not read this key.
 
 Owning domain means the domain whose home path this tag is found
 under.
@@ -139,6 +143,25 @@ The domain's suspend event channel. The use of a suspend event channel
 is optional at the domain's discression. If it is not used then this
 path will be left blank.
 
+#### ~/hvmloader/generation-id-address = ADDRESS [r,HVM,INTERNAL]
+
+The hexadecimal representation of the address of the domain's
+"generation id".
+
+XXX Where is this specified.
+
+#### ~/hvmloader/bios = ("rombios"|"seabios"|"OVMF") [HVM,INTERNAL]
+
+The BIOS used by this domain.
+
+#### ~/platform/* [HVM,INTERNAL]
+
+Various platform properties. 
+
+* acpi -- is ACPI enabled for this domain
+* acpi_s3 -- is ACPI S3 support enabled for this domain
+* acpi_s4 -- is ACPI S4 support enabled for this domain
+
 ### Frontend device paths
 
 Paravirtual device frontends are generally specified by their own
@@ -201,6 +224,22 @@ directory within the XenStore hierarchy. Usually this is under
 A virtual block device backend. Described by
 [xen/include/public/io/blkif.h][BLKIF]
 
+Uses the in-kernel blkback driver.
+
+#### ~/backend/qdisk/$DOMID/$DEVID/* []
+
+A virtual block device backend. Described by
+[xen/include/public/io/blkif.h][BLKIF]
+
+Uses the qemu based disk backend.
+
+#### ~/backend/tap/$DOMID/$DEVID/* []
+
+A virtual block device backend. Described by
+[xen/include/public/io/blkif.h][BLKIF]
+
+Uses the in-kernel blktap (v1) disk backend (deprecated).
+
 #### ~/backend/vfb/$DOMID/$DEVID/* []
 
 A virtual framebuffer backend. Described by
@@ -274,12 +313,16 @@ The domains name.
 
 ### /vm/$UUID/image/* []
 
-Various information relating to the domain builder.
+Various information relating to the domain builder used for this guest.
 
 ### /vm/$UUID/start_time = INTEGER "." INTEGER []
 
 The time which the guest was started in SECONDS.MICROSECONDS format
 
+### /vm/$UUID/rtc/timeoffset = ""|INTEGER [HVM,INTERNAL]
+
+The guest's virtual time offset from UTC in seconds.
+
 ## Platform-Level paths
 
 ### libxl Specific Paths
-- 
1.7.2.5

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths
  2012-07-30 14:03 [DOCDAY PATCH] docs: initial documentation for xenstore paths Ian Campbell
  2012-07-30 16:11 ` [DOCDAY PATCH] docs: update xenstore-paths.markdown with HVM paths Ian Campbell
@ 2012-08-09 10:01 ` Ian Campbell
  2012-08-09 13:38   ` Ian Jackson
  2012-08-09 14:02   ` Ian Jackson
  2012-08-09 21:02 ` Guest knowledge of own domid [was: docs: initial documentation for xenstore paths] Daniel De Graaf
  2 siblings, 2 replies; 17+ messages in thread
From: Ian Campbell @ 2012-08-09 10:01 UTC (permalink / raw)
  To: xen-devel; +Cc: Ian Jackson

On Mon, 2012-07-30 at 15:03 +0100, Ian Campbell wrote:
> This is based upon my inspection of a system with a single PV domain running
> and is therefore very incomplete.
> 
> There are several things I'm not sure of here, mostly marked with XXX in the
> text.
> 
> In particular:
> 
>  - We seem to expose various things to the guest which really it has no need to
>    know (at least not via xenstore). e.g. its own domid, its device model pid,
>    the size of the video ram, store port and gref.
>  - Missing reference for ~/device-model/*
>  - Missing protocol reference for ~/control/shutdown
>  - What is the distinction between /vm/UUID and /local/domain/DOMID
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

Any thoughts on this or the follow up?

Ian J is this machine-readable in a way which is useful for whatever it
is you wanted to machine read it into?

> ---
>  docs/INDEX                        |    1 +
>  docs/misc/xenstore-paths.markdown |  294 +++++++++++++++++++++++++++++++++++++
>  2 files changed, 295 insertions(+), 0 deletions(-)
> 
> diff --git a/docs/INDEX b/docs/INDEX
> index 5a0a2c2..f5ccae2 100644
> --- a/docs/INDEX
> +++ b/docs/INDEX
> @@ -12,6 +12,7 @@ misc/kexec_and_kdump		Kexec and Kdump for Xen
>  misc/tscmode			TSC Mode HOWTO
>  misc/vbd-interface		Xen Guest Disk (VBD) Interface
>  misc/xenstore			Xenstore protocol specification
> +misc/xenstore-paths		Xenstore path documentation
>  misc/xl-disk-configuration	XL Disk Configuration
>  misc/xl-network-configuration	XL Network Configuration
>  misc/distro_mapping		Distro Directory Layouts
> diff --git a/docs/misc/xenstore-paths.markdown b/docs/misc/xenstore-paths.markdown
> index e69de29..967ed7b 100644
> --- a/docs/misc/xenstore-paths.markdown
> +++ b/docs/misc/xenstore-paths.markdown
> @@ -0,0 +1,294 @@
> +# XenStore Paths
> +
> +This document attempts to defines all the paths which are in common
> +use by either guests, front-/back-end drivers, toolstacks etc.
> +
> +The XenStore wire protocol itself is described in
> +[xenstore.txt](xenstore.txt).
> +
> +## Notation
> +
> +This document is intended to be partially machine readable, such that
> +test system etc can use it to validate whether the xenstore paths used
> +by a test are allowable etc.
> +
> +Therefore the following notation conventions apply:
> +
> +A xenstore path is generically defined as:
> +
> +        PATH = VALUES [TAGS]
> +    
> +        PATH/* [TAGS]
> +
> +The first syntax defines a simple path with a single value. The second
> +syntax defines an aggregated set of paths which are usually described
> +externally to this document. The text will give a pointer to the
> +appropriate external documentation.
> +
> +PATH can contain simple regex constructs following the POSIX regexp
> +syntax described in regexp(7). In addition the following additional
> +wild card names are defined and are evaluated before regexp expansion:
> +
> +* ~ -- expands to an arbitrary a domain's home path (described below).
> +  Only valid at the begining of a path.
> +* $DEVID -- a per-device type device identifier. Typically an integer.
> +* $DOMID -- a domain identifier, an integer. Typically this refers to
> +  the "other" domain. i.e. ~ refers to the domain providing a service
> +  while $DOMID is the consumer of that service.
> +* $UUID -- a UUID in the form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
> +
> +VALUES are strings and can take the following forms:
> +
> +* PATH -- a XenStore path.
> +* STRING -- an arbitrary string.
> +* INTEGER -- the decimal representation of an integer.
> + * MEMKB -- the decimal representation of a number of kilobytes.
> + * EVTCHN -- the decimal representation of an event channel.
> + * GNTREF -- the decimal representation of a grant reference.
> +* "a literal string" -- literal strings are contained within quotes.
> +* (VALUE | VALUE | ... ) -- a set of alternatives. Alternatives are
> +  separated by a "|" and all the alternatives are enclosed in "(" and
> +  ")".
> +
> +Additional TAGS may follow as a comma separated set of the following
> +tags enclosed in square brackets.
> +
> +* w -- Path is writable by the containing domain, that is the domain
> +  whose home path ~ this key is under or which /vm/$UUID refers to. By
> +  default paths under both of these locationsare read only for the
> +  domain.
> +* HVM -- Path is valid for HVM domains only
> +* PV --  Path is valid for PV domains only
> +* BACKEND -- Path is valid for a backend domain (AKA driver domain)
> +
> +Owning domain means the domain whose home path this tag is found
> +under.
> +
> +Lack of either a __HVM__ or __PV__ tag indicates that the path is
> +valid for either type of domain (including PVonHVM and similar mixed
> +domain types).
> +
> +## Domain Home Path
> +
> +Every domain has a home path within the xenstore hierarchy. This is
> +the path where the majority of the domain-visible information about
> +each domain is stored.
> +
> +This path is:
> +    
> +      /local/domain/$DOMID
> +
> +All non-absolute paths are relative to this path.
> +
> +Although this path could be considered a "Home Directory" for the
> +domain it would not usually be writable by the domain. The tools will
> +create writable subdirectories as necessary.
> +
> +## Per Domain Paths
> +
> +## General Paths
> +
> +#### ~/vm = PATH []
> +
> +A pointer back to the domain's /vm/$UUID path (described below).
> +
> +#### ~/name = STRING []
> +
> +The guests name.
> +
> +#### ~/domid = INTEGER   []
> +
> +The domain's own ID.
> +
> +XXX why is this exposed to the guest here?
> +
> +#### ~/image/device-model-pid = INTEGER   [r]
> +
> +The process ID of the device model associated with this domain, if it
> +has one.
> +
> +XXX why is this visible to the guest?
> +
> +#### ~/cpu/[0-9]+/availability = ("online"|"offline") [PV]
> +
> +One node for each virtual CPU up to the guest's configured
> +maximum. Valid values are "online" and "offline". 
> +
> +#### ~/memory/static-max = MEMKB []
> +
> +Specifies a static maximum amount memory which this domain should
> +expect to be given. In the absence of in-guest memory hotplug support
> +this set on domain boot and is usually the maximum amount of RAM which
> +a guest can make use of .
> +
> +#### ~/memory/target = MEMKB []
> +
> +The current balloon target for the domain. The balloon driver within the guest is expected to make every effort 
> +
> +#### ~/memory/videoram = MEMKB [HVM]
> +
> +The size of the video RAM this domain is configured with.
> +
> +XXX why is this exposed to the guest here instead of as a PCI BAR or
> +some other property of the virtual GFX card? Perhaps should be
> +non-guest visible.
> +
> +#### ~/device/suspend/event-channel = ""|EVTCHN [w]
> +
> +The domain's suspend event channel. The use of a suspend event channel
> +is optional at the domain's discression. If it is not used then this
> +path will be left blank.
> +
> +### Frontend device paths
> +
> +Paravirtual device frontends are generally specified by their own
> +directory within the XenStore hierarchy. Usually this is under
> +~/device/$TYPE/$DEVID although there are exceptions, e.g. ~/console
> +for the first PV console.
> +
> +#### ~/device/vbd/$DEVID/* []
> +
> +A virtual block device frontend. Described by
> +[xen/include/public/io/blkif.h][BLKIF]
> +
> +#### ~/device/vfb/$DEVID/* []
> +
> +A virtual framebuffer frontend. Described by
> +[xen/include/public/io/fbif.h][FBIF]
> +
> +#### ~/device/vkbd/$DEVID/* []
> +
> +A virtual keyboard device frontend. Described by
> +[xen/include/public/io/kbdif.h][KBDIF]
> +
> +#### ~/device/vif/$DEVID/* []
> +
> +A virtual network device frontend. Described by
> +[xen/include/public/io/netif.h][NETIF]
> +
> +#### ~/console/* []
> +
> +The primary PV console device. Described in [console.txt](console.txt)
> +
> +#### ~/device/console/$DEVID/* []
> +
> +A secondary PV console device. Described in [console.txt](console.txt)
> +
> +#### ~/device/serial/$DEVID/* [HVM]
> +
> +An emulated serial device
> +
> +#### ~/store/port = EVTCHN []
> +
> +The event channel used by the domains connection to XenStore.
> +
> +XXX why is this exposed to the guest?
> +
> +#### ~/store/ring-ref = GNTREF []
> +
> +The grant reference of the domain's XenStore ring.
> +
> +XXX why is this exposed to the guest?
> +
> +### Backend Device Paths
> +
> +Paravirtual device backends are generally specified by their own
> +directory within the XenStore hierarchy. Usually this is under
> +~/backend/$TYPE/$DOMID/$DEVID.
> +
> +#### ~/backend/vbd/$DOMID/$DEVID/* []
> +
> +A virtual block device backend. Described by
> +[xen/include/public/io/blkif.h][BLKIF]
> +
> +#### ~/backend/vfb/$DOMID/$DEVID/* []
> +
> +A virtual framebuffer backend. Described by
> +[xen/include/public/io/fbif.h][FBIF]
> +
> +#### ~/backend/vkbd/$DOMID/$DEVID/* []
> +
> +A virtual keyboard device backend. Described by
> +[xen/include/public/io/kbdif.h][KBDIF]
> +
> +#### ~/backend/vif/$DOMID/$DEVID/* []
> +
> +A virtual network device backend. Described by
> +[xen/include/public/io/netif.h][NETIF]
> +
> +#### ~/backend/console/$DOMID/$DEVID/* []
> +
> +A PV console backend. Described in [console.txt](console.txt)
> +
> +#### ~/device-model/$DOMID/* []
> +
> +Information relating to device models running in the domain. $DOMID is
> +the target domain of the device model.
> +
> +XXX where is the contents of this directory specified?
> +
> +#### ~/libxl/disable_udev = ("1"|"0") []
> +
> +Indicates whether device hotplug scripts in this domain should be run
> +by udev ("0") or will be run by the toolstack directly ("1").
> +
> +### Platform Feature and Control Paths
> +
> +#### ~/control/shutdown = (""|COMMAND) [w]
> +
> +This is the PV shutdown control node. A toolstack can write various
> +commands here to cause various guest shutdown, reboot or suspend
> +activities. The guest acknowledges a request by writing the empty
> +string back to the command node.
> +
> +XXX where is this protocol and the valid keys defined?
> +
> +#### ~/control/platform-feature-multiprocessor-suspend = (0|1) []
> +
> +Indicates to the guest that this platform supports the multiprocessor
> +suspend feature.
> +
> +#### ~/control/platform-feature-xs\_reset\_watches = (0|1) []
> +
> +Indicates to the guest that this platform supports the
> +XS_RESET_WATCHES xenstore message. See xen/include/public/io/xs\_wire.h
> +for the XenStore wire protocol definition.
> +
> +### Domain Controlled Paths
> +
> +#### ~/data/* [w]
> +
> +A domain writable path. Available for arbitrary domain use.
> +
> +## Virtual Machine paths
> +
> +XXX somehow describe how /vm is different to /local/domain/
> +
> +### /vm/$UUID/uuid = UUID []
> +
> +Value is the same UUID as the path.
> +
> +### /vm/$UUID/name = STRING []
> +
> +The domains name.
> +
> +### /vm/$UUID/image/* []
> +
> +Various information relating to the domain builder.
> +
> +### /vm/$UUID/start_time = INTEGER "." INTEGER []
> +
> +The time which the guest was started in SECONDS.MICROSECONDS format
> +
> +## Platform-Level paths
> +
> +### libxl Specific Paths
> +
> +#### /libxl/$DOMID/dm-version ("qemu_xen"|"qemu_xen_traditional") = []
> +
> +The device model version for a domain.
> +
> +[BLKIF]: http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,blkif.h.html
> +[FBIF]: http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,fbif.h.html
> +[KBDIF]: http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,kbdif.h.html
> +[NETIF]: http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,netif.h.html

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths
  2012-08-09 10:01 ` [DOCDAY PATCH] docs: initial documentation for xenstore paths Ian Campbell
@ 2012-08-09 13:38   ` Ian Jackson
  2012-08-17 14:35     ` Ian Campbell
  2012-08-09 14:02   ` Ian Jackson
  1 sibling, 1 reply; 17+ messages in thread
From: Ian Jackson @ 2012-08-09 13:38 UTC (permalink / raw)
  To: Ian Campbell; +Cc: xen-devel

Ian Campbell writes ("Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths"):
> On Mon, 2012-07-30 at 15:03 +0100, Ian Campbell wrote:
> > This is based upon my inspection of a system with a single PV domain running
> > and is therefore very incomplete.
> > 
> > There are several things I'm not sure of here, mostly marked with XXX in the
> > text.
> > 
> > In particular:
> > 
> >  - We seem to expose various things to the guest which really it has no need to
> >    know (at least not via xenstore). e.g. its own domid, its device model pid,
> >    the size of the video ram, store port and gref.

Maybe we need to have a "???" or "?deprecated" tag ?  That would avoid
us documenting things which we don't want people to use.

> >  - Missing reference for ~/device-model/*
> >  - Missing protocol reference for ~/control/shutdown

These can wait for now.

> >  - What is the distinction between /vm/UUID and /local/domain/DOMID

I think the former should be abolished (eventually).

> Ian J is this machine-readable in a way which is useful for whatever it
> is you wanted to machine read it into?

I think it will do.

Thanks,
Ian.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths
  2012-08-09 10:01 ` [DOCDAY PATCH] docs: initial documentation for xenstore paths Ian Campbell
  2012-08-09 13:38   ` Ian Jackson
@ 2012-08-09 14:02   ` Ian Jackson
  2012-08-17 15:05     ` Ian Campbell
  1 sibling, 1 reply; 17+ messages in thread
From: Ian Jackson @ 2012-08-09 14:02 UTC (permalink / raw)
  To: Ian Campbell; +Cc: xen-devel

Ian Campbell writes ("Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths"):
...
> > --- a/docs/misc/xenstore-paths.markdown
> > +++ b/docs/misc/xenstore-paths.markdown
> > @@ -0,0 +1,294 @@
...
> > +PATH can contain simple regex constructs following the POSIX regexp
> > +syntax described in regexp(7). In addition the following additional
> > +wild card names are defined and are evaluated before regexp expansion:

Can we use a restricted perl re syntax ?  That avoids weirdness with
the rules for \.  Also how does this interact with markdown ?

> > +#### ~/image/device-model-pid = INTEGER   [r]

This [r] tag is not defined above.  I assume you mean "readonly to the
domain" but that's the default.  Left over from an earlier version ?

> > +The process ID of the device model associated with this domain, if it
> > +has one.
> > +
> > +XXX why is this visible to the guest?

I think some of these things were put here just because there wasn't
another place for the toolstack to store things.  See also the
arbitrary junk stored by scripts in the device backend directories.

> > +#### ~/cpu/[0-9]+/availability = ("online"|"offline") [PV]
> > +
> > +One node for each virtual CPU up to the guest's configured
> > +maximum. Valid values are "online" and "offline". 

Should have a cross-reference to the cpu online/offline protocol,
which appears to be in xen/include/public/vcpu.h.  It doesn't seem to
be fully documented yet.

> > +#### ~/memory/static-max = MEMKB []
> > +
> > +Specifies a static maximum amount memory which this domain should
> > +expect to be given. In the absence of in-guest memory hotplug support
> > +this set on domain boot and is usually the maximum amount of RAM which
> > +a guest can make use of .

This should have a cross-reference to the documentation defining
static-max etc.  I thought we had some in tree but I can't seem to
find it.  The best I can find is docs/man/xl.cfg.pod.5.

> > +#### ~/memory/target = MEMKB []
> > +
> > +The current balloon target for the domain. The balloon driver within the guest is expected to make every effort 

every effort to ... ?

The interaction with the Xen maximum should be stated, preferably by
cross-reference.  In general it might be better to have a single place
where all these values and their semantics are written down ?

> > +#### ~/device/suspend/event-channel = ""|EVTCHN [w]
> > +
> > +The domain's suspend event channel. The use of a suspend event channel
> > +is optional at the domain's discression. If it is not used then this
> > +path will be left blank.

May it be ENOENT ?  Does the toolstack create it as "" then ?

> > +#### ~/device/serial/$DEVID/* [HVM]
> > +
> > +An emulated serial device

You should presumably add
    XXX documentation for the protocol needed
here.

> > +#### ~/store/port = EVTCHN []
> > +
> > +The event channel used by the domains connection to XenStore.

Apostrophe.

> > +XXX why is this exposed to the guest?

Is there really only one event channel ?  Ie the same evtchn is used
to signal to xenstore that the guest has sent a command, and to signal
the guest that xenstore has written the response ?

Anyway surely this is something the guest needs to know.  Why it's in
xenstore is a bit of a mystery since you can't use xenstore without it
and it's in the start_info.

Is this the same value as start_info.store_evtchn ?  Cross reference ?

> > +#### ~/store/ring-ref = GNTREF []
> > +
> > +The grant reference of the domain's XenStore ring.
> > +
> > +XXX why is this exposed to the guest?

See above.

> > +#### ~/device-model/$DOMID/* []
> > +
> > +Information relating to device models running in the domain. $DOMID is
> > +the target domain of the device model.
> > +
> > +XXX where is the contents of this directory specified?

I think it's not specified anywhere.  It's ad-hoc.  The guest
shouldn't need to see it but exposing it readonly is probably
harmless.  Except perhaps for the vnc password ?

> > +## Virtual Machine paths
> > +
> > +XXX somehow describe how /vm is different to /local/domain/

See my other email.

> > +### /vm/$UUID/uuid = UUID []
> > +
> > +Value is the same UUID as the path.
> > +
> > +### /vm/$UUID/name = STRING []
> > +
> > +The domains name.

IMO this should be
  (a) in /local/domain/$DOMID
  (b) also a copy in /byname/$NAME = $DOMID   for fast lookup
but not in 4.2.

Guests shouldn't rely on it.  In fact do guests actually need anything
from here ?

Ian.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Guest knowledge of own domid [was: docs: initial documentation for xenstore paths]
  2012-07-30 14:03 [DOCDAY PATCH] docs: initial documentation for xenstore paths Ian Campbell
  2012-07-30 16:11 ` [DOCDAY PATCH] docs: update xenstore-paths.markdown with HVM paths Ian Campbell
  2012-08-09 10:01 ` [DOCDAY PATCH] docs: initial documentation for xenstore paths Ian Campbell
@ 2012-08-09 21:02 ` Daniel De Graaf
  2012-08-09 21:26   ` Jean Guyader
  2 siblings, 1 reply; 17+ messages in thread
From: Daniel De Graaf @ 2012-08-09 21:02 UTC (permalink / raw)
  To: Ian Campbell; +Cc: xen-devel

On 07/30/2012 10:03 AM, Ian Campbell wrote:
> This is based upon my inspection of a system with a single PV domain running
> and is therefore very incomplete.
> 
> There are several things I'm not sure of here, mostly marked with XXX in the
> text.
> 
> In particular:
> 
>  - We seem to expose various things to the guest which really it has no need to
>    know (at least not via xenstore). e.g. its own domid, its device model pid,
>    the size of the video ram, store port and gref.

If the domid key is unneeded/removed, is there a recommended method for
a guest to query its own domid? I don't see a hypercall that returns it
directly, although there is one to return the guest's UUID - which seems
much less useful for a guest to know about itself.

While hypercalls are fairly consistent about accepting DOMID_SELF, a 
domain does occasionally need to know its own ID: xenstore permission
changes do not accept DOMID_SELF, and if two domains are attempting to
set up communication such as V4V or vchan, they need to be able to tell
their peer what domain ID to use.

It is possible for a domain to query its own domain ID indirectly, so it 
would be difficult to argue that a domain should not be able to obtain 
its own ID. One method for a domain to query its own ID is to create an
unbound event channel with remote_domid = DOMID_SELF, and then execute
evtchn_status on the event channel in order to see the resolved domain 
id. Querying Xenstore permissions on a newly-created key will show the
local domain as the first entry. Less reliably, the backend paths for
all xenbus devices contain the local and remote domain IDs.

-- 
Daniel De Graaf
National Security Agency

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Guest knowledge of own domid [was: docs: initial documentation for xenstore paths]
  2012-08-09 21:02 ` Guest knowledge of own domid [was: docs: initial documentation for xenstore paths] Daniel De Graaf
@ 2012-08-09 21:26   ` Jean Guyader
  2012-08-16 11:28     ` Ian Campbell
  0 siblings, 1 reply; 17+ messages in thread
From: Jean Guyader @ 2012-08-09 21:26 UTC (permalink / raw)
  To: Daniel De Graaf; +Cc: Ian Campbell, xen-devel

On 9 August 2012 22:02, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote:
> On 07/30/2012 10:03 AM, Ian Campbell wrote:
>> This is based upon my inspection of a system with a single PV domain running
>> and is therefore very incomplete.
>>
>> There are several things I'm not sure of here, mostly marked with XXX in the
>> text.
>>
>> In particular:
>>
>>  - We seem to expose various things to the guest which really it has no need to
>>    know (at least not via xenstore). e.g. its own domid, its device model pid,
>>    the size of the video ram, store port and gref.
>
> If the domid key is unneeded/removed, is there a recommended method for
> a guest to query its own domid? I don't see a hypercall that returns it
> directly, although there is one to return the guest's UUID - which seems
> much less useful for a guest to know about itself.
>
> While hypercalls are fairly consistent about accepting DOMID_SELF, a
> domain does occasionally need to know its own ID: xenstore permission
> changes do not accept DOMID_SELF, and if two domains are attempting to
> set up communication such as V4V or vchan, they need to be able to tell
> their peer what domain ID to use.
>

That is one way for doing it another way would be to use a name
resolution system
a bit like a DNS. A system like that would need to live where the VM
are created and destroyed
(probably dom0 or a domain builder VM).
The server could be using vchan, v4v or even a shared XenStore node,
but I think we
need something like that.

In the long run it's much better to rely on a name instead of a domid
because domids can
change throughout the VM life cycle (reboot, hibernate, save/restore,
migration, ...).

Jean

> It is possible for a domain to query its own domain ID indirectly, so it
> would be difficult to argue that a domain should not be able to obtain
> its own ID. One method for a domain to query its own ID is to create an
> unbound event channel with remote_domid = DOMID_SELF, and then execute
> evtchn_status on the event channel in order to see the resolved domain
> id. Querying Xenstore permissions on a newly-created key will show the
> local domain as the first entry. Less reliably, the backend paths for
> all xenbus devices contain the local and remote domain IDs.
>

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Guest knowledge of own domid [was: docs: initial documentation for xenstore paths]
  2012-08-09 21:26   ` Jean Guyader
@ 2012-08-16 11:28     ` Ian Campbell
  2012-08-16 14:33       ` Daniel De Graaf
  0 siblings, 1 reply; 17+ messages in thread
From: Ian Campbell @ 2012-08-16 11:28 UTC (permalink / raw)
  To: Jean Guyader; +Cc: Daniel De Graaf, xen-devel

On Thu, 2012-08-09 at 22:26 +0100, Jean Guyader wrote:
> On 9 August 2012 22:02, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote:
> > On 07/30/2012 10:03 AM, Ian Campbell wrote:
> >> This is based upon my inspection of a system with a single PV domain running
> >> and is therefore very incomplete.
> >>
> >> There are several things I'm not sure of here, mostly marked with XXX in the
> >> text.
> >>
> >> In particular:
> >>
> >>  - We seem to expose various things to the guest which really it has no need to
> >>    know (at least not via xenstore). e.g. its own domid, its device model pid,
> >>    the size of the video ram, store port and gref.
> >
> > If the domid key is unneeded/removed, is there a recommended method for
> > a guest to query its own domid? I don't see a hypercall that returns it
> > directly, although there is one to return the guest's UUID - which seems
> > much less useful for a guest to know about itself.
> >
> > While hypercalls are fairly consistent about accepting DOMID_SELF, a
> > domain does occasionally need to know its own ID: xenstore permission
> > changes do not accept DOMID_SELF, 

I wonder if that would be a worthwhile protocol extension.

> and if two domains are attempting to
> > set up communication such as V4V or vchan, they need to be able to tell
> > their peer what domain ID to use.

That's trickier.

I suppose they could rendezvous via /vm/$UUID? Although there has been
talk of removing that path in the future.

> >
> 
> That is one way for doing it another way would be to use a name
> resolution system
> a bit like a DNS. A system like that would need to live where the VM
> are created and destroyed
> (probably dom0 or a domain builder VM).
> The server could be using vchan, v4v or even a shared XenStore node,
> but I think we
> need something like that.
> 
> In the long run it's much better to rely on a name instead of a domid
> because domids can
> change throughout the VM life cycle (reboot, hibernate, save/restore,
> migration, ...).

Right, this is the main reason to avoid building a reliance on domid
into a protocol.

> 
> Jean
> 
> > It is possible for a domain to query its own domain ID indirectly, so it
> > would be difficult to argue that a domain should not be able to obtain
> > its own ID. One method for a domain to query its own ID is to create an
> > unbound event channel with remote_domid = DOMID_SELF, and then execute
> > evtchn_status on the event channel in order to see the resolved domain
> > id. Querying Xenstore permissions on a newly-created key will show the
> > local domain as the first entry. Less reliably, the backend paths for
> > all xenbus devices contain the local and remote domain IDs.
> >

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Guest knowledge of own domid [was: docs: initial documentation for xenstore paths]
  2012-08-16 11:28     ` Ian Campbell
@ 2012-08-16 14:33       ` Daniel De Graaf
  2012-08-16 14:36         ` Ian Campbell
  0 siblings, 1 reply; 17+ messages in thread
From: Daniel De Graaf @ 2012-08-16 14:33 UTC (permalink / raw)
  To: Ian Campbell; +Cc: Jean Guyader, xen-devel

On 08/16/2012 07:28 AM, Ian Campbell wrote:
> On Thu, 2012-08-09 at 22:26 +0100, Jean Guyader wrote:
>> On 9 August 2012 22:02, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote:
>>> On 07/30/2012 10:03 AM, Ian Campbell wrote:
>>>> This is based upon my inspection of a system with a single PV domain running
>>>> and is therefore very incomplete.
>>>>
>>>> There are several things I'm not sure of here, mostly marked with XXX in the
>>>> text.
>>>>
>>>> In particular:
>>>>
>>>>  - We seem to expose various things to the guest which really it has no need to
>>>>    know (at least not via xenstore). e.g. its own domid, its device model pid,
>>>>    the size of the video ram, store port and gref.
>>>
>>> If the domid key is unneeded/removed, is there a recommended method for
>>> a guest to query its own domid? I don't see a hypercall that returns it
>>> directly, although there is one to return the guest's UUID - which seems
>>> much less useful for a guest to know about itself.
>>>
>>> While hypercalls are fairly consistent about accepting DOMID_SELF, a
>>> domain does occasionally need to know its own ID: xenstore permission
>>> changes do not accept DOMID_SELF, 
> 
> I wonder if that would be a worthwhile protocol extension.

It might be, although it's never required. After checking where this would
be useful, it looks like the caller could just call a get-permissions on
the node before set-permissions, and just avoid modification to the owner.
Since non-privileged xenstore clients cannot modify the owner of a xenstore
key, and cannot change permissions on keys they do not own, this change
would just end up saving one call to xenstore. Privileged domains already
take advantage of their override capabilities and usually do not add domain
0 to the node permissions list, so it's not useful there.

>> and if two domains are attempting to
>>> set up communication such as V4V or vchan, they need to be able to tell
>>> their peer what domain ID to use.
> 
> That's trickier.
> 
> I suppose they could rendezvous via /vm/$UUID? Although there has been
> talk of removing that path in the future.

The /vm/$UUID path isn't currently useful for this, since it doesn't maintain
domain IDs (just names) and doesn't contain writable sub-keys for a domain
to use. I also don't think such a sub-key should be added; it makes more
sense to keep all of a domain's modifiable keys under its home path.

Perhaps this could be changed to another identifier-to-domid mapping, like
the proposed addition of a location to map name to domid? 

The toolstack would maintain something like:
  /local/by-name/$name == domid
  /local/by-uuid/$uuid == domid
  /local/domain/$domid/name - same as existing
  /local/domain/$domid/uuid - ? maybe unneeded, as it's available from Xen.

>>>
>>
>> That is one way for doing it another way would be to use a name
>> resolution system
>> a bit like a DNS. A system like that would need to live where the VM
>> are created and destroyed
>> (probably dom0 or a domain builder VM).
>> The server could be using vchan, v4v or even a shared XenStore node,
>> but I think we
>> need something like that.
>>
>> In the long run it's much better to rely on a name instead of a domid
>> because domids can
>> change throughout the VM life cycle (reboot, hibernate, save/restore,
>> migration, ...).
> 
> Right, this is the main reason to avoid building a reliance on domid
> into a protocol.
> 

Both ends of any xen-based communication need to handle each of these events
in order to re-establish grants/events or v4v rings/ports. The domid does
need to be treated as a short-term identifier of a domain in a protocol that
expects to continue to work across such events.

>>
>> Jean
>>
>>> It is possible for a domain to query its own domain ID indirectly, so it
>>> would be difficult to argue that a domain should not be able to obtain
>>> its own ID. One method for a domain to query its own ID is to create an
>>> unbound event channel with remote_domid = DOMID_SELF, and then execute
>>> evtchn_status on the event channel in order to see the resolved domain
>>> id. Querying Xenstore permissions on a newly-created key will show the
>>> local domain as the first entry. Less reliably, the backend paths for
>>> all xenbus devices contain the local and remote domain IDs.
>>>

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Guest knowledge of own domid [was: docs: initial documentation for xenstore paths]
  2012-08-16 14:33       ` Daniel De Graaf
@ 2012-08-16 14:36         ` Ian Campbell
  2012-08-16 14:52           ` Daniel De Graaf
  0 siblings, 1 reply; 17+ messages in thread
From: Ian Campbell @ 2012-08-16 14:36 UTC (permalink / raw)
  To: Daniel De Graaf; +Cc: Jean Guyader, xen-devel

On Thu, 2012-08-16 at 15:33 +0100, Daniel De Graaf wrote:

> >> and if two domains are attempting to
> >>> set up communication such as V4V or vchan, they need to be able to tell
> >>> their peer what domain ID to use.
> > 
> > That's trickier.
> > 
> > I suppose they could rendezvous via /vm/$UUID? Although there has been
> > talk of removing that path in the future.
> 
> The /vm/$UUID path isn't currently useful for this, since it doesn't maintain
> domain IDs (just names) and doesn't contain writable sub-keys for a domain
> to use. I also don't think such a sub-key should be added; it makes more
> sense to keep all of a domain's modifiable keys under its home path.
> 
> Perhaps this could be changed to another identifier-to-domid mapping, like
> the proposed addition of a location to map name to domid? 
> 
> The toolstack would maintain something like:
>   /local/by-name/$name == domid
>   /local/by-uuid/$uuid == domid

This second one is a bit like the existing /vm/$uuid/domid.

I think I would go with:

  /local/by-name/$name == /local/domain/$domid
  /local/by-uuid/$uuid == /local/domain/$domid

though, So that you can just read it and use it without interpreting it.

>   /local/domain/$domid/name - same as existing
>   /local/domain/$domid/uuid - ? maybe unneeded, as it's available from Xen.

Is it available for other domains via xen, or just yourself?

Ian.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Guest knowledge of own domid [was: docs: initial documentation for xenstore paths]
  2012-08-16 14:36         ` Ian Campbell
@ 2012-08-16 14:52           ` Daniel De Graaf
  2012-08-17  8:05             ` Ian Campbell
  0 siblings, 1 reply; 17+ messages in thread
From: Daniel De Graaf @ 2012-08-16 14:52 UTC (permalink / raw)
  To: Ian Campbell; +Cc: Jean Guyader, xen-devel

On 08/16/2012 10:36 AM, Ian Campbell wrote:
> On Thu, 2012-08-16 at 15:33 +0100, Daniel De Graaf wrote:
> 
>>>> and if two domains are attempting to
>>>>> set up communication such as V4V or vchan, they need to be able to tell
>>>>> their peer what domain ID to use.
>>>
>>> That's trickier.
>>>
>>> I suppose they could rendezvous via /vm/$UUID? Although there has been
>>> talk of removing that path in the future.
>>
>> The /vm/$UUID path isn't currently useful for this, since it doesn't maintain
>> domain IDs (just names) and doesn't contain writable sub-keys for a domain
>> to use. I also don't think such a sub-key should be added; it makes more
>> sense to keep all of a domain's modifiable keys under its home path.
>>
>> Perhaps this could be changed to another identifier-to-domid mapping, like
>> the proposed addition of a location to map name to domid? 
>>
>> The toolstack would maintain something like:
>>   /local/by-name/$name == domid
>>   /local/by-uuid/$uuid == domid
> 
> This second one is a bit like the existing /vm/$uuid/domid.

That key isn't being populated by xl on my system, so I didn't know it existed.

> 
> I think I would go with:
> 
>   /local/by-name/$name == /local/domain/$domid
>   /local/by-uuid/$uuid == /local/domain/$domid
> 
> though, So that you can just read it and use it without interpreting it.

In that case, you would need to parse the domid out of the string in order
to use it in hypercalls (grant, event, v4v). The frontend/backend paths use
a distinct frontend-id/backend-id key for the domain ID, but we are trying
to avoid this since populating this key would mean the domain populating it
needs to know its own domain ID.

>>   /local/domain/$domid/name - same as existing
>>   /local/domain/$domid/uuid - ? maybe unneeded, as it's available from Xen.
> 
> Is it available for other domains via xen, or just yourself?
> 

Yourself and anyone who can call getdomaininfo (which is usually just dom0).
This is actually the same as the xenstore permissions on keys such as
/local/domain/$id/name. Changing these may need consideration, because on
public hosting servers, you might not want to allow domains to be enumerated
by name.

-- 
Daniel De Graaf
National Security Agency

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Guest knowledge of own domid [was: docs: initial documentation for xenstore paths]
  2012-08-16 14:52           ` Daniel De Graaf
@ 2012-08-17  8:05             ` Ian Campbell
  0 siblings, 0 replies; 17+ messages in thread
From: Ian Campbell @ 2012-08-17  8:05 UTC (permalink / raw)
  To: Daniel De Graaf; +Cc: Jean Guyader, xen-devel

On Thu, 2012-08-16 at 15:52 +0100, Daniel De Graaf wrote:
> On 08/16/2012 10:36 AM, Ian Campbell wrote:
> > On Thu, 2012-08-16 at 15:33 +0100, Daniel De Graaf wrote:
> > 
> >>>> and if two domains are attempting to
> >>>>> set up communication such as V4V or vchan, they need to be able to tell
> >>>>> their peer what domain ID to use.
> >>>
> >>> That's trickier.
> >>>
> >>> I suppose they could rendezvous via /vm/$UUID? Although there has been
> >>> talk of removing that path in the future.
> >>
> >> The /vm/$UUID path isn't currently useful for this, since it doesn't maintain
> >> domain IDs (just names) and doesn't contain writable sub-keys for a domain
> >> to use. I also don't think such a sub-key should be added; it makes more
> >> sense to keep all of a domain's modifiable keys under its home path.
> >>
> >> Perhaps this could be changed to another identifier-to-domid mapping, like
> >> the proposed addition of a location to map name to domid? 
> >>
> >> The toolstack would maintain something like:
> >>   /local/by-name/$name == domid
> >>   /local/by-uuid/$uuid == domid
> > 
> > This second one is a bit like the existing /vm/$uuid/domid.
> 
> That key isn't being populated by xl on my system, so I didn't know it existed.

I could have sworn I'd seen it recently, but don't now. Maybe I had
started xend at some point.

> > I think I would go with:
> > 
> >   /local/by-name/$name == /local/domain/$domid
> >   /local/by-uuid/$uuid == /local/domain/$domid
> > 
> > though, So that you can just read it and use it without interpreting it.
> 
> In that case, you would need to parse the domid out of the string in order
> to use it in hypercalls (grant, event, v4v). The frontend/backend paths use
> a distinct frontend-id/backend-id key for the domain ID, but we are trying
> to avoid this since populating this key would mean the domain populating it
> needs to know its own domain ID.

Right, dang ;-)

> >>   /local/domain/$domid/name - same as existing
> >>   /local/domain/$domid/uuid - ? maybe unneeded, as it's available from Xen.
> > 
> > Is it available for other domains via xen, or just yourself?
> > 
> 
> Yourself and anyone who can call getdomaininfo (which is usually just dom0).
> This is actually the same as the xenstore permissions on keys such as
> /local/domain/$id/name. Changing these may need consideration, because on
> public hosting servers, you might not want to allow domains to be enumerated
> by name.

Sure.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths
  2012-08-09 13:38   ` Ian Jackson
@ 2012-08-17 14:35     ` Ian Campbell
  0 siblings, 0 replies; 17+ messages in thread
From: Ian Campbell @ 2012-08-17 14:35 UTC (permalink / raw)
  To: Ian Jackson; +Cc: xen-devel

On Thu, 2012-08-09 at 14:38 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths"):
> > On Mon, 2012-07-30 at 15:03 +0100, Ian Campbell wrote:
> > > This is based upon my inspection of a system with a single PV domain running
> > > and is therefore very incomplete.
> > > 
> > > There are several things I'm not sure of here, mostly marked with XXX in the
> > > text.
> > > 
> > > In particular:
> > > 
> > >  - We seem to expose various things to the guest which really it has no need to
> > >    know (at least not via xenstore). e.g. its own domid, its device model pid,
> > >    the size of the video ram, store port and gref.
> 
> Maybe we need to have a "???" or "?deprecated" tag ?  That would avoid
> us documenting things which we don't want people to use.

deprecated would be useful in some places.

I think we have other places where a key is visible to the guest e.g.
because hvmloader uses it, which we can't really "deprecate" as a means
of hiding them. Perhaps "internal" or "private" in addition to
deprecate?.

> > >  - What is the distinction between /vm/UUID and /local/domain/DOMID
> 
> I think the former should be abolished (eventually).

Is there a distinction between "domain" and "vm" which is relevant here.
Is a VM potentially composed of several domains? (e.g. the guest, the
stub domain, a. n. other helper domain etc)

Ian.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths
  2012-08-09 14:02   ` Ian Jackson
@ 2012-08-17 15:05     ` Ian Campbell
  2012-08-25  8:54       ` Joseph Glanville
  2012-09-24 11:35       ` Ian Jackson
  0 siblings, 2 replies; 17+ messages in thread
From: Ian Campbell @ 2012-08-17 15:05 UTC (permalink / raw)
  To: Ian Jackson; +Cc: xen-devel

On Thu, 2012-08-09 at 15:02 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths"):
> ...
> > > --- a/docs/misc/xenstore-paths.markdown
> > > +++ b/docs/misc/xenstore-paths.markdown
> > > @@ -0,0 +1,294 @@
> ...
> > > +PATH can contain simple regex constructs following the POSIX regexp
> > > +syntax described in regexp(7). In addition the following additional
> > > +wild card names are defined and are evaluated before regexp expansion:
> 
> Can we use a restricted perl re syntax ?  That avoids weirdness with
> the rules for \.

Is "restricted perl re syntax" a well defined thing (reference?) or do
you just mean perlre(1)--?

What's the weirdness with \.?

> Also how does this interact with markdown ?

The html version looks ok after a brief inspection.

> > > +#### ~/image/device-model-pid = INTEGER   [r]
> 
> This [r] tag is not defined above.  I assume you mean "readonly to the
> domain" but that's the default.  Left over from an earlier version ?

Yes, it's vestigial. Remove it.

> 
> > > +The process ID of the device model associated with this domain, if it
> > > +has one.
> > > +
> > > +XXX why is this visible to the guest?
> 
> I think some of these things were put here just because there wasn't
> another place for the toolstack to store things.  See also the
> arbitrary junk stored by scripts in the device backend directories.

Should we define a proper home for these? e.g. /$toolstack/$domid?

> > > +#### ~/cpu/[0-9]+/availability = ("online"|"offline") [PV]
> > > +
> > > +One node for each virtual CPU up to the guest's configured
> > > +maximum. Valid values are "online" and "offline". 
> 
> Should have a cross-reference to the cpu online/offline protocol,
> which appears to be in xen/include/public/vcpu.h.  It doesn't seem to
> be fully documented yet.

vcpu.h has the hypercalls which are the mechanism by which a guest
brings a cpu up/down but nothing on the xenstore protocol which might
cause it to do so.

I don't think a reference currently exists for that protocol. This
probably belongs in the same (non-existent) protocol doc as
~/control/shutdown in so much as it is a toolstack<->guest kernel
protocol.

> > > +#### ~/memory/static-max = MEMKB []
> > > +
> > > +Specifies a static maximum amount memory which this domain should
> > > +expect to be given. In the absence of in-guest memory hotplug support
> > > +this set on domain boot and is usually the maximum amount of RAM which
> > > +a guest can make use of .
> 
> This should have a cross-reference to the documentation defining
> static-max etc.  I thought we had some in tree but I can't seem to
> find it.  The best I can find is docs/man/xl.cfg.pod.5.

I think you might be thinking of tools/libxl/libxl_memory.txt.

Shall we move that doc to docs/misc?

> 
> > > +#### ~/memory/target = MEMKB []
> > > +
> > > +The current balloon target for the domain. The balloon driver within the guest is expected to make every effort 
> 
> every effort to ... ?

err. yes. I appear  to have got distracted there ...

Perhaps:

        every effort to ... reach this target

? but I'm not sure that is strictly correct, a guest can use less if it
wants to. So perhaps

        every effort to ... not use more than this
        
? seems clumsy though.

> 
> The interaction with the Xen maximum should be stated, preferably by
> cross-reference.  In general it might be better to have a single place
> where all these values and their semantics are written down ?
> 
> > > +#### ~/device/suspend/event-channel = ""|EVTCHN [w]
> > > +
> > > +The domain's suspend event channel. The use of a suspend event channel
> > > +is optional at the domain's discression. If it is not used then this
> > > +path will be left blank.
> 
> May it be ENOENT ?  Does the toolstack create it as "" then ?

libxl seems to *mkdir* it:
    libxl__xs_mkdir(gc, t,
                    libxl__sprintf(gc, "%s/device/suspend/event-channel", dom_path),
                    rwperm, ARRAY_SIZE(rwperm));

which I suppose is the same as writing it as "" (unless there is some
subtle xenstore semantic difference I'm not thinking of)

If xend writes this key then I can't find it. I rather suspect the
~/device/suspend is guest writeable in that case (but I can't find that
either).

While grepping around I noticed xs_suspend_evtchn_port which reads this.
Seems like an odd place for it...

> 
> > > +#### ~/device/serial/$DEVID/* [HVM]
> > > +
> > > +An emulated serial device
> 
> You should presumably add
>     XXX documentation for the protocol needed
> here.

I think this is in docs/misc/console.txt along with the PV stuff, so
I've added that as a reference.

> 
> > > +#### ~/store/port = EVTCHN []
> > > +
> > > +The event channel used by the domains connection to XenStore.
> 
> Apostrophe.
> 
> > > +XXX why is this exposed to the guest?
> 
> Is there really only one event channel ?  Ie the same evtchn is used
> to signal to xenstore that the guest has sent a command, and to signal
> the guest that xenstore has written the response ?

Yes, event channels are bidirectional so that's quite common.

> Anyway surely this is something the guest needs to know.  Why it's in
> xenstore is a bit of a mystery since you can't use xenstore without it
> and it's in the start_info.

I should have written "why is this exposed to the guest via xenstore?"

> Is this the same value as start_info.store_evtchn ?  Cross reference ?

I'd be semi inclined to ditch/deprecate it unless we can figure out what
it is for -- as you say there is something of a chicken and egg problem
with using it.

> 
> > > +#### ~/store/ring-ref = GNTREF []
> > > +
> > > +The grant reference of the domain's XenStore ring.
> > > +
> > > +XXX why is this exposed to the guest?
> 
> See above.

Yup, the same issues.

> > > +#### ~/device-model/$DOMID/* []
> > > +
> > > +Information relating to device models running in the domain. $DOMID is
> > > +the target domain of the device model.
> > > +
> > > +XXX where is the contents of this directory specified?
> 
> I think it's not specified anywhere.  It's ad-hoc.  The guest
> shouldn't need to see it but exposing it readonly is probably
> harmless.  Except perhaps for the vnc password ?

vnc password appears to go into /vm/$uuid/vncpass (looking at libxl code
only).

AFAIK it does nothing special with the perms, but /vm/$uuid is not guest
readable (perms are "n0") so I think that works out ok.

I wonder if that's part of the point of /vm/$uuid.

> > > +### /vm/$UUID/uuid = UUID []
> > > +
> > > +Value is the same UUID as the path.
> > > +
> > > +### /vm/$UUID/name = STRING []
> > > +
> > > +The domains name.
> 
> IMO this should be
>   (a) in /local/domain/$DOMID
>   (b) also a copy in /byname/$NAME = $DOMID   for fast lookup
> but not in 4.2.
> 
> Guests shouldn't rely on it.  In fact do guests actually need anything
> from here ?

I'd say definitely not, but it has existed with xend for many years so
I'd be surprised if something hadn't crept in somewhere :-(

Ian.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths
  2012-08-17 15:05     ` Ian Campbell
@ 2012-08-25  8:54       ` Joseph Glanville
  2012-09-24 11:35       ` Ian Jackson
  1 sibling, 0 replies; 17+ messages in thread
From: Joseph Glanville @ 2012-08-25  8:54 UTC (permalink / raw)
  To: Ian Campbell; +Cc: Ian Jackson, xen-devel

On 18 August 2012 01:05, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Thu, 2012-08-09 at 15:02 +0100, Ian Jackson wrote:
>> Ian Campbell writes ("Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths"):
>> ...
>> > > --- a/docs/misc/xenstore-paths.markdown
>> > > +++ b/docs/misc/xenstore-paths.markdown
>> > > @@ -0,0 +1,294 @@
>> ...
>> > > +PATH can contain simple regex constructs following the POSIX regexp
>> > > +syntax described in regexp(7). In addition the following additional
>> > > +wild card names are defined and are evaluated before regexp expansion:
>>
>> Can we use a restricted perl re syntax ?  That avoids weirdness with
>> the rules for \.
>
> Is "restricted perl re syntax" a well defined thing (reference?) or do
> you just mean perlre(1)--?
>
> What's the weirdness with \.?
>
>> Also how does this interact with markdown ?
>
> The html version looks ok after a brief inspection.
>
>> > > +#### ~/image/device-model-pid = INTEGER   [r]
>>
>> This [r] tag is not defined above.  I assume you mean "readonly to the
>> domain" but that's the default.  Left over from an earlier version ?
>
> Yes, it's vestigial. Remove it.
>
>>
>> > > +The process ID of the device model associated with this domain, if it
>> > > +has one.
>> > > +
>> > > +XXX why is this visible to the guest?
>>
>> I think some of these things were put here just because there wasn't
>> another place for the toolstack to store things.  See also the
>> arbitrary junk stored by scripts in the device backend directories.
>
> Should we define a proper home for these? e.g. /$toolstack/$domid?
>
>> > > +#### ~/cpu/[0-9]+/availability = ("online"|"offline") [PV]
>> > > +
>> > > +One node for each virtual CPU up to the guest's configured
>> > > +maximum. Valid values are "online" and "offline".
>>
>> Should have a cross-reference to the cpu online/offline protocol,
>> which appears to be in xen/include/public/vcpu.h.  It doesn't seem to
>> be fully documented yet.
>
> vcpu.h has the hypercalls which are the mechanism by which a guest
> brings a cpu up/down but nothing on the xenstore protocol which might
> cause it to do so.
>
> I don't think a reference currently exists for that protocol. This
> probably belongs in the same (non-existent) protocol doc as
> ~/control/shutdown in so much as it is a toolstack<->guest kernel
> protocol.
>
>> > > +#### ~/memory/static-max = MEMKB []
>> > > +
>> > > +Specifies a static maximum amount memory which this domain should
>> > > +expect to be given. In the absence of in-guest memory hotplug support
>> > > +this set on domain boot and is usually the maximum amount of RAM which
>> > > +a guest can make use of .
>>
>> This should have a cross-reference to the documentation defining
>> static-max etc.  I thought we had some in tree but I can't seem to
>> find it.  The best I can find is docs/man/xl.cfg.pod.5.
>
> I think you might be thinking of tools/libxl/libxl_memory.txt.
>
> Shall we move that doc to docs/misc?
>
>>
>> > > +#### ~/memory/target = MEMKB []
>> > > +
>> > > +The current balloon target for the domain. The balloon driver within the guest is expected to make every effort
>>
>> every effort to ... ?
>
> err. yes. I appear  to have got distracted there ...
>
> Perhaps:
>
>         every effort to ... reach this target
>
> ? but I'm not sure that is strictly correct, a guest can use less if it
> wants to. So perhaps
>
>         every effort to ... not use more than this
>
> ? seems clumsy though.
>
>>
>> The interaction with the Xen maximum should be stated, preferably by
>> cross-reference.  In general it might be better to have a single place
>> where all these values and their semantics are written down ?
>>
>> > > +#### ~/device/suspend/event-channel = ""|EVTCHN [w]
>> > > +
>> > > +The domain's suspend event channel. The use of a suspend event channel
>> > > +is optional at the domain's discression. If it is not used then this
>> > > +path will be left blank.
>>
>> May it be ENOENT ?  Does the toolstack create it as "" then ?
>
> libxl seems to *mkdir* it:
>     libxl__xs_mkdir(gc, t,
>                     libxl__sprintf(gc, "%s/device/suspend/event-channel", dom_path),
>                     rwperm, ARRAY_SIZE(rwperm));
>
> which I suppose is the same as writing it as "" (unless there is some
> subtle xenstore semantic difference I'm not thinking of)
>
> If xend writes this key then I can't find it. I rather suspect the
> ~/device/suspend is guest writeable in that case (but I can't find that
> either).
>
> While grepping around I noticed xs_suspend_evtchn_port which reads this.
> Seems like an odd place for it...
>
>>
>> > > +#### ~/device/serial/$DEVID/* [HVM]
>> > > +
>> > > +An emulated serial device
>>
>> You should presumably add
>>     XXX documentation for the protocol needed
>> here.
>
> I think this is in docs/misc/console.txt along with the PV stuff, so
> I've added that as a reference.
>
>>
>> > > +#### ~/store/port = EVTCHN []
>> > > +
>> > > +The event channel used by the domains connection to XenStore.
>>
>> Apostrophe.
>>
>> > > +XXX why is this exposed to the guest?
>>
>> Is there really only one event channel ?  Ie the same evtchn is used
>> to signal to xenstore that the guest has sent a command, and to signal
>> the guest that xenstore has written the response ?
>
> Yes, event channels are bidirectional so that's quite common.
>
>> Anyway surely this is something the guest needs to know.  Why it's in
>> xenstore is a bit of a mystery since you can't use xenstore without it
>> and it's in the start_info.
>
> I should have written "why is this exposed to the guest via xenstore?"
>
>> Is this the same value as start_info.store_evtchn ?  Cross reference ?
>
> I'd be semi inclined to ditch/deprecate it unless we can figure out what
> it is for -- as you say there is something of a chicken and egg problem
> with using it.
>
>>
>> > > +#### ~/store/ring-ref = GNTREF []
>> > > +
>> > > +The grant reference of the domain's XenStore ring.
>> > > +
>> > > +XXX why is this exposed to the guest?
>>
>> See above.
>
> Yup, the same issues.
>
>> > > +#### ~/device-model/$DOMID/* []
>> > > +
>> > > +Information relating to device models running in the domain. $DOMID is
>> > > +the target domain of the device model.
>> > > +
>> > > +XXX where is the contents of this directory specified?
>>
>> I think it's not specified anywhere.  It's ad-hoc.  The guest
>> shouldn't need to see it but exposing it readonly is probably
>> harmless.  Except perhaps for the vnc password ?
>
> vnc password appears to go into /vm/$uuid/vncpass (looking at libxl code
> only).
>
> AFAIK it does nothing special with the perms, but /vm/$uuid is not guest
> readable (perms are "n0") so I think that works out ok.
>
> I wonder if that's part of the point of /vm/$uuid.

What has /vm/$UUID been used for historically?

I find it useful if you set your own UUIDs as it provides a consistent
path across guest reboots (which ofcourse change the domid)
A /byname shortcut sounds good as a replacement if /vm/$UUID goes away.

>
>> > > +### /vm/$UUID/uuid = UUID []
>> > > +
>> > > +Value is the same UUID as the path.
>> > > +
>> > > +### /vm/$UUID/name = STRING []
>> > > +
>> > > +The domains name.
>>
>> IMO this should be
>>   (a) in /local/domain/$DOMID
>>   (b) also a copy in /byname/$NAME = $DOMID   for fast lookup
>> but not in 4.2.
>>
>> Guests shouldn't rely on it.  In fact do guests actually need anything
>> from here ?
>
> I'd say definitely not, but it has existed with xend for many years so
> I'd be surprised if something hadn't crept in somewhere :-(
>
> Ian.
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel



-- 
CTO | Orion Virtualisation Solutions | www.orionvm.com.au
Phone: 1300 56 99 52 | Mobile: 0428 754 846

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths
  2012-08-17 15:05     ` Ian Campbell
  2012-08-25  8:54       ` Joseph Glanville
@ 2012-09-24 11:35       ` Ian Jackson
  2012-09-24 15:07         ` Ian Campbell
  1 sibling, 1 reply; 17+ messages in thread
From: Ian Jackson @ 2012-09-24 11:35 UTC (permalink / raw)
  To: Ian Campbell; +Cc: xen-devel

Ian Campbell writes ("Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths"):
> On Thu, 2012-08-09 at 15:02 +0100, Ian Jackson wrote:
> > Ian Campbell writes ("Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths"):
> > ...
> > > > --- a/docs/misc/xenstore-paths.markdown
> > > > +++ b/docs/misc/xenstore-paths.markdown
> > > > @@ -0,0 +1,294 @@
> > ...
> > > > +PATH can contain simple regex constructs following the POSIX regexp
> > > > +syntax described in regexp(7). In addition the following additional
> > > > +wild card names are defined and are evaluated before regexp expansion:
> > 
> > Can we use a restricted perl re syntax ?  That avoids weirdness with
> > the rules for \.
> 
> Is "restricted perl re syntax" a well defined thing (reference?) or do
> you just mean perlre(1)--?

I mean pcre, or perlre(1), probably.

> What's the weirdness with \.?

In Perl syntax, a punctuation character preceded by \ is always a
literal, and all metacharacters are unbackslashed punctuation.

In regexp(7) you need to remember which of ( ) | [ ] . & $ need
\-escaping to be literals and which need \-annotating to be
metacharacters.  (And there are various dialects of this too; for
example Emacs regexps require \ in front of a different subset of the
punctuation.)

I don't particularly care about all the fancy (?:...) etc. extensions,
but being able to write the regexp without referring to the manual, or
experimenting, is very good - particularly in a document which will be
tested at most rather indirectly.

> > > > +The process ID of the device model associated with this domain, if it
> > > > +has one.
> > > > +
> > > > +XXX why is this visible to the guest?
> > 
> > I think some of these things were put here just because there wasn't
> > another place for the toolstack to store things.  See also the
> > arbitrary junk stored by scripts in the device backend directories.
> 
> Should we define a proper home for these? e.g. /$toolstack/$domid?

Yes, we should, but the purpose of the doc is to document what we do
now, not what we will do :-).  I just replied to explain your XXX.

> > Should have a cross-reference to the cpu online/offline protocol,
> > which appears to be in xen/include/public/vcpu.h.  It doesn't seem to
> > be fully documented yet.
> 
> vcpu.h has the hypercalls which are the mechanism by which a guest
> brings a cpu up/down but nothing on the xenstore protocol which might
> cause it to do so.
> 
> I don't think a reference currently exists for that protocol. This
> probably belongs in the same (non-existent) protocol doc as
> ~/control/shutdown in so much as it is a toolstack<->guest kernel
> protocol.

Right.  Well, then a cross-reference to vcpu.h is in order along with
remarks along the lines you quote about the lack of a proper protocol
document.

> > This should have a cross-reference to the documentation defining
> > static-max etc.  I thought we had some in tree but I can't seem to
> > find it.  The best I can find is docs/man/xl.cfg.pod.5.
> 
> I think you might be thinking of tools/libxl/libxl_memory.txt.
> 
> Shall we move that doc to docs/misc?

Good idea.

> Perhaps:
> 
>         every effort to ... reach this target
> 
> ? but I'm not sure that is strictly correct, a guest can use less if it
> wants to. So perhaps
> 
>         every effort to ... not use more than this
>         
> ? seems clumsy though.

:-).  These all seem fine to me.

> > > > +#### ~/device/suspend/event-channel = ""|EVTCHN [w]
> > > > +
> > > > +The domain's suspend event channel. The use of a suspend event channel
> > > > +is optional at the domain's discression. If it is not used then this
> > > > +path will be left blank.
> > 
> > May it be ENOENT ?  Does the toolstack create it as "" then ?
> 
> libxl seems to *mkdir* it:
>     libxl__xs_mkdir(gc, t,
>                     libxl__sprintf(gc, "%s/device/suspend/event-channel", dom_path),
>                     rwperm, ARRAY_SIZE(rwperm));
> 
> which I suppose is the same as writing it as "" (unless there is some
> subtle xenstore semantic difference I'm not thinking of)

As docs/misc/xenstore.txt says:

 If a particular path exists, all of its parents do too.  Every
 existing path maps to a possibly empty value, and may also have zero
 or more immediate children.  There is thus no particular distinction
 between directories and leaf nodes.  However, it is conventional not
 to store nonempty values at nodes which also have children.

The difference between MKDIR and WRITE is simply that if there is an
existing node, WRITE overwrites its value and MKDIR leaves it
unchanged.  See the doc.

> > > > +XXX why is this exposed to the guest?
> > 
> > Is there really only one event channel ?  Ie the same evtchn is used
> > to signal to xenstore that the guest has sent a command, and to signal
> > the guest that xenstore has written the response ?
> 
> Yes, event channels are bidirectional so that's quite common.

Oh, I didn't realise they were bidirectional.

> > Anyway surely this is something the guest needs to know.  Why it's in
> > xenstore is a bit of a mystery since you can't use xenstore without it
> > and it's in the start_info.
> 
> I should have written "why is this exposed to the guest via xenstore?"

Right.  OK.

> > Is this the same value as start_info.store_evtchn ?  Cross reference ?
> 
> I'd be semi inclined to ditch/deprecate it unless we can figure out what
> it is for -- as you say there is something of a chicken and egg problem
> with using it.

I think it's fine to deprecate it.

> > I think it's not specified anywhere.  It's ad-hoc.  The guest
> > shouldn't need to see it but exposing it readonly is probably
> > harmless.  Except perhaps for the vnc password ?
> 
> vnc password appears to go into /vm/$uuid/vncpass (looking at libxl code
> only).

Yuk.  We want to abolish /vm/$uuid/ surely.

> AFAIK it does nothing special with the perms, but /vm/$uuid is not guest
> readable (perms are "n0") so I think that works out ok.
> 
> I wonder if that's part of the point of /vm/$uuid.

Perhaps we should make a new directory for this.  We do seem to have
quite a bit of cruft in our system which attempting to write this
document is revealing.

The right answer is probably to mention it in the doc as "likely to
move".

> > > > +### /vm/$UUID/uuid = UUID []
> > > > +
> > > > +Value is the same UUID as the path.
> > > > +
> > > > +### /vm/$UUID/name = STRING []
> > > > +
> > > > +The domains name.
> > 
> > IMO this should be
> >   (a) in /local/domain/$DOMID
> >   (b) also a copy in /byname/$NAME = $DOMID   for fast lookup
> > but not in 4.2.
> > 
> > Guests shouldn't rely on it.  In fact do guests actually need anything
> > from here ?
> 
> I'd say definitely not, but it has existed with xend for many years so
> I'd be surprised if something hadn't crept in somewhere :-(

Yers.

We should at least say that guests shouldn't use it.

Ian.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths
  2012-09-24 11:35       ` Ian Jackson
@ 2012-09-24 15:07         ` Ian Campbell
  0 siblings, 0 replies; 17+ messages in thread
From: Ian Campbell @ 2012-09-24 15:07 UTC (permalink / raw)
  To: Ian Jackson; +Cc: xen-devel

On Mon, 2012-09-24 at 12:35 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths"):
> > On Thu, 2012-08-09 at 15:02 +0100, Ian Jackson wrote:
> > > Ian Campbell writes ("Re: [DOCDAY PATCH] docs: initial documentation for xenstore paths"):
> > > ...
> > > > > --- a/docs/misc/xenstore-paths.markdown
> > > > > +++ b/docs/misc/xenstore-paths.markdown
> > > > > @@ -0,0 +1,294 @@
> > > ...
> > > > > +PATH can contain simple regex constructs following the POSIX regexp
> > > > > +syntax described in regexp(7). In addition the following additional
> > > > > +wild card names are defined and are evaluated before regexp expansion:
> > > 
> > > Can we use a restricted perl re syntax ?  That avoids weirdness with
> > > the rules for \.
> > 
> > Is "restricted perl re syntax" a well defined thing (reference?) or do
> > you just mean perlre(1)--?
> 
> I mean pcre, or perlre(1), probably.
> 
> > What's the weirdness with \.?
> 
> In Perl syntax, a punctuation character preceded by \ is always a
> literal, and all metacharacters are unbackslashed punctuation.

Great, I think I can cope with that.

I suspect I've mostly used that syntax already anyhow.

> In regexp(7) you need to remember which of ( ) | [ ] . & $ need
> \-escaping to be literals and which need \-annotating to be
> metacharacters.  (And there are various dialects of this too; for
> example Emacs regexps require \ in front of a different subset of the
> punctuation.)

Yes, I don't think I've ever done a regexp search and replace in emacs
successfully on my first attempt.

> I don't particularly care about all the fancy (?:...) etc. extensions,
> but being able to write the regexp without referring to the manual, or
> experimenting, is very good - particularly in a document which will be
> tested at most rather indirectly.
> 
> > > > > +The process ID of the device model associated with this domain, if it
> > > > > +has one.
> > > > > +
> > > > > +XXX why is this visible to the guest?
> > > 
> > > I think some of these things were put here just because there wasn't
> > > another place for the toolstack to store things.  See also the
> > > arbitrary junk stored by scripts in the device backend directories.
> > 
> > Should we define a proper home for these? e.g. /$toolstack/$domid?
> 
> Yes, we should, but the purpose of the doc is to document what we do
> now, not what we will do :-).  I just replied to explain your XXX.

Thanks. I've tagged this one as INTERNAL.

> > > This should have a cross-reference to the documentation defining
> > > static-max etc.  I thought we had some in tree but I can't seem to
> > > find it.  The best I can find is docs/man/xl.cfg.pod.5.
> > 
> > I think you might be thinking of tools/libxl/libxl_memory.txt.
> > 
> > Shall we move that doc to docs/misc?
> 
> Good idea.

I'll incorporate this move into the next version of the patch.

> > Perhaps:
> > 
> >         every effort to ... reach this target
> > 
> > ? but I'm not sure that is strictly correct, a guest can use less if it
> > wants to. So perhaps
> > 
> >         every effort to ... not use more than this
> >         
> > ? seems clumsy though.
> 
> :-).  These all seem fine to me.

I went with "make every effort to every effort use no more
than this amount of RAM". I don't think there's any point in requiring a
guest to use more RAM than it wants to.

> > > I think it's not specified anywhere.  It's ad-hoc.  The guest
> > > shouldn't need to see it but exposing it readonly is probably
> > > harmless.  Except perhaps for the vnc password ?
> > 
> > vnc password appears to go into /vm/$uuid/vncpass (looking at libxl code
> > only).
> 
> Yuk.  We want to abolish /vm/$uuid/ surely.
> 
> > AFAIK it does nothing special with the perms, but /vm/$uuid is not guest
> > readable (perms are "n0") so I think that works out ok.
> > 
> > I wonder if that's part of the point of /vm/$uuid.
> 
> Perhaps we should make a new directory for this.  We do seem to have
> quite a bit of cruft in our system which attempting to write this
> document is revealing.
> 
> The right answer is probably to mention it in the doc as "likely to
> move".

I've noticed that nothing in here appears to be readable by guests.
Therefore I have marked them all with new tags INTERNAL (guest
should/must not read)  and "n" (guest's can't even read).

Thanks for the review/discussion. V2 of this patch to follow shortly.

Ian.

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2012-09-24 15:07 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-07-30 14:03 [DOCDAY PATCH] docs: initial documentation for xenstore paths Ian Campbell
2012-07-30 16:11 ` [DOCDAY PATCH] docs: update xenstore-paths.markdown with HVM paths Ian Campbell
2012-08-09 10:01 ` [DOCDAY PATCH] docs: initial documentation for xenstore paths Ian Campbell
2012-08-09 13:38   ` Ian Jackson
2012-08-17 14:35     ` Ian Campbell
2012-08-09 14:02   ` Ian Jackson
2012-08-17 15:05     ` Ian Campbell
2012-08-25  8:54       ` Joseph Glanville
2012-09-24 11:35       ` Ian Jackson
2012-09-24 15:07         ` Ian Campbell
2012-08-09 21:02 ` Guest knowledge of own domid [was: docs: initial documentation for xenstore paths] Daniel De Graaf
2012-08-09 21:26   ` Jean Guyader
2012-08-16 11:28     ` Ian Campbell
2012-08-16 14:33       ` Daniel De Graaf
2012-08-16 14:36         ` Ian Campbell
2012-08-16 14:52           ` Daniel De Graaf
2012-08-17  8:05             ` Ian Campbell

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.