pkg/53095: xentools must remove xenstored/tdb on reboot

["Greg A. Woods" <woods%planix.ca@localhost>]

>Number:         53095
>Category:       pkg
>Synopsis:       xentools must remove xenstored/tdb on reboot
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Mar 12 21:20:00 +0000 2018
>Originator:     "Greg A. Woods" <woods%planix.ca@localhost>
>Release:        NetBSD 7.1
>Organization:
Planix, Inc.; Kelowna, BC; Canada
>Environment:
System: NetBSD xentral 7.1 NetBSD 7.1 (XEN3_DOM0.201703111743Z) amd64
Architecture: x86_64
Machine: amd64
$ /usr/sbin/pkg_info -I xentools\*
xentools48-4.8.0    Userland Tools for Xen 4.8.x
>Description:

The xenstored database file, currently /var/lib/xenstored/tdb, contains
runtime information stored in Domain-0 on behalf of the Xen management
commands (e.g. "xl") and the various DomUs.

It contains nothing persistent.

However it has a very fragile format ("Trivial DataBase") and it is
apparently easily corrupted during a crash.

It must be re-initialized from scratch after every reboot.

Note that there is no way to verify the integrity of a stale left-over
TDB file however it can always be re-created fully at boot time.

The "libxl_json" files now in /var/lib/xen are ephemeral too, as they
are just runtime copies of configuration data saved at domain creation
time and are only used to restore configuration during the "restart",
"restart-rename", and "soft-reset" actions that are executed for various
"on_*" events (e.g. at domain death).


>How-To-Repeat:

Crash a Xen server at any inopportune time and end up in a situation
where Xen domains cannot be started, with varying results from simple
hangs in "xl", possibly deadlocks in the kernel, as well as more obvious
errors such as:

04:56 [1.401] # xl create -c /usr/pkg/etc/xen/central.conf
Parsing config from /usr/pkg/etc/xen/central.conf
libxl: error: libxl_xshelp.c:175:libxl__xs_mknod: xenstore write failed: `/libxl/7/device' = '': No space left on device
libxl: error: libxl.c:456:libxl__domain_rename: failed to write new name `central' for domain 7 previously named `(null)'
libxl: error: libxl_create.c:928:initiate_domain_create: cannot make domain: -3
libxl: error: libxl.c:1575:libxl__destroy_domid: non-existant domain 7
libxl: error: libxl.c:1534:domain_destroy_callback: unable to destroy guest with domid 7
libxl: error: libxl.c:1463:domain_destroy_cb: destruction of domain 7 failed


>Fix:

Previously it has been proposed that XEN_LIB_STORED be changed to
/var/run/xenstored.

	https://mail-index.netbsd.org/port-xen/2016/03/24/msg008876.html

See below for a more complete proposal.

In the mean time this simple fix suffices (and keeps the old tdb file
for post-crash analysis):

--- xencommons.sh.~1.1.~	2017-03-30 02:15:09.000000000 -0700
+++ xencommons.sh	2018-03-12 11:22:36.269083716 -0700
@@ -17,6 +17,8 @@
 XENSTORED_PIDFILE="/var/run/xenstored.pid"
 XENCONSOLED_PIDFILE="/var/run/xenconsoled.pid"
 
+XEN_LIB_STORED="/var/lib/xenstored"
+
 BINDIR=@PREFIX@/bin
 SBINDIR=@PREFIX@/sbin
 LIBEXEC_BIN=@PREFIX@/libexec/xen/bin
@@ -25,6 +27,8 @@
 {
 	mkdir -p /var/run/xen || exit 1
 	mkdir -p /var/run/xenstored || exit 1
+
+	mv -f ${XEN_LIB_STORED}/tdb ${XEN_LIB_STORED}/tdb-previous
 }
 
 xen_startcmd()



The following patches (these from xen HEAD at commit-id a823a5280f) are
perhaps more appropriate for upstream (and would probably be preferred
over the patch suggested in the email linked above):

5 files changed, 7 insertions(+), 6 deletions(-)
m4/paths.m4                     | 5 +++--
tools/libxl/libxl_dom.c         | 2 +-
tools/xenstore/xs_lib.c         | 2 +-

modified   m4/paths.m4
@@ -135,8 +135,9 @@ XEN_SCRIPT_DIR=$XEN_CONFIG_DIR/scripts
 AC_SUBST(XEN_SCRIPT_DIR)
 
 case "$host_os" in
-*freebsd*) XEN_LOCK_DIR=$localstatedir/lib ;;
-*netbsd*) XEN_LOCK_DIR=$localstatedir/lib ;;
+*freebsd*) XEN_LOCK_DIR=$rundir_path ;;
+*netbsd*) XEN_LOCK_DIR=$rundir_path ;;
+*openbsd*) XEN_LOCK_DIR=$rundir_path ;;
 *) XEN_LOCK_DIR=$localstatedir/lock ;;
 esac
 AC_SUBST(XEN_LOCK_DIR)
modified   tools/libxl/libxl_dom.c
@@ -1361,7 +1361,7 @@ const char *libxl__userdata_path(libxl__gc *gc, uint32_t domid,
         goto out;
     }
     uuid_string = GCSPRINTF(LIBXL_UUID_FMT, LIBXL_UUID_BYTES(info.uuid));
-    path = GCSPRINTF(XEN_LIB_DIR "/userdata-%s.%u.%s.%s",
+    path = GCSPRINTF(XEN_RUN_DIR "/userdata-%s.%u.%s.%s",
                      wh, domid, uuid_string, userdata_userid);
 
  out:
modified   tools/xenstore/xs_lib.c
@@ -52,7 +52,7 @@ static const char *xs_daemon_path(void)
 const char *xs_daemon_tdb(void)
 {
 	static char buf[PATH_MAX];
-	snprintf(buf, sizeof(buf), "%s/tdb", xs_daemon_rootdir());
+	snprintf(buf, sizeof(buf), "%s/tdb", xs_daemon_rundir());
 	return buf;
 }