[B.A.T.M.A.N.] running batman-adv in a openvz VE

[B.A.T.M.A.N.] running batman-adv in a openvz VE

[Bjoern Franke]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

is there any possibility to run batman-adv in a openvz VE? Must it run
as kernel module or is there any way to run it in userspace?

tia
bjo
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAku3DEYACgkQazwuXcm17arGEQCfbd/8ILJVSy0ZGMXqqpApXzy9
NSwAn32A2t8qILgQvH/5Fqxia8Y24+HK
=gjf6
-----END PGP SIGNATURE-----

Re: [B.A.T.M.A.N.] running batman-adv in a openvz VE

[Sven Eckelmann]

[-- Attachment #1: Type: Text/Plain, Size: 835 bytes --]

On Saturday 03 April 2010 11:37:15 Bjoern Franke wrote:
> is there any possibility to run batman-adv in a openvz VE?

I have never used openvz, but adjusted some protocols to deal with openvz 
aware kernels (which means that they used and checked their namespace - so 
nothing really fancy).

What must be done for device drivers to make them work on OpenVZ? Have you 
tried it and where have found you problems?

> Must it run
> as kernel module or is there any way to run it in userspace?

There was a proof-of-concept version of batman-adv running in userland. This 
work was abandoned in favor of the in kernel implementation (better 
integration/better latency/less overhead/.... - stuff which is even worse on 
embedded systems - see Linus comments on the tinc vpn throughput on DIR300).

Best regards,
	Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [B.A.T.M.A.N.] running batman-adv in a openvz VE

[Marek Lindner]

On Saturday 03 April 2010 17:37:15 Bjoern Franke wrote:
> is there any possibility to run batman-adv in a openvz VE? Must it run
> as kernel module or is there any way to run it in userspace?

There was a userspace implementation but we abandoned this approach simply 
because it is too slow. Copying packets from userspace to kernelland and back 
is a real performance killer.

However, we are working towards supporting multiple mesh networks running on 
the same host which could solve your problem (I don't know openvz well enough 
to be 100% sure) but don't expect this feature very soon. It is work in 
progress. :-)

Cheers,
Marek

Re: [B.A.T.M.A.N.] running batman-adv in a openvz VE

[Bjoern Franke]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

thanks for your reply!

> I have never used openvz, but adjusted some protocols to deal with openvz 
> aware kernels (which means that they used and checked their namespace - so 
> nothing really fancy).
>
> What must be done for device drivers to make them work on OpenVZ? Have you 
> tried it and where have found you problems?

OpenVZ-Containers use the same kernel as the host system and I can build
and load the module on the host, but I don't know how to access
/proc/net/batman-adv from within the container. The container uses a own
proc-fs.

> There was a proof-of-concept version of batman-adv running in userland. This 
> work was abandoned in favor of the in kernel implementation (better 
> integration/better latency/less overhead/.... - stuff which is even worse on 
> embedded systems - see Linus comments on the tinc vpn throughput on DIR300).

Ah ok.

best regards
Bjoern
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAku3OAQACgkQazwuXcm17apu4gCggYD8pg+Spx8XIgRmB3MDa8c2
vV4Ani8LgXUXInXRm0pOZd9g9jtslE7J
=fu2x
-----END PGP SIGNATURE-----

Re: [B.A.T.M.A.N.] running batman-adv in a openvz VE

[Sven Eckelmann]

[-- Attachment #1: Type: Text/Plain, Size: 1745 bytes --]

Bjoern Franke wrote:
> Hi,
> 
> thanks for your reply!
> 
> > I have never used openvz, but adjusted some protocols to deal with openvz
> > aware kernels (which means that they used and checked their namespace -
> > so nothing really fancy).
> >
> > What must be done for device drivers to make them work on OpenVZ? Have
> > you tried it and where have found you problems?
> 
> OpenVZ-Containers use the same kernel as the host system and I can build
> and load the module on the host, but I don't know how to access
> /proc/net/batman-adv from within the container. The container uses a own
> proc-fs.

Ok, that is correct. I wanted to check how to work around such problems and 
how to implement it in the new configuration api/multiple bat-device 
implementation for 0.3 (or later) - but unfortunately  OpenVZ for 2.6.32 was 
only good to crash my system very hard. (but it is at least a known upstream 
bug :) )
So it can take a little bit longer until I really checked what we can do here.

Just some small question: Why do you want batman-adv in a VE? batman-adv runs 
entirely inside the kernel. This means there is no benefit in running it in a 
"secure" environment because it doesn't run there. The real hardware used for 
the mesh is also part of the host and not the ve (unless you do some kind of 
device sharing with VE_NUMBER.conf - which seems to be a little bit 
overcomplicated).

To me it sounds easier and more straight forward to use veth inside the kernel 
and do the configuration outside (it only configures the kernel - so no 
security or separation benefits). The userspace applications like tinc can now 
run inside the ve with the correct setup in the host.

Best regards,
	Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [B.A.T.M.A.N.] running batman-adv in a openvz VE

[Sven Eckelmann]

[-- Attachment #1.1: Type: Text/Plain, Size: 3880 bytes --]

Sven Eckelmann wrote:
> > OpenVZ-Containers use the same kernel as the host system and I can build
> > and load the module on the host, but I don't know how to access
> > /proc/net/batman-adv from within the container. The container uses a own
> > proc-fs.
> 
> Ok, that is correct. I wanted to check how to work around such problems and
> how to implement it in the new configuration api/multiple bat-device
> implementation for 0.3 (or later) - but unfortunately  OpenVZ for 2.6.32
>  was only good to crash my system very hard. (but it is at least a known
>  upstream bug :) )
> So it can take a little bit longer until I really checked what we can do
>  here.

Just to document some stuff I ran into:

Ok, was a little bit work to get 2.6.32 working with openvz and without these 
nasty vfs crashes, but now it boots up correctly.  The first thing I checked 
was if it is possible to create bridge devices in VEs - and the result was 
quite impressive. It can create a socket to create the bridge, but the ioctl 
fails. The problem seems to be the get_exec_env()->features doesn't have 
VE_FEATURE_BRIDGE set (ENOTTY).

vzctl set 777 --features "bridges:on" --save

should fix it - but is not supported by the current vzctl. But even after 
using this feature it fails with EPERM when using that ioctl - which should 
only come when we don't have the capability CAP_NET_ADMIN. I find it a little 
bit irritating since capabilities should not be enabled - but openvz seems to 
activate it for VEs and somebody must enable it using

vzctl set 777 --capability "NET_ADMIN:on" --save

... which can only be done when the VE is stopped (which is different to the 
--features stuff, which irritates me even more).

So, now we have everything - lets try `brctl addbr br0` - and what would we 
expect to happen? Yes, after that day that it just crash my complete kernel in 
br_sysfs_addbr.... Good time to stop anything practical using openvz and 
concentrate on bare source code and ignoring the reality :)

Note for Marek:

proc and sysfs is created using init_ve_proc and init_ve_sysfs for the VE. It 
doesn't look like there is much stuff done there. You must use 
register_pernet_subsys to create functions which are called when a network 
namespace is created or deleted. This could be used to create the appropriate 
files in /proc/net - should work with /sys too (but haven't found something 
which uses it). It is documented in net/core/net_namespace.c. There are more 
namespaces in the current openvz kernel, but I don't think that they are 
interesting for us.

We should get informed about the removal of a device only in the namespace 
using hard_if_event - so it should not be a problem when the namespace 
disappears. Also we only use init_net in hard_interface.c when we add a 
interface - so the actual namespace should not be a problem. But I am a little 
bit curious why we only deactivate the interface in  hard_if_event when it 
gets NETDEV_UNREGISTER. I would think that batman_if->net_dev of that 
interface isn't valid anymore and we aren't allowed to access it. So removing 
it should be the right choice.

I started to write some tests, but then noticed quite fast that (at least in 
0.2.1) all variables to store the proc filesystem information are global 
variables - which will not work.

proc_interface_write must also check the net_device structure and not the 
actual string (if it is allowed to share a batX device between host and VE).

I've added a small (and really unclean) test of the pernet stuff - just to get 
some feeling what must be done to use it. Haven't really done anything useful 
due to openvz's way of telling me that it don't want to cooperate... at least 
there is now some idea how we (ok... you) can deal with the net namespaces :)

Best regards,
	Sven

[-- Attachment #1.2: pernet_proctest.patch --]
[-- Type: text/x-patch, Size: 4353 bytes --]

diff -ruN batman-adv-kernelland-0.2.1/proc.c batadv_namespace//proc.c
--- batman-adv-kernelland-0.2.1/proc.c	2010-03-22 21:33:55.000000000 +0000
+++ batadv_namespace//proc.c	2010-04-03 22:06:07.000000000 +0000
@@ -556,44 +556,35 @@
 	.release	= single_release,
 };
 
-void cleanup_procfs(void)
+static void __net_exit batmanadv_net_exit(struct net *net)
 {
-	if (proc_transt_global_file)
 		remove_proc_entry(PROC_FILE_TRANST_GLOBAL, proc_batman_dir);
 
-	if (proc_transt_local_file)
 		remove_proc_entry(PROC_FILE_TRANST_LOCAL, proc_batman_dir);
 
-	if (proc_originators_file)
 		remove_proc_entry(PROC_FILE_ORIGINATORS, proc_batman_dir);
 
-	if (proc_orig_interval_file)
 		remove_proc_entry(PROC_FILE_ORIG_INTERVAL, proc_batman_dir);
 
-	if (proc_interface_file)
 		remove_proc_entry(PROC_FILE_INTERFACES, proc_batman_dir);
 
-	if (proc_vis_data_file)
 		remove_proc_entry(PROC_FILE_VIS_DATA, proc_batman_dir);
 
-	if (proc_vis_srv_file)
 		remove_proc_entry(PROC_FILE_VIS_SRV, proc_batman_dir);
 
-	if (proc_aggr_file)
 		remove_proc_entry(PROC_FILE_AGGR, proc_batman_dir);
 
-	if (proc_batman_dir)
 #ifdef __NET_NET_NAMESPACE_H
-		remove_proc_entry(PROC_ROOT_DIR, init_net.proc_net);
+		remove_proc_entry(PROC_ROOT_DIR, net->proc_net);
 #else
 		remove_proc_entry(PROC_ROOT_DIR, proc_net);
 #endif
 }
 
-int setup_procfs(void)
+static int __net_init batmanadv_net_init(struct net *net)
 {
 #ifdef __NET_NET_NAMESPACE_H
-	proc_batman_dir = proc_mkdir(PROC_ROOT_DIR, init_net.proc_net);
+	proc_batman_dir = proc_mkdir(PROC_ROOT_DIR, net->proc_net);
 #else
 	proc_batman_dir = proc_mkdir(PROC_ROOT_DIR, proc_net);
 #endif
@@ -610,7 +601,7 @@
 		proc_interface_file->proc_fops = &proc_interfaces_fops;
 	} else {
 		printk(KERN_ERR "batman-adv: Registering the '/proc/net/%s/%s' file failed\n", PROC_ROOT_DIR, PROC_FILE_INTERFACES);
-		cleanup_procfs();
+		batmanadv_net_exit(net);
 		return -EFAULT;
 	}
 
@@ -621,7 +612,7 @@
 		proc_orig_interval_file->proc_fops = &proc_orig_interval_fops;
 	} else {
 		printk(KERN_ERR "batman-adv: Registering the '/proc/net/%s/%s' file failed\n", PROC_ROOT_DIR, PROC_FILE_ORIG_INTERVAL);
-		cleanup_procfs();
+		batmanadv_net_exit(net);
 		return -EFAULT;
 	}
 
@@ -631,7 +622,7 @@
 		proc_originators_file->proc_fops = &proc_originators_fops;
 	} else {
 		printk(KERN_ERR "batman-adv: Registering the '/proc/net/%s/%s' file failed\n", PROC_ROOT_DIR, PROC_FILE_ORIGINATORS);
-		cleanup_procfs();
+		batmanadv_net_exit(net);
 		return -EFAULT;
 	}
 
@@ -641,7 +632,7 @@
 		proc_transt_local_file->proc_fops = &proc_transt_local_fops;
 	} else {
 		printk(KERN_ERR "batman-adv: Registering the '/proc/net/%s/%s' file failed\n", PROC_ROOT_DIR, PROC_FILE_TRANST_LOCAL);
-		cleanup_procfs();
+		batmanadv_net_exit(net);
 		return -EFAULT;
 	}
 
@@ -651,7 +642,7 @@
 		proc_transt_global_file->proc_fops = &proc_transt_global_fops;
 	} else {
 		printk(KERN_ERR "batman-adv: Registering the '/proc/net/%s/%s' file failed\n", PROC_ROOT_DIR, PROC_FILE_TRANST_GLOBAL);
-		cleanup_procfs();
+		batmanadv_net_exit(net);
 		return -EFAULT;
 	}
 
@@ -662,7 +653,7 @@
 		proc_vis_srv_file->proc_fops = &proc_vis_srv_fops;
 	} else {
 		printk(KERN_ERR "batman-adv: Registering the '/proc/net/%s/%s' file failed\n", PROC_ROOT_DIR, PROC_FILE_VIS_SRV);
-		cleanup_procfs();
+		batmanadv_net_exit(net);
 		return -EFAULT;
 	}
 
@@ -672,7 +663,7 @@
 		proc_vis_data_file->proc_fops = &proc_vis_data_fops;
 	} else {
 		printk(KERN_ERR "batman-adv: Registering the '/proc/net/%s/%s' file failed\n", PROC_ROOT_DIR, PROC_FILE_VIS_DATA);
-		cleanup_procfs();
+		batmanadv_net_exit(net);
 		return -EFAULT;
 	}
 
@@ -682,9 +673,25 @@
 		proc_aggr_file->proc_fops = &proc_aggr_fops;
 	} else {
 		printk(KERN_ERR "batman-adv: Registering the '/proc/net/%s/%s' file failed\n", PROC_ROOT_DIR, PROC_FILE_AGGR);
-		cleanup_procfs();
+		batmanadv_net_exit(net);
 		return -EFAULT;
 	}
 
 	return 0;
 }
+
+static struct pernet_operations batmanadv_net_ops = {
+	.init = batmanadv_net_init,
+	.exit = batmanadv_net_exit,
+};
+
+
+int setup_procfs(void)
+{
+	return register_pernet_subsys(&batmanadv_net_ops);
+}
+
+void cleanup_procfs(void)
+{
+	return unregister_pernet_subsys(&batmanadv_net_ops);
+}

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [B.A.T.M.A.N.] running batman-adv in a openvz VE

[Marek Lindner]

On Sunday 04 April 2010 06:29:24 Sven Eckelmann wrote:
> Also we only use init_net in hard_interface.c when we add a 
> interface - so the actual namespace should not be a problem. But I am a
>  little  bit curious why we only deactivate the interface in  hard_if_event
>  when it gets NETDEV_UNREGISTER. I would think that batman_if->net_dev of
>  that interface isn't valid anymore and we aren't allowed to access it. So
>  removing it should be the right choice.

The idea was to support some buggy (wifi) drivers better. Back in the layer 3 
days I had a driver (not exactly sure which one) that would disappear for a 
second and then come back immediately afterwards. In that case we don't want 
to purge all routing information over and over again.


> I started to write some tests, but then noticed quite fast that (at least
>  in  0.2.1) all variables to store the proc filesystem information are
>  global variables - which will not work.

Yes, this is why I mentioned the multiple mesh support in one of my earlier 
mails. The current stable module (0.2.x) is pretty static and cluttered with 
global variables which I started to clean up.

On the other hand I don't think it is worthwhile to invest much time into the 
proc-fs implementation since it is phasing out. I even thought about 
backporting the sysfs patches to the 0.2.2 branch, so that we can have all 
kernel interface changes while batman-adv resides in the staging tree. Later 
it will be much harder.

Regards,
Marek