RFC: unit tests for kernel GTP module

RFC: unit tests for kernel GTP module

[Harald Welte]

Dear GTP-interested folks,

I would love to somehow get towards some degree of unit testing (or even
"continuous integration") for teh kernel GTP code.

We currently have the original code in the kernel, we had some recent
small fixes and now are getting more patches into place.  With
relatively few active users out there (and probably none of them in
production yet), it's particularly easy to introduce regressions while
working on the code.  Also, having tested new code even against a
test set with limited covrage could help to get more confidence in new
patches and thus get them merged sooner.

Using tools like sgsnemu of OpenGGSN and the command line tools included
in libgtpnl, it should be possibel to cook up some scripts for testing.
Even the most basic set of tests would be an improvement over what we
have now.  One could also think about pcap replay to test with
hand-crafted or real-world packets from other GTP implementations.

As much as I'd like to put something like this into place myself, I
don't think I will be able to work much on this in the near future. The
GTP module at this point is a pure hobby and contrary to some years ago
while I started it, I don't have any contract work in the GTP area at
this point, so other projects currently unfortunately get more
attention.

So in case somebody among the GTP-interested parties (Travelping, OAI,
...) would want to do something in terms of testing, I'd be more than
happy if somebody would step ahead.  Otherwise it's all just vapourware
going to end up on my ever-growing TODO list :/

Also, if netdev folks have some ideas/pointers about possible
frameworks/tools for this kind of testing [it must exist for at least
some other kernel networking code?]: Please let me know.  I'd be
interested to have a look if there's something that can be used as a
basis (starting network namespaces, sending/transmitting packets, test
case startup/teardown, ...)

My "old school" approach would have been to start one or multiple
user-mode-linux kernels (those that are to be tested), and then have
scripts that set up a gtp socket and gtp tunnels via the libgtp command
line tools, and throw packets at that.   But I'm sure there must be
quite powerful frameworks for that kind of testing in the 21st century?
How do other tunneling implementations handle this?

Regards,
	Harald
-- 
- Harald Welte <laforge@gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)

Re: RFC: unit tests for kernel GTP module

[Ido Schimmel]

Hi,

On Thu, Feb 16, 2017 at 11:08:01PM +0100, Harald Welte wrote:
> Also, if netdev folks have some ideas/pointers about possible
> frameworks/tools for this kind of testing [it must exist for at least
> some other kernel networking code?]: Please let me know.  I'd be
> interested to have a look if there's something that can be used as a
> basis (starting network namespaces, sending/transmitting packets, test
> case startup/teardown, ...)

I'm not sure it meets all your needs, but we use LNST [1] to test our
code. You can see the tests (recipes) here [2]. You can create network
namespaces, send / receive packets, run arbitrary shell commands and
use existing test modules (e.g., ping, netperf) which return pass /
fail.

Adding Jiri and Ondrej, who is currently in the process of refactoring
the code and also familiar with more LNST use cases, such as those
employed by Redhat.

1. https://github.com/jpirko/lnst
2. https://github.com/jpirko/lnst/tree/master/recipes/

Re: RFC: unit tests for kernel GTP module

[Andreas Schultz]

Hi,

----- On Feb 16, 2017, at 11:08 PM, laforge laforge@gnumonks.org wrote:

> Dear GTP-interested folks,
> 
> I would love to somehow get towards some degree of unit testing (or even
> "continuous integration") for teh kernel GTP code.
> 
> We currently have the original code in the kernel, we had some recent
> small fixes and now are getting more patches into place.  With
> relatively few active users out there (and probably none of them in
> production yet), it's particularly easy to introduce regressions while
> working on the code.  Also, having tested new code even against a
> test set with limited covrage could help to get more confidence in new
> patches and thus get them merged sooner.

We do run the current kernel code in production. The version I have
been push patches for will be deployed into prod soonish.

Automated testing is something we are also looking at. Currently, we
have cooperation going on with a vendor of a an RAN and EPC protocol
testing solution. The goal is to have a automated testing setup for
our GGSN/PDN-GW OpenSource project. That setup will also exercise the
kernel parts.
The test suite is proprietary, so we can only share the results but
not the test setup itself.

> Using tools like sgsnemu of OpenGGSN and the command line tools included
> in libgtpnl, it should be possibel to cook up some scripts for testing.
> Even the most basic set of tests would be an improvement over what we
> have now.  One could also think about pcap replay to test with
> hand-crafted or real-world packets from other GTP implementations.

We more or less removed static GTP tunnels. The tools in libgtpnl
only work when an application is keeping the GTP socket alive.

Andreas

> As much as I'd like to put something like this into place myself, I
> don't think I will be able to work much on this in the near future. The
> GTP module at this point is a pure hobby and contrary to some years ago
> while I started it, I don't have any contract work in the GTP area at
> this point, so other projects currently unfortunately get more
> attention.
> 
> So in case somebody among the GTP-interested parties (Travelping, OAI,
> ...) would want to do something in terms of testing, I'd be more than
> happy if somebody would step ahead.  Otherwise it's all just vapourware
> going to end up on my ever-growing TODO list :/
> 
> Also, if netdev folks have some ideas/pointers about possible
> frameworks/tools for this kind of testing [it must exist for at least
> some other kernel networking code?]: Please let me know.  I'd be
> interested to have a look if there's something that can be used as a
> basis (starting network namespaces, sending/transmitting packets, test
> case startup/teardown, ...)
> 
> My "old school" approach would have been to start one or multiple
> user-mode-linux kernels (those that are to be tested), and then have
> scripts that set up a gtp socket and gtp tunnels via the libgtp command
> line tools, and throw packets at that.   But I'm sure there must be
> quite powerful frameworks for that kind of testing in the 21st century?
> How do other tunneling implementations handle this?
> 
> Regards,
>	Harald
> --
> - Harald Welte <laforge@gnumonks.org>           http://laforge.gnumonks.org/
> ============================================================================
> "Privacy in residential applications is a desirable marketing option."
>                                                   (ETSI EN 300 175-7 Ch. A6)

Re: RFC: unit tests for kernel GTP module

[Harald Welte]

Hi Andreas,

On Fri, Feb 17, 2017 at 04:02:59PM +0100, Andreas Schultz wrote:
> The test suite is proprietary, so we can only share the results but
> not the test setup itself.

it would be great to have some CI setup where both current stable as
well as a development branch of the code is tested, and reports
published regularly.  Not sure how realistic that is.

> We more or less removed static GTP tunnels. The tools in libgtpnl
> only work when an application is keeping the GTP socket alive.

Well, then those tools need to be adapted accordingly.  That's what I
also indicated in other mails:  Changes in the kernel GTP code should
always be followed-up with correspondign changes in libgtpnl and
associated tools.  I mean, the tool could just open the socket and then
continue to run until terminated explicitly...

I think there's a lot of value in some very low-level tools for testing
and experimentation, without the complexity of configuring + running an
Erlang GGSN/P-GW with all its dependencies.

-- 
- Harald Welte <laforge@gnumonks.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)