RadioTap Archive on lore.kernel.org
 help / color / Atom feed
From: Harald Welte <laforge-TgoAw6mPHtdg9hUCZPvPmw@public.gmane.org>
To: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
Cc: openbsc-qjLDD68F18N4m7O/Vxda39i2O/JbrIOy@public.gmane.org,
	radiotap-S783fYmB3Ccdnm+yROfE0A@public.gmane.org,
	linux-wireless-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	"Subash Abhinov Kasiviswanathan"
	<subashab-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>,
	"Dan Williams" <dcbw-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
	"Bjørn Mork" <bjorn-yOkvZcmFvRU@public.gmane.org>,
	netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	"Sean Tranchetti"
	<stranche-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>,
	"Aleksander Morgado"
	<aleksander-Dvg4H30XQSRVIjRurl1/8g@public.gmane.org>
Subject: Re: gsmtap design/extensions?
Date: Thu, 11 Apr 2019 01:32:13 +0200
Message-ID: <20190410233213.GN25552@nataraja> (raw)
In-Reply-To: <cf07ae24c436f92769f9289d208f01846ebe8826.camel-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

Hi Johannes,

On Tue, Apr 09, 2019 at 03:50:45PM +0200, Johannes Berg wrote:
> As I'm looking into adding a generic cell modem framework to the linux
> kernel (to create session netdevs etc.), I started looking for a
> metadata encapsulation, a la Radiotap (I'm a wifi guy :-) ).

Is there any discussion about "session netdevs, etc." anywhere?  What exactly
do you have in mind for that "generic cell modem framework"?  I'm quite
curious to learn more about it and happy to provide feedback from the
perspective of my cellular protocols/specs/systems knowledge.

> So obviously, I found gsmtap, but for my use case it doesn't really
> address most of the interesting data, 

I have no knowledge of your use case, so I don't feel like I can
comment on that :/

> 1) Why the design with encapsulating it in UDP? Radiotap is just a raw
>    header without IP etc. in front, and you use it with tcpdump,
>    wireshark or similar tools on the local system. What's the value in
>    having something "network transparent"?

GSMTAP was designed as a format to encapsulate protocols normally not spoken over IP
(such as classic GSM radio protocols, e.g. the Layer 2 LAPDm over GSM Um)
inside an IP transport.

The existing implementations of such protocols all live outside of the
kernel but in userspace.  So you either have

a) a pure SDR software implementation of a GSM phy, whether on the MS/UE
   side or on the BTS/network side, or whether in sniffing mode, running
   as a userspace process.  This could e.g. be airprobe or gr-gsm

b) a OsmocomBB phone attached over serial port running the Osmocom Layer1
   firmware on the baseband processor, with some userspace program on a Linux
   PC implementing higher layers

c) GSM base station software such as osmo-bts, running in userspace

d) Using an old nokia phone attached via serial cable to a Linux PC running
   dct3-gsmtap.

So for any of those, if you want to provide real-time data streams, you have to
somehow transmit those over some kind of network technology.   One could have
done raw ethernet frames or raw IP frames, but using UDP seemed straight-forward
as one can create UDP socket from non-privileged processes.

We also have a virtual physical layer between OsmocomBB and OsmoBTS called
virt_phy where the GSMTAP over multicast IP is used to simulate/virtualize
the entire Layer1 / PHY / radio interface between phone and base station.

Once again, all related network elements are implemented in userspace,
and having an 

> 2) The format of gsmtap doesn't seem very extensible, but I guess a new
>    version could be made that has a TLV-based format or so. I'd have
>    argued that a new version isn't even needed, but the length field is
>    only 8 bits right now which seems too short.

Yes, it's a known problem.  The format was originally designed for GSM,
that is circuit switched common and dedicated channels on the GSM Um interface.
It was later extended for GPRS, TETRA, SIM card protocol traces and many others,
but all of that was an afterthought.

For sure any future version should be more extensible and e.g. use TLVs

> (speaking of versions - the docs say "version, set to 0x01 currently"
> but "#define GSMTAP_VERSION 0x02")

Sorry, it's the usual "developer changes code but not comment". patches
are welcome, we use gerrit.osmocom.org :)

> 3) Does the packet data follow the gsmtap header? It's not really clear
>    to me based on reading the wireshark code.

Sure, the packet data follows the GSMTAP header, and the type of data
is defined by the GSMTAP type / sub-type as per the specific use case.  As
you can see, there's 18 TYPE by now (each of which has at least one
program/implementation generating the data).

I think the best way to learn about GSMTAP is to use any of the many
programs that support it by now.  I think not only the examples above
use it, but meanwhile many others like the independently-developed OpenBTS
project that's unrelated to Osmocom, as are other projects implementing LTE.

> In particular, the data I'm thinking of is higher-level things, like the
> session ID for a frame when it's going through the kernel, or perhaps a
> flow label on RX, etc.

I'm not quite sure how that relates to GSM?  GSMTAP so far was intended
to encapsulate ETSI/3GPP messages inside UDP/IP, particularly messages of
protocols that don't traditionally are transported over IP baesd transports.

Having said, we're open to extending the scope - it just all needs to make
sense

> Also, vendor-specific data would be useful, e.g. to encapsulate the
> device-specific headers like QMI, where such metadata is encapsulated in
> a vendor- or device-specific way, which you'd want to see for debugging
> certain things, but for other things the generic "session ID" type
> information - encoded in a vendor-agnostic way - would be better to show
> in wireshark.

I'm really not following you here.  What's a "session ID"?

In terms of vendor-specific encapsulations: We do have a GSMTAP sub-type
for Qualcomm DIAG, as well as the osmo-qcdiag program to take DIAG from
Qualcomm chips + put it in GSMTAP and I do have plenty of experimental
wireshark dissectors for various parts of the DIAG protocl in a branch
on git.osmocom.org.  This just never went anywhere complete enough that
I'd consider merging it.

> Since it doesn't seem possible to use gsmtap in the current version,
> would it make sense to define a new gsmtap that (say) has version 3 or
> something, followed by an overall length and TLVs? 

By all means.  Vadim has once presented some ideas/plans about exactly
that at an Osmocom conference, but I don't think any related work
ever started.

> I do note that this
> wouldn't be compatible with the current wireshark code as it doesn't
> check the version, just shows it...

If that's the case that's sad, and I should have paid more attention to
it when originally writing it.  We should get a related fix into
wireshark ASAP then.  But then, the current dissector would just state
something like unsupported version instead of showing some garbage it
cannot parse.  Either way, you will of course need new sources and
sink side implemetations.

> Or would it make more sense to define a new ARPHDR_WWANTAP like
> ARPHDR_IEEE80211_RADIOTAP and just use that instead of encapsulating in
> IP/UDP, and then have a completely new (extensible) protocol inside of
> that? 

No userspace source would ever be able to generate such data and stream
it real-time into wireshark, would it?  Sure, I can write pcap file with
such ARPHDR_* values, but I could never do this in real-time.  For many
but not all use cases, that's really what it is: A vehicle to stream
real-time non-IP protocol traces into wireshark so it can visualize
the protocol traces.

Regards,
	Harald
-- 
- Harald Welte <laforge-TgoAw6mPHtdg9hUCZPvPmw@public.gmane.org>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)

  parent reply index

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-09 13:50 Johannes Berg
     [not found] ` <CAPEnp=bHTVARr7WA9Qkq4=y7kk8_FLWhrkqKOEXubMO4X=BHCw@mail.gmail.com>
     [not found]   ` <CAPEnp=bHTVARr7WA9Qkq4=y7kk8_FLWhrkqKOEXubMO4X=BHCw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2019-04-10  7:23     ` Johannes Berg
     [not found]       ` <46474c61d7748042cc0a1f23773186786020638e.camel-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
2019-04-10  7:57         ` Marcel Holtmann
2019-04-10 23:45         ` Harald Welte
     [not found]       ` <6F1998DC-EFD2-4145-BD81-A80F9DC7ED2D@holtmann.org>
     [not found]         ` <6F1998DC-EFD2-4145-BD81-A80F9DC7ED2D-kz+m5ild9QBg9hUCZPvPmw@public.gmane.org>
2019-04-12 12:12           ` Johannes Berg
     [not found]             ` <1d64c578cd5b254d301cf1cac82f32a062916888.camel-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
2019-04-12 19:49               ` Marcel Holtmann
     [not found]             ` <D4D12CF9-9CE5-46FB-9738-F89DB6B6F9EA@holtmann.org>
     [not found]               ` <D4D12CF9-9CE5-46FB-9738-F89DB6B6F9EA-kz+m5ild9QBg9hUCZPvPmw@public.gmane.org>
2019-04-15  9:11                 ` Johannes Berg
     [not found]                   ` <92e8e142b6d441c1c995abc57d64ad7b7747a688.camel-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
2019-04-15 10:29                     ` Bjørn Mork
     [not found]                       ` <87d0ln1s0a.fsf-3F4PFWf5pNjpjLOzFPqGjWGXanvQGlWp@public.gmane.org>
2019-04-15 10:41                         ` Johannes Berg
     [not found]       ` <20190410234555.GO25552@nataraja>
2019-04-12 17:15         ` Johannes Berg
     [not found]           ` <f0a64bb751b8c601f58130a7f3a3158d70f18f63.camel-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
2019-04-12 19:54             ` Guy Harris
2019-04-13  7:12             ` Harald Welte
2019-04-13  7:49               ` Guy Harris
     [not found]           ` <196E881D-AB4A-4569-9C1F-5BCB41FE832F@alum.mit.edu>
     [not found]             ` <196E881D-AB4A-4569-9C1F-5BCB41FE832F-FrUbXkNCsVf2fBVCVOL8/A@public.gmane.org>
2019-04-12 22:47               ` Guy Harris
     [not found]                 ` <1D6FE33B-DC15-47D8-A1AD-E95608300FDC-FrUbXkNCsVf2fBVCVOL8/A@public.gmane.org>
2019-04-13  6:41                   ` Harald Welte
     [not found]                 ` <20190413064152.GB24451@nataraja>
2019-04-13  7:36                   ` Guy Harris
     [not found] ` <cf07ae24c436f92769f9289d208f01846ebe8826.camel-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
2019-04-10  6:35   ` Vadim Yanitskiy
2019-04-10 23:32   ` Harald Welte [this message]
2019-04-12 12:24     ` Johannes Berg
     [not found]       ` <1462659018bc40830efbe2348791b8df45b54cff.camel-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
2019-04-12 19:48         ` Guy Harris
2019-04-13  7:35         ` Harald Welte
     [not found]       ` <20190413073505.GD24451@nataraja>
2019-04-13  7:55         ` Guy Harris
2019-04-15  9:26         ` Johannes Berg
     [not found]       ` <1089142F-2966-4C41-921B-465FBA721E79@alum.mit.edu>
     [not found]         ` <1089142F-2966-4C41-921B-465FBA721E79-FrUbXkNCsVf2fBVCVOL8/A@public.gmane.org>
2019-04-15  9:28           ` Johannes Berg

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190410233213.GN25552@nataraja \
    --to=laforge-tgoaw6mphtdg9huczpvpmw@public.gmane.org \
    --cc=aleksander-Dvg4H30XQSRVIjRurl1/8g@public.gmane.org \
    --cc=bjorn-yOkvZcmFvRU@public.gmane.org \
    --cc=dcbw-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
    --cc=johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org \
    --cc=linux-wireless-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=openbsc-qjLDD68F18N4m7O/Vxda39i2O/JbrIOy@public.gmane.org \
    --cc=radiotap-S783fYmB3Ccdnm+yROfE0A@public.gmane.org \
    --cc=stranche-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org \
    --cc=subashab-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

RadioTap Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/radiotap/0 radiotap/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 radiotap radiotap/ https://lore.kernel.org/radiotap \
		radiotap@radiotap.org
	public-inbox-index radiotap

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.netbsd.radiotap


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git