use of radiotap bit 14?

use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 311 bytes --]

Our radiotap header in Linux defines bit 14 as
IEEE80211_RADIOTAP_RX_FLAGS; however running wireshark on it tells me
that this bit means "FCS in header". Does anybody know which use is
correct, if any?

Maybe we should simply skip to bit 32 or something and start redefining
things properly?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: use of radiotap bit 14?

[Pavel Roskin]

On Fri, 2007-08-31 at 00:16 +0200, Johannes Berg wrote:
> Our radiotap header in Linux defines bit 14 as
> IEEE80211_RADIOTAP_RX_FLAGS; however running wireshark on it tells me
> that this bit means "FCS in header". Does anybody know which use is
> correct, if any?

As far as I know, "FCS in header" is a FreeBSD thing, which came to
Linux in MadWifi.  "Rx flags" comes from Linux Libertas driver.

The standard uses the later:
http://netbsd.gw.com/cgi-bin/man-cgi?ieee80211_radiotap+9+NetBSD-current

MadWifi has removed the non-standard use of bit 14.  Now it's time to
fix wireshark.

> Maybe we should simply skip to bit 32 or something and start redefining
> things properly?

And what would be "properly"?  How would it prevent drivers from adding
non-standard fields?

-- 
Regards,
Pavel Roskin

Re: use of radiotap bit 14?

[Sam Leffler]

Pavel Roskin wrote:
> On Fri, 2007-08-31 at 00:16 +0200, Johannes Berg wrote:
>   
>> Our radiotap header in Linux defines bit 14 as
>> IEEE80211_RADIOTAP_RX_FLAGS; however running wireshark on it tells me
>> that this bit means "FCS in header". Does anybody know which use is
>> correct, if any?
>>     
>
> As far as I know, "FCS in header" is a FreeBSD thing, which came to
> Linux in MadWifi.  "Rx flags" comes from Linux Libertas driver.
>
> The standard uses the later:
> http://netbsd.gw.com/cgi-bin/man-cgi?ieee80211_radiotap+9+NetBSD-current
>
> MadWifi has removed the non-standard use of bit 14.  Now it's time to
> fix wireshark.
>
>   
>> Maybe we should simply skip to bit 32 or something and start redefining
>> things properly?
>>     
>
> And what would be "properly"?  How would it prevent drivers from adding
> non-standard fields?
>
>   
freebsd never had "fcs in header".

Re: use of radiotap bit 14?

[Sam Leffler]

Sam Leffler wrote:
> Pavel Roskin wrote:
>> On Fri, 2007-08-31 at 00:16 +0200, Johannes Berg wrote:
>>  
>>> Our radiotap header in Linux defines bit 14 as
>>> IEEE80211_RADIOTAP_RX_FLAGS; however running wireshark on it tells me
>>> that this bit means "FCS in header". Does anybody know which use is
>>> correct, if any?
>>>     
>>
>> As far as I know, "FCS in header" is a FreeBSD thing, which came to
>> Linux in MadWifi.  "Rx flags" comes from Linux Libertas driver.
>>
>> The standard uses the later:
>> http://netbsd.gw.com/cgi-bin/man-cgi?ieee80211_radiotap+9+NetBSD-current
>>
>> MadWifi has removed the non-standard use of bit 14.  Now it's time to
>> fix wireshark.
>>
>>  
>>> Maybe we should simply skip to bit 32 or something and start redefining
>>> things properly?
>>>     
>>
>> And what would be "properly"?  How would it prevent drivers from adding
>> non-standard fields?
>>
>>   
> freebsd never had "fcs in header".

Let me rephrase.  From the CVS commit that removed it's definiteion on 
freebsd:

o remove reference to IEEE80211_RADIOTAP_FCS; it was never used, instead
  the flags are marked with IEEE80211_RADIOTAP_F_FCS to indicate whether
  or not FCS is present

I used to try and sync against other .h files.  I no longer find it useful.

    Sam

Re: use of radiotap bit 14?

[Gerald Combs]

Pavel Roskin wrote:
> On Fri, 2007-08-31 at 00:16 +0200, Johannes Berg wrote:
>> Our radiotap header in Linux defines bit 14 as
>> IEEE80211_RADIOTAP_RX_FLAGS; however running wireshark on it tells me
>> that this bit means "FCS in header". Does anybody know which use is
>> correct, if any?
> 
> As far as I know, "FCS in header" is a FreeBSD thing, which came to
> Linux in MadWifi.  "Rx flags" comes from Linux Libertas driver.
> 
> The standard uses the later:
> http://netbsd.gw.com/cgi-bin/man-cgi?ieee80211_radiotap+9+NetBSD-current
> 
> MadWifi has removed the non-standard use of bit 14.  Now it's time to
> fix wireshark.

The problem is a little more widespread than that.  Packet-radiotap.h in 
the Wireshark sources has the following comments:

/*
  * AAAAAAAAAAAAAAAAAAAAAAAAAARGH.
  *
  * The current FreeBSD ieee80211_radiotap.h has IEEE80211_RADIOTAP_XCHANNEL
  * as 14.
  *
  * The current NetBSD ieee80211_radiotap.h has IEEE80211_RADIOTAP_RX_FLAGS
  * as 14.
  *
  * The current OpenBSD ieee80211_radiotap.h has IEEE80211_RADIOTAP_FCS as
  * 14.
  *
  * NetBSD and OpenBSD also differ on what comes *after* 14.
  *
  * They all use the same DLT_ value for "802.11+radiotap".
  *
  * This is all wonderfully appreciated by those of us who write code to
  * read files containing packets with radiotap headers.  I will see if
  * I can apply a little cluebat-fu here.
  */

and

     /* XXX - IEEE80211_RADIOTAP_FCS is used by MadWifi and AirPcap, but
      * was never officially assigned. */

I'll open a ticket on Wireshark's Bugzilla so we (Wireshark) don't lose 
track of this before the next release.  I'll talk to the guys here 
(CACE) about fixing AirPcap as well.

An earlier email from David Young discussed placing the Radiotap 
documentation in a more "official" location -- I think this would help 
immensely.  For a developer starting from scratch, it's not immediately 
obvious where the canonical Radiotap specification is and it's very easy 
to get started down the wrong path.  For example, if NetBSD Radiotap 
header and man page show up in Google's rankings for "radiotap" and 
"radiotap specification", they're pretty far down the list.

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 678 bytes --]

On Fri, 2007-08-31 at 09:50 -0700, Gerald Combs wrote:

> An earlier email from David Young discussed placing the Radiotap 
> documentation in a more "official" location -- I think this would help 
> immensely.  For a developer starting from scratch, it's not immediately 
> obvious where the canonical Radiotap specification is and it's very easy 
> to get started down the wrong path.  For example, if NetBSD Radiotap 
> header and man page show up in Google's rankings for "radiotap" and 
> "radiotap specification", they're pretty far down the list.

I second that. If somebody has a domain registered, I can host a wiki
over with linuxwireless.org.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: use of radiotap bit 14?

[Pavel Roskin]

On Fri, 2007-08-31 at 23:57 +0200, Johannes Berg wrote:

> I second that. If somebody has a domain registered, I can host a wiki
> over with linuxwireless.org.

David Young owns radiotap.org, which is probably the best domain for the
radiotap standard.

-- 
Regards,
Pavel Roskin

Re: use of radiotap bit 14?

[David Young]

On Fri, Aug 31, 2007 at 11:57:55PM +0200, Johannes Berg wrote:
> On Fri, 2007-08-31 at 09:50 -0700, Gerald Combs wrote:
> 
> > An earlier email from David Young discussed placing the Radiotap 
> > documentation in a more "official" location -- I think this would help 
> > immensely.  For a developer starting from scratch, it's not immediately 
> > obvious where the canonical Radiotap specification is and it's very easy 
> > to get started down the wrong path.  For example, if NetBSD Radiotap 
> > header and man page show up in Google's rankings for "radiotap" and 
> > "radiotap specification", they're pretty far down the list.
> 
> I second that. If somebody has a domain registered, I can host a wiki
> over with linuxwireless.org.

Thank you for the offer!  Beggars cannot be choosers, but perhaps the
WireShark wiki is closer to "neutral ground" for a spec that the various
operating systems (*BSD, Linux, Solaris??) will share?  (Gerald?)

Can we grant access to the radiotap wiki page to only active radiotap
developers?

Dave

-- 
David Young             OJC Technologies
dyoung-eZodSLrBbDpBDgjK7y7TUQ@public.gmane.org      Urbana, IL * (217) 278-3933 ext 24

Re: use of radiotap bit 14?

[Gerald Combs]

David Young wrote:
> On Fri, Aug 31, 2007 at 11:57:55PM +0200, Johannes Berg wrote:
>> On Fri, 2007-08-31 at 09:50 -0700, Gerald Combs wrote:
>>
>>> An earlier email from David Young discussed placing the Radiotap 
>>> documentation in a more "official" location -- I think this would help 
>>> immensely.  For a developer starting from scratch, it's not immediately 
>>> obvious where the canonical Radiotap specification is and it's very easy 
>>> to get started down the wrong path.  For example, if NetBSD Radiotap 
>>> header and man page show up in Google's rankings for "radiotap" and 
>>> "radiotap specification", they're pretty far down the list.
>> I second that. If somebody has a domain registered, I can host a wiki
>> over with linuxwireless.org.
> 
> Thank you for the offer!  Beggars cannot be choosers, but perhaps the
> WireShark wiki is closer to "neutral ground" for a spec that the various
> operating systems (*BSD, Linux, Solaris??) will share?  (Gerald?)
> 
> Can we grant access to the radiotap wiki page to only active radiotap
> developers?

The wiki software we're using (MoinMoin) lets you add ACL groups pretty easily.
 Just let me know (offline or online) who needs access to the page.

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 780 bytes --]

On Wed, 2007-09-12 at 12:17 -0500, David Young wrote:

> > I second that. If somebody has a domain registered, I can host a wiki
> > over with linuxwireless.org.
> 
> Thank you for the offer!  Beggars cannot be choosers, but perhaps the
> WireShark wiki is closer to "neutral ground" for a spec that the various
> operating systems (*BSD, Linux, Solaris??) will share?  (Gerald?)

Oh of course, but I should've made clearer that I wasn't suggesting
making it a page in the linuxwireless.org wiki but rather offering to
host the radiotap.org domain as a name-virtual host on the same machine
against the same wiki engine.

But do feel free to do whatever you like better. Maybe having a whole
domain dedicated for maybe a few dozen pages is overkill :)

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 722 bytes --]


> > Thank you for the offer!  Beggars cannot be choosers, but perhaps the
> > WireShark wiki is closer to "neutral ground" for a spec that the various
> > operating systems (*BSD, Linux, Solaris??) will share?  (Gerald?)
> 
> Oh of course, but I should've made clearer that I wasn't suggesting
> making it a page in the linuxwireless.org wiki but rather offering to
> host the radiotap.org domain as a name-virtual host on the same machine
> against the same wiki engine.

I've gone ahead and set up a new wiki at
http://radiotap.sipsolutions.net/ which will also work as
http://radiotap.org/ and http://www.radiotap.org/ if the DNS is changed
accordingly.

Feel free to ignore me, of course.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: use of radiotap bit 14?

[David Young]

On Mon, Nov 26, 2007 at 05:58:21PM +0100, Johannes Berg wrote:
> 
> > > Thank you for the offer!  Beggars cannot be choosers, but perhaps the
> > > WireShark wiki is closer to "neutral ground" for a spec that the various
> > > operating systems (*BSD, Linux, Solaris??) will share?  (Gerald?)
> > 
> > Oh of course, but I should've made clearer that I wasn't suggesting
> > making it a page in the linuxwireless.org wiki but rather offering to
> > host the radiotap.org domain as a name-virtual host on the same machine
> > against the same wiki engine.
> 
> I've gone ahead and set up a new wiki at
> http://radiotap.sipsolutions.net/ which will also work as
> http://radiotap.org/ and http://www.radiotap.org/ if the DNS is changed
> accordingly.
> 
> Feel free to ignore me, of course.

Thanks, I have pointed www.radiotap.org and www.radiotap.net at
radiotap.sipsolutions.net.  I believe the latter name is without a
virtual host.

BTW, I have been preparing a "canonical" radiotap parser.  I will send
it to the list shortly.

Dave

-- 
David Young             OJC Technologies
dyoung-eZodSLrBbDpBDgjK7y7TUQ@public.gmane.org      Urbana, IL * (217) 278-3933 ext 24

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 468 bytes --]


> Thanks, I have pointed www.radiotap.org and www.radiotap.net at
> radiotap.sipsolutions.net.  I believe the latter name is without a
> virtual host.

Fixed. I didn't know you had .net as well. If you need any wiki config
changes etc. let me know. Right now, you need to be logged in to edit,
but I can change that too if you want.

> BTW, I have been preparing a "canonical" radiotap parser.  I will send
> it to the list shortly.

Neat :)

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: use of radiotap bit 14?

[David Young]

On Fri, Aug 31, 2007 at 12:16:54AM +0200, Johannes Berg wrote:
> Our radiotap header in Linux defines bit 14 as
> IEEE80211_RADIOTAP_RX_FLAGS; however running wireshark on it tells me
> that this bit means "FCS in header". Does anybody know which use is
> correct, if any?
> 
> Maybe we should simply skip to bit 32 or something and start redefining
> things properly?

There are technical reasons that we cannot simply skip.  I have a couple
of technical solutions in mind.

Solution 1: introduce a new bit, m, that nobody has used so far.
        Let it stand for the presence of a 32-bit magic number.
        When an interpreter sees a header whose presence bit 14 is
        set, let it test presence bit m.  If m is not present, let the
        interpreter abort.  If m is present, let the interpreter skip
        forward until it finds the magic at a 32-bit boundary.  Let it
        continue processing presence bits > m in the usual way.

Solution 2: increase the radiotap version number, and start over from
        bit 0.  Now we need a "legacy" parser for v0, 0 <= bit <= 13,
        and a "next gen" parser for v1.  No v0 header with bits >=
        14 set can be interpreted without ambiguity.

        2a: adopt all of the legacy presence bits 0 through 13 in v1
        2b: adopt some of the legacy presence bits in v1
        2c: start from scratch for v1

We could also use both Solution 1 and Solution 2; let interpreters treat
every v0 bit >= 14 as vendor-specific.

Dave

-- 
David Young             OJC Technologies
dyoung-eZodSLrBbDpBDgjK7y7TUQ@public.gmane.org      Urbana, IL * (217) 278-3933 ext 24

Re: use of radiotap bit 14?

[Pavel Roskin]

Hello, David!

On Wed, 2007-09-12 at 13:06 -0500, David Young wrote:

> There are technical reasons that we cannot simply skip.  I have a couple
> of technical solutions in mind.

Before we start addressing the technical side, we should probably
consider the "political" side.  Did the documentation specify where
unapproved extensions should be encoded?  Were the extensions added in
violation of the rules?  How widespread is the non-standard use of bit
14?  What can be done to avoid such situations in the future?

I would hate to see the protocol changed to work around the symptoms
without addressing the real problem.

I would probably try to put the workaround completely in userspace.  I
think the bit 14 abusers can be detected by the fact that bit 14 is the
highest byte they set and that they allocate 2 extra bytes for the
header (since FCS is 32 whereas RX flags is 16 bit).

Failing that, I think we can just obsolete bit 14 and use something else
(e.g. bit 18) for RX flags, or squeeze RX flags into the TX flags field,
perhaps into the upper byte.  I think it's much less intrusive than
adding any "magic" that would need special interpretation.

My preference is to change the protocol version only if we are going to
support radically new media or change something fundamental in the way
the header data is aligned.  Just adding "magic" would probably require
the protocol change.

-- 
Regards,
Pavel Roskin

Re: use of radiotap bit 14?

[David Young]

On Wed, Sep 12, 2007 at 03:03:58PM -0400, Pavel Roskin wrote:
> Hello, David!
> 
> On Wed, 2007-09-12 at 13:06 -0500, David Young wrote:
> 
> > There are technical reasons that we cannot simply skip.  I have a couple
> > of technical solutions in mind.
> 
> Before we start addressing the technical side, we should probably
> consider the "political" side.  Did the documentation specify where
> unapproved extensions should be encoded?  Were the extensions added in
> violation of the rules?  How widespread is the non-standard use of bit
> 14?  What can be done to avoid such situations in the future?
> 
> I would hate to see the protocol changed to work around the symptoms
> without addressing the real problem.

By the nature of radiotap, one cannot introduce an "unapproved extension"
at bit n without throwing off the interpretation of bits n+1 and above, be
they approved or not.  I do not think this could escape an implementor's
attention.  Radiotap did not ever provide for unapproved extensions.

IIRC, the documentation in the manual page has always named me as the
point of contact for extensions.  As we know, the manual page has not
been widely disseminated.  The header file, which has been copied around
a lot, has (ordinarily) contained my copyright, but it did not mention
any procedure for adding new fields.

I think that the divergence starting at bit 14 happened in this way:

        1 I miscommunicated.  I added the _FCS field to the header file
          in NetBSD in the middle of a discussion; subsequent discussion
          killed _FCS off.  I only removed _FCS when someone pointed
          out that I had mistakenly left it in.  The manual page has
          not always been adequate, and I have not been publicized it
          very well.

        2 Private radiotap development "leaked" into public.  That is
          how the _RX_FLAGS and _TX_FLAGS fields, which were not
          intentionally intended for public consumption, came to be in
          some Linux drivers.

        3 All communication about new radiotap fields has been reactive
          instead of proactive.

        4 Radiotap development had no "center."  You could not go
          to one place to find the standard, discussions, etc.

I think that having the wiki and mailing list will help #3 and #4.
Adding a method for embedding vendor-specific fields will help #2.
Centralizing the documentation at a Wiki will help it get the oversight
it needs; I hope that will protect against a mistake such as #1, and it
will provide the center.

I believe we are moving in the right direction to solve the political
problem.  I think that participation needs to widen, and I think that
we need to make good on "rough consensus and running code" by writing
more interpreter code for WireShark and TCPDump.

> I would probably try to put the workaround completely in userspace.  I
> think the bit 14 abusers can be detected by the fact that bit 14 is the
> highest byte they set and that they allocate 2 extra bytes for the
> header (since FCS is 32 whereas RX flags is 16 bit).

It's worse than that.  OpenBSD uses several bits.

Dave

-- 
David Young             OJC Technologies
dyoung-eZodSLrBbDpBDgjK7y7TUQ@public.gmane.org      Urbana, IL * (217) 278-3933 ext 24

Re: use of radiotap bit 14?

[Guy Harris]

David Young wrote:

> It's worse than that.  OpenBSD uses several bits.

I've sent Reyk Floeter mail about this, as he appears to be the person who
added those bits.  I haven't heard from him; I assume he's not on the
radiotap list.

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1071 bytes --]

On Wed, 2007-09-12 at 13:06 -0500, David Young wrote:

> Solution 2: increase the radiotap version number, and start over from
>         bit 0.  Now we need a "legacy" parser for v0, 0 <= bit <= 13,
>         and a "next gen" parser for v1.  No v0 header with bits >=
>         14 set can be interpreted without ambiguity.
> 
>         2a: adopt all of the legacy presence bits 0 through 13 in v1
>         2b: adopt some of the legacy presence bits in v1
>         2c: start from scratch for v1

Personally, I favour solution 2a as it allows us to easily port parsers
and even share code, but before we do that, we need a good way to allow
vendor-specific extensions. I've been thinking that a format with an
explicit length field like 802.11 information elements would have been
easier for this (think 802.11 vendor-specific IEs) but I'm sure we can
come up with a solution for vendor-specific fields, for example breaking
the "bit ordering" == "element ordering" for them and requiring them to
be last at all times or something similar.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: use of radiotap bit 14?

[David Young]

On Thu, Sep 13, 2007 at 09:29:42AM +0200, Johannes Berg wrote:
> On Wed, 2007-09-12 at 13:06 -0500, David Young wrote:
> 
> > Solution 2: increase the radiotap version number, and start over from
> >         bit 0.  Now we need a "legacy" parser for v0, 0 <= bit <= 13,
> >         and a "next gen" parser for v1.  No v0 header with bits >=
> >         14 set can be interpreted without ambiguity.
> > 
> >         2a: adopt all of the legacy presence bits 0 through 13 in v1
> >         2b: adopt some of the legacy presence bits in v1
> >         2c: start from scratch for v1
> 
> Personally, I favour solution 2a as it allows us to easily port parsers
> and even share code,

Johannes,

My problem with #2 right now is that you and I are acting as stronger
advocates for the folks who adopted conflicting radiotap numbers than
they themselves are.  Meanwhile, v0 has some momentum: FreeBSD has
introduced useful new fields, FreeBSD and NetBSD are in agreement,
and there are NO interpreters in tcpdump or in wireshark for any field
past 15.  Let's stick with v0 until somebody objects.

> but before we do that, we need a good way to allow
> vendor-specific extensions. I've been thinking that a format with an
> explicit length field like 802.11 information elements would have been
> easier for this (think 802.11 vendor-specific IEs) but I'm sure we can
> come up with a solution for vendor-specific fields, for example breaking
> the "bit ordering" == "element ordering" for them and requiring them to
> be last at all times or something similar.

I propose to add two new presence bits, 29 and 30.  Let them both reset
the bitmap index to 0.  That is, the bits in following next 32-bit
presence word are interpreted as bits in 0..31, not n..31+n.  We must
use both of these bits in conjunction with the extension bit, bit 31,
or else they will have no effect.

Let bit 29 make the interpreter change to a vendor namespace, and let
bit 30 make the interpreter change back to the default radiotap namespace.
Some detailed field specs follow.

Vendor Namespace Field (29)

        presence bit IEEE80211_RADIOTAP_NSVENDOR = 29
        1-byte alignment
        5 bytes long
        scope: this bit is reserved in all namespaces, and it is
               interpreted identically in every namespace

        The Vendor Namespace Field contains three sub-fields.  The first
        sub-field is 3 bytes long.  It contains the vendor's IEEE 802
        Organizationally Unique Identifier (OUI).  The fourth byte is
        a vendor-specific "namespace selector."

        Before it resumes interpretation of presence bits in the following
        32-bit presence words, if any, the interpreter shall reset its
        presence-bitmap index to 0, and change to the vendor namespace
        specified by the OUI and selector.

        The fifth byte, skip_length, tells the interpreter how many bytes
        of data after the end of the Vendor Namespace Field can only be
        interpreted according to the vendor namespace.  If a radiotap
        header changes to a namespace that the interpreter does not
        understand, and back, the interpreter may resume interpretation
        in the new namespace by skipping skip_length data bytes after
        the end of the Vendor Namespace Field.

Reset Namespace Field (30)

        presence bit IEEE80211_RADIOTAP_NSRESET = 30
        1-byte alignment
        0 bytes long
        scope: this bit is reserved in all namespaces, and it is
               interpreted identically in every namespace

        Upon interpreting this field, the interpreter shall reset its
        presence-bitmap index to 0, and change to the default radiotap
        namespace, before it interprets subsequent presence-bitmap words.

If we adopt Reset/Vendor Namespace, it is possible for a radiotap field
to appear twice or more in one radiotap header.  I believe that this is
a feature, and I have some ideas for using Reset Namespace to specify /
record transmit strategies.

Dave

-- 
David Young             OJC Technologies
dyoung-eZodSLrBbDpBDgjK7y7TUQ@public.gmane.org      Urbana, IL * (217) 278-3933 ext 24

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 3787 bytes --]

On Sun, 2007-10-07 at 18:55 -0500, David Young wrote:

> My problem with #2 right now is that you and I are acting as stronger
> advocates for the folks who adopted conflicting radiotap numbers than
> they themselves are.  Meanwhile, v0 has some momentum: FreeBSD has
> introduced useful new fields, FreeBSD and NetBSD are in agreement,
> and there are NO interpreters in tcpdump or in wireshark for any field
> past 15.  Let's stick with v0 until somebody objects.

Ok. I just need a bunch of fields in the Linux code for injection
purposes and am actually developing userspace code to parse/create
these. Also, here I am actually using bit 14/15/16/17 as RX/TX flags,
RTS/DATA retries. It would help much if we could go forward with hosting
the radiotap header "on neutral ground" and standardise these fields, no
matter how.

> I propose to add two new presence bits, 29 and 30.  Let them both reset
> the bitmap index to 0.  That is, the bits in following next 32-bit
> presence word are interpreted as bits in 0..31, not n..31+n.  We must
> use both of these bits in conjunction with the extension bit, bit 31,
> or else they will have no effect.
> 
> Let bit 29 make the interpreter change to a vendor namespace, and let
> bit 30 make the interpreter change back to the default radiotap namespace.

This doesn't seem to require two bits, why not just use a single one to
mean "next bitmap is vendor namespace"?

> Some detailed field specs follow.
> 
> Vendor Namespace Field (29)
> 
>         presence bit IEEE80211_RADIOTAP_NSVENDOR = 29
>         1-byte alignment
>         5 bytes long
>         scope: this bit is reserved in all namespaces, and it is
>                interpreted identically in every namespace
> 
>         The Vendor Namespace Field contains three sub-fields.  The first
>         sub-field is 3 bytes long.  It contains the vendor's IEEE 802
>         Organizationally Unique Identifier (OUI).  The fourth byte is
>         a vendor-specific "namespace selector."
> 
>         Before it resumes interpretation of presence bits in the following
>         32-bit presence words, if any, the interpreter shall reset its
>         presence-bitmap index to 0, and change to the vendor namespace
>         specified by the OUI and selector.
> 
>         The fifth byte, skip_length, tells the interpreter how many bytes
>         of data after the end of the Vendor Namespace Field can only be
>         interpreted according to the vendor namespace.  If a radiotap
>         header changes to a namespace that the interpreter does not
>         understand, and back, the interpreter may resume interpretation
>         in the new namespace by skipping skip_length data bytes after
>         the end of the Vendor Namespace Field.
> 
> Reset Namespace Field (30)
> 
>         presence bit IEEE80211_RADIOTAP_NSRESET = 30
>         1-byte alignment
>         0 bytes long
>         scope: this bit is reserved in all namespaces, and it is
>                interpreted identically in every namespace
> 
>         Upon interpreting this field, the interpreter shall reset its
>         presence-bitmap index to 0, and change to the default radiotap
>         namespace, before it interprets subsequent presence-bitmap words.
> 
> If we adopt Reset/Vendor Namespace, it is possible for a radiotap field
> to appear twice or more in one radiotap header.  I believe that this is
> a feature, and I have some ideas for using Reset Namespace to specify /
> record transmit strategies.

Ah, I see, ok, yes, now I know why you want two bits. Sounds good to me,
although I think you could add some ascii art example as this
description is quite procedural and gives little overview :)

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1375 bytes --]


> > My problem with #2 right now is that you and I are acting as stronger
> > advocates for the folks who adopted conflicting radiotap numbers than
> > they themselves are.  Meanwhile, v0 has some momentum: FreeBSD has
> > introduced useful new fields, FreeBSD and NetBSD are in agreement,
> > and there are NO interpreters in tcpdump or in wireshark for any field
> > past 15.  Let's stick with v0 until somebody objects.
> 
> Ok. I just need a bunch of fields in the Linux code for injection
> purposes and am actually developing userspace code to parse/create
> these. Also, here I am actually using bit 14/15/16/17 as RX/TX flags,
> RTS/DATA retries. It would help much if we could go forward with hosting
> the radiotap header "on neutral ground" and standardise these fields, no
> matter how.
> 
> > I propose to add two new presence bits, 29 and 30.  Let them both reset
> > the bitmap index to 0.  That is, the bits in following next 32-bit
> > presence word are interpreted as bits in 0..31, not n..31+n.  We must
> > use both of these bits in conjunction with the extension bit, bit 31,
> > or else they will have no effect.

Ping? Any news on this? Proper 11w implementation in Linux may require
Linux-specific fields to specify which interface a frame belongs to (ie.
which interface the code should use to look up keys etc.)

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1933 bytes --]

David,

> I propose to add two new presence bits, 29 and 30.  Let them both reset
> the bitmap index to 0.  That is, the bits in following next 32-bit
> presence word are interpreted as bits in 0..31, not n..31+n.  We must
> use both of these bits in conjunction with the extension bit, bit 31,
> or else they will have no effect.

The more I think about your proposal the more sense does it make :)

One thing occurred to me: this extension with bit 31 is interesting in
that it leads to holes in the numbering because number 63 can never be
assigned (it corresponds to 31 in the second bitmap). Similar things
will happen when 29/30 are reserved.

Since we haven't yet assigned anything above 23, maybe we should reserve
the flag bits 24..31 for global bits so future extensions like the
global bits 29/30 you defined could be done? As you certainly noticed
when defining the bits 29/30, this is perfectly compatible with current
parsers because they have to stop parsing at unknown bits anyway. This
would allow also allow us to declare that the bits in the second bitmap
start being numbered from 24 rather than 32 (because it doesn't matter
if existing parsers think it's an unknown bit 32 when it really is the
unknown bit 24). Of course the remaining 5 bits 24..28 would have to be
reserved and set zero, parsers would have to stop parsing when
encountering any of them set they cannot handle.

Basically this would lead to the following layout:

| n0..23 (24 bits) | g0..7 (8 bits) | n24..47 (24 bits) | g0..7 (8 bits) | ...

rather than

| 0..28 (29 bits) | 29..31 (3 bits) | | 32..60 (29 bits) | 29..31 (3 bits) | ...

where n/g indicate that they're in the numbering/global space.

I'm only proposing to do the split as 24/8 because it seems more natural
to have a whole byte reserved in the upper part although it is likely
we'll never need the extra five global bits.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1310 bytes --]

> Reset Namespace Field (30)
> 
>         presence bit IEEE80211_RADIOTAP_NSRESET = 30
>         1-byte alignment
>         0 bytes long
>         scope: this bit is reserved in all namespaces, and it is
>                interpreted identically in every namespace
> 
>         Upon interpreting this field, the interpreter shall reset its
>         presence-bitmap index to 0, and change to the default radiotap
>         namespace, before it interprets subsequent presence-bitmap words.

Maybe we can find somebody whose OUI we can borrow, then we could define
that the namespace selector 0 within that OUI selects the radiotap
namespace rather than having an extra bit for it. I think that would be
simpler to parse and understand.

That way, we could do everything with a single bit. Since that takes
quite a bit of space (5 bytes plus padding for each switch) the reset
bit would be useful independently to reset within a namespace. It might
make sense to define it to have a field too, namely a u8 that indicates
which offset to reset to: field value * 32 (or 24 if you accept my other
proposal)

If there is somebody willing to donate part of their radiotap namespace,
having an "experimental" selector and a selectors for each OS would
probably be useful as well.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 408 bytes --]


> and there are NO interpreters in tcpdump or in wireshark for any field
> past 15.  Let's stick with v0 until somebody objects.

That's actually no longer true, wireshark interprets field 18 now :/

> I propose to add two new presence bits, 29 and 30.
[...]

I'm trying to capture this with examples here:
http://www.radiotap.org/Discussion/Vendor-Extensions

Look there in a bit.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 856 bytes --]


> > I propose to add two new presence bits, 29 and 30.
> [...]
> 
> I'm trying to capture this with examples here:
> http://www.radiotap.org/Discussion/Vendor-Extensions
> 
> Look there in a bit.

Done. While writing that, I noticed that it is not strictly required to
reserve two bits in every it_present field and added the following under
a "Discussion" headline:

There is no need to reserve two bits in every it_present bitmap if one
is willing to expend slightly more space for vendor namespaces: Say the
reset bit is 30 instead of 29 and vendor-namespace is a regular
free-running bit number (e.g. 20) that is reserved in every namespace.
Then, to switch into a vendor namespace, you first reset the radiotap
namespace to 0 using flagbit 30 and then switch to the vendor namespace
using a normal field (e.g. 20).

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 994 bytes --]

Ok, I tried collecting all the information, and here's what I found so
far:

http://www.radiotap.org/suggested-fields

Bits 14, 15 and 16 are defined twice.

Can we declare bits 14 through 18 reserved (i.e. may not be used) and
continue life at 19? Current parsers would have to be changed (the only
one I found is wireshark and internal ones like hostapd, Linux kernel)
to treat the presence of those bits to be an abort-condition and as an
error if there are any above them, current generators would have to stop
generating those fields over time and instead generate standardised
versions.

For me, of course, it would be more comfortable to stick with RX/TX
flags in 14/15 and RTS retries in 16, but since it's not widely used in
Linux yet I can easily discard them.

We need to reach a decision though, this is blocking further work on 11w
and AP mode in Linux. I'm willing to lead such an effort if you let me
"take over" maintainership of the standard.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 899 bytes --]

On Thu, 2008-06-19 at 19:44 +0200, Johannes Berg wrote:
> Ok, I tried collecting all the information, and here's what I found so
> far:
> 
> http://www.radiotap.org/suggested-fields
> 
> Bits 14, 15 and 16 are defined twice.

I just noticed that at least wireshark also defines two non-standard
bits for the 'flags' field (field bit 1):

#define IEEE80211_RADIOTAP_F_BADFCS     0x40    /* does not pass FCS check */
#define IEEE80211_RADIOTAP_F_SHORTGI    0x80    /* HT short GI */


Also, it defines that if the rate (field bit 2) is >= 0x80 (or rather,
has bit 0x80 set) then the lower 4 bits are used to define some HT rate?

Where the hell did that come from?

Crap. I'm starting to consider just cycling the version number for linux
(to 42!) and you guys can play catch-up for each OS by itself rather
than us trying to figure out wtf. we should do.

:(

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[Gerald Combs]

Johannes Berg wrote:
> I just noticed that at least wireshark also defines two non-standard
> bits for the 'flags' field (field bit 1):
> 
> #define IEEE80211_RADIOTAP_F_BADFCS     0x40    /* does not pass FCS check */
> #define IEEE80211_RADIOTAP_F_SHORTGI    0x80    /* HT short GI */
> 
> 
> Also, it defines that if the rate (field bit 2) is >= 0x80 (or rather,
> has bit 0x80 set) then the lower 4 bits are used to define some HT rate?
> 
> Where the hell did that come from?

FreeBSD: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/net80211/ieee80211_radiotap.h

Re: use of radiotap bit 14?

[Guy Harris]

Gerald Combs wrote:

> FreeBSD: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/net80211/ieee80211_radiotap.h

In particular, note both bits, as well as the comment for 
IEEE80211_RADIOTAP_RATE:

  * IEEE80211_RADIOTAP_RATE              uint8_t         500kb/s or index
  *
  *      Tx/Rx data rate.  If bit 0x80 is set then it represents an
  *      an MCS index and not an IEEE rate.

The BADFCS bit, and the comment for IEEE80211_RADIOTAP_RATE, but not the 
SHORTGI bit, are also in the NetBSD header:

	http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/net80211/ieee80211_radiotap.h

OpenBSD has none of them.

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 798 bytes --]

On Thu, 2008-06-19 at 11:34 -0700, Guy Harris wrote:
> Gerald Combs wrote:
> 
> > FreeBSD: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/net80211/ieee80211_radiotap.h
> 
> In particular, note both bits, as well as the comment for 
> IEEE80211_RADIOTAP_RATE:
> 
>   * IEEE80211_RADIOTAP_RATE              uint8_t         500kb/s or index
>   *
>   *      Tx/Rx data rate.  If bit 0x80 is set then it represents an
>   *      an MCS index and not an IEEE rate.
> 
> The BADFCS bit, and the comment for IEEE80211_RADIOTAP_RATE, but not the 
> SHORTGI bit, are also in the NetBSD header:
> 
> 	http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/net80211/ieee80211_radiotap.h
> 
> OpenBSD has none of them.

Ok, thank you both, I had only checked out the respective man-pages.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 391 bytes --]


> I just noticed that at least wireshark also defines two non-standard
> bits for the 'flags' field (field bit 1):
> 
> #define IEEE80211_RADIOTAP_F_BADFCS     0x40    /* does not pass FCS check */

Incidentally, the RX flags (field 14) have a flag for that too... Should
we remove that from the proposal and use this one instead so we don't
upset even more people?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[Pavel Roskin]

On Thu, 2008-06-19 at 19:44 +0200, Johannes Berg wrote:
> Ok, I tried collecting all the information, and here's what I found so
> far:
> 
> http://www.radiotap.org/suggested-fields
> 
> Bits 14, 15 and 16 are defined twice.

If we call those fields "suggested", we deceive ourselves.  Suggested
fields should not come with bit numbers.  There bit numbers should be
assigned only when the suggestion is accepted.

> Can we declare bits 14 through 18 reserved (i.e. may not be used) and
> continue life at 19?

Perhaps you could add your proposal without a bit number and then ask
for the numbers to be assigned?  If 19 is the lowest uncontested number,
then it would be assigned.

There is no need to wait with your proposals until the contested fields
controversy is resolved.

-- 
Regards,
Pavel Roskin

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

On Thu, 2008-06-19 at 14:13 -0400, Pavel Roskin wrote:
> On Thu, 2008-06-19 at 19:44 +0200, Johannes Berg wrote:
> > Ok, I tried collecting all the information, and here's what I found so
> > far:
> > 
> > http://www.radiotap.org/suggested-fields
> > 
> > Bits 14, 15 and 16 are defined twice.
> 
> If we call those fields "suggested", we deceive ourselves.  Suggested
> fields should not come with bit numbers.  There bit numbers should be
> assigned only when the suggestion is accepted.

Well, yes, but it seemed prudent to not introduce a third
"used-but-not-standardised" category and these bit numbers reflect the
fact that they are indeed used with those bit numbers in varying OSes.

I hope the audience of this website is smart enough to realise that.

> Perhaps you could add your proposal without a bit number and then ask
> for the numbers to be assigned?  If 19 is the lowest uncontested number,
> then it would be assigned.
> 
> There is no need to wait with your proposals until the contested fields
> controversy is resolved.

Well, I don't have a proposal yet, those fields that are there are
sufficient if we also accept the vendor-namespaces idea that I'm just
trying to put into understandable words.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[Pavel Roskin]

On Thu, 2008-06-19 at 20:18 +0200, Johannes Berg wrote:

> Well, yes, but it seemed prudent to not introduce a third
> "used-but-not-standardised" category and these bit numbers reflect the
> fact that they are indeed used with those bit numbers in varying OSes.

OK.  And proper proposals should have "unassigned" under "Bit Number".

> I hope the audience of this website is smart enough to realise that.

Sure, but the contested fields set a bad example.  It may be better to
write "don't do it again" in some places.

> > Perhaps you could add your proposal without a bit number and then ask
> > for the numbers to be assigned?  If 19 is the lowest uncontested number,
> > then it would be assigned.
> > 
> > There is no need to wait with your proposals until the contested fields
> > controversy is resolved.
> 
> Well, I don't have a proposal yet, those fields that are there are
> sufficient if we also accept the vendor-namespaces idea that I'm just
> trying to put into understandable words.

There should be a local/testing/vendor area or bit numbers so that the
proposals can be tested before they are submitted.  Another approach
would be to have an extra header consisting of items in the format like:

header length, in bytes
number of items (N)
vendor id for item 1
length of item 1
type of item 1
data of item 1
...
vendor id for item N
length of item N
type of item N
data of item N

And the radiotap header would indicate whether the extra header is
present.  That would be very flexible for testing (item type could be
incremented when something changes).

-- 
Regards,
Pavel Roskin

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]


> > Well, yes, but it seemed prudent to not introduce a third
> > "used-but-not-standardised" category and these bit numbers reflect the
> > fact that they are indeed used with those bit numbers in varying OSes.
> 
> OK.  And proper proposals should have "unassigned" under "Bit Number".

Yeah, I'll add a note.

> > I hope the audience of this website is smart enough to realise that.
> 
> Sure, but the contested fields set a bad example.  It may be better to
> write "don't do it again" in some places.

Ok :)

> > Well, I don't have a proposal yet, those fields that are there are
> > sufficient if we also accept the vendor-namespaces idea that I'm just
> > trying to put into understandable words.
> 
> There should be a local/testing/vendor area or bit numbers so that the
> proposals can be tested before they are submitted.  Another approach
> would be to have an extra header consisting of items in the format like:

[...]

Dave had come up with a different proposal which I just captured on the
wiki at http://www.radiotap.org/Discussion/Vendor-Extensions

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[David Young]

On Thu, Jun 19, 2008 at 07:44:59PM +0200, Johannes Berg wrote:
> Ok, I tried collecting all the information, and here's what I found so
> far:
> 
> http://www.radiotap.org/suggested-fields
> 
> Bits 14, 15 and 16 are defined twice.
> 
> Can we declare bits 14 through 18 reserved (i.e. may not be used) and
> continue life at 19? Current parsers would have to be changed (the only
> one I found is wireshark and internal ones like hostapd, Linux kernel)
> to treat the presence of those bits to be an abort-condition and as an
> error if there are any above them, current generators would have to stop
> generating those fields over time and instead generate standardised
> versions.
> 
> For me, of course, it would be more comfortable to stick with RX/TX
> flags in 14/15 and RTS retries in 16, but since it's not widely used in
> Linux yet I can easily discard them.

Johannes,

I favor these assignments:

14 /RX flags
15 /TX flags
16 /RTS retries
17 /data retries
18 /XChannel 

> We need to reach a decision though, this is blocking further work on 11w
> and AP mode in Linux. I'm willing to lead such an effort if you let me
> "take over" maintainership of the standard.

I think that what you mean is to lead a discussion of fields 14 and
beyond?  I reckon that no fewer than 4 persons or organizations have
"taken over" maintainership of the standard by now, and that is how we got
into the current mess, with the same bits used for different purposes. :-)

I initiated the mailing list so that there is one place to propose new
fields for discussion and adoption by the community of radiotap users.
It's not here so that anybody, least of all you and I, can tell everybody
else what the fields are.

I don't think that reaching agreement on fields 14 and greater is
necessarily so hard.  Here are three reasons why.

First, Guy Harris invited a lot of people to this discussion list who
seemed to have a stake in radiotap, and some of them did not show up.
As I said before, we do not have to act as advocates for people who don't
show up.  If the person who uses bit 14 for the flavor of packets will
not stand up for it here, then there is no use wringing our hands over
the conflicting assignment.

Agreement is in everybody's interest.  Disagreement is not.

This is not the IETF, but I think that "rough consensus and running
code" is a perfectly fine way to decide what fields we adopt, and there
is neither a consensus nor running code for some fields.  I think that
the lack of code makes the current "conflicts" easy to resolve: just how
much pain can we cause by assigning field 15 to Rx flags if some obscure
driver emits field 15 equal to a packet's smell, but neither WireShark
nor TCPDump interprets it at all?

Dave

-- 
David Young             OJC Technologies
dyoung-eZodSLrBbDpBDgjK7y7TUQ@public.gmane.org      Urbana, IL * (217) 278-3933 ext 24

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 2313 bytes --]

On Thu, 2008-06-19 at 13:56 -0500, David Young wrote:

> I favor these assignments:
> 
> 14 /RX flags
> 15 /TX flags
> 16 /RTS retries
> 17 /data retries
> 18 /XChannel 

That'd be great to me. Want me to declare them that on the wiki, move
them over to the defined-fields and add the other ones as "suggested
fields" without assigned numbers?

> I think that what you mean is to lead a discussion of fields 14 and
> beyond?  I reckon that no fewer than 4 persons or organizations have
> "taken over" maintainership of the standard by now, and that is how we got
> into the current mess, with the same bits used for different purposes. :-)
> 
> I initiated the mailing list so that there is one place to propose new
> fields for discussion and adoption by the community of radiotap users.
> It's not here so that anybody, least of all you and I, can tell everybody
> else what the fields are.

Yeah, I know, it's a mess and will always be unless we discuss things
here and on the wiki.

> I don't think that reaching agreement on fields 14 and greater is
> necessarily so hard.  Here are three reasons why.
> 
> First, Guy Harris invited a lot of people to this discussion list who
> seemed to have a stake in radiotap, and some of them did not show up.
> As I said before, we do not have to act as advocates for people who don't
> show up.  If the person who uses bit 14 for the flavor of packets will
> not stand up for it here, then there is no use wringing our hands over
> the conflicting assignment.
> 
> Agreement is in everybody's interest.  Disagreement is not.
> 
> This is not the IETF, but I think that "rough consensus and running
> code" is a perfectly fine way to decide what fields we adopt, and there
> is neither a consensus nor running code for some fields.  I think that
> the lack of code makes the current "conflicts" easy to resolve: just how
> much pain can we cause by assigning field 15 to Rx flags if some obscure
> driver emits field 15 equal to a packet's smell, but neither WireShark
> nor TCPDump interprets it at all?

True.

Problem is, wireshark actually takes bit 14 to mean "FCS in header".
Easily changed though if Gerald is willing to take such a patch, I can
cook up a patch to make it parse the fields above.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[Guy Harris]

Johannes Berg wrote:

> Problem is, wireshark actually takes bit 14 to mean "FCS in header".
> Easily changed though if Gerald is willing to take such a patch, I can
> cook up a patch to make it parse the fields above.

OpenBSD defines 14 as IEEE80211_RADIOTAP_FCS.  However, a search through 
the top-of-CVS-tree OpenBSD kernel code shows nothing that *uses* that 
presence bit, and neither top-of-tree FreeBSD nor NetBSD use it (I 
assume from what Sam Leffler said that no FreeBSD release did, and given 
that NetBSD uses it for other purposes I assume no NetBSD release did), 
and I assume, given that you're proposing this, that Linux doesn't use 
it for that purpose.

Given that I've gotten no response from the OpenBSD people from my mail 
and bug report about presence-bit collisions, I'd say Wireshark should 
conform to the standard, and if that breaks analysis of captures from 
OpenBSD systems, too bad.

(If that becomes an issue serious enough to care about, we could either 
get a new DLT_ value for standard radiotap headers, and use that, or rev 
the radiotap header version, but that wouldn't help with existing captures.)

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 940 bytes --]

On Thu, 2008-06-19 at 12:33 -0700, Guy Harris wrote:
> Johannes Berg wrote:
> 
> > Problem is, wireshark actually takes bit 14 to mean "FCS in header".
> > Easily changed though if Gerald is willing to take such a patch, I can
> > cook up a patch to make it parse the fields above.
> 
> OpenBSD defines 14 as IEEE80211_RADIOTAP_FCS.  However, a search through 
> the top-of-CVS-tree OpenBSD kernel code shows nothing that *uses* that 
> presence bit, and neither top-of-tree FreeBSD nor NetBSD use it (I 
> assume from what Sam Leffler said that no FreeBSD release did, and given 
> that NetBSD uses it for other purposes I assume no NetBSD release did), 
> and I assume, given that you're proposing this, that Linux doesn't use 
> it for that purpose.

Ok, cool. Yes, we use the RX flags meaning for bit 14. I guess we
shouldn't have started doing that but that stuff was there before I
understood the issue.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[David Young]

On Thu, Jun 19, 2008 at 09:10:33PM +0200, Johannes Berg wrote:
> On Thu, 2008-06-19 at 13:56 -0500, David Young wrote:
> 
> > I favor these assignments:
> > 
> > 14 /RX flags
> > 15 /TX flags
> > 16 /RTS retries
> > 17 /data retries
> > 18 /XChannel 
> 
> That'd be great to me. Want me to declare them that on the wiki, move
> them over to the defined-fields and add the other ones as "suggested
> fields" without assigned numbers?

Well, you and I agree, but let us give everyone else a chance. :-)
Let's do this:

1 define the alignment, width, interpretation of the fields (I believe
  this is done, somewhere.)
2 produce implementations wireshark and/or tcpdump and for one or
  more driver
3 post fields and code here, and on the wiki
4 leave 7 days for discussion (ordinarily, this should be longer)
5 if the proposal withstands discussion, re-post proposal in final form
6 adopt the proposal in 7 days if there are no further objections
  (discussion is over, however, 7 days lets everyone verify that the
  proposal and amendments, as discussed, have become the standard)

> Problem is, wireshark actually takes bit 14 to mean "FCS in header".

I'd forgotten about that, but I think that we still have some flexibility.

Dave

-- 
David Young             OJC Technologies
dyoung-eZodSLrBbDpBDgjK7y7TUQ@public.gmane.org      Urbana, IL * (217) 278-3933 ext 24

Re: use of radiotap bit 14?

[Guy Harris]

David Young wrote:
> On Thu, Jun 19, 2008 at 09:10:33PM +0200, Johannes Berg wrote:
>> Problem is, wireshark actually takes bit 14 to mean "FCS in header".
> 
> I'd forgotten about that, but I think that we still have some flexibility.

As per my mail, we can just take that out; OpenBSD defines it as "FCS in 
header", but doesn't appear to use it, and I don't know of any other 
system that uses it as "FCS in header".

Gerald, should we just remove the code that dissects 
IEEE80211_RADIOTAP_FCS from the 1.0.1 Wireshark release, in preparation 
for later releases interpreting presence bit 14 as RX flags?

The collision problem with 15 and 16 is worse - OpenBSD defines 15 as 
IEEE80211_RADIOTAP_HWQUEUE, and defines 16 as IEEE80211_RADIOTAP_RSSI, 
and several of their drivers use one or the other or both.  Wireshark 
currently doesn't dissect them; it should dissect them according to the 
standard.

Re: use of radiotap bit 14?

[Loris Degioanni]

Guy Harris wrote:

> David Young wrote:
>> On Thu, Jun 19, 2008 at 09:10:33PM +0200, Johannes Berg wrote:
>>> Problem is, wireshark actually takes bit 14 to mean "FCS in header".
>>
>> I'd forgotten about that, but I think that we still have some 
>> flexibility.
> 
> As per my mail, we can just take that out; OpenBSD defines it as "FCS in 
> header", but doesn't appear to use it, and I don't know of any other 
> system that uses it as "FCS in header".
> 
> Gerald, should we just remove the code that dissects 
> IEEE80211_RADIOTAP_FCS from the 1.0.1 Wireshark release, in preparation 
> for later releases interpreting presence bit 14 as RX flags?
> 

AirPcap uses the IEEE80211_RADIOTAP_FCS flag. We can remove it, but all 
the trace files produced with airpcap+radiotap until now include it.

Loris

Re: use of radiotap bit 14?

[Guy Harris]

Loris Degioanni wrote:

> AirPcap uses the IEEE80211_RADIOTAP_FCS flag. We can remove it, but all 
> the trace files produced with airpcap+radiotap until now include it.

Oh, well.

Is it time to either rev the version number or pick a new DLT_ for 
"standard" radiotap?

That doesn't help with existing captures - we'd probably have to add a 
preference setting to control how to interpret bit 14 (and bits 15 and 
16, if we ever add support for them, given that there's a collision 
between the NetBSD and OpenBSD uses of them) - but it'd at least give us 
a new DLT_ or version number value to use in the future, and anybody who 
wants tcpdump or Wireshark to interpret any presence bits in a 
non-standard fashion in files with that DLT_ value/version number can be 
politely told "no, ask for an assignment from the radiotap list".

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 399 bytes --]


> AirPcap uses the IEEE80211_RADIOTAP_FCS flag. We can remove it, but all 
> the trace files produced with airpcap+radiotap until now include it.

Is that really going to be a problem? It'd be trivial to write a filter
that rewrites such existing files to have the FCS at end (I never
understood the need for in-header FCS anyway.) And even if you don't,
you only lose the FCS?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14? [mostly 11n]

[Sam Leffler]

Guy Harris wrote:
> David Young wrote:
>> On Thu, Jun 19, 2008 at 09:10:33PM +0200, Johannes Berg wrote:
>>> Problem is, wireshark actually takes bit 14 to mean "FCS in header".
>>
>> I'd forgotten about that, but I think that we still have some 
>> flexibility.
>
> As per my mail, we can just take that out; OpenBSD defines it as "FCS 
> in header", but doesn't appear to use it, and I don't know of any 
> other system that uses it as "FCS in header".

Right, as I told you that mechanism was only ever defined by netbsd. 
Pulling the FCS out of band is not a good idea.  Presumably obsd just 
brought it over.

>
> Gerald, should we just remove the code that dissects 
> IEEE80211_RADIOTAP_FCS from the 1.0.1 Wireshark release, in 
> preparation for later releases interpreting presence bit 14 as RX flags?

As Guy mentioned I worked with him to resolve at least one conflict 
between various systems.  The definitions in freebsd have been in use 
for a while and have shipped in released versions of the os so there are 
many data files "in the field" and unless there's some way to maintain 
backwards compatibility it's unlikely they'll change.

As to shortgi et al, I've had simple 11n sniffing support for ~2 years 
and sent Gerald+Guy my mods to provide the most basic support in 
wireshark (haven't pushed tcpdump mods upstream year but they are in 
freebsd cvs).  Since then I've done more involved changes for wireshark 
but haven't pushed them back because I first wanted to leverage the PPI 
dissector's support for reconstructing AMPDU aggregates.  However my 
other changes build on the earlier work and that work addresses issues 
more general than 11n so I believe they are worth keeping.

>
> The collision problem with 15 and 16 is worse - OpenBSD defines 15 as 
> IEEE80211_RADIOTAP_HWQUEUE, and defines 16 as IEEE80211_RADIOTAP_RSSI, 
> and several of their drivers use one or the other or both.  Wireshark 
> currently doesn't dissect them; it should dissect them according to 
> the standard.

I gave up trying to deal w/ obsd and tell anyone that comes to me w/ 
data files from them that they need to convert formats if they want to 
inspect the data.  I know that doesn't help but given the way radiotap 
is designed (ATM) I don't see a better solution.

    Sam

Re: use of radiotap bit 14?

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1908 bytes --]

On Thu, 2008-06-19 at 14:39 -0500, David Young wrote:
> On Thu, Jun 19, 2008 at 09:10:33PM +0200, Johannes Berg wrote:
> > On Thu, 2008-06-19 at 13:56 -0500, David Young wrote:
> > 
> > > I favor these assignments:
> > > 
> > > 14 /RX flags
> > > 15 /TX flags
> > > 16 /RTS retries
> > > 17 /data retries
> > > 18 /XChannel 
> > 
> > That'd be great to me. Want me to declare them that on the wiki, move
> > them over to the defined-fields and add the other ones as "suggested
> > fields" without assigned numbers?
> 
> Well, you and I agree, but let us give everyone else a chance. :-)

Heh yeah, sure.

> Let's do this:
> 
> 1 define the alignment, width, interpretation of the fields (I believe
>   this is done, somewhere.)

http://www.radiotap.org/suggested-fields
click any of the links, there you have it.

I haven't decided yet whether radiotap.org should maintain a header
file. It would be nice for source-code interoperability, but in practice
the different OSes use different header files already...

> 2 produce implementations wireshark and/or tcpdump and for one or
>   more driver

Linux is actually shipping code now that will produce RX_FLAGS (in bit
14) with the IEEE80211_RADIOTAP_F_RX_BADFCS flag, but there's
IEEE80211_RADIOTAP_F_BADFCS in wireshark so maybe we should use that
instead.

> 3 post fields and code here, and on the wiki

I can do that.

> 4 leave 7 days for discussion (ordinarily, this should be longer)
> 5 if the proposal withstands discussion, re-post proposal in final form
> 6 adopt the proposal in 7 days if there are no further objections
>   (discussion is over, however, 7 days lets everyone verify that the
>   proposal and amendments, as discussed, have become the standard)

Sounds good. Should we document that process with a normal period of a
month before adoption after discussion/reposts?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: use of radiotap bit 14?

[Scott Raynel]

Hi all,
On 20/06/2008, at 6:56 AM, David Young wrote:

> On Thu, Jun 19, 2008 at 07:44:59PM +0200, Johannes Berg wrote:
>> Ok, I tried collecting all the information, and here's what I found  
>> so
>> far:
>>
>> http://www.radiotap.org/suggested-fields
>>
>> Bits 14, 15 and 16 are defined twice.
>>
>> Can we declare bits 14 through 18 reserved (i.e. may not be used) and
>> continue life at 19? Current parsers would have to be changed (the  
>> only
>> one I found is wireshark and internal ones like hostapd, Linux  
>> kernel)
>> to treat the presence of those bits to be an abort-condition and as  
>> an
>> error if there are any above them, current generators would have to  
>> stop
>> generating those fields over time and instead generate standardised
>> versions.
>>
>> For me, of course, it would be more comfortable to stick with RX/TX
>> flags in 14/15 and RTS retries in 16, but since it's not widely  
>> used in
>> Linux yet I can easily discard them.
>
> Johannes,
>
> I favor these assignments:
>
> 14 /RX flags
> 15 /TX flags
> 16 /RTS retries
> 17 /data retries
> 18 /XChannel

For what it's worth, back in March 2007 I spent some time making sure  
that madwifi used these bit assignments (minus XChannel, which I  
hadn't heard of back then). I took the NetBSD headers to be the  
"official" source of the Radiotap standard.

Previously we had been using the in-header FCS flag, which I got rid of.

So, madwifi has been using these bit assignments for some time now -  
hopefully one more reason to not change them :)

Cheers,

--
Scott Raynel
WAND Network Research Group
Department of Computer Science
University of Waikato
New Zealand