[RFC] Penumbra - Enabling unencrypted broadcasts alongside normal traffic

[RFC] Penumbra - Enabling unencrypted broadcasts alongside normal traffic

[Andy Green]

Hi folks -

Introduction
------------

I have been working on a GPL project called Penumbra

http://penumbra.warmcat.com

The idea is simple, to allow unencrypted wireless broadcast packets to 
be generated and received in userspace, while the normal traffic remains 
privately encrypted as before.  This allows direct intercommunication 
between boxes on differently protected private networks (on the same 
channel) while not disturbing or reducing security of a normal encrypted 
association.

Due to the broadcast nature of the protocol, there are no station 
identities at all.  All broadcast packets use the same MAC and there is 
no concept of addressability, at least not at the protocol level.  There 
are no retries at the transport protocol level either since there may be 
many listeners --  situations where there are multiple listeners in 
range of the broadcasts are very well served.  Higher levels on protocol 
can implement tunneling other protocols and retransmission requests, but 
that is out of scope of what is needed in the 80211 stack.

We have a userspace daemon that allows anonymous filesharing using the 
protocol, but there are many other applications possible for machines to 
be able to remain secure and private for normal traffic, yet be able to 
also communicate locally unencrypted when they choose without going 
through the Internet, an ISP or other paid -- and metered -- gateway.

There is a start on a formal RFC for the broadcast transport protocol here:

http://penumbra.warmcat.com/penumbra-rfc-0.0.3.txt


Status
------

Currently I have this working for several wireless devices using the old 
stack (IPW2200, IPW3945, and ZD1211RW -- BCM43XX is also done but does 
not always receive for reasons I didn't discover), and I have it working 
on the new stack for iwlwifi and RT73USB (RT2571 chipset) too now.  The 
old stack drivers are patched without any changes to the old stack, but 
for the new 80211 stack, the additional code is mostly tucked away in 
the stack.  You can see examples of the patching required in both the 
stack and the driver for the rt73usb case here:

http://penumbra.warmcat.com/rt2x00-cvs-2007021403-penumbra-0.0.3.patch

There is a related userspace daemon also available which communicates 
via an embedded https server with the user and allows arbitrary files to 
be shared around using the broadcast protocol.  The daemon bridges 
traffic between multiple wireless devices on different channels seamlessly.

All patches and the penumbrad daemon are available here:

http://penumbra.warmcat.com/_twk/tiki-index.php?page=Downloads

Performance at the moment for a iwlwifi <-> IPW2200 testcase is between 
100KBytes/sec and 200KBytes/sec transfer, tested while downloading 
simultaneously at 1MByte/sec using a WPA encrypted association to a 
Belkin AP.  So any questions about whether it is possible, or can work 
on a range of devices, or can operate simultaneously with WPA are 
answered: it works great.


Implementation
--------------

The unencrypted broadcast packets are indicated by having a "Magic MAC" 
address in their IEEE80211 Header Addr<n> fields.  The Magic MAC for 
Penumbra is 13:22:33:44:55:66 (the IEEE had something to say about our 
original choice of 11:22.. :-O ).

Here is the path for a packet using the new stack patched to support the 
protocol.

  - Userspace transmits by creating a PF_PACKET / SOCK_RAW socket and 
prepending an Ethernet header with the Magic MAC in it and send()ing it. 
  Reed-Solomon coding is optionally used in userspace in case the 
receiver allows to see damaged packets.

  - The stack detects the magic MAC, and prepends a fixed IEEE80211 
header to the skb, basically addr1-> FF:FF:FF:FF:FF:FF, addr2 and addr3 
-> 13:22:33:44:55:66 and encryption is turned off.

  - The wireless driver gets the packet for transmission, recognizes the 
Magic MAC, disables retries and sets the transmission rate (currently 
fixed 54Mbps, but this will change) and transmits the packet as a broadcast

For receive:

  - The wireless device is set to hardware promiscuous

  - When an incoming packet is seen with the magic MAC it has a fake 
fixed Ethernet, IP and UDP header prepended to it.  IP and UDP checksums 
are inserted so the packet is clean.  The packet always looks like it is 
coming from UDP 0.0.0.0:61441 (port 0xF001) and is directed to 
255.255.255.255:61442 (port 0xF002).  The packet is subject to iptables 
rules as usual.

  - Userspace listens on the right interface at UDP 61442 with a normal 
socket and receives the packets as usual.

There is some more detail and information here (it is about the old 
stack implementation currently though)

http://penumbra.warmcat.com/_twk/tiki-index.php?page=Implementation


Issues
------

I hope there is some interest in this new capability for wireless 
devices in Linux.  To get any kind of widespread use, the capability 
needs to be already available in stock kernels and drivers so that the 
user only needs to open iptables and run a userspace daemon rather than 
patch his wireless drivers and stack.  So this is my reason for 
presenting this Request For Comments, to see if it can be considered.

Some of the issues that need working on/consensus if there is interest:

  - Hardware promisc is needed to catch the packets because they are not 
tagged as coming from the local AP.  All the devices I looked at can do 
hardware promisc separately from the logical promisc for Monitor mode. 
But perhaps this needs to be enabled and disabled in some modal way.

  - The way the packets are captured does not integrate at the moment 
with the rx handler list concept, because it needs access to args that 
do not come to the "pre" rx handlers where it has to capture.  No doubt 
there are multiple other implementation issues that can be hammered out 
easily enough if there is interest.

  - Right this second there are two patches, one for the iwlwifi case 
and one for rt2x00, since they are using different versions of the new 
80211 stack.

  - The first byte of the payload is reserved for holding the desired TX 
rate sent up from userspace, as well as indications about any ECC coding 
that is present.  But today only a fixed 54Mbps packet rate is 
implemented.  Despite all the stations are anonymous, the upper levels 
of the protocol can guess the best rate to send a particular packet 
depending on what has been going on, and so allow the system to work 
over larger distances than 54Mbps only will allow.

...  no doubt there will be more assuming there is any interest ;-)

Thanks for reading this far and I hope it will be possible by adding 
these relatively small changes to the stack to enable some cool 
additional uses for existing wireless devices.

-Andy

Re: [RFC] Penumbra - Enabling unencrypted broadcasts alongside normal traffic

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 2930 bytes --]

Hi,

Just a few comments. I'll leave aside the issues with 802.11 here hoping
you've thought about that. There are issues with admission control and
similar things, for example.

First, allow me to comment on some things from other sources:

> The payload is protected using a 255:223 Reed-Solomon? Forward Error
> Correction coding. This is capable to correct any 16 symbol errors
> over each 223 bytes of payload. To maximize the benefit of this
> protection, bulk data packets are limited to 213 bytes of actual
> payload (and a 10-byte header), giving a 255 byte encoded packet
> payload so each fragment fits inside a single error correction coding
> cycle. The Reed-Solomon? coding also means there is no need for a
> payload CRC.

802.11 frames are always protected by a 32-bit CRC and will be discarded
by hardware (in most cases) if that doesn't match. This is unnecessary
overhead.

> The unencrypted broadcast packets are indicated by having a "Magic MAC" 
> address in their IEEE80211 Header Addr<n> fields.  The Magic MAC for 
> Penumbra is 13:22:33:44:55:66 (the IEEE had something to say about our 
> original choice of 11:22.. :-O ).

How about registering a OUI or getting someone to donate a MAC address
instead of using a locally administered one?

>   - Userspace transmits by creating a PF_PACKET / SOCK_RAW socket and 
> prepending an Ethernet header with the Magic MAC in it and send()ing it. 

I don't see why you couldn't use the packet injection stuff we'll be
needing anyway for userspace MLME.

>   - The wireless driver gets the packet for transmission, recognizes the 
> Magic MAC, disables retries and sets the transmission rate (currently 
> fixed 54Mbps, but this will change) and transmits the packet as a broadcast

You'd be able to control these parameters then.

>   - When an incoming packet is seen with the magic MAC it has a fake 
> fixed Ethernet, IP and UDP header prepended to it.  IP and UDP checksums 
> are inserted so the packet is clean.  The packet always looks like it is 
> coming from UDP 0.0.0.0:61441 (port 0xF001) and is directed to 
> 255.255.255.255:61442 (port 0xF002).  The packet is subject to iptables 
> rules as usual.

Similarly, why not have userspace use a monitor interface directly?

> To get any kind of widespread use, the capability 
> needs to be already available in stock kernels and drivers so that the 
> user only needs to open iptables and run a userspace daemon rather than 
> patch his wireless drivers and stack.

I don't think that once we have packet injection in place for userspace
MLME (well, we even have it now) you'll need to do any modifications at
all. You'll just need to do more stuff in userspace. I also don't see
why iptables should see these packets that are explicitly not IP. In
fact, I think such packets should not be seen by the networking stack at
all.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [RFC] Penumbra - Enabling unencrypted broadcasts alongside normal traffic

[Andy Green]

Johannes Berg wrote:

Hi -

Thanks for your comments.

> Just a few comments. I'll leave aside the issues with 802.11 here hoping
> you've thought about that. There are issues with admission control and
> similar things, for example.

No I didn't consider these elements, because in effect the broadcasts 
are no different from uncontrolled traffic from another network on the 
same channel.

> First, allow me to comment on some things from other sources:
> 
>> The payload is protected using a 255:223 Reed-Solomon? Forward Error
>> Correction coding. This is capable to correct any 16 symbol errors
>> over each 223 bytes of payload. To maximize the benefit of this
>> protection, bulk data packets are limited to 213 bytes of actual
>> payload (and a 10-byte header), giving a 255 byte encoded packet
>> payload so each fragment fits inside a single error correction coding
>> cycle. The Reed-Solomon? coding also means there is no need for a
>> payload CRC.
> 
> 802.11 frames are always protected by a 32-bit CRC and will be discarded
> by hardware (in most cases) if that doesn't match. This is unnecessary
> overhead.

That's right that some devices are hiding frames with failed CRCs, but 
it isn't true of all of them.  In the case where they aren't hiding the 
broken frames, the Reed-Solomon coding allows the broadcast to be 
recovered in many cases.  One of the bits on the first byte of the 
packet is reserved to show if the sending station has ever seen a broken 
packet being received, so a responding station can decide whether to use 
the ECC or not in reply.

In the case where a broadcasting station has decided to "use the 
airtime", it makes sense to try to recover the packet if possible since 
the airtime was already blocked.  I found that typical errors were runs 
of bits getting trashed, the R-S coding can recover up to 16 bytes of 
damage in such runs.  The benefit of it is greater as the number of 
listeners in range increases, or the bitrate used for the transmission 
decreases (and the potentially lost airtime therefore goes up).

>> The unencrypted broadcast packets are indicated by having a "Magic MAC" 
>> address in their IEEE80211 Header Addr<n> fields.  The Magic MAC for 
>> Penumbra is 13:22:33:44:55:66 (the IEEE had something to say about our 
>> original choice of 11:22.. :-O ).
> 
> How about registering a OUI or getting someone to donate a MAC address
> instead of using a locally administered one?

I contacted the IEEE about it some weeks ago, their response was 
basically to tell me not to use the 11: space but this locally 
administered one.  They did not respond to my requests for a donation of 
one measly MAC address, although I did not exhaust all the contacts 
there yet.  I can't afford to register a OUI myself, but I have a 
customer with an allocation that can probably spare one.  An advantage 
of using locally administered though is that the LSByte of the MAC could 
be overloaded to indicate other broadcast protocols using the same 
technique, if only the first 5 bytes of the MAC are matched.

>>   - Userspace transmits by creating a PF_PACKET / SOCK_RAW socket and 
>> prepending an Ethernet header with the Magic MAC in it and send()ing it. 
> 
> I don't see why you couldn't use the packet injection stuff we'll be
> needing anyway for userspace MLME.

Well that would be fine :-)  The current implementation is just what I 
could make work in a reasonable way with the old 80211 stack and then I 
ported it to the new one.

The old 80211 stack method uses a kernel module that hooks the 
hard_start_xmit() for network interfaces that have wireless extensions 
on them, using the network interface notification API in Linux.  I chose 
to come at it like that because this is similar to the method that I 
hope will work on Windows.

>>   - The wireless driver gets the packet for transmission, recognizes the 
>> Magic MAC, disables retries and sets the transmission rate (currently 
>> fixed 54Mbps, but this will change) and transmits the packet as a broadcast
> 
> You'd be able to control these parameters then.

Sounds good!  Does this exist yet?  How does one use it?

>>   - When an incoming packet is seen with the magic MAC it has a fake 
>> fixed Ethernet, IP and UDP header prepended to it.  IP and UDP checksums 
>> are inserted so the packet is clean.  The packet always looks like it is 
>> coming from UDP 0.0.0.0:61441 (port 0xF001) and is directed to 
>> 255.255.255.255:61442 (port 0xF002).  The packet is subject to iptables 
>> rules as usual.
> 
> Similarly, why not have userspace use a monitor interface directly?

The answer is again because the old stack did not offer the concept of a 
logical Monitor action on the same device that is associated as INFRA. 
But allowing "out of the box" operation is critical so I will try this 
method also.  I saw it mentioned that you can create these by writing 
down /sys.

>> To get any kind of widespread use, the capability 
>> needs to be already available in stock kernels and drivers so that the 
>> user only needs to open iptables and run a userspace daemon rather than 
>> patch his wireless drivers and stack.
> 
> I don't think that once we have packet injection in place for userspace
> MLME (well, we even have it now) you'll need to do any modifications at
> all. You'll just need to do more stuff in userspace. I also don't see
> why iptables should see these packets that are explicitly not IP. In
> fact, I think such packets should not be seen by the networking stack at
> all.

Well this is a perfectly fine solution for me if the need for all my 
patches should evaporate and things "just work" for endusers, so I am 
interested about the state of the injection stuff.

-Andy

Re: [RFC] Penumbra - Enabling unencrypted broadcasts alongside normal traffic

[Jouni Malinen]

On Tue, Feb 27, 2007 at 11:48:46AM +0000, Andy Green wrote:

> There is a start on a formal RFC for the broadcast transport protocol here:
> 
> http://penumbra.warmcat.com/penumbra-rfc-0.0.3.txt

Could you please extend the description of "Wireless Packet Format" to
define all fields in the 802.11 header. I would especially be interested
in seeing what is in the FromDS and ToDS fields of the frame control.
I'm assuming this is using normal data frames, but that (type and
subtype fields) should also be described clearly.

>  - The wireless driver gets the packet for transmission, recognizes the 
> Magic MAC, disables retries and sets the transmission rate (currently 
> fixed 54Mbps, but this will change) and transmits the packet as a broadcast

So this frame is to be sent by the non-AP STAs directly as a broadcast
frame, not through the AP which is the normal way of transmitting
frames(?)

> I hope there is some interest in this new capability for wireless 
> devices in Linux.  To get any kind of widespread use, the capability 
> needs to be already available in stock kernels and drivers so that the 
> user only needs to open iptables and run a userspace daemon rather than 
> patch his wireless drivers and stack.  So this is my reason for 
> presenting this Request For Comments, to see if it can be considered.

You would need to add "and configure the wlan driver/802.11 stack to
enable this". I don't think this has to be disabled by default.

Is this only for Linux at the moment or is someone looking at supporting
this with other OSes?

>  - Hardware promisc is needed to catch the packets because they are not 
> tagged as coming from the local AP.  All the devices I looked at can do 
> hardware promisc separately from the logical promisc for Monitor mode. 
> But perhaps this needs to be enabled and disabled in some modal way.

This has to be disabled by default. There is a reason for such a
filtering in wlan designs and just enabling promiscuous mode at 802.11
level is going to considerable increase power needs for the device which
is not acceptable for many low-power devices.

-- 
Jouni Malinen                                            PGP id EFC895FA

Re: [RFC] Penumbra - Enabling unencrypted broadcasts alongside normal traffic

[Andy Green]

Jouni Malinen wrote:

Hi Jouni -

>> There is a start on a formal RFC for the broadcast transport protocol here:
>>
>> http://penumbra.warmcat.com/penumbra-rfc-0.0.3.txt
> 
> Could you please extend the description of "Wireless Packet Format" to
> define all fields in the 802.11 header. I would especially be interested
> in seeing what is in the FromDS and ToDS fields of the frame control.
> I'm assuming this is using normal data frames, but that (type and
> subtype fields) should also be described clearly.

I extended it to this:

  frame_control: 08 01               (FCTL_DATA/STYPE_DATA/TODS)
    duration_id: 00 00
          addr1: FF:FF:FF:FF:FF:FF
          addr2: 13:22:33:44:55:66
          addr3: 13:22:33:44:55:66
       seq_ctrl: 00:00

The TODS will always be accepted for transmission by even a pedantic 
device in INFRA mode, and any combination will be accepted in MONITOR 
mode.  Whereas one of the devices (IPW3945?  I forget) would not issue 
packets with FROMDS in INFRA mode IIRC.

>>  - The wireless driver gets the packet for transmission, recognizes the 
>> Magic MAC, disables retries and sets the transmission rate (currently 
>> fixed 54Mbps, but this will change) and transmits the packet as a broadcast
> 
> So this frame is to be sent by the non-AP STAs directly as a broadcast
> frame, not through the AP which is the normal way of transmitting
> frames(?)

Yes -- even further there is nothing in the broadcast frame to identify 
the sender or recipient.  The broadcast frame is not directed at 
anything either: any number of stations are either in range to accept it 
or not.  If you're not in range, but through one or more hops you are in 
range of someone that is, a store-and-forward system that rides of top 
of this enables intercommunication anyway.  So your street or building 
or carpark or train or airplane can be a more interesting place, 
hopefully even if you are not associated or have nothing to associate to.

>> I hope there is some interest in this new capability for wireless 
>> devices in Linux.  To get any kind of widespread use, the capability 
>> needs to be already available in stock kernels and drivers so that the 
>> user only needs to open iptables and run a userspace daemon rather than 
>> patch his wireless drivers and stack.  So this is my reason for 
>> presenting this Request For Comments, to see if it can be considered.
> 
> You would need to add "and configure the wlan driver/802.11 stack to
> enable this". I don't think this has to be disabled by default.

Yes if I understood it I can bring up the monitor interface when the 
daemon is started and take it down when it is stopped.

> Is this only for Linux at the moment or is someone looking at supporting
> this with other OSes?

It will really become more useful with Windows support.  I looked a 
little way into it and found they seem to have a hookable minidriver 
idea for XP SP2 and Vista.  I am hoping somebody with experience on that 
side will write it once the system is easily available on Linux.

>>  - Hardware promisc is needed to catch the packets because they are not 
>> tagged as coming from the local AP.  All the devices I looked at can do 
>> hardware promisc separately from the logical promisc for Monitor mode. 
>> But perhaps this needs to be enabled and disabled in some modal way.
> 
> This has to be disabled by default. There is a reason for such a
> filtering in wlan designs and just enabling promiscuous mode at 802.11
> level is going to considerable increase power needs for the device which
> is not acceptable for many low-power devices.

Fair enough: it seems that creating the logical Monitor interface for rx 
will and using that instead of the current system of private monitoring 
with hardware promisc will take care of that as well, it will only be 
doing promisc while the daemon runs and the interface is up.

-Andy

Re: Question about packet injection

[Andy Green]

Hi folks -

As suggested here I migrated my daemon to using a per-physical device 
monitor mode interface it creates and destroys from usermode, together 
with libpcap for filtering and receiving broadcasts and that seems to be 
working okay.  So that is half the battle to get the broadcast system 
working with mac80211 out of the box without patches.

However, if I inject a captured packet - with a PRISM header prepended - 
down the Monitor mode interface using pcap_inject(), I see it appear on 
the libpcap captured output okay, but it does not go out on the air AFAICT.

Is this in fact the correct way to try to inject a packet with fine 
control over its encryption and rate?  If so is it expected to currently 
work on iwlwifi 0.0.8 or rt2x00 (rt73usb) from CVS 2008-02-14?

-Andy

Re: Question about packet injection

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 315 bytes --]

On Thu, 2007-03-01 at 15:11 +0000, Andy Green wrote:

> Is this in fact the correct way to try to inject a packet with fine 
> control over its encryption and rate?

Right you need to create a management interface and dump it into there I
think. Subject to change, still under discussion though.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: Question about packet injection

[Ivo van Doorn]

On Thursday 01 March 2007 16:11, Andy Green wrote:
> Hi folks -
> 
> As suggested here I migrated my daemon to using a per-physical device 
> monitor mode interface it creates and destroys from usermode, together 
> with libpcap for filtering and receiving broadcasts and that seems to be 
> working okay.  So that is half the battle to get the broadcast system 
> working with mac80211 out of the box without patches.
> 
> However, if I inject a captured packet - with a PRISM header prepended - 
> down the Monitor mode interface using pcap_inject(), I see it appear on 
> the libpcap captured output okay, but it does not go out on the air AFAICT.
> 
> Is this in fact the correct way to try to inject a packet with fine 
> control over its encryption and rate?  If so is it expected to currently 
> work on iwlwifi 0.0.8 or rt2x00 (rt73usb) from CVS 2008-02-14?

Note that rt73usb is currently broken, I know that RX isn't working
because scanning fails. Chances are quite high that TX is failing too.
I am still looking into that issue, but it might be the reason why
packet injection is failing for you.

Ivo

Re: Question about packet injection

[Andy Green]

Ivo van Doorn wrote:
> On Thursday 01 March 2007 16:11, Andy Green wrote:
>> Hi folks -
>>
>> As suggested here I migrated my daemon to using a per-physical device 
>> monitor mode interface it creates and destroys from usermode, together 
>> with libpcap for filtering and receiving broadcasts and that seems to be 
>> working okay.  So that is half the battle to get the broadcast system 
>> working with mac80211 out of the box without patches.
>>
>> However, if I inject a captured packet - with a PRISM header prepended - 
>> down the Monitor mode interface using pcap_inject(), I see it appear on 
>> the libpcap captured output okay, but it does not go out on the air AFAICT.
>>
>> Is this in fact the correct way to try to inject a packet with fine 
>> control over its encryption and rate?  If so is it expected to currently 
>> work on iwlwifi 0.0.8 or rt2x00 (rt73usb) from CVS 2008-02-14?
> 
> Note that rt73usb is currently broken, I know that RX isn't working
> because scanning fails. Chances are quite high that TX is failing too.
> I am still looking into that issue, but it might be the reason why
> packet injection is failing for you.

The 2007-02-14 rt73usb CVS works okay for RX, it does iwlist wlan0 scans 
alright here anyway with the AP shown.  I have had that version issuing 
unencrypted broadcasts too by hacking the stack and the driver slightly. 
  Maybe it was some Valentines Day magic :-)  But it doesn't associate 
in a stable way with the AP using WPA on wpa_supplicant with the dscape 
patch.

Anyway it seems the problem is one does not inject for TX on the Monitor 
mode interface (/me goes away to grep and google what a Management mode 
interface does, thanks Johannes)

-Andy