Hidden SSID's

Hidden SSID's

[Larry Finger]

Is there a convention regarding the information that a wireless MAC layer should provide when
reporting scan data from an AP with a hidden SSID?

In ieee80211, the software inserts the string "<hidden>" for such an AP, which seems to give
wpa_supplicant fits because it rejects the SSID before even looking at the encryption data. Is this
the normal convention?

Thanks,

Larry

Re: Hidden SSID's

[Dan Williams]

On Fri, 2007-01-26 at 21:48 -0600, Larry Finger wrote:
> Is there a convention regarding the information that a wireless MAC layer should provide when
> reporting scan data from an AP with a hidden SSID?
> 
> In ieee80211, the software inserts the string "<hidden>" for such an AP, which seems to give
> wpa_supplicant fits because it rejects the SSID before even looking at the encryption data. Is this
> the normal convention?

No.  It's wrong.  It is not normal convention.  The one and only
_correct_ way of reporting a hidden SSID is to not report the SSID at
all.  The only item that needs to be reported with WEXT is the BSSID,
and the client app assumes that if the SSID is not received for a given
scan result, that the driver doesn't have an SSID for that BSSID.

I really, really don't know why ieee80211 uses <hidden>, but it's a pain
in the ass and should NOT be done for d80211.  I don't know if we can
ever remove it from ieee80211 though for backwards compat reasons.

Dan

> Thanks,
> 
> Larry
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Hidden SSID's

[Larry Finger]

Dan Williams wrote:
> On Fri, 2007-01-26 at 21:48 -0600, Larry Finger wrote:
>> Is there a convention regarding the information that a wireless MAC layer should provide when
>> reporting scan data from an AP with a hidden SSID?
>>
>> In ieee80211, the software inserts the string "<hidden>" for such an AP, which seems to give
>> wpa_supplicant fits because it rejects the SSID before even looking at the encryption data. Is this
>> the normal convention?
> 
> No.  It's wrong.  It is not normal convention.  The one and only
> _correct_ way of reporting a hidden SSID is to not report the SSID at
> all.  The only item that needs to be reported with WEXT is the BSSID,
> and the client app assumes that if the SSID is not received for a given
> scan result, that the driver doesn't have an SSID for that BSSID.
> 
> I really, really don't know why ieee80211 uses <hidden>, but it's a pain
> in the ass and should NOT be done for d80211.  I don't know if we can
> ever remove it from ieee80211 though for backwards compat reasons.

The situation might not be too bad. The routine in ieee80211 has two direct callers, ipw2100 and
ipw2200, and is called through ieee80211softmac by bcm43xx and zd1211rw. I'll contact the
maintainers and see if we can reach an agreement regarding removal of the bad info.

Thanks for your response.

Larry

Re: Hidden SSID's

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 367 bytes --]

On Sat, 2007-01-27 at 07:08 -0500, Dan Williams wrote:

> I really, really don't know why ieee80211 uses <hidden>, but it's a pain
> in the ass and should NOT be done for d80211.  I don't know if we can
> ever remove it from ieee80211 though for backwards compat reasons.

Ugh. /me makes a note for the cfg80211/we compat layer. This is a
mess :(

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: Hidden SSID's

[Larry Finger]

Dan Williams wrote:
> On Fri, 2007-01-26 at 21:48 -0600, Larry Finger wrote:
>> Is there a convention regarding the information that a wireless MAC layer should provide when
>> reporting scan data from an AP with a hidden SSID?
>>
>> In ieee80211, the software inserts the string "<hidden>" for such an AP, which seems to give
>> wpa_supplicant fits because it rejects the SSID before even looking at the encryption data. Is this
>> the normal convention?
> 
> No.  It's wrong.  It is not normal convention.  The one and only
> _correct_ way of reporting a hidden SSID is to not report the SSID at
> all.  The only item that needs to be reported with WEXT is the BSSID,
> and the client app assumes that if the SSID is not received for a given
> scan result, that the driver doesn't have an SSID for that BSSID.
> 
> I really, really don't know why ieee80211 uses <hidden>, but it's a pain
> in the ass and should NOT be done for d80211.  I don't know if we can
> ever remove it from ieee80211 though for backwards compat reasons.

Is there something funny about Cisco APs with hidden SSID? I made a patch for ieee80211 and softmac
that works on my AP when its SSID is hidden. It also works for Paul Marks on his personal AP, but
fails for the Cisco units at Purdue. His interface is a 4311 and there may be a signal strength
issue, but we are not sure.

My patch fixes ieee80211 without breaking the previous behavior.

Larry

Re: Hidden SSID's

[Dan Williams]

On Sun, 2007-01-28 at 22:28 +0100, Johannes Berg wrote:
> On Sat, 2007-01-27 at 07:08 -0500, Dan Williams wrote:
> 
> > I really, really don't know why ieee80211 uses <hidden>, but it's a pain
> > in the ass and should NOT be done for d80211.  I don't know if we can
> > ever remove it from ieee80211 though for backwards compat reasons.
> 
> Ugh. /me makes a note for the cfg80211/we compat layer. This is a
> mess :(

Well, there's no way a userspace program could depend on all hidden SSID
APs having the <hidden> tag, since if you stick in another,
non-ieee80211-stack card it won't be like that.  So I don't think we
should care about <hidden> in d80211, but I don't think we can remove it
from ieee80211 either.  The only case where we'll care about it is if we
move to common scan-result processing code, and there we may have to put
a compat flag in that the driver can set or something.  But we should
definitely _not_ use <hidden> in d80211 or cfg80211, because any program
depending on <hidden> just won't work with anything other than an ipw
card.

Dan

Re: Hidden SSID's

[Jouni Malinen]

On Mon, Jan 29, 2007 at 08:00:11AM -0500, Dan Williams wrote:

> Well, there's no way a userspace program could depend on all hidden SSID
> APs having the <hidden> tag, since if you stick in another,
> non-ieee80211-stack card it won't be like that.  So I don't think we
> should care about <hidden> in d80211, but I don't think we can remove it
> from ieee80211 either.

Use of '<hidden>' is just not acceptable. IMHO, it should be removed
from everywhere, including net/ieee80211. The sooner this is done, the
better.

-- 
Jouni Malinen                                            PGP id EFC895FA

Re: Hidden SSID's

[Dan Williams]

On Mon, 2007-01-29 at 19:09 -0800, Jouni Malinen wrote:
> On Mon, Jan 29, 2007 at 08:00:11AM -0500, Dan Williams wrote:
> 
> > Well, there's no way a userspace program could depend on all hidden SSID
> > APs having the <hidden> tag, since if you stick in another,
> > non-ieee80211-stack card it won't be like that.  So I don't think we
> > should care about <hidden> in d80211, but I don't think we can remove it
> > from ieee80211 either.
> 
> Use of '<hidden>' is just not acceptable. IMHO, it should be removed
> from everywhere, including net/ieee80211. The sooner this is done, the
> better.

You're probably right.  Lets just pull it out of ieee80211 and be done.

dan

Re: Hidden SSID's

[Larry Finger]

Dan Williams wrote:
> On Mon, 2007-01-29 at 19:09 -0800, Jouni Malinen wrote:
>> On Mon, Jan 29, 2007 at 08:00:11AM -0500, Dan Williams wrote:
>>
>>> Well, there's no way a userspace program could depend on all hidden SSID
>>> APs having the <hidden> tag, since if you stick in another,
>>> non-ieee80211-stack card it won't be like that.  So I don't think we
>>> should care about <hidden> in d80211, but I don't think we can remove it
>>> from ieee80211 either.
>> Use of '<hidden>' is just not acceptable. IMHO, it should be removed
>> from everywhere, including net/ieee80211. The sooner this is done, the
>> better.
> 
> You're probably right.  Lets just pull it out of ieee80211 and be done.
Before it gets pulled, please look at this patch.

Larry

=======================================



When an AP has a hidden SSID, ieee80211 fails, at least with wpa_supplicant,
which searches through the scan data looking for a particular ssid. Because
ieee80211 has substituted a false ssid, namely "<hidden>", wpa_supplicant
cannot authenticate. This behavior is fixed by adding a new argument to
ieee80211_translate_scan that contains the expected ssid. A new routine,
ieee80211_wx_get_scan_essid, has an additional argument that contains the essid
of the AP that wpa_supplicant is trying to find. The existing routine,
ieee80211_wx_get_scan, calls the new one with the false ssid. The code in
ieee80211softmac is also modified to use the new routine and has been tested
with bcm43xx.

Signed-off-by: Larry Finger <Larry.Finger@lwfinger.net>
---

Index: linux-2.6/include/net/ieee80211.h
===================================================================
--- linux-2.6.orig/include/net/ieee80211.h
+++ linux-2.6/include/net/ieee80211.h
@@ -946,6 +946,11 @@ struct ieee80211_network {
 	struct list_head list;
 };

+struct ieee80211_essid {
+	u8 len;
+	char data[IW_ESSID_MAX_SIZE];
+};
+
 enum ieee80211_state {
 	IEEE80211_UNINITIALIZED = 0,
 	IEEE80211_INITIALIZED,
@@ -1296,6 +1301,10 @@ extern const struct ieee80211_channel *i
 extern int ieee80211_wx_get_scan(struct ieee80211_device *ieee,
 				 struct iw_request_info *info,
 				 union iwreq_data *wrqu, char *key);
+extern int ieee80211_wx_get_scan_essid(struct ieee80211_device *ieee,
+				 struct iw_request_info *info,
+				 union iwreq_data *wrqu, char *key,
+				 struct ieee80211_essid *essid);
 extern int ieee80211_wx_set_encode(struct ieee80211_device *ieee,
 				   struct iw_request_info *info,
 				   union iwreq_data *wrqu, char *key);
Index: linux-2.6/net/ieee80211/ieee80211_wx.c
===================================================================
--- linux-2.6.orig/net/ieee80211/ieee80211_wx.c
+++ linux-2.6/net/ieee80211/ieee80211_wx.c
@@ -44,7 +44,8 @@ static const char *ieee80211_modes[] = {
 #define MAX_CUSTOM_LEN 64
 static char *ieee80211_translate_scan(struct ieee80211_device *ieee,
 					   char *start, char *stop,
-					   struct ieee80211_network *network)
+					   struct ieee80211_network *network,
+					   struct ieee80211_essid *essid)
 {
 	char custom[MAX_CUSTOM_LEN];
 	char *p;
@@ -65,10 +66,10 @@ static char *ieee80211_translate_scan(st
 	iwe.cmd = SIOCGIWESSID;
 	iwe.u.data.flags = 1;
 	if (network->flags & NETWORK_EMPTY_ESSID) {
-		iwe.u.data.length = sizeof("<hidden>");
-		start = iwe_stream_add_point(start, stop, &iwe, "<hidden>");
+		iwe.u.data.length = min(essid->len, (u8) IW_ESSID_MAX_SIZE);
+		start = iwe_stream_add_point(start, stop, &iwe, essid->data);
 	} else {
-		iwe.u.data.length = min(network->ssid_len, (u8) 32);
+		iwe.u.data.length = min(network->ssid_len, (u8) IW_ESSID_MAX_SIZE);
 		start = iwe_stream_add_point(start, stop, &iwe, network->ssid);
 	}

@@ -247,9 +248,15 @@ static char *ieee80211_translate_scan(st

 #define SCAN_ITEM_SIZE 128

-int ieee80211_wx_get_scan(struct ieee80211_device *ieee,
+static struct ieee80211_essid hidden_essid = {
+	.len  = sizeof ("<hidden>"),
+	.data = {"<hidden>"},
+};
+
+int ieee80211_wx_get_scan_essid(struct ieee80211_device *ieee,
 			  struct iw_request_info *info,
-			  union iwreq_data *wrqu, char *extra)
+			  union iwreq_data *wrqu, char *extra,
+			  struct ieee80211_essid *essid)
 {
 	struct ieee80211_network *network;
 	unsigned long flags;
@@ -272,7 +279,7 @@ int ieee80211_wx_get_scan(struct ieee802

 		if (ieee->scan_age == 0 ||
 		    time_after(network->last_scanned + ieee->scan_age, jiffies))
-			ev = ieee80211_translate_scan(ieee, ev, stop, network);
+			ev = ieee80211_translate_scan(ieee, ev, stop, network, essid);
 		else
 			IEEE80211_DEBUG_SCAN("Not showing network '%s ("
 					     MAC_FMT ")' due to age (%dms).\n",
@@ -294,6 +301,13 @@ int ieee80211_wx_get_scan(struct ieee802
 	return err;
 }

+int ieee80211_wx_get_scan(struct ieee80211_device *ieee,
+			  struct iw_request_info *info,
+			  union iwreq_data *wrqu, char *extra)
+{
+	return ieee80211_wx_get_scan_essid(ieee, info, wrqu, extra,  &hidden_essid);
+}
+
 int ieee80211_wx_set_encode(struct ieee80211_device *ieee,
 			    struct iw_request_info *info,
 			    union iwreq_data *wrqu, char *keybuf)
@@ -834,6 +848,7 @@ EXPORT_SYMBOL(ieee80211_wx_set_encodeext
 EXPORT_SYMBOL(ieee80211_wx_get_encodeext);

 EXPORT_SYMBOL(ieee80211_wx_get_scan);
+EXPORT_SYMBOL(ieee80211_wx_get_scan_essid);
 EXPORT_SYMBOL(ieee80211_wx_set_encode);
 EXPORT_SYMBOL(ieee80211_wx_get_encode);

Index: linux-2.6/net/ieee80211/softmac/ieee80211softmac_wx.c
===================================================================
--- linux-2.6.orig/net/ieee80211/softmac/ieee80211softmac_wx.c
+++ linux-2.6/net/ieee80211/softmac/ieee80211softmac_wx.c
@@ -37,6 +37,10 @@ ieee80211softmac_wx_trigger_scan(struct
 				 char *extra)
 {
 	struct ieee80211softmac_device *sm = ieee80211_priv(net_dev);
+	struct iw_scan_req *req = (struct iw_scan_req *) extra;
+
+	sm->scan_essid.len = req->essid_len;
+	memcpy(sm->scan_essid.data, req->essid, req->essid_len);
 	return ieee80211softmac_start_scan(sm);
 }
 EXPORT_SYMBOL_GPL(ieee80211softmac_wx_trigger_scan);
@@ -59,7 +63,7 @@ ieee80211softmac_wx_get_scan_results(str
 		return -EAGAIN;
 	}
 	spin_unlock_irqrestore(&sm->lock, flags);
-	return ieee80211_wx_get_scan(sm->ieee, info, data, extra);
+	return ieee80211_wx_get_scan_essid(sm->ieee, info, data, extra, &sm->scan_essid);
 }
 EXPORT_SYMBOL_GPL(ieee80211softmac_wx_get_scan_results);

Index: linux-2.6/include/net/ieee80211softmac.h
===================================================================
--- linux-2.6.orig/include/net/ieee80211softmac.h
+++ linux-2.6/include/net/ieee80211softmac.h
@@ -209,6 +209,10 @@ struct ieee80211softmac_device {

 	/* we'll need something about beacons here too, for AP or ad-hoc modes */

+	/* keep track of the essid for scanning so that we can associate with
+	 * APs that are hidden */
+	struct ieee80211_essid scan_essid;
+
 	/* Transmission rates to be used by the driver.
 	 * The SoftMAC figures out the best possible rates.
 	 * The driver just needs to read them.

Re: Hidden SSID's

[Jouni Malinen]

On Mon, Jan 29, 2007 at 10:52:20PM -0600, Larry Finger wrote:

> When an AP has a hidden SSID, ieee80211 fails, at least with wpa_supplicant,
> which searches through the scan data looking for a particular ssid. Because
> ieee80211 has substituted a false ssid, namely "<hidden>", wpa_supplicant
> cannot authenticate. This behavior is fixed by adding a new argument to
> ieee80211_translate_scan that contains the expected ssid.

Would this be replacing the SSID of all BSSes in scan results with the
SSID for which the latest per-SSID scan was issued? If yes, this does
not sound any better than the current behavior. The driver/802.11 code
should not replace the SSID value with anything else than the value
received from the AP.

In case of hidden SSIDs, the 802.11 implementation should maintain a
list of BSSes found during the scan(s) and update the SSID (in most
cases, by creating a new BSS entry) with the SSID from Probe Response
frames. In other words, if the scan is done for a specific SSID (Probe
Request includes that SSID), the AP that is using hidden SSIDs will
likely include the SSID in Probe Response and data from that Probe
Response can be used to fill in the missing pieces for the <BSSID,SSID>
pair.

Generating false scan results by locally guessing what the SSID
could be is just plain wrong. The scan results need to be based on real
frames from the APs.

-- 
Jouni Malinen                                            PGP id EFC895FA

Re: Hidden SSID's

[Larry Finger]

Jouni Malinen wrote:
> On Mon, Jan 29, 2007 at 10:52:20PM -0600, Larry Finger wrote:
> 
>> When an AP has a hidden SSID, ieee80211 fails, at least with wpa_supplicant,
>> which searches through the scan data looking for a particular ssid. Because
>> ieee80211 has substituted a false ssid, namely "<hidden>", wpa_supplicant
>> cannot authenticate. This behavior is fixed by adding a new argument to
>> ieee80211_translate_scan that contains the expected ssid.
> 
> Would this be replacing the SSID of all BSSes in scan results with the
> SSID for which the latest per-SSID scan was issued? If yes, this does
> not sound any better than the current behavior. The driver/802.11 code
> should not replace the SSID value with anything else than the value
> received from the AP.

Any AP with a hidden SSID will only respond to probe requests that specify its SSID, and will ignore
any other probes. In addition, the response will have an empty SSID field. These responses are the
only ones in which a substitution would occur. These are the same responses where the current code
sends back the "<hidden>" pseudo-SSID. My change would put the correct one there.

> In case of hidden SSIDs, the 802.11 implementation should maintain a
> list of BSSes found during the scan(s) and update the SSID (in most
> cases, by creating a new BSS entry) with the SSID from Probe Response
> frames. In other words, if the scan is done for a specific SSID (Probe
> Request includes that SSID), the AP that is using hidden SSIDs will
> likely include the SSID in Probe Response and data from that Probe
> Response can be used to fill in the missing pieces for the <BSSID,SSID>
> pair.
> 
> Generating false scan results by locally guessing what the SSID
> could be is just plain wrong. The scan results need to be based on real
> frames from the APs.

We aren't guessing. The response frame with the empty SSID field must have come from the AP with the
SSID we want. Filling in the expected value is just making it easier for the user-space tools.

Larry

Re: Hidden SSID's

[Jouni Malinen]

On Sun, Jan 28, 2007 at 04:18:01PM -0600, Larry Finger wrote:

> Is there something funny about Cisco APs with hidden SSID?

Yes, there is.. Or well, "hidden SSID" is funny concept in itself, but
anyway.. Some APs set the SSID IE to an array of 0x00 octets with the
length of the original SSID when "hiding" the SSID while others just use
0-length SSID.

-- 
Jouni Malinen                                            PGP id EFC895FA

Re: Hidden SSID's

[Jouni Malinen]

On Tue, Jan 30, 2007 at 01:08:29AM -0600, Larry Finger wrote:

> Any AP with a hidden SSID will only respond to probe requests that specify its SSID, and will ignore
> any other probes. In addition, the response will have an empty SSID field. These responses are the
> only ones in which a substitution would occur. These are the same responses where the current code
> sends back the "<hidden>" pseudo-SSID. My change would put the correct one there.

Is the SSID from the probe response really used here? Your patch did not
look like that.. The SSID from the last scan request command may not be
the one that triggered the last scan (e.g., one could request a new scan
without specifying an SSID).

> We aren't guessing. The response frame with the empty SSID field must have come from the AP with the
> SSID we want. Filling in the expected value is just making it easier for the user-space tools.

I don't see how the proposed patch would be using the correct SSID value
in all cases. Especially cases where there are multiple APs using hidden
SSIDs, but with different real SSID values and cases where multiple scan
requests are being processed would be likely to leave windows open for
reporting incorrect SSID.

-- 
Jouni Malinen                                            PGP id EFC895FA

Re: Hidden SSID's

[Larry Finger]

Jouni Malinen wrote:
> On Tue, Jan 30, 2007 at 01:08:29AM -0600, Larry Finger wrote:
> 
>> Any AP with a hidden SSID will only respond to probe requests that specify its SSID, and will ignore
>> any other probes. In addition, the response will have an empty SSID field. These responses are the
>> only ones in which a substitution would occur. These are the same responses where the current code
>> sends back the "<hidden>" pseudo-SSID. My change would put the correct one there.
> 
> Is the SSID from the probe response really used here? Your patch did not
> look like that.. The SSID from the last scan request command may not be
> the one that triggered the last scan (e.g., one could request a new scan
> without specifying an SSID).

If one does the equivalent of 'iwlist eth1 scan essid myssid', then a probe response with
NETWORK_EMPTY_ESSID set in the network flags will have 'myssid' returned in the SSID field of the
returned buffer. If the input command were 'iwlist eth1 scan', then an empty SSID would be returned
under the same circumstances. My code saves the SSID that is in the extra argument of the
SIOCSIWSCAN call, and uses that in the SIOCGIWSCAN call.
> 
>> We aren't guessing. The response frame with the empty SSID field must have come from the AP with the
>> SSID we want. Filling in the expected value is just making it easier for the user-space tools.
> 
> I don't see how the proposed patch would be using the correct SSID value
> in all cases. Especially cases where there are multiple APs using hidden
> SSIDs, but with different real SSID values and cases where multiple scan
> requests are being processed would be likely to leave windows open for
> reporting incorrect SSID.

I can think of one instance where the wrong value could be reported. That is if some other STA
probes a different hidden AP just when we have sent a probe request. For WPA this should not cause a
problem as wpa_supplicant will sort that out while authenticating.

What is the method that should be used to associated with a given hidden AP?

Larry

Re: Hidden SSID's

[Jouni Malinen]

On Tue, Jan 30, 2007 at 08:35:18PM -0600, Larry Finger wrote:

> If one does the equivalent of 'iwlist eth1 scan essid myssid', then a probe response with
> NETWORK_EMPTY_ESSID set in the network flags will have 'myssid' returned in the SSID field of the
> returned buffer. If the input command were 'iwlist eth1 scan', then an empty SSID would be returned
> under the same circumstances. My code saves the SSID that is in the extra argument of the
> SIOCSIWSCAN call, and uses that in the SIOCGIWSCAN call.

Well, yes, but SIOCSIWSCAN and SIOCGIWSCAN calls are not in any way
linked together.. You could have two user space programs asking for a
scan of different SSID at more or less the same time and then have them
read the results with SIOCGIWSCAN. At this point, the SSID from the last
SIOCSIWSCAN would be returned for all APs that are in the scan list
without an SSID. This may not be the correct SSID and can produce quite
confusing results. I don't see this as an improvement over just removing
the "<hidden>" which will at least provide consistent results.

The proper fix for this is to use the information from the AP (Probe
Response frames) and create a local BSS list that has entries with SSID
updated from Probe Responses. If the underlying hardware/firmware does
not allow such handling, this kind of mapping of SSIDs from scan request
to scan results should probably in the hardware driver that would have
chance of mapping results from firmware to request for a given SSID. At
that point, it might be feasible to change the SSID in scan results, but
I see only problems for the case where this is done at higher layer.

> What is the method that should be used to associated with a given hidden AP?

The 802.11 stack should fill in the proper SSID data from Probe
Responses and maintain a local BSS list, i.e., the hidden APs would be
marked with proper SSID whenever Probe Responses are processed. See,
e.g., how driver/net/wireless/hostap/*.c use local->bss_list that will
be filled with information from both Beacon and Probe Response frames or
how wireless-dev.git net/d80211/ieee80211_sta.c handles updating of BSS
entry from both Beacon and Probe Response frames (i.e., do not allow
Beacon frames to replace data from Probe Response frames) in
ieee80211_rx_bss_info().

In other words, the hidden SSIDs are resolved in the BSS list based on
information from Probe Responses and user space programs will get proper
information in the scan results regardless of how the real SSID was
learned.

-- 
Jouni Malinen                                            PGP id EFC895FA