[PATCH 1/1] ip.7: Add "special and reserved addresses" section

[PATCH 1/1] ip.7: Add "special and reserved addresses" section

[Seth David Schoen]

Add a more detailed description of the IP addresses that have
a special meaning in Internet standards, and how these affect
Linux.

Signed-off-by: Seth David Schoen <schoen@loyalty.org>
Suggested-by: John Gilmore <gnu@toad.com>
---
 man7/ip.7 | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/man7/ip.7 b/man7/ip.7
index d9299eb9e..478cda269 100644
--- a/man7/ip.7
+++ b/man7/ip.7
@@ -39,7 +39,7 @@
 .\"	    commit 76e21053b5bf33a07c76f99d27a74238310e3c71
 .\"	    Author: Erich E. Hoover <ehoover@mines.edu>
 .\"
-.TH IP  7 2020-11-01 "Linux" "Linux Programmer's Manual"
+.TH IP  7 2021-03-10 "Linux" "Linux Programmer's Manual"
 .SH NAME
 ip \- Linux IPv4 protocol implementation
 .SH SYNOPSIS
@@ -232,6 +232,7 @@ In particular, this means that you need to call
 on the number that is assigned to a port.
 All address/port manipulation
 functions in the standard library work in network byte order.
+.SS Special and reserved addresses
 .PP
 There are several special addresses:
 .B INADDR_LOOPBACK
@@ -245,6 +246,25 @@ means any address for binding;
 means any host and has the same effect on bind as
 .B INADDR_ANY
 for historical reasons.
+.PP
+Internet standards have also traditionally reserved various
+addresses for particular uses. The addresses
+in the ranges 0.0.0.0 through 0.255.255.255 and 240.0.0.0 through
+255.255.255.254 (0/8 and 240/4 in CIDR notation) are reserved globally
+(but Linux permits addresses within these ranges, other than 0.0.0.0,
+to be assigned to an interface and used like other unicast addresses).
+All addresses in 127.0.0.0 through 127.255.255.255
+("127/8") are treated as loopback addresses akin to the standardized
+local loopback address 127.0.0.1, while addresses in 224.0.0.0 through
+239.255.255.255 ("224/4") are dedicated to multicast use.
+.PP
+On any locally-attached IP subnet, the lowest-numbered address and
+highest-numbered address (e.g., the .0 and .255 addresses on a subnet
+with netmask 255.255.255.0) are both designated as broadcast addresses.
+These cannot usefully be assigned to an interface, and can only be
+addressed with a socket on which the
+.B SO_BROADCAST
+option has been explicitly enabled.
 .SS Socket options
 IP supports some protocol-specific socket options that can be set with
 .BR setsockopt (2)
-- 
2.25.1

Re: [PATCH 1/1] ip.7: Add "special and reserved addresses" section

[Florian Weimer]

* Seth David Schoen:

> +On any locally-attached IP subnet, the lowest-numbered address and
> +highest-numbered address (e.g., the .0 and .255 addresses on a subnet
> +with netmask 255.255.255.0) are both designated as broadcast addresses.
> +These cannot usefully be assigned to an interface, and can only be
> +addressed with a socket on which the
> +.B SO_BROADCAST
> +option has been explicitly enabled.

I think the broadcast address is actually settable to some degree, and
/31 subnets do not have them.  I wouldn't be surprised if kernel
behavior also depended on the network device.

Re: [PATCH 1/1] ip.7: Add "special and reserved addresses" section

[Seth David Schoen]

Florian Weimer writes:

> * Seth David Schoen:
> 
> > +On any locally-attached IP subnet, the lowest-numbered address and
> > +highest-numbered address (e.g., the .0 and .255 addresses on a subnet
> > +with netmask 255.255.255.0) are both designated as broadcast addresses.
> > +These cannot usefully be assigned to an interface, and can only be
> > +addressed with a socket on which the
> > +.B SO_BROADCAST
> > +option has been explicitly enabled.
> 
> I think the broadcast address is actually settable to some degree, and
> /31 subnets do not have them.  I wouldn't be surprised if kernel
> behavior also depended on the network device.

Thanks!  We're somewhat familiar with that because we are currently proposing
to change this behavior. :-)

Both things you noted are true: you can change broadcast addresses with
ifconfig or ip (e.g. ip addr add x/y broadcast z dev d), and /31 subnets
indeed don't have them.  This is defined at

https://github.com/torvalds/linux/blob/master/net/ipv4/fib_frontend.c#L1125

I'm not familiar with a way in which it depends on the network device
other than the /31 issue.  At least, this particular logic isn't
affected by other aspects of the network device type.

Could you suggest a way that these details could usefully be described
here?  The ability to change the broadcast address is very little-used,
so I'm not sure many readers would benefit from details about it here,
but I also wouldn't want to mislead them about that.  Maybe just
changing it to "are both designated, by default, as broadcast
addresses"?

Conceivably it could say something like


On any locally-attached IP subnet that contains more than two IP
addresses, the lowest-numbered address and highest-numbered address
(e.g., the .0 and .255 addresses on a subnet with netmask 255.255.255.0)
are both designated as broadcast addresses by default. These cannot
usefully be assigned to an interface, and can only be addressed with a
socket on which the
.B SO_BROADCAST
option has been explicitly enabled.

Re: [PATCH 1/1] ip.7: Add "special and reserved addresses" section

[Seth David Schoen]

Seth David Schoen writes:

> Both things you noted are true: you can change broadcast addresses with
> ifconfig or ip (e.g. ip addr add x/y broadcast z dev d), and /31 subnets
> indeed don't have them.  This is defined at
> 
> https://github.com/torvalds/linux/blob/master/net/ipv4/fib_frontend.c#L1125

I would like to add to this by noting that the automatic definition
of the lowest and highest addresses as broadcast is run here
_unconditionally_, whether or not the user specifies an explicit
broadcast address.

So, if you run something like

ip link set eno1 up
ip addr add 192.168.17.17/24 broadcast 192.168.17.23 dev eno1

then "ifconfig eno1" will show

[...]
inet 192.168.17.17  netmask 255.255.255.0  broadcast 192.168.17.23
[...]

and "ip addr show eno1" will show

[...]
inet 192.168.17.17/24 brd 192.168.17.23 scope global eno1
[...]

However, even though 192.168.17.23 is indicated as "the" broadcast
address, both 192.168.17.0 and 192.168.17.255 are still designated as
broadcast addresses on this link by the kernel!

$ ping 192.168.17.0
ping: Do you want to ping broadcast? Then -b. If not, check your local firewall rules
$ ping 192.168.17.23
ping: Do you want to ping broadcast? Then -b. If not, check your local firewall rules
$ ping 192.168.17.255
ping: Do you want to ping broadcast? Then -b. If not, check your local firewall rules
$ ping 192.168.17.17
PING 192.168.17.17 (192.168.17.17) 56(84) bytes of data.
64 bytes from 192.168.17.17: icmp_seq=1 ttl=64 time=0.065 ms
[...]

I'm unaware that there's even any way to _remove_ that designation via
existing userspace interfaces.  If not, we can't, in fact, "change" a
broadcast address so much as designate an additional one.  You can
attempt to retroactively mark a broadcast-capable interface as
non-broadcast capable with "ifconfig eno1 -broadcast", but it appears to
me that the kernel refuses to accept this (maybe this behavior does
depend on the interface type).

$ sudo ifconfig eno1 -broadcast
Warning: Interface eno1 still in BROADCAST mode.

One might argue that setting the default broadcast addresses in addition
to the broadcast address requested by the user is a kernel bug.  I think
this is a somewhat complicated question, but it doesn't necessarily need
to be resolved, in part because there are almost no practical applications
for setting an additional non-default broadcast address manually.  (You
could conceivably use it to simulate the behavior of IPv4 or IPv6 multicast
groups on a local network segment, in an awkward way where you would
probably be better off using actual multicast instead.)

I think a slight change to the language in my previous message would
capture this, like

On any locally-attached IP subnet that contains more than two IP
addresses, the lowest-numbered address and highest-numbered address
(e.g., the .0 and .255 addresses on a subnet with netmask 255.255.255.0)
are both automatically designated as broadcast addresses by the kernel.
These cannot usefully be assigned to an interface, and can only be addressed
with a socket on which the
.B SO_BROADCAST
option has been explicitly enabled.

-- and of course I'm happy for any corrections if I've missed or
misdescribed any subtleties here.

Re: [PATCH 1/1] ip.7: Add "special and reserved addresses" section

[John Gilmore]

Perhaps the reason there is an interface for setting the broadcast
address is so that it can be explicitly set to the lowest address (e.g.
for compatability with 4.2BSD)?  The ability to set an arbitrary
broadcast address does seem rather useless otherwise.

Sounds like a good email discussion to pull some historical info out of
netdev participants and/or the Internet-History mailing list.

Hmm -- after our patch, when the lowest address is no longer reserved
for broadcast, you can still make the lowest address into a second
broadcast address by setting it with ifconfig!  That actually argues
that our patch is even more innocuous; if anybody in the world actually
needs the lowest address to be broadcast, they already have a userspace
way to specify that.

(By the way, the kernel code looks to me that if the interface broadcast
address is set to 255.255.255.255 then it is NOT enabled, so only the
default broadcast addresses are enabled.  That doesn't seem to be
documented either.)

I don't know that the documentation that we added about how Linux
support for reserved addresses varies from the RFCs needs to wait for
these broadcast-related questions to be resolved.  We can further
improve that documentation once we know if and when "setting the
broadcast address on an interface" has or used to have any usefulness.

	John

Re: [PATCH 1/1] ip.7: Add "special and reserved addresses" section

[Florian Weimer]

* Seth David Schoen:

> Both things you noted are true: you can change broadcast addresses with
> ifconfig or ip (e.g. ip addr add x/y broadcast z dev d), and /31 subnets
> indeed don't have them.  This is defined at

There is also the questions of netmask that aren't in the CIDR style
(so 255.255.0.255 instead of 255.255.255.0).  Are they still
supported?

> I'm not familiar with a way in which it depends on the network device
> other than the /31 issue.  At least, this particular logic isn't
> affected by other aspects of the network device type.

Are there network devices that are neither point-to-point, nor do they
have broadcast support?

> Could you suggest a way that these details could usefully be described
> here?  The ability to change the broadcast address is very little-used,
> so I'm not sure many readers would benefit from details about it here,
> but I also wouldn't want to mislead them about that.  Maybe just
> changing it to "are both designated, by default, as broadcast
> addresses"?

My main concern is that the language should not imply that something
cannot happen if it actually can.  Otherwise programmers will end up
using this guidance and create software that won't work in such
configurations.

Re: [PATCH 1/1] ip.7: Add "special and reserved addresses" section

[Seth David Schoen]

Florian Weimer writes:

> * Seth David Schoen:
> 
> > Both things you noted are true: you can change broadcast addresses with
> > ifconfig or ip (e.g. ip addr add x/y broadcast z dev d), and /31 subnets
> > indeed don't have them.  This is defined at
> 
> There is also the questions of netmask that aren't in the CIDR style
> (so 255.255.0.255 instead of 255.255.255.0).  Are they still
> supported?

Nope -- I tried setting such a mask with ifconfig and got EINVAL back
from the kernel, while iproute2 doesn't even allow specifying a netmask
as opposed to a prefix length.

This EINVAL comes from here

https://github.com/torvalds/linux/blob/4ee998b0ef8b6d7b1267cd4d953182224929abba/net/ipv4/devinet.c#L1214

where the bad_mask() macro is checked to see if the mask is valid.  (I
haven't reasoned about the way bad_mask() is implemented, but the purpose
of the check seems to be whether the requested mask is CIDR-valid.)

> > I'm not familiar with a way in which it depends on the network device
> > other than the /31 issue.  At least, this particular logic isn't
> > affected by other aspects of the network device type.
> 
> Are there network devices that are neither point-to-point, nor do they
> have broadcast support?

We've talked about this question a bit in our project before.  I don't
believe so, but I can look into it further.  It's logically conceivable
that there could be such devices.

> > Could you suggest a way that these details could usefully be described
> > here?  The ability to change the broadcast address is very little-used,
> > so I'm not sure many readers would benefit from details about it here,
> > but I also wouldn't want to mislead them about that.  Maybe just
> > changing it to "are both designated, by default, as broadcast
> > addresses"?
> 
> My main concern is that the language should not imply that something
> cannot happen if it actually can.  Otherwise programmers will end up
> using this guidance and create software that won't work in such
> configurations.

That makes sense.  Are you OK with the "are both designated, by default,
as broadcast addresses" or similar language?

-- 
Seth David Schoen <schoen@loyalty.org>      |  Qué empresa fácil no pensar
     http://www.loyalty.org/~schoen/        |  en un tigre, reflexioné.
                                            |        -- Borges, "El Zahir"

Re: [PATCH 1/1] ip.7: Add "special and reserved addresses" section

[Seth David Schoen]

Seth David Schoen writes:

> > Are there network devices that are neither point-to-point, nor do they
> > have broadcast support?
> 
> We've talked about this question a bit in our project before.  I don't
> believe so, but I can look into it further.  It's logically conceivable
> that there could be such devices.

I found that there are several address family and/or interface types in the
Linux kernel now that always refuse the SIOCSIFBRDADDR ioctl (with EINVAL).
However, none of them are IPv4 and so they're all outside the scope of the
ip(7) man page.  (They are x25, ax25, rose, netrom, and qrtr, each of
which has its own address family instead of AF_INET, namely AF_X25,
AF_AX25, AF_ROSE, AF_NETROM, and AF_QIPCRTR.)

I haven't found anywhere else in the kernel where setting a broadcast
address will be rejected as inapplicable.  That is, you can always at
least complete the ioctl, so if it doesn't work because it's
inapplicable at a lower layer, that seems like a bug.

The existing man page for ip-link(8) seems to assume that an IP link is
either broadcast-capable or marked point-to-point:

       broadcast LLADDRESS

       brd LLADDRESS

       peer LLADDRESS
              change the link layer broadcast address or the peer address when
              the interface is POINTOPOINT.

(These two cases seem to be taken as exhaustive and mutually exclusive
here.)

The more complete answer that I'm looking into now will depend on
whether you can get an interface that can run IPv4 but that has neither
IFF_BROADCAST nor IFF_POINTOPOINT in its flags.

-- 
Seth David Schoen <schoen@loyalty.org>      |  Qué empresa fácil no pensar
     http://www.loyalty.org/~schoen/        |  en un tigre, reflexioné.
                                            |        -- Borges, "El Zahir"

Re: [PATCH 1/1] ip.7: Add "special and reserved addresses" section

[John Gilmore]

> > > Are there network devices that are neither point-to-point, nor do they
> > > have broadcast support?

It seems to me that cellular radio systems are neither point-to-point,
nor do they support broadcast.  And they keep moving further and further
toward being IP-based (e.g. phone calls on your LTE phone use SIP over
UDP over IP over cellular!).  But given the proprietary nature of most
cellphone radio chips and their firmware, it's possible that Linux has
never built a fully capable network interface to talk to them.  I don't
think we should attempt to rule it out, though.

	John
	

Re: [PATCH 1/1] ip.7: Add "special and reserved addresses" section

[Seth David Schoen]

John Gilmore writes:

> > > > Are there network devices that are neither point-to-point, nor do they
> > > > have broadcast support?
> 
> It seems to me that cellular radio systems are neither point-to-point,
> nor do they support broadcast.  And they keep moving further and further
> toward being IP-based (e.g. phone calls on your LTE phone use SIP over
> UDP over IP over cellular!).  But given the proprietary nature of most
> cellphone radio chips and their firmware, it's possible that Linux has
> never built a fully capable network interface to talk to them.  I don't
> think we should attempt to rule it out, though.

I found in include/uapi/linux/if_link.h the following classification
about this:

   IFF_LOOPBACK, IFF_BROADCAST and IFF_POINTOPOINT are
   more not changeable by user. They describe link media
   characteristics and set by device driver.

   Comments:
   - Combination IFF_BROADCAST|IFF_POINTOPOINT is invalid
   - If neither of these three flags are set;
     the interface is NBMA.

https://en.wikipedia.org/wiki/Non-broadcast_multiple-access_network

The likeliest example there in terms of running IP directly over such a
network would presumably be ATM, which had a whole series of RFCs about
how to make this work, and which is supported by the Linux kernel (e.g.
net/atm/clip.c, for "Classical IP over ATM" following RFC 1577).  That
includes, for example, a non-broadcast-based ARP mechanism!

I'll try to figure out what a Linux ATM interface does about IP broadcasts.
It seems like the RFC 1577 instruction is to still allow IP broadcasts
but accept that they won't be mapped onto link-layer broadcasts:

   ATM does not support broadcast addressing, therefore there are no
   mappings available from IP broadcast addresses to ATM broadcast
   services. Note: this lack of mapping does not restrict members from
   transmitting or receiving IP datagrams specifying any of the four
   standard IP broadcast address forms as described in [8].  Members,
   upon receiving an IP broadcast or IP subnet broadcast for their LIS,
   MUST process the packet as if addressed to that station.

In other IP-over-ATM methods it appears that broadcasting is emulated by
a lower layer.

https://www.cse.wustl.edu/~jain/cis788-97/ftp/ip_over_atm/index.html#3

[PATCH v2] ip.7: Add "special and reserved addresses" section

[Seth David Schoen]

Add a more detailed description of the IP addresses that have
a special meaning in Internet standards, and how these affect
Linux.

This revision notes details that network devices may not support
broadcasting at the link layer (and how to check).

Signed-off-by: Seth David Schoen <schoen@loyalty.org>
---
 man7/ip.7 | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/man7/ip.7 b/man7/ip.7
index 7eee2811e..27df71ba4 100644
--- a/man7/ip.7
+++ b/man7/ip.7
@@ -233,6 +233,7 @@ on the number that is assigned to a port.
 All address/port manipulation
 functions in the standard library work in network byte order.
 .PP
+.SS Special and reserved addresses
 There are several special addresses:
 .B INADDR_LOOPBACK
 (127.0.0.1)
@@ -245,6 +246,44 @@ means any address for binding;
 means any host and has the same effect on bind as
 .B INADDR_ANY
 for historical reasons.
+.PP
+Internet standards have also traditionally reserved various
+addresses for particular uses. The addresses
+in the ranges 0.0.0.0 through 0.255.255.255 and 240.0.0.0 through
+255.255.255.254 (0/8 and 240/4 in CIDR notation) are reserved globally
+(but Linux permits addresses within these ranges, other than 0.0.0.0, to
+be assigned to an interface and used like other unicast addresses).
+All addresses in 127.0.0.0 through 127.255.255.255 ("127/8")
+are treated as loopback addresses akin to the standardized local
+loopback address 127.0.0.1, while addresses in 224.0.0.0 through
+239.255.255.255 ("224/4") are dedicated to multicast use.
+.PP
+On any locally-attached IP subnet that contains more than two IP
+addresses, the lowest-numbered address and highest-numbered address
+(e.g., the .0 and .255 addresses on a subnet with netmask 255.255.255.0)
+are both designated as broadcast addresses by default. These cannot
+usefully be assigned to an interface as its own address, and can only
+be addressed with a socket on which the
+.B SO_BROADCAST
+option has been explicitly enabled. The ability to actually send and
+receive broadcasts at the link layer is available on most transports
+(like Ethernet), but may not be implemented on some non-broadcast
+multiple access (NBMA) networks, such as some methods of sending IP
+over ATM. Individual network interfaces (see
+.BR netdevice (7))
+are designated as
+.BR IFF_LOOPBACK ,
+.BR IFF_BROADCAST ,
+.BR IFF_POINTOPOINT ,
+or, more rarely, none of these three (NBMA); only those designated
+.B IFF_BROADCAST
+possess the ability to send and receive broadcasts at the link
+layer. These flags can be accessed with
+.BR getifaddrs (3)
+or viewed with
+.BR ip-link (8)
+or
+.BR ifconfig (8).
 .SS Socket options
 IP supports some protocol-specific socket options that can be set with
 .BR setsockopt (2)
-- 
2.25.1

[RESEND PATCH v2] ip.7: Add "special and reserved addresses" section

[Seth David Schoen]

Add a more detailed description of the IP addresses that have
a special meaning in Internet standards, and how these affect
Linux.

This revision notes details that network devices may not support
broadcasting at the link layer (and how to check).

Signed-off-by: Seth David Schoen <schoen@loyalty.org>
---
 man7/ip.7 | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/man7/ip.7 b/man7/ip.7
index 7eee2811e..27df71ba4 100644
--- a/man7/ip.7
+++ b/man7/ip.7
@@ -233,6 +233,7 @@ on the number that is assigned to a port.
 All address/port manipulation
 functions in the standard library work in network byte order.
 .PP
+.SS Special and reserved addresses
 There are several special addresses:
 .B INADDR_LOOPBACK
 (127.0.0.1)
@@ -245,6 +246,44 @@ means any address for binding;
 means any host and has the same effect on bind as
 .B INADDR_ANY
 for historical reasons.
+.PP
+Internet standards have also traditionally reserved various
+addresses for particular uses. The addresses
+in the ranges 0.0.0.0 through 0.255.255.255 and 240.0.0.0 through
+255.255.255.254 (0/8 and 240/4 in CIDR notation) are reserved globally
+(but Linux permits addresses within these ranges, other than 0.0.0.0, to
+be assigned to an interface and used like other unicast addresses).
+All addresses in 127.0.0.0 through 127.255.255.255 ("127/8")
+are treated as loopback addresses akin to the standardized local
+loopback address 127.0.0.1, while addresses in 224.0.0.0 through
+239.255.255.255 ("224/4") are dedicated to multicast use.
+.PP
+On any locally-attached IP subnet that contains more than two IP
+addresses, the lowest-numbered address and highest-numbered address
+(e.g., the .0 and .255 addresses on a subnet with netmask 255.255.255.0)
+are both designated as broadcast addresses by default. These cannot
+usefully be assigned to an interface as its own address, and can only
+be addressed with a socket on which the
+.B SO_BROADCAST
+option has been explicitly enabled. The ability to actually send and
+receive broadcasts at the link layer is available on most transports
+(like Ethernet), but may not be implemented on some non-broadcast
+multiple access (NBMA) networks, such as some methods of sending IP
+over ATM. Individual network interfaces (see
+.BR netdevice (7))
+are designated as
+.BR IFF_LOOPBACK ,
+.BR IFF_BROADCAST ,
+.BR IFF_POINTOPOINT ,
+or, more rarely, none of these three (NBMA); only those designated
+.B IFF_BROADCAST
+possess the ability to send and receive broadcasts at the link
+layer. These flags can be accessed with
+.BR getifaddrs (3)
+or viewed with
+.BR ip-link (8)
+or
+.BR ifconfig (8).
 .SS Socket options
 IP supports some protocol-specific socket options that can be set with
 .BR setsockopt (2)
-- 
2.25.1