ppp_async: ioctl to set MTU needed

ppp_async: ioctl to set MTU needed

[Matthias-Christian Ott]

Some months ago I filed the following bug report (with some sentences
corrected) on the Kernel Bug Tracker but have received no reaction
reaction so far:

Summary: ppp_async: ioctl to set MTU needed

The MTU of a channel of a Multilink PPP link is set from the MRU of a
Configure-Ack LCP packet. However, just because the is able to receive
packets of a certain size it doesn't mean that the link or the sender
are able to transmit packets of that size, so the received MTU of the
channel should not be set to the MRU of the peer.

A practical example is PPP encapsulated in GRE or L2TP over IPv6
(without fragmentation). Peer A sets its MRU to the link MTU minus
maximum IP header length minus GRE/L2TP header length to be able to
receive packets of maximum length over PPP. Peer B connects to peer A
and the maximum payload size, which is derived from the Path MTU, of the
tunnel between peers A and B is lower than the MRU of peer A. If peer B
sends a packet P smaller than the MRU of peer A and larger than the
maximum payload size of the IP packet encapsulating P, it will exceed
the Path MTU and thus will be rejected and never arrive at peer A. Such
scenario is common for VPNs that use PPP or to transport PPP sessions of
broadband/Internet subscribers (e.g. if one ISP manages the physical
network and another ISP provides Internet access and is the endpoint of
the PPP session that runs over the physical network and the PPP session
is transported over IPv6). IPv4 was able to fragment IP packets larger
than the Path MTU, so setting the MTU to the MRU of peer a worked. With
IPv6 (without fragmentation) this no longer works.

As a workaround I patched pppd and sent a Configure-Nak with the correct
MRU in the options of the packet in the hope that the remote peer would
adjust its MRU to the MRU suggested in the Configure-Nak and that the
PPP driver would in turn set the MTU of the channel correctly. However,
this didn't work because the remote peer simply sent the original
Configure-Request again and again without adjusting its MRU.
Unfortunately, I don't know which PPP implementation the remote peer uses.


I understand that PPP is not as interesting as in the 1990s any longer.
Nonetheless I would like to have this issue fixed and I would be happy
to create a provide for the kernel module and pppd if I'm certain that
the patch would get accepted (with minor modifications). So it would
help if somebody in charge of pppd or the kernel module comment on my
proposal to introduce an ioctl for this. Thanks!

Regards,
Matthias-Christian

[1] https://bugzilla.kernel.org/show_bug.cgi?idˆ101

Re: ppp_async: ioctl to set MTU needed

[James Carlson]

On 01/13/15 12:33, Matthias-Christian Ott wrote:
> Some months ago I filed the following bug report (with some sentences
> corrected) on the Kernel Bug Tracker but have received no reaction
> reaction so far:
> 
> Summary: ppp_async: ioctl to set MTU needed

It would be nice to see complete configuration options and a trace or
debug log of the problem.  I followed what you posted, but being 100%
sure of the scenario and the issues is pretty important when trying to
talk about fixes.

Have you tried just setting the pppd "mtu" option?  That should force
pppd to set an MTU of that value (unless the peer's MRU is even smaller).

I realize that's not a perfect fix.  A perfect fix would have some
mechanism for the underlying transport to signal changes in effective
MTU (e.g., Path MTU data) so that this value could be altered at the
network layer as needed.  Unfortunately, there's currently no such
mechanism, as far as I know.

> As a workaround I patched pppd and sent a Configure-Nak with the correct
> MRU in the options of the packet in the hope that the remote peer would
> adjust its MRU to the MRU suggested in the Configure-Nak and that the
> PPP driver would in turn set the MTU of the channel correctly. However,
> this didn't work because the remote peer simply sent the original
> Configure-Request again and again without adjusting its MRU.
> Unfortunately, I don't know which PPP implementation the remote peer uses.

I would have been surprised if that had worked.  Sending a Configure-Nak
for a smaller MRU is (to me, at least) a nonsense request.  The peer's
offer of a "too large" MRU is just fine and should be accepted.  You're
under no obligation ever to send a packet as large as the maximum your
peer may be willing to accept, so the local MTU can be lower than the
peer's MRU.  It really only makes sense to negotiate the peer's MRU
upwards, not downwards, and then only *if* the advertised value is too
small for the desired purpose of the link and if there's some hope that
it'll either comply or fail in a manner that makes sense to the user.

So, yes, ignoring your request for a smaller MRU sounds like the
behavior of a somewhat reasonable (or not entirely wrong) peer.  Better
still would be to omit the option on subsequent Configure-Request
messages, and thus insist on the default.

-- 
James Carlson         42.703N 71.076W         <carlsonj@workingcode.com>

Re: ppp_async: ioctl to set MTU needed

[Christoph Schulz]

Hello!

Matthias-Christian Ott schrieb am Tue, 13 Jan 2015 18:33:04 +0100:

> The MTU of a channel of a Multilink PPP link is set from the MRU of a
> Configure-Ack LCP packet. However, just because the is able to receive
> packets of a certain size it doesn't mean that the link or the sender
> are able to transmit packets of that size, so the received MTU of the
> channel should not be set to the MRU of the peer.

Well, this seems to be not so easy. I just studied the Kernel source  
code, the pppd source code and the PPP RFCs, so I feel rather  
well-informed in a way ;-)

What does the "mtu" option really do? In a word: (almost) nothing! It  
only restricts the MTU of the underlying network interface that is  
finally created after LCP negotation has completed successfully. You  
can see that the MTU value is stored in lcp_allowoptions[0].mru:

     { "mtu", o_int, &lcp_allowoptions[0].mru,
       "Set our MTU", OPT_LIMITS, NULL, MAXMRU, MINMRU },

(lcp.c, lines 132-133), and it is only used when setting the MTU of  
the network interface. The non-multilink case:

     /*
      * Set our MTU to the smaller of the MTU we wanted and
      * the MRU our peer wanted.  If we negotiated an MRU,
      * set our MRU to the larger of value we wanted and
      * the value we got in the negotiation.
      * Note on the MTU: the link MTU can be the MRU the peer wanted,
      * the interface MTU is set to the lowest of that, the
      * MTU we want to use, and our link MRU.
      */
     mtu = ho->neg_mru? ho->mru: PPP_MRU;
     mru = go->neg_mru? MAX(wo->mru, go->mru): PPP_MRU;
#ifdef HAVE_MULTILINK
     if (!(multilink && go->neg_mrru && ho->neg_mrru))
#endif /* HAVE_MULTILINK */
	netif_set_mtu(f->unit, MIN(MIN(mtu, mru), ao->mru));
                                ^^^                ^^^^^^^

(lcp.c, lines 1911-1925) and

		/* not doing multilink */
		if (go->neg_mrru)
			notice("oops, multilink negotiated only for receive");
		mtu = ho->neg_mru? ho->mru: PPP_MRU;
--->		if (mtu > ao->mru)
--->			mtu = ao->mru;

(multilink.c, lines 128-133). The multilink case:

	mtu = MIN(ho->mrru, ao->mru);
               ^^^           ^^^^^^^
	[...]
	netif_set_mtu(0, mtu);

(multilink.c, lines 184 and 234). So ao->mru (or the "mtu" value)  
effectively caps the MTU (due to the various MIN(...) macro  
invocations), but this does NOT mean that a larger peer MRU cannot be  
successfully negotiated! You see this in the lcp_reqci() function  
where CONFREQs are ACKed NAKed or REJected:

	case CI_MRU:
	    if (!ao->neg_mru ||		/* Allow option? */
		cilen != CILEN_SHORT) {	/* Check CI length */
		orc = CONFREJ;		/* Reject CI */
		break;
	    }
	    GETSHORT(cishort, p);	/* Parse MRU */

	    /*
	     * He must be able to receive at least our minimum.
	     * No need to check a maximum.  If he sends a large number,
	     * we'll just ignore it.
	     */
	    if (cishort < MINMRU) {
		orc = CONFNAK;		/* Nak CI */
		PUTCHAR(CI_MRU, nakp);
		PUTCHAR(CILEN_SHORT, nakp);
		PUTSHORT(MINMRU, nakp);	/* Give him a hint */
		break;
	    }
	    ho->neg_mru = 1;		/* Remember he sent MRU */
	    ho->mru = cishort;		/* And remember value */

(lcp.c, lines 1556-1577). So the "mtu" value does not have any  
consequences for the negotiated peer MRU. And, consequently, it does  
not influence the value the kernel uses for the channel MTU either, as  
the kernel uses the ACKed value:

	/* process the options in the confack */
	data += 4;
	dlen -= 4;
	/* data[0] is code, data[1] is length */
	while (dlen >= 2 && dlen >= data[1] && data[1] >= 2) {
		switch (data[0]) {
		case LCP_MRU:
			val = get_unaligned_be16(data + 2);
			if (inbound)
				ap->mru = val;
			else
				ap->chan.mtu = val;
			break;

(Kernel 3.17.8, ppp_async.c, lines 995-1007).

Where are we now? I think your problem lies in this comment in lcp_reqci():

	    /*
	     * He must be able to receive at least our minimum.
	     * No need to check a maximum.  If he sends a large number,
---> !!	     * we'll just ignore it.
	     */

You think each PPP implementation has to NAK when a MRU being larger  
than our MTU is REQuested by the peer. Obviously this isn't handled  
this way, at least not by pppd. However, I think your problem is  
twofold. You did not write how big your desired MTU (at the side of  
peer B) really is. I bet that it is smaller than 1500, as you write it  
is smaller than "link MTU minus maximum IP header length minus  
GRE/L2TP header length". However, RFC 1661 requires that each  
implementation MUST accept packets that are 1500 bytes long:

6.1.  Maximum-Receive-Unit (MRU)
[...]
       The default value is 1500 octets.  If smaller packets are
       requested, an implementation MUST still be able to receive the
       full 1500 octet information field in case link synchronization is
       lost.

So each attempt to force peer A to request a MRU < 1500 is formally  
wrong as it might be ignored -- even if the peer ACKs this MRU.


Regards,

Christoph Schulz

Re: ppp_async: ioctl to set MTU needed

[James Carlson]

On 01/13/15 15:57, Christoph Schulz wrote:
> (lcp.c, lines 132-133), and it is only used when setting the MTU of the
> network interface. The non-multilink case:
[...]
>     netif_set_mtu(f->unit, MIN(MIN(mtu, mru), ao->mru));
>                                ^^^                ^^^^^^^

That part looks good to me.  Capping at ao->mru means that the user can
specify the "mtu" option and force the actual link MTU to be anything he
wants (up to the peer's advertised MRU).

> (multilink.c, lines 128-133). The multilink case:
> 
>     mtu = MIN(ho->mrru, ao->mru);
>               ^^^           ^^^^^^^
>     [...]
>     netif_set_mtu(0, mtu);

That also looks good.  Again, the user can specify anything he wants
with the "mtu" option.

That's why I suggested setting "mtu" in the pppd options as a reasonable
work-around.

> 6.1.  Maximum-Receive-Unit (MRU)
> [...]
>       The default value is 1500 octets.  If smaller packets are
>       requested, an implementation MUST still be able to receive the
>       full 1500 octet information field in case link synchronization is
>       lost.
> 
> So each attempt to force peer A to request a MRU < 1500 is formally
> wrong as it might be ignored -- even if the peer ACKs this MRU.

Not so much "wrong" as "pointless."  Why bother to ask for less?  You
don't have to pad the packets out to the full MRU, so why do you care
what the peer has advertised as a maximum?  All that matters is that (by
acking the value) you've promised not to send anything bigger.  By
adhering to a smaller limit, you're certainly in compliance with the
peer's limitations.

Note that the "synchronization" bit there is obscure, but refers to the
possibility of LCP renegotiation.  For some (many?) implementations, LCP
renegotiation (unfortunately!) causes link failure, so everyone avoids it.

In truth, the reason you're required to support an MRU of 1500 is that
the peer can always just Configure-Reject the option.  If he does that,
then what do you do?  If the peer rejects it, that's the end; the option
ends up with the default value, which is 1500.

-- 
James Carlson         42.703N 71.076W         <carlsonj@workingcode.com>

Re: ppp_async: ioctl to set MTU needed

[Christoph Schulz]

Hello!

James Carlson schrieb am Tue, 13 Jan 2015 16:31:29 -0500:

> Not so much "wrong" as "pointless."  Why bother to ask for less?  You
> don't have to pad the packets out to the full MRU, so why do you care
> what the peer has advertised as a maximum?  All that matters is that (by
> acking the value) you've promised not to send anything bigger.  By
> adhering to a smaller limit, you're certainly in compliance with the
> peer's limitations.

Yes. However, the problem lies in the mechanism the Linux kernel  
determines the MTU of a link in a multilink bundle: It simply takes  
the peer's MRU we ACKed. Obviously this is not correct if we are bound  
to use a smaller (channel) MTU due to other constraints (tunnel  
overhead). But there is currently no possibility to tell the Linux  
kernel what channel MTU to use.


Regards,

Christoph Schulz

Re: ppp_async: ioctl to set MTU needed

[James Carlson]

On 01/13/15 16:56, Christoph Schulz wrote:
> Hello!
> 
> James Carlson schrieb am Tue, 13 Jan 2015 16:31:29 -0500:
> 
>> Not so much "wrong" as "pointless."  Why bother to ask for less?  You
>> don't have to pad the packets out to the full MRU, so why do you care
>> what the peer has advertised as a maximum?  All that matters is that (by
>> acking the value) you've promised not to send anything bigger.  By
>> adhering to a smaller limit, you're certainly in compliance with the
>> peer's limitations.
> 
> Yes. However, the problem lies in the mechanism the Linux kernel
> determines the MTU of a link in a multilink bundle: It simply takes the
> peer's MRU we ACKed. Obviously this is not correct if we are bound to
> use a smaller (channel) MTU due to other constraints (tunnel overhead).
> But there is currently no possibility to tell the Linux kernel what
> channel MTU to use.

Ah, ok.  Are you referring to the individual link MTU rather than the
bundle's overall MTU?

If so, then, yes, that does look like a problem.  It should be possible
to limit individual link MTU values as desired, but I don't see a good
way to do it.

But, for what it's worth, running MP over a tunneled link sounds like
extreme weirdness to me.  Unless you're trying to set new records for
packet loss and link latency, why would you do that?

-- 
James Carlson         42.703N 71.076W         <carlsonj@workingcode.com>

Re: ppp_async: ioctl to set MTU needed

[Christoph Schulz]

Hello!

James Carlson schrieb am Tue, 13 Jan 2015 17:22:47 -0500:

> Ah, ok.  Are you referring to the individual link MTU rather than the
> bundle's overall MTU?

Yes.

>
> If so, then, yes, that does look like a problem.  It should be possible
> to limit individual link MTU values as desired, but I don't see a good
> way to do it.

Me neither, at least without introducing a system-specific ioctl that  
would have to be explicitly called by pppd. However, perhaps anyone  
has a better idea...

By the way, how does Solaris solve this problem? I see that PPPIO_MTU  
is sent in tty_send_config() in sys-solaris.c for setting the link  
MTU. Is that used when doing multilink PPP? The corresponding function  
in sys-linux.c does not send the link MTU to the kernel, it only saves  
the value in a global static variable "link_mtu" (line 1250) which is  
not used anywhere else.

>
> But, for what it's worth, running MP over a tunneled link sounds like
> extreme weirdness to me.  Unless you're trying to set new records for
> packet loss and link latency, why would you do that?

Don't ask me, ask Matthias-Christian ;-)

But nevertheless there is no obvious reason why it should be  
impossible to make it work.


Regards,

Christoph

Re: ppp_async: ioctl to set MTU needed

[Christoph Schulz]

Hello!

Christoph Schulz schrieb am Tue, 13 Jan 2015 23:56:25 +0100:

> By the way, how does Solaris solve this problem? [...]

Never mind, I forgot that pppd supports multilink PPP only under Linux.


Regards,

Christoph Sculz

Re: ppp_async: ioctl to set MTU needed

[Matthias-Christian Ott]

On 2015-01-13 21:57, Christoph Schulz wrote:
> Matthias-Christian Ott schrieb am Tue, 13 Jan 2015 18:33:04 +0100:
> 
>> The MTU of a channel of a Multilink PPP link is set from the MRU of a
>> Configure-Ack LCP packet. However, just because the is able to receive
>> packets of a certain size it doesn't mean that the link or the sender
>> are able to transmit packets of that size, so the received MTU of the
>> channel should not be set to the MRU of the peer.
> 
> Well, this seems to be not so easy. I just studied the Kernel source
> code, the pppd source code and the PPP RFCs, so I feel rather
> well-informed in a way ;-)

I dont't doubt that.

> Where are we now? I think your problem lies in this comment in lcp_reqci():
> 
>         /*
>          * He must be able to receive at least our minimum.
>          * No need to check a maximum.  If he sends a large number,
> ---> !!         * we'll just ignore it.
>          */
> 
> You think each PPP implementation has to NAK when a MRU being larger
> than our MTU is REQuested by the peer. Obviously this isn't handled this
> way, at least not by pppd. However, I think your problem is twofold. You
> did not write how big your desired MTU (at the side of peer B) really
> is. I bet that it is smaller than 1500, as you write it is smaller than
> "link MTU minus maximum IP header length minus GRE/L2TP header length".
> However, RFC 1661 requires that each implementation MUST accept packets
> that are 1500 bytes long:
> 
> 6.1.  Maximum-Receive-Unit (MRU)
> [...]
>       The default value is 1500 octets.  If smaller packets are
>       requested, an implementation MUST still be able to receive the
>       full 1500 octet information field in case link synchronization is
>       lost.
> 
> So each attempt to force peer A to request a MRU < 1500 is formally
> wrong as it might be ignored -- even if the peer ACKs this MRU.

It seems that I read over this than and what I wanted to achieve is
impossible with PPP. I think setting the channel MTU to the MRU is
unaffected by this and is still wrong though. From RFC 1661:

6.1.  Maximum-Receive-Unit (MRU)

[...]

    This option is used to indicate an implementation capability.
    The peer is not required to maximize the use of the capacity.
    For example, when a MRU is indicated which is 2048 octets, the
    peer is not required to send any packet with 2048 octets.  The
    peer need not Configure-Nak to indicate that it will only send
    smaller packets, since the implementation will always require
    support for at least 1500 octets.

Regards,
Matthias-Christian

Re: ppp_async: ioctl to set MTU needed

[Matthias-Christian Ott]

On 2015-01-14 00:24, Matthias-Christian Ott wrote:
> It seems that I read over this than and what I wanted to achieve is
> impossible with PPP. I think setting the channel MTU to the MRU is
> unaffected by this and is still wrong though. From RFC 1661:
> 
> 6.1.  Maximum-Receive-Unit (MRU)
> 
> [...]
> 
>     This option is used to indicate an implementation capability.
>     The peer is not required to maximize the use of the capacity.
>     For example, when a MRU is indicated which is 2048 octets, the
>     peer is not required to send any packet with 2048 octets.  The
>     peer need not Configure-Nak to indicate that it will only send
>     smaller packets, since the implementation will always require
>     support for at least 1500 octets.

Another aspect is that if Multilink PPP is used, it might be wrong to
set the MTU based on the MRU. If both peers know that the link MTU is
smaller than 1500 octets and want to encapsulate payloads larger than
the link MTU (that is the interface MTU is bigger than the channel MTU
and Multilink PPP fragments payloads larger the channel MTU), they can
adjust their MRRU and channel MTU accordingly and thus never send PPP
packets larger than the link MTU. If the channel MTU is set from the
MRU, this is not possible.

Regards,
Matthias-Christian

Re: ppp_async: ioctl to set MTU needed

[James Carlson]

On 1/13/2015 6:04 PM, Christoph Schulz wrote:
> Hello!
> 
> Christoph Schulz schrieb am Tue, 13 Jan 2015 23:56:25 +0100:
> 
>> By the way, how does Solaris solve this problem? [...]
> 
> Never mind, I forgot that pppd supports multilink PPP only under Linux.

Yes, exactly.  At Sun, we had a multilink implementation, but there
wasn't much call for it.  The best reason to use MP is to bond together
the two B channels in Basic-Rate ISDN or two or more B channels in
Primary-Rate ISDN.  Those links tend to be tightly controlled in terms
of delay and error rate, so they work well for MP.

And not many people ended up caring too much about ISDN.  (Yeah, I know
there are still aficionados out there ...)

If because the segmentation/reassembly process in MP works only as well
as the worst-performing link, it tends to be an overall loss if you have
links that differ.

A better plan is to use multipath IP routes -- ECMP across multiple
independent ordinary (non-MP) PPP links -- if you have links that differ.

-- 
James Carlson         42.703N 71.076W         <carlsonj@workingcode.com>