PPP compression

PPP compression

[arthurpaulino]

Hello,

I saw this e-mall address at http://ppp.samba.org/resources.html.
We do have some questions about ppp. About ppp compression, to be more
specific.

We need to decompress the payload of some packets to verify if they are
encrypted on Android devices.

We configured a PPTP VPN server with no encryption. The compression
algorithm negotiated during the connection phase is Deflate.

For instance, in these packets

19:59:09.883213 IP 191.10.97.74 > 200.129.43.196: GREv1, call 1152, seq
106, length 51: compressed PPP data
	0x0000:  4500 0047 cc42 4000 402f 59ab bf0a 614a
	0x0010:  c881 2bc4 3001 880b 0027 0480 0000 006a
	0x0020:  ff03 00fd 005c 02f7 1e3f 8327 ce85 cfc1
	0x0030:  13e8 4b86 80ff 8d1c 6eb0 dee3 7f23 e4de
	0x0040:  e3ef 4258 ef11 00
19:59:09.884437 IP 191.10.97.74 > 200.129.43.196: GREv1, call 1152, seq
107, length 43: compressed PPP data
	0x0000:  4500 003f cc43 4000 402f 59b2 bf0a 614a
	0x0010:  c881 2bc4 3001 880b 001f 0480 0000 006b
	0x0020:  ff03 00fd 005d 026b 5e04 4edd 294b e09a
	0x0030:  5f31 04e8 0547 0ac0 34d7 4fc3 ae19 00
19:59:09.885609 IP 191.10.97.74 > 200.129.43.196: GREv1, call 1152, seq
108, length 44: compressed PPP data
	0x0000:  4500 0040 cc44 4000 402f 59b0 bf0a 614a
	0x0010:  c881 2bc4 3001 880b 0020 0480 0000 006c
	0x0020:  ff03 00fd 005e 026b 3e06 ae27 1c56 c035
	0x0030:  bf66 0858 c168 ab03 d3ac 781a bb66 0000

We figured out that hex FD means that the payload is compressed. Then
comes a coherency counter (fd 005c, fd 005d, fd 005e). And then, the
compressed payload itself.

Is there a way to call the deflate algorithm implemented on the linux
kernel? Are we following the right path? We're trying to avoid
implementing
Deflate from zero.

Thank you

Re: PPP compression

[James Carlson]

On 12/18/14 15:49, arthurpaulino wrote:
> We configured a PPTP VPN server with no encryption. The compression
> algorithm negotiated during the connection phase is Deflate.

The simplest method (of course) would be to disable compression and just
avoid the issue.

But assuming you don't want to do that, there are two other options:

- If you're actually running pppd, you could use the "record" option
  to record raw data and then dump using pppdump.  That tool has a
  "-d" option to decompress this kind of data.

- If you're not running pppd, and have just GRE-encapsulated data
  captured off the network, then I suggest writing a small tool that
  will extract the PPP frames (ff 03 ...) and rewrite them to a file
  in the very simple format that pppdump expects so that you can use
  pppdump to display them.

The file format can be deduced by looking at the code in pppdump.c.
For a really trivial converter (payload only), you can use:

  01 LL LL ...   - sent data, where LLLL is the number of bytes in
                   big endian (network byte order) format.
  02 LL LL ...   - received data, same format.

Note that pppdump expects the data to be AHDLC encoded (!) so you
have to do that before doing the encoding above.  AHDLC is pretty
simple; replace any instance of 7D or 7E in the data with 7D 5D or 7D 5E
(respectively) and then end the frame with 7E.

-- 
James Carlson         42.703N 71.076W         <carlsonj@workingcode.com>

Re: PPP compression

[Arthur Paulino]

[-- Attachment #1: Type: text/plain, Size: 2003 bytes --]

Hi,

I tried to use pppdump with -p and -d. The input file I used was the 
pcap file (packets.pcap) generated by tcpdump.
Unfortunately, pppdump prints almost the exact payload as the compressed 
ppp data, meaning it's not decompressing the payload when it should be.

I attached:
-the original pcap file (packets.pcap);
-a file containing the payload in hex (HEX_packets.txt);
-a readable text file containing the packets (packets.txt)
-and the pppdump_output.txt

I don't know what I'm doing wrong. Could you help me out?

On 18/12/2014 18:24, James Carlson wrote:
> On 12/18/14 15:49, arthurpaulino wrote:
>> We configured a PPTP VPN server with no encryption. The compression
>> algorithm negotiated during the connection phase is Deflate.
> The simplest method (of course) would be to disable compression and just
> avoid the issue.
>
> But assuming you don't want to do that, there are two other options:
>
> - If you're actually running pppd, you could use the "record" option
>    to record raw data and then dump using pppdump.  That tool has a
>    "-d" option to decompress this kind of data.
>
> - If you're not running pppd, and have just GRE-encapsulated data
>    captured off the network, then I suggest writing a small tool that
>    will extract the PPP frames (ff 03 ...) and rewrite them to a file
>    in the very simple format that pppdump expects so that you can use
>    pppdump to display them.
>
> The file format can be deduced by looking at the code in pppdump.c.
> For a really trivial converter (payload only), you can use:
>
>    01 LL LL ...   - sent data, where LLLL is the number of bytes in
>                     big endian (network byte order) format.
>    02 LL LL ...   - received data, same format.
>
> Note that pppdump expects the data to be AHDLC encoded (!) so you
> have to do that before doing the encoding above.  AHDLC is pretty
> simple; replace any instance of 7D or 7E in the data with 7D 5D or 7D 5E
> (respectively) and then end the frame with 7E.
>


[-- Attachment #2: packets.zip --]
[-- Type: application/x-zip-compressed, Size: 12722 bytes --]

Re: PPP compression

[James Carlson]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

On 12/19/14 12:28, Arthur Paulino wrote:
> I tried to use pppdump with -p and -d. The input file I used was the
> pcap file (packets.pcap) generated by tcpdump.

As I said, pppdump and pcap formats are not at all the same.  You'll
have to convert to go this route.

Attached is a quick-and-dirty program I wrote to convert from the Linux
libpcap variant and PPTP encapsulation you seem to be using and simple
pppdump format.  I didn't bother with timestamps or other bits.  Maybe
it'll work for you.

-- 
James Carlson         42.703N 71.076W         <carlsonj@workingcode.com>

[-- Attachment #2: pcap2pppd.c --]
[-- Type: text/x-csrc, Size: 8725 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <inttypes.h>
#include <netinet/in.h>

#define MAGIC 0xa1b2c3d4
#define RMAGIC 0xd4c3b2a1

#define ETHERNET 1
#define LINUX_SLL 113

#define ESCAPED(x)	((x) == 0x7D || (x) == 0x7E)
#define PUTESC(x, o)	\
	if (ESCAPED(x)) { \
		putc(0x7D, (o)); \
		putc((x) ^ 0x20, (o)); \
	} else { \
		putc((x), (o)); \
	}

typedef struct pcap_hdr_s {
        uint32_t magic_number;   /* magic number */
        uint16_t version_major;  /* major version number */
        uint16_t version_minor;  /* minor version number */
        int32_t  thiszone;       /* GMT to local correction */
        uint32_t sigfigs;        /* accuracy of timestamps */
        uint32_t snaplen;        /* max length of captured packets, in octets */
        uint32_t network;        /* data link type */
} pcap_hdr_t;

typedef struct pcaprec_hdr_s {
	uint32_t ts_sec;
	uint32_t ts_usec;
	uint32_t incl_len;
	uint32_t orig_len;
} pcaprec_hdr_t;

typedef struct linux_sll_s {
	uint16_t packettype;	/* big endian */
	uint16_t arphrd;	/* big endian */
	uint16_t ll_addr_len;	/* big endian */
	uint8_t ll_addr[8];
	uint16_t protocoltype;	/* big endian */
} linux_sll_t;

#define PROTO_IP 0x0800
#define PROTO_PPP 0x880B

typedef enum {
	to_us = 0,
	bcast_by_other,
	mcast_by_other,
	other_to_other,
	from_us
} ptype_t;

static u_short fcstab[256] = {
	0x0000,	0x1189,	0x2312,	0x329b,	0x4624,	0x57ad,	0x6536,	0x74bf,
	0x8c48,	0x9dc1,	0xaf5a,	0xbed3,	0xca6c,	0xdbe5,	0xe97e,	0xf8f7,
	0x1081,	0x0108,	0x3393,	0x221a,	0x56a5,	0x472c,	0x75b7,	0x643e,
	0x9cc9,	0x8d40,	0xbfdb,	0xae52,	0xdaed,	0xcb64,	0xf9ff,	0xe876,
	0x2102,	0x308b,	0x0210,	0x1399,	0x6726,	0x76af,	0x4434,	0x55bd,
	0xad4a,	0xbcc3,	0x8e58,	0x9fd1,	0xeb6e,	0xfae7,	0xc87c,	0xd9f5,
	0x3183,	0x200a,	0x1291,	0x0318,	0x77a7,	0x662e,	0x54b5,	0x453c,
	0xbdcb,	0xac42,	0x9ed9,	0x8f50,	0xfbef,	0xea66,	0xd8fd,	0xc974,
	0x4204,	0x538d,	0x6116,	0x709f,	0x0420,	0x15a9,	0x2732,	0x36bb,
	0xce4c,	0xdfc5,	0xed5e,	0xfcd7,	0x8868,	0x99e1,	0xab7a,	0xbaf3,
	0x5285,	0x430c,	0x7197,	0x601e,	0x14a1,	0x0528,	0x37b3,	0x263a,
	0xdecd,	0xcf44,	0xfddf,	0xec56,	0x98e9,	0x8960,	0xbbfb,	0xaa72,
	0x6306,	0x728f,	0x4014,	0x519d,	0x2522,	0x34ab,	0x0630,	0x17b9,
	0xef4e,	0xfec7,	0xcc5c,	0xddd5,	0xa96a,	0xb8e3,	0x8a78,	0x9bf1,
	0x7387,	0x620e,	0x5095,	0x411c,	0x35a3,	0x242a,	0x16b1,	0x0738,
	0xffcf,	0xee46,	0xdcdd,	0xcd54,	0xb9eb,	0xa862,	0x9af9,	0x8b70,
	0x8408,	0x9581,	0xa71a,	0xb693,	0xc22c,	0xd3a5,	0xe13e,	0xf0b7,
	0x0840,	0x19c9,	0x2b52,	0x3adb,	0x4e64,	0x5fed,	0x6d76,	0x7cff,
	0x9489,	0x8500,	0xb79b,	0xa612,	0xd2ad,	0xc324,	0xf1bf,	0xe036,
	0x18c1,	0x0948,	0x3bd3,	0x2a5a,	0x5ee5,	0x4f6c,	0x7df7,	0x6c7e,
	0xa50a,	0xb483,	0x8618,	0x9791,	0xe32e,	0xf2a7,	0xc03c,	0xd1b5,
	0x2942,	0x38cb,	0x0a50,	0x1bd9,	0x6f66,	0x7eef,	0x4c74,	0x5dfd,
	0xb58b,	0xa402,	0x9699,	0x8710,	0xf3af,	0xe226,	0xd0bd,	0xc134,
	0x39c3,	0x284a,	0x1ad1,	0x0b58,	0x7fe7,	0x6e6e,	0x5cf5,	0x4d7c,
	0xc60c,	0xd785,	0xe51e,	0xf497,	0x8028,	0x91a1,	0xa33a,	0xb2b3,
	0x4a44,	0x5bcd,	0x6956,	0x78df,	0x0c60,	0x1de9,	0x2f72,	0x3efb,
	0xd68d,	0xc704,	0xf59f,	0xe416,	0x90a9,	0x8120,	0xb3bb,	0xa232,
	0x5ac5,	0x4b4c,	0x79d7,	0x685e,	0x1ce1,	0x0d68,	0x3ff3,	0x2e7a,
	0xe70e,	0xf687,	0xc41c,	0xd595,	0xa12a,	0xb0a3,	0x8238,	0x93b1,
	0x6b46,	0x7acf,	0x4854,	0x59dd,	0x2d62,	0x3ceb,	0x0e70,	0x1ff9,
	0xf78f,	0xe606,	0xd49d,	0xc514,	0xb1ab,	0xa022,	0x92b9,	0x8330,
	0x7bc7,	0x6a4e,	0x58d5,	0x495c,	0x3de3,	0x2c6a,	0x1ef1,	0x0f78
};

static void
debug(const char *fmt, ...)
{
#if 0
	va_list ap;

	va_start(ap, fmt);
	vprintf(fmt, ap);
	va_end(ap);
#endif
}

static uint32_t
readu32(uint32_t *vptr, int reversed)
{
	uint32_t value = *vptr;

	if (reversed)
		value = (value << 24) | ((value << 8) & 0xff0000) |
			((value >> 8) & 0xff00) | (value >> 24);
	return value;
}

static void
process(FILE *fin, FILE *fout, char *filename)
{
	pcap_hdr_t hdr;
	size_t rlen, slen;
	int reversed;
	uint32_t network;
	uint8_t *packet;
	uint32_t addr_from = 0; /* arbitrary to/from */

	rlen = fread(&hdr, 1, sizeof (hdr), fin);
	if (rlen < sizeof (hdr)) {
		fprintf(stderr, "pcap2pppd: missing header on %s\n", filename);
		return;
	}
	if (hdr.magic_number == MAGIC) {
		reversed = 0;
	} else if (hdr.magic_number == RMAGIC) {
		reversed = 1;
	} else {
		fprintf(stderr, "pcap2pppd: bad magic number on %s\n", filename);
		return;
	}
	network = readu32(&hdr.network, reversed);
	if (network != LINUX_SLL) {
		fprintf(stderr, "pcap2pppd: unsupported network type %d in %s\n", network, filename);
		return;
	}
	slen = readu32(&hdr.snaplen, reversed);
	if (slen < 1024)
		slen = 1024;
	else if (slen > 65536)
		slen = 65536;
	packet = malloc(slen);
	for (;;) {
		struct {
			pcaprec_hdr_t pcaprec;
			linux_sll_t sll;
		} phdr;
		uint32_t plen;
		int hlen, i;
		uint16_t gtype, ilen, fcs;
		uint32_t ipaddr;

		rlen = fread(&phdr, 1, sizeof (phdr), fin);
		if (rlen < sizeof (phdr))
			break;
		plen = readu32(&phdr.pcaprec.incl_len, reversed);
		if (plen > slen)
			plen = slen;
		/* pcap's packet length includes the Linux SLL header */
		if (plen < sizeof (phdr.sll)) {
			fprintf(stderr, "pcap2pppd: packet without SLL in %s\n", filename);
			break;
		}
		plen -= sizeof (phdr.sll);
		/* now read the actual payload from the file */
		rlen = fread(packet, 1, plen, fin);
		if (rlen < plen) {
			fprintf(stderr, "pcap2pppd: truncated packet at end of %s\n", filename);
			break;
		}
		if (ntohs(phdr.sll.protocoltype) != PROTO_IP) {
			debug("ignore sll proto %d\n",
			      ntohs(phdr.sll.protocoltype));
			continue;
		}
		/* check for IPv4 */
		if ((packet[0] & 0xF0) != 0x40) {
			debug("ignore bad IP type %02x\n", packet[0]);
			continue;
		}
		hlen = (packet[0] & 0xF) << 2;
		if (hlen < 20 || hlen > rlen) {
			debug("ignore bad IP header len %d\n", hlen);
			continue;
		}
		ilen = (packet[2] << 8) + packet[3];
		if (hlen > ilen) {
			debug("ignore bad IP header len %d\n", hlen);
			continue;
		}
		if (ilen > rlen)
			ilen = rlen;
		ipaddr = (packet[12] << 24) | (packet[13] << 16) |
			(packet[14] << 8) | packet[15];
		if (addr_from == 0) {
			debug("arbitrarily choosing %08x as sender.\n", ipaddr);
			addr_from = ipaddr;
		}
		/* check for some form of GRE */
		if (packet[9] != 0x2F) {
			debug("ignore non-GRE protocol %02x\n", packet[9]);
			continue;
		}
		if (ilen < hlen + 2) {
			debug("ignore GRE missing header (%d < %d)\n",
			      ilen, hlen + 2);
			continue;
		}
		/* check for PPTP's non-standard GRE */
		if (!(packet[hlen] & 0x20) || (packet[hlen + 1] & 3) != 1) {
			debug("ignore non-PPTP GRE\n");
			continue;
		}
		gtype = (packet[hlen + 2] << 8) | packet[hlen + 3];
		if (gtype != PROTO_PPP) {
			debug("ignore unexpected GRE payload type %04x\n",
			      gtype);
			continue;
		}
		/* This next line would be correct for real GRE. */
		/* hlen += (packet[hlen] & 0x80) ? 8 : 4; */
		/* This is for Microsoft's weird PPTP GRE. */
		hlen += 8 + ((packet[hlen] & 0x10) ? 4 : 0) +
			((packet[hlen + 1] & 0x80) ? 4 : 0);
		if (ilen <= hlen) {
			debug("ignore GRE without payload\n");
			continue;
		}
		/* Encode the remainder */
		if (phdr.sll.packettype == from_us) {
			putc(0x01, fout);
		} else if (phdr.sll.packettype == to_us) {
			putc(0x02, fout);
		} else {
			putc((ipaddr == addr_from ? 0x01 : 0x02), fout);
		}
		plen = ilen - hlen;
		debug("encoding packet of length %u\n", plen);
		/* first, scan for escapes and add them up */
		fcs = 0xFFFF;
		for (i = hlen; i < ilen; i++) {
			if (ESCAPED(packet[i]))
				plen++;
			fcs = (fcs >> 8) ^ fcstab[(fcs ^ packet[i]) & 0xFF];
		}
		fcs ^= 0xFFFF;
		/* then account for the final FCS and 0x7E */
		plen += 3;
		if (ESCAPED((fcs >> 8) & 0xFF))
			plen++;
		if (ESCAPED(fcs & 0xFF))
			plen++;
		putc((plen >> 8) & 0xFF, fout);
		putc(plen & 0xFF, fout);
		while (hlen < ilen) {
			PUTESC(packet[hlen], fout);
			hlen++;
		}
		PUTESC(fcs & 0xFF, fout);
		PUTESC((fcs >> 8) & 0xFF, fout);
		putc(0x7E, fout);
	}
	free(packet);
}

int
main(int argc, char **argv)
{
	int retv = -1;

	if (argc <= 1) {
		process(stdin, stdout, "stdin");
		retv = 0;
	} else {
		char *filename;
		while ((filename = *++argv) != NULL) {
			FILE *fin = fopen(filename, "r");
			FILE *fout;

			if (fin == NULL) {
				perror(filename);
			} else {
				char *cp = strstr(filename, ".pcap");
				size_t slen = strlen(filename);
				char *oname;

				if (cp != NULL)
					slen = cp - filename;
				oname = malloc(slen + 9);
				strncpy(oname, filename, slen);
				strcpy(oname + slen, ".pppdump");
				fout = fopen(oname, "w");
				if (fout == NULL) {
					perror(oname);
				} else {
					process(fin, fout, filename);
					retv = 0;
					fclose(fout);
				}
				fclose(fin);
				free(oname);
			}
		}
	}
	return retv;
}

Re: PPP compression

[Michael Richardson]

James Carlson <carlsonj@workingcode.com> wrote:
    >> I tried to use pppdump with -p and -d. The input file I used was the
    >> pcap file (packets.pcap) generated by tcpdump.

    > As I said, pppdump and pcap formats are not at all the same.  You'll
    > have to convert to go this route.

    > Attached is a quick-and-dirty program I wrote to convert from the Linux
    > libpcap variant and PPTP encapsulation you seem to be using and simple
    > pppdump format.  I didn't bother with timestamps or other bits.  Maybe
    > it'll work for you.

Do you think it would be worth having a tcpdump DLT type for pppdump stuff,
and just wrap it all up into a pcap container?

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        | network architect  [
]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [

Re: PPP compression

[James Carlson]

On 12/21/14 13:37, Michael Richardson wrote:
> James Carlson <carlsonj@workingcode.com> wrote:
>     >> I tried to use pppdump with -p and -d. The input file I used was the
>     >> pcap file (packets.pcap) generated by tcpdump.
> 
>     > As I said, pppdump and pcap formats are not at all the same.  You'll
>     > have to convert to go this route.
> 
>     > Attached is a quick-and-dirty program I wrote to convert from the Linux
>     > libpcap variant and PPTP encapsulation you seem to be using and simple
>     > pppdump format.  I didn't bother with timestamps or other bits.  Maybe
>     > it'll work for you.
> 
> Do you think it would be worth having a tcpdump DLT type for pppdump stuff,
> and just wrap it all up into a pcap container?

It's an interesting idea, and I could see how there might be cases where
it may even be helpful, but I think it's the wrong thing to do.

These are fundamentally different things.  The pppdump stuff, at its
heart, is really an asynchronous serial recording mechanism.  It doesn't
know anything about packets or frames or other such L2 nonsense.  It
just records bytes going in and out in a format that can preserve
interesting information such as timing.

The original purpose of it was to record low-level async data to reveal
communications problems that caused AHDLC to go off the rails.  You
would not (ordinarily) want to use it to debug some sort of PPP, IP, or
higher-level problem.  Instead, you'd use the existing raw packet
mechanisms to record the packets.

In fact, when you're using pppd's "record" option (which saves data in
the pppdump format), it actually forks up a separate relay process and
runs PPP over a pty pair instead of the user's specified serial port.
That relay process is what saves the data while copying data back and
forth between the serial port and the pty.  It's a brutal trip back and
forth in and out of the kernel multiple times on each packet, and the
original data are (unfortunately) sometimes mangled in the process due
to pty and serial port issues.

It would probably be nice to have some sort of raw recording mechanism
for AHDLC's in-kernel decoder (and perhaps HDLC frames as well, for PPP
over synchronous lines) so that packet capture tools can get better
low-level data, but I don't think this is the right long-term mechanism.

I suggested pppdump as a way out for this particular case because this
one user wants to decode compressed data, and this is one thing that
pppdump happens to know how to do (albeit in a very limited way).

tcpdump datalink type values are, I think, a higher-level concept.

A better path forward for the original poster's problem would be to
extend the existing wireshark decoders for PPP, which already know about
LCP negotiation and the existence of compressed data, so that they know
how to decode at least the freely-available compressed data formats.  It
looks like a nice little project for someone.  I suppose if I find a
whole bunch of time at some point, I might do it, but don't let that
stop anyone else from trying!

-- 
James Carlson         42.703N 71.076W         <carlsonj@workingcode.com>

Re: PPP compression

[arthurpaulino]

[-- Attachment #1: Type: text/plain, Size: 3881 bytes --]

First, I appreciate all your support.

I tried to convert from .pcap to .pppdump and use pppdump (with -p and -d)
to decompress the data.
But again, the output contains the same data as the compressed packets.
The files are attached.

I have some questions:

1 - I tried to use pppd with the recording option (pppd record
output.pppdump), but all it did was print
"~�}#�!}!}!} }4}"}&} } } } }%}&v��}<}'}"}(}">*~" over and over on the
terminal. Also, no output file was saved. Am I doing something wrong?

2 - How does pppdump know that he has to use deflate for decompression?

On Mon, 22 Dec 2014 07:34:31 -0500, James Carlson
<carlsonj@workingcode.com> wrote:
> On 12/21/14 13:37, Michael Richardson wrote:
>> James Carlson <carlsonj@workingcode.com> wrote:
>>     >> I tried to use pppdump with -p and -d. The input file I used was
>>     >> the
>>     >> pcap file (packets.pcap) generated by tcpdump.
>> 
>>     > As I said, pppdump and pcap formats are not at all the same. 
>>     > You'll
>>     > have to convert to go this route.
>> 
>>     > Attached is a quick-and-dirty program I wrote to convert from the
>>     > Linux
>>     > libpcap variant and PPTP encapsulation you seem to be using and
>>     > simple
>>     > pppdump format.  I didn't bother with timestamps or other bits. 
>>     > Maybe
>>     > it'll work for you.
>> 
>> Do you think it would be worth having a tcpdump DLT type for pppdump
>> stuff,
>> and just wrap it all up into a pcap container?
> 
> It's an interesting idea, and I could see how there might be cases where
> it may even be helpful, but I think it's the wrong thing to do.
> 
> These are fundamentally different things.  The pppdump stuff, at its
> heart, is really an asynchronous serial recording mechanism.  It doesn't
> know anything about packets or frames or other such L2 nonsense.  It
> just records bytes going in and out in a format that can preserve
> interesting information such as timing.
> 
> The original purpose of it was to record low-level async data to reveal
> communications problems that caused AHDLC to go off the rails.  You
> would not (ordinarily) want to use it to debug some sort of PPP, IP, or
> higher-level problem.  Instead, you'd use the existing raw packet
> mechanisms to record the packets.
> 
> In fact, when you're using pppd's "record" option (which saves data in
> the pppdump format), it actually forks up a separate relay process and
> runs PPP over a pty pair instead of the user's specified serial port.
> That relay process is what saves the data while copying data back and
> forth between the serial port and the pty.  It's a brutal trip back and
> forth in and out of the kernel multiple times on each packet, and the
> original data are (unfortunately) sometimes mangled in the process due
> to pty and serial port issues.
> 
> It would probably be nice to have some sort of raw recording mechanism
> for AHDLC's in-kernel decoder (and perhaps HDLC frames as well, for PPP
> over synchronous lines) so that packet capture tools can get better
> low-level data, but I don't think this is the right long-term mechanism.
> 
> I suggested pppdump as a way out for this particular case because this
> one user wants to decode compressed data, and this is one thing that
> pppdump happens to know how to do (albeit in a very limited way).
> 
> tcpdump datalink type values are, I think, a higher-level concept.
> 
> A better path forward for the original poster's problem would be to
> extend the existing wireshark decoders for PPP, which already know about
> LCP negotiation and the existence of compressed data, so that they know
> how to decode at least the freely-available compressed data formats.  It
> looks like a nice little project for someone.  I suppose if I find a
> whole bunch of time at some point, I might do it, but don't let that
> stop anyone else from trying!

[-- Attachment #2: packets.zip --]
[-- Type: application/zip, Size: 5425 bytes --]

Re: PPP compression

[James Carlson]

On 12/22/14 13:06, arthurpaulino wrote:
> First, I appreciate all your support.
> 
> I tried to convert from .pcap to .pppdump and use pppdump (with -p and -d)
> to decompress the data.
> But again, the output contains the same data as the compressed packets.
> The files are attached.

Looking at the traces provided, it seems to be just a partial trace, so
there's no way anyone could decode it.

It's important to know that data compression is a stateful operation.
It retains an LZ string dictionary between packets.  As a result, if you
don't have all of the data from the very beginning of the connection,
any attempt to decode it is going to go terribly awry.

For that reason, pppdump won't attempt to decode unless it sees the
initial handshake and CCP negotiation.

That's why I said in the first place that the *SIMPLEST* option to debug
the higher-level data is to disable compression.  Any system built on
non-proprietary standards should work fine with data compression disabled.

> I have some questions:
> 
> 1 - I tried to use pppd with the recording option (pppd record
> output.pppdump), but all it did was print
> "~�}#�!}!}!} }4}"}&} } } } }%}&v��}<}'}"}(}">*~" over and over on the
> terminal. Also, no output file was saved. Am I doing something wrong?

That result indicates that no tty was provided for pppd to use, so it's
falling back to the default -- which is your current tty.  I don't know
what specific options you were using or what issues there are on your
system, so it's hard for me to tell what might be wrong here.

> 2 - How does pppdump know that he has to use deflate for decompression?

It watches the CCP negotiation.  See pppdump.c for details.  It's a
pretty simple program.

For what it's worth, I'd be at least mildly surprised if someone was
running Deflate with PPTP.  That'd be a weird combination of things --
Deflate comes from the open standards world, but PPTP is a Microsoft
proprietary thing that relies on a number of Microsoft proprietary
extensions.

Are you sure the data are compressed with Deflate as originally stated?

Note that pppdump knows only about the open standards -- BSD Compress
and Deflate algorithms.  If this link is using MPPC instead (Microsoft's
proprietary data compression algorithm), then you're sunk.  I don't know
of any simple tools that currently deal with that situation.

Assuming that's the problem, it's possible that there are third party
network analyzers out there that can handle MPPC over PPP over PPTP.  I
haven't researched the state of the art for commercial network
monitoring gear in many years now.

-- 
James Carlson         42.703N 71.076W         <carlsonj@workingcode.com>