* PPP compression @ 2014-12-18 20:49 arthurpaulino 2014-12-18 21:24 ` James Carlson ` (6 more replies) 0 siblings, 7 replies; 8+ messages in thread From: arthurpaulino @ 2014-12-18 20:49 UTC (permalink / raw) To: linux-ppp Hello, I saw this e-mall address at http://ppp.samba.org/resources.html. We do have some questions about ppp. About ppp compression, to be more specific. We need to decompress the payload of some packets to verify if they are encrypted on Android devices. We configured a PPTP VPN server with no encryption. The compression algorithm negotiated during the connection phase is Deflate. For instance, in these packets 19:59:09.883213 IP 191.10.97.74 > 200.129.43.196: GREv1, call 1152, seq 106, length 51: compressed PPP data 0x0000: 4500 0047 cc42 4000 402f 59ab bf0a 614a 0x0010: c881 2bc4 3001 880b 0027 0480 0000 006a 0x0020: ff03 00fd 005c 02f7 1e3f 8327 ce85 cfc1 0x0030: 13e8 4b86 80ff 8d1c 6eb0 dee3 7f23 e4de 0x0040: e3ef 4258 ef11 00 19:59:09.884437 IP 191.10.97.74 > 200.129.43.196: GREv1, call 1152, seq 107, length 43: compressed PPP data 0x0000: 4500 003f cc43 4000 402f 59b2 bf0a 614a 0x0010: c881 2bc4 3001 880b 001f 0480 0000 006b 0x0020: ff03 00fd 005d 026b 5e04 4edd 294b e09a 0x0030: 5f31 04e8 0547 0ac0 34d7 4fc3 ae19 00 19:59:09.885609 IP 191.10.97.74 > 200.129.43.196: GREv1, call 1152, seq 108, length 44: compressed PPP data 0x0000: 4500 0040 cc44 4000 402f 59b0 bf0a 614a 0x0010: c881 2bc4 3001 880b 0020 0480 0000 006c 0x0020: ff03 00fd 005e 026b 3e06 ae27 1c56 c035 0x0030: bf66 0858 c168 ab03 d3ac 781a bb66 0000 We figured out that hex FD means that the payload is compressed. Then comes a coherency counter (fd 005c, fd 005d, fd 005e). And then, the compressed payload itself. Is there a way to call the deflate algorithm implemented on the linux kernel? Are we following the right path? We're trying to avoid implementing Deflate from zero. Thank you ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: PPP compression 2014-12-18 20:49 PPP compression arthurpaulino @ 2014-12-18 21:24 ` James Carlson 2014-12-19 17:28 ` Arthur Paulino ` (5 subsequent siblings) 6 siblings, 0 replies; 8+ messages in thread From: James Carlson @ 2014-12-18 21:24 UTC (permalink / raw) To: linux-ppp On 12/18/14 15:49, arthurpaulino wrote: > We configured a PPTP VPN server with no encryption. The compression > algorithm negotiated during the connection phase is Deflate. The simplest method (of course) would be to disable compression and just avoid the issue. But assuming you don't want to do that, there are two other options: - If you're actually running pppd, you could use the "record" option to record raw data and then dump using pppdump. That tool has a "-d" option to decompress this kind of data. - If you're not running pppd, and have just GRE-encapsulated data captured off the network, then I suggest writing a small tool that will extract the PPP frames (ff 03 ...) and rewrite them to a file in the very simple format that pppdump expects so that you can use pppdump to display them. The file format can be deduced by looking at the code in pppdump.c. For a really trivial converter (payload only), you can use: 01 LL LL ... - sent data, where LLLL is the number of bytes in big endian (network byte order) format. 02 LL LL ... - received data, same format. Note that pppdump expects the data to be AHDLC encoded (!) so you have to do that before doing the encoding above. AHDLC is pretty simple; replace any instance of 7D or 7E in the data with 7D 5D or 7D 5E (respectively) and then end the frame with 7E. -- James Carlson 42.703N 71.076W <carlsonj@workingcode.com> ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: PPP compression 2014-12-18 20:49 PPP compression arthurpaulino 2014-12-18 21:24 ` James Carlson @ 2014-12-19 17:28 ` Arthur Paulino 2014-12-20 21:24 ` James Carlson ` (4 subsequent siblings) 6 siblings, 0 replies; 8+ messages in thread From: Arthur Paulino @ 2014-12-19 17:28 UTC (permalink / raw) To: linux-ppp [-- Attachment #1: Type: text/plain, Size: 2003 bytes --] Hi, I tried to use pppdump with -p and -d. The input file I used was the pcap file (packets.pcap) generated by tcpdump. Unfortunately, pppdump prints almost the exact payload as the compressed ppp data, meaning it's not decompressing the payload when it should be. I attached: -the original pcap file (packets.pcap); -a file containing the payload in hex (HEX_packets.txt); -a readable text file containing the packets (packets.txt) -and the pppdump_output.txt I don't know what I'm doing wrong. Could you help me out? On 18/12/2014 18:24, James Carlson wrote: > On 12/18/14 15:49, arthurpaulino wrote: >> We configured a PPTP VPN server with no encryption. The compression >> algorithm negotiated during the connection phase is Deflate. > The simplest method (of course) would be to disable compression and just > avoid the issue. > > But assuming you don't want to do that, there are two other options: > > - If you're actually running pppd, you could use the "record" option > to record raw data and then dump using pppdump. That tool has a > "-d" option to decompress this kind of data. > > - If you're not running pppd, and have just GRE-encapsulated data > captured off the network, then I suggest writing a small tool that > will extract the PPP frames (ff 03 ...) and rewrite them to a file > in the very simple format that pppdump expects so that you can use > pppdump to display them. > > The file format can be deduced by looking at the code in pppdump.c. > For a really trivial converter (payload only), you can use: > > 01 LL LL ... - sent data, where LLLL is the number of bytes in > big endian (network byte order) format. > 02 LL LL ... - received data, same format. > > Note that pppdump expects the data to be AHDLC encoded (!) so you > have to do that before doing the encoding above. AHDLC is pretty > simple; replace any instance of 7D or 7E in the data with 7D 5D or 7D 5E > (respectively) and then end the frame with 7E. > [-- Attachment #2: packets.zip --] [-- Type: application/x-zip-compressed, Size: 12722 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: PPP compression 2014-12-18 20:49 PPP compression arthurpaulino 2014-12-18 21:24 ` James Carlson 2014-12-19 17:28 ` Arthur Paulino @ 2014-12-20 21:24 ` James Carlson 2014-12-21 18:37 ` Michael Richardson ` (3 subsequent siblings) 6 siblings, 0 replies; 8+ messages in thread From: James Carlson @ 2014-12-20 21:24 UTC (permalink / raw) To: linux-ppp [-- Attachment #1: Type: text/plain, Size: 578 bytes --] On 12/19/14 12:28, Arthur Paulino wrote: > I tried to use pppdump with -p and -d. The input file I used was the > pcap file (packets.pcap) generated by tcpdump. As I said, pppdump and pcap formats are not at all the same. You'll have to convert to go this route. Attached is a quick-and-dirty program I wrote to convert from the Linux libpcap variant and PPTP encapsulation you seem to be using and simple pppdump format. I didn't bother with timestamps or other bits. Maybe it'll work for you. -- James Carlson 42.703N 71.076W <carlsonj@workingcode.com> [-- Attachment #2: pcap2pppd.c --] [-- Type: text/x-csrc, Size: 8725 bytes --] #include <stdio.h> #include <stdlib.h> #include <stdarg.h> #include <string.h> #include <inttypes.h> #include <netinet/in.h> #define MAGIC 0xa1b2c3d4 #define RMAGIC 0xd4c3b2a1 #define ETHERNET 1 #define LINUX_SLL 113 #define ESCAPED(x) ((x) == 0x7D || (x) == 0x7E) #define PUTESC(x, o) \ if (ESCAPED(x)) { \ putc(0x7D, (o)); \ putc((x) ^ 0x20, (o)); \ } else { \ putc((x), (o)); \ } typedef struct pcap_hdr_s { uint32_t magic_number; /* magic number */ uint16_t version_major; /* major version number */ uint16_t version_minor; /* minor version number */ int32_t thiszone; /* GMT to local correction */ uint32_t sigfigs; /* accuracy of timestamps */ uint32_t snaplen; /* max length of captured packets, in octets */ uint32_t network; /* data link type */ } pcap_hdr_t; typedef struct pcaprec_hdr_s { uint32_t ts_sec; uint32_t ts_usec; uint32_t incl_len; uint32_t orig_len; } pcaprec_hdr_t; typedef struct linux_sll_s { uint16_t packettype; /* big endian */ uint16_t arphrd; /* big endian */ uint16_t ll_addr_len; /* big endian */ uint8_t ll_addr[8]; uint16_t protocoltype; /* big endian */ } linux_sll_t; #define PROTO_IP 0x0800 #define PROTO_PPP 0x880B typedef enum { to_us = 0, bcast_by_other, mcast_by_other, other_to_other, from_us } ptype_t; static u_short fcstab[256] = { 0x0000, 0x1189, 0x2312, 0x329b, 0x4624, 0x57ad, 0x6536, 0x74bf, 0x8c48, 0x9dc1, 0xaf5a, 0xbed3, 0xca6c, 0xdbe5, 0xe97e, 0xf8f7, 0x1081, 0x0108, 0x3393, 0x221a, 0x56a5, 0x472c, 0x75b7, 0x643e, 0x9cc9, 0x8d40, 0xbfdb, 0xae52, 0xdaed, 0xcb64, 0xf9ff, 0xe876, 0x2102, 0x308b, 0x0210, 0x1399, 0x6726, 0x76af, 0x4434, 0x55bd, 0xad4a, 0xbcc3, 0x8e58, 0x9fd1, 0xeb6e, 0xfae7, 0xc87c, 0xd9f5, 0x3183, 0x200a, 0x1291, 0x0318, 0x77a7, 0x662e, 0x54b5, 0x453c, 0xbdcb, 0xac42, 0x9ed9, 0x8f50, 0xfbef, 0xea66, 0xd8fd, 0xc974, 0x4204, 0x538d, 0x6116, 0x709f, 0x0420, 0x15a9, 0x2732, 0x36bb, 0xce4c, 0xdfc5, 0xed5e, 0xfcd7, 0x8868, 0x99e1, 0xab7a, 0xbaf3, 0x5285, 0x430c, 0x7197, 0x601e, 0x14a1, 0x0528, 0x37b3, 0x263a, 0xdecd, 0xcf44, 0xfddf, 0xec56, 0x98e9, 0x8960, 0xbbfb, 0xaa72, 0x6306, 0x728f, 0x4014, 0x519d, 0x2522, 0x34ab, 0x0630, 0x17b9, 0xef4e, 0xfec7, 0xcc5c, 0xddd5, 0xa96a, 0xb8e3, 0x8a78, 0x9bf1, 0x7387, 0x620e, 0x5095, 0x411c, 0x35a3, 0x242a, 0x16b1, 0x0738, 0xffcf, 0xee46, 0xdcdd, 0xcd54, 0xb9eb, 0xa862, 0x9af9, 0x8b70, 0x8408, 0x9581, 0xa71a, 0xb693, 0xc22c, 0xd3a5, 0xe13e, 0xf0b7, 0x0840, 0x19c9, 0x2b52, 0x3adb, 0x4e64, 0x5fed, 0x6d76, 0x7cff, 0x9489, 0x8500, 0xb79b, 0xa612, 0xd2ad, 0xc324, 0xf1bf, 0xe036, 0x18c1, 0x0948, 0x3bd3, 0x2a5a, 0x5ee5, 0x4f6c, 0x7df7, 0x6c7e, 0xa50a, 0xb483, 0x8618, 0x9791, 0xe32e, 0xf2a7, 0xc03c, 0xd1b5, 0x2942, 0x38cb, 0x0a50, 0x1bd9, 0x6f66, 0x7eef, 0x4c74, 0x5dfd, 0xb58b, 0xa402, 0x9699, 0x8710, 0xf3af, 0xe226, 0xd0bd, 0xc134, 0x39c3, 0x284a, 0x1ad1, 0x0b58, 0x7fe7, 0x6e6e, 0x5cf5, 0x4d7c, 0xc60c, 0xd785, 0xe51e, 0xf497, 0x8028, 0x91a1, 0xa33a, 0xb2b3, 0x4a44, 0x5bcd, 0x6956, 0x78df, 0x0c60, 0x1de9, 0x2f72, 0x3efb, 0xd68d, 0xc704, 0xf59f, 0xe416, 0x90a9, 0x8120, 0xb3bb, 0xa232, 0x5ac5, 0x4b4c, 0x79d7, 0x685e, 0x1ce1, 0x0d68, 0x3ff3, 0x2e7a, 0xe70e, 0xf687, 0xc41c, 0xd595, 0xa12a, 0xb0a3, 0x8238, 0x93b1, 0x6b46, 0x7acf, 0x4854, 0x59dd, 0x2d62, 0x3ceb, 0x0e70, 0x1ff9, 0xf78f, 0xe606, 0xd49d, 0xc514, 0xb1ab, 0xa022, 0x92b9, 0x8330, 0x7bc7, 0x6a4e, 0x58d5, 0x495c, 0x3de3, 0x2c6a, 0x1ef1, 0x0f78 }; static void debug(const char *fmt, ...) { #if 0 va_list ap; va_start(ap, fmt); vprintf(fmt, ap); va_end(ap); #endif } static uint32_t readu32(uint32_t *vptr, int reversed) { uint32_t value = *vptr; if (reversed) value = (value << 24) | ((value << 8) & 0xff0000) | ((value >> 8) & 0xff00) | (value >> 24); return value; } static void process(FILE *fin, FILE *fout, char *filename) { pcap_hdr_t hdr; size_t rlen, slen; int reversed; uint32_t network; uint8_t *packet; uint32_t addr_from = 0; /* arbitrary to/from */ rlen = fread(&hdr, 1, sizeof (hdr), fin); if (rlen < sizeof (hdr)) { fprintf(stderr, "pcap2pppd: missing header on %s\n", filename); return; } if (hdr.magic_number == MAGIC) { reversed = 0; } else if (hdr.magic_number == RMAGIC) { reversed = 1; } else { fprintf(stderr, "pcap2pppd: bad magic number on %s\n", filename); return; } network = readu32(&hdr.network, reversed); if (network != LINUX_SLL) { fprintf(stderr, "pcap2pppd: unsupported network type %d in %s\n", network, filename); return; } slen = readu32(&hdr.snaplen, reversed); if (slen < 1024) slen = 1024; else if (slen > 65536) slen = 65536; packet = malloc(slen); for (;;) { struct { pcaprec_hdr_t pcaprec; linux_sll_t sll; } phdr; uint32_t plen; int hlen, i; uint16_t gtype, ilen, fcs; uint32_t ipaddr; rlen = fread(&phdr, 1, sizeof (phdr), fin); if (rlen < sizeof (phdr)) break; plen = readu32(&phdr.pcaprec.incl_len, reversed); if (plen > slen) plen = slen; /* pcap's packet length includes the Linux SLL header */ if (plen < sizeof (phdr.sll)) { fprintf(stderr, "pcap2pppd: packet without SLL in %s\n", filename); break; } plen -= sizeof (phdr.sll); /* now read the actual payload from the file */ rlen = fread(packet, 1, plen, fin); if (rlen < plen) { fprintf(stderr, "pcap2pppd: truncated packet at end of %s\n", filename); break; } if (ntohs(phdr.sll.protocoltype) != PROTO_IP) { debug("ignore sll proto %d\n", ntohs(phdr.sll.protocoltype)); continue; } /* check for IPv4 */ if ((packet[0] & 0xF0) != 0x40) { debug("ignore bad IP type %02x\n", packet[0]); continue; } hlen = (packet[0] & 0xF) << 2; if (hlen < 20 || hlen > rlen) { debug("ignore bad IP header len %d\n", hlen); continue; } ilen = (packet[2] << 8) + packet[3]; if (hlen > ilen) { debug("ignore bad IP header len %d\n", hlen); continue; } if (ilen > rlen) ilen = rlen; ipaddr = (packet[12] << 24) | (packet[13] << 16) | (packet[14] << 8) | packet[15]; if (addr_from == 0) { debug("arbitrarily choosing %08x as sender.\n", ipaddr); addr_from = ipaddr; } /* check for some form of GRE */ if (packet[9] != 0x2F) { debug("ignore non-GRE protocol %02x\n", packet[9]); continue; } if (ilen < hlen + 2) { debug("ignore GRE missing header (%d < %d)\n", ilen, hlen + 2); continue; } /* check for PPTP's non-standard GRE */ if (!(packet[hlen] & 0x20) || (packet[hlen + 1] & 3) != 1) { debug("ignore non-PPTP GRE\n"); continue; } gtype = (packet[hlen + 2] << 8) | packet[hlen + 3]; if (gtype != PROTO_PPP) { debug("ignore unexpected GRE payload type %04x\n", gtype); continue; } /* This next line would be correct for real GRE. */ /* hlen += (packet[hlen] & 0x80) ? 8 : 4; */ /* This is for Microsoft's weird PPTP GRE. */ hlen += 8 + ((packet[hlen] & 0x10) ? 4 : 0) + ((packet[hlen + 1] & 0x80) ? 4 : 0); if (ilen <= hlen) { debug("ignore GRE without payload\n"); continue; } /* Encode the remainder */ if (phdr.sll.packettype == from_us) { putc(0x01, fout); } else if (phdr.sll.packettype == to_us) { putc(0x02, fout); } else { putc((ipaddr == addr_from ? 0x01 : 0x02), fout); } plen = ilen - hlen; debug("encoding packet of length %u\n", plen); /* first, scan for escapes and add them up */ fcs = 0xFFFF; for (i = hlen; i < ilen; i++) { if (ESCAPED(packet[i])) plen++; fcs = (fcs >> 8) ^ fcstab[(fcs ^ packet[i]) & 0xFF]; } fcs ^= 0xFFFF; /* then account for the final FCS and 0x7E */ plen += 3; if (ESCAPED((fcs >> 8) & 0xFF)) plen++; if (ESCAPED(fcs & 0xFF)) plen++; putc((plen >> 8) & 0xFF, fout); putc(plen & 0xFF, fout); while (hlen < ilen) { PUTESC(packet[hlen], fout); hlen++; } PUTESC(fcs & 0xFF, fout); PUTESC((fcs >> 8) & 0xFF, fout); putc(0x7E, fout); } free(packet); } int main(int argc, char **argv) { int retv = -1; if (argc <= 1) { process(stdin, stdout, "stdin"); retv = 0; } else { char *filename; while ((filename = *++argv) != NULL) { FILE *fin = fopen(filename, "r"); FILE *fout; if (fin == NULL) { perror(filename); } else { char *cp = strstr(filename, ".pcap"); size_t slen = strlen(filename); char *oname; if (cp != NULL) slen = cp - filename; oname = malloc(slen + 9); strncpy(oname, filename, slen); strcpy(oname + slen, ".pppdump"); fout = fopen(oname, "w"); if (fout == NULL) { perror(oname); } else { process(fin, fout, filename); retv = 0; fclose(fout); } fclose(fin); free(oname); } } } return retv; } ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: PPP compression 2014-12-18 20:49 PPP compression arthurpaulino ` (2 preceding siblings ...) 2014-12-20 21:24 ` James Carlson @ 2014-12-21 18:37 ` Michael Richardson 2014-12-22 12:34 ` James Carlson ` (2 subsequent siblings) 6 siblings, 0 replies; 8+ messages in thread From: Michael Richardson @ 2014-12-21 18:37 UTC (permalink / raw) To: linux-ppp James Carlson <carlsonj@workingcode.com> wrote: >> I tried to use pppdump with -p and -d. The input file I used was the >> pcap file (packets.pcap) generated by tcpdump. > As I said, pppdump and pcap formats are not at all the same. You'll > have to convert to go this route. > Attached is a quick-and-dirty program I wrote to convert from the Linux > libpcap variant and PPTP encapsulation you seem to be using and simple > pppdump format. I didn't bother with timestamps or other bits. Maybe > it'll work for you. Do you think it would be worth having a tcpdump DLT type for pppdump stuff, and just wrap it all up into a pcap container? -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works | network architect [ ] mcr@sandelman.ca http://www.sandelman.ca/ | ruby on rails [ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: PPP compression 2014-12-18 20:49 PPP compression arthurpaulino ` (3 preceding siblings ...) 2014-12-21 18:37 ` Michael Richardson @ 2014-12-22 12:34 ` James Carlson 2014-12-22 18:06 ` arthurpaulino 2014-12-22 19:09 ` James Carlson 6 siblings, 0 replies; 8+ messages in thread From: James Carlson @ 2014-12-22 12:34 UTC (permalink / raw) To: linux-ppp On 12/21/14 13:37, Michael Richardson wrote: > James Carlson <carlsonj@workingcode.com> wrote: > >> I tried to use pppdump with -p and -d. The input file I used was the > >> pcap file (packets.pcap) generated by tcpdump. > > > As I said, pppdump and pcap formats are not at all the same. You'll > > have to convert to go this route. > > > Attached is a quick-and-dirty program I wrote to convert from the Linux > > libpcap variant and PPTP encapsulation you seem to be using and simple > > pppdump format. I didn't bother with timestamps or other bits. Maybe > > it'll work for you. > > Do you think it would be worth having a tcpdump DLT type for pppdump stuff, > and just wrap it all up into a pcap container? It's an interesting idea, and I could see how there might be cases where it may even be helpful, but I think it's the wrong thing to do. These are fundamentally different things. The pppdump stuff, at its heart, is really an asynchronous serial recording mechanism. It doesn't know anything about packets or frames or other such L2 nonsense. It just records bytes going in and out in a format that can preserve interesting information such as timing. The original purpose of it was to record low-level async data to reveal communications problems that caused AHDLC to go off the rails. You would not (ordinarily) want to use it to debug some sort of PPP, IP, or higher-level problem. Instead, you'd use the existing raw packet mechanisms to record the packets. In fact, when you're using pppd's "record" option (which saves data in the pppdump format), it actually forks up a separate relay process and runs PPP over a pty pair instead of the user's specified serial port. That relay process is what saves the data while copying data back and forth between the serial port and the pty. It's a brutal trip back and forth in and out of the kernel multiple times on each packet, and the original data are (unfortunately) sometimes mangled in the process due to pty and serial port issues. It would probably be nice to have some sort of raw recording mechanism for AHDLC's in-kernel decoder (and perhaps HDLC frames as well, for PPP over synchronous lines) so that packet capture tools can get better low-level data, but I don't think this is the right long-term mechanism. I suggested pppdump as a way out for this particular case because this one user wants to decode compressed data, and this is one thing that pppdump happens to know how to do (albeit in a very limited way). tcpdump datalink type values are, I think, a higher-level concept. A better path forward for the original poster's problem would be to extend the existing wireshark decoders for PPP, which already know about LCP negotiation and the existence of compressed data, so that they know how to decode at least the freely-available compressed data formats. It looks like a nice little project for someone. I suppose if I find a whole bunch of time at some point, I might do it, but don't let that stop anyone else from trying! -- James Carlson 42.703N 71.076W <carlsonj@workingcode.com> ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: PPP compression 2014-12-18 20:49 PPP compression arthurpaulino ` (4 preceding siblings ...) 2014-12-22 12:34 ` James Carlson @ 2014-12-22 18:06 ` arthurpaulino 2014-12-22 19:09 ` James Carlson 6 siblings, 0 replies; 8+ messages in thread From: arthurpaulino @ 2014-12-22 18:06 UTC (permalink / raw) To: linux-ppp [-- Attachment #1: Type: text/plain, Size: 3881 bytes --] First, I appreciate all your support. I tried to convert from .pcap to .pppdump and use pppdump (with -p and -d) to decompress the data. But again, the output contains the same data as the compressed packets. The files are attached. I have some questions: 1 - I tried to use pppd with the recording option (pppd record output.pppdump), but all it did was print "~�}#�!}!}!} }4}"}&} } } } }%}&v��}<}'}"}(}">*~" over and over on the terminal. Also, no output file was saved. Am I doing something wrong? 2 - How does pppdump know that he has to use deflate for decompression? On Mon, 22 Dec 2014 07:34:31 -0500, James Carlson <carlsonj@workingcode.com> wrote: > On 12/21/14 13:37, Michael Richardson wrote: >> James Carlson <carlsonj@workingcode.com> wrote: >> >> I tried to use pppdump with -p and -d. The input file I used was >> >> the >> >> pcap file (packets.pcap) generated by tcpdump. >> >> > As I said, pppdump and pcap formats are not at all the same. >> > You'll >> > have to convert to go this route. >> >> > Attached is a quick-and-dirty program I wrote to convert from the >> > Linux >> > libpcap variant and PPTP encapsulation you seem to be using and >> > simple >> > pppdump format. I didn't bother with timestamps or other bits. >> > Maybe >> > it'll work for you. >> >> Do you think it would be worth having a tcpdump DLT type for pppdump >> stuff, >> and just wrap it all up into a pcap container? > > It's an interesting idea, and I could see how there might be cases where > it may even be helpful, but I think it's the wrong thing to do. > > These are fundamentally different things. The pppdump stuff, at its > heart, is really an asynchronous serial recording mechanism. It doesn't > know anything about packets or frames or other such L2 nonsense. It > just records bytes going in and out in a format that can preserve > interesting information such as timing. > > The original purpose of it was to record low-level async data to reveal > communications problems that caused AHDLC to go off the rails. You > would not (ordinarily) want to use it to debug some sort of PPP, IP, or > higher-level problem. Instead, you'd use the existing raw packet > mechanisms to record the packets. > > In fact, when you're using pppd's "record" option (which saves data in > the pppdump format), it actually forks up a separate relay process and > runs PPP over a pty pair instead of the user's specified serial port. > That relay process is what saves the data while copying data back and > forth between the serial port and the pty. It's a brutal trip back and > forth in and out of the kernel multiple times on each packet, and the > original data are (unfortunately) sometimes mangled in the process due > to pty and serial port issues. > > It would probably be nice to have some sort of raw recording mechanism > for AHDLC's in-kernel decoder (and perhaps HDLC frames as well, for PPP > over synchronous lines) so that packet capture tools can get better > low-level data, but I don't think this is the right long-term mechanism. > > I suggested pppdump as a way out for this particular case because this > one user wants to decode compressed data, and this is one thing that > pppdump happens to know how to do (albeit in a very limited way). > > tcpdump datalink type values are, I think, a higher-level concept. > > A better path forward for the original poster's problem would be to > extend the existing wireshark decoders for PPP, which already know about > LCP negotiation and the existence of compressed data, so that they know > how to decode at least the freely-available compressed data formats. It > looks like a nice little project for someone. I suppose if I find a > whole bunch of time at some point, I might do it, but don't let that > stop anyone else from trying! [-- Attachment #2: packets.zip --] [-- Type: application/zip, Size: 5425 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: PPP compression 2014-12-18 20:49 PPP compression arthurpaulino ` (5 preceding siblings ...) 2014-12-22 18:06 ` arthurpaulino @ 2014-12-22 19:09 ` James Carlson 6 siblings, 0 replies; 8+ messages in thread From: James Carlson @ 2014-12-22 19:09 UTC (permalink / raw) To: linux-ppp On 12/22/14 13:06, arthurpaulino wrote: > First, I appreciate all your support. > > I tried to convert from .pcap to .pppdump and use pppdump (with -p and -d) > to decompress the data. > But again, the output contains the same data as the compressed packets. > The files are attached. Looking at the traces provided, it seems to be just a partial trace, so there's no way anyone could decode it. It's important to know that data compression is a stateful operation. It retains an LZ string dictionary between packets. As a result, if you don't have all of the data from the very beginning of the connection, any attempt to decode it is going to go terribly awry. For that reason, pppdump won't attempt to decode unless it sees the initial handshake and CCP negotiation. That's why I said in the first place that the *SIMPLEST* option to debug the higher-level data is to disable compression. Any system built on non-proprietary standards should work fine with data compression disabled. > I have some questions: > > 1 - I tried to use pppd with the recording option (pppd record > output.pppdump), but all it did was print > "~�}#�!}!}!} }4}"}&} } } } }%}&v��}<}'}"}(}">*~" over and over on the > terminal. Also, no output file was saved. Am I doing something wrong? That result indicates that no tty was provided for pppd to use, so it's falling back to the default -- which is your current tty. I don't know what specific options you were using or what issues there are on your system, so it's hard for me to tell what might be wrong here. > 2 - How does pppdump know that he has to use deflate for decompression? It watches the CCP negotiation. See pppdump.c for details. It's a pretty simple program. For what it's worth, I'd be at least mildly surprised if someone was running Deflate with PPTP. That'd be a weird combination of things -- Deflate comes from the open standards world, but PPTP is a Microsoft proprietary thing that relies on a number of Microsoft proprietary extensions. Are you sure the data are compressed with Deflate as originally stated? Note that pppdump knows only about the open standards -- BSD Compress and Deflate algorithms. If this link is using MPPC instead (Microsoft's proprietary data compression algorithm), then you're sunk. I don't know of any simple tools that currently deal with that situation. Assuming that's the problem, it's possible that there are third party network analyzers out there that can handle MPPC over PPP over PPTP. I haven't researched the state of the art for commercial network monitoring gear in many years now. -- James Carlson 42.703N 71.076W <carlsonj@workingcode.com> ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2014-12-22 19:09 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-12-18 20:49 PPP compression arthurpaulino 2014-12-18 21:24 ` James Carlson 2014-12-19 17:28 ` Arthur Paulino 2014-12-20 21:24 ` James Carlson 2014-12-21 18:37 ` Michael Richardson 2014-12-22 12:34 ` James Carlson 2014-12-22 18:06 ` arthurpaulino 2014-12-22 19:09 ` James Carlson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).