bpf_csum_diff - R3 offset is outside of the packet

bpf_csum_diff - R3 offset is outside of the packet

[Francesco Ruta (fruta)]

Hello,

I am trying to use bpf_csum_diff() to recompute a corrupted checksum in XDP; unfortunately, in this specific case, I am unable to just perform an incremental update.
I understand that the verifier will reject code that uses dynamic length, but even the stripped-down test shown below is rejected.
I tried directly in assembly adding extra range checks for the r3 register to placate the most phobic verifier to no avail.
Is there any workaround -or any alternative?
BR
F.

SEC("test")
int intercept(struct xdp_md *ctx) {
    void * data = (void*)(long)ctx->data;
    void * dataEnd = (void*)(long)ctx->data_end;
    void * dataPnt = data;
    if (dataPnt >= dataEnd) {
        return XDP_ABORTED;
    }
    __u32 dataLen = (__be32 *)dataEnd - (__be32 *)dataPnt;
    if (dataLen > sizeof(__be32)){
        (void)bpf_csum_diff(0, 0, (__be32 *)dataPnt, sizeof(__be32), 0);
   }
    return XDP_PASS;
}
char _license[] SEC("license") = "GPL";

Prog section 'test' rejected: Permission denied (13)!
- Type:         6
- Instructions: 18 (0 over limit)
- License:      GPL

Verifier analysis:

0: (b7) r6 = 1
1: (61) r2 = *(u32 *)(r1 +4)
2: (61) r3 = *(u32 *)(r1 +0)
3: (3d) if r3 >= r2 goto pc+12
R1=ctx(id=0,off=0,imm=0) R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=0,imm=0) R6=inv1 R10=fp0,call_-1
4: (1f) r2 -= r3
5: (18) r1 = 0x3fffffffc
7: (5f) r2 &= r1
8: (b7) r6 = 2
9: (b7) r1 = 17
10: (2d) if r1 > r2 goto pc+5
R1=inv17 R2=inv(id=0,umin_value=17,umax_value=17179869180,var_off=(0x0; 0x3fffffffc)) R3=pkt(id=0,off=0,r=0,imm=0) R6=inv2 R10=fp0,call_-1
11: (b7) r1 = 0
12: (b7) r2 = 0
13: (b7) r4 = 4
14: (b7) r5 = 0
15: (85) call bpf_csum_diff#28
invalid access to packet, off=0 size=4, R3(id=0,off=0,r=0)
R3 offset is outside of the packet

uname -sr
Linux 5.0.0-1022-gke

Re: bpf_csum_diff - R3 offset is outside of the packet

[Anton Protopopov]

Hi Francesco,

пт, 6 дек. 2019 г. в 19:12, Francesco Ruta (fruta) <fruta@cisco.com>:
>
> Hello,
>
> I am trying to use bpf_csum_diff() to recompute a corrupted checksum in XDP; unfortunately, in this specific case, I am unable to just perform an incremental update.
> I understand that the verifier will reject code that uses dynamic length, but even the stripped-down test shown below is rejected.
> I tried directly in assembly adding extra range checks for the r3 register to placate the most phobic verifier to no avail.
> Is there any workaround -or any alternative?
> BR
> F.
>
> SEC("test")
> int intercept(struct xdp_md *ctx) {
>     void * data = (void*)(long)ctx->data;
>     void * dataEnd = (void*)(long)ctx->data_end;
>     void * dataPnt = data;
>     if (dataPnt >= dataEnd) {
>         return XDP_ABORTED;
>     }
>     __u32 dataLen = (__be32 *)dataEnd - (__be32 *)dataPnt;
>     if (dataLen > sizeof(__be32)){
>         (void)bpf_csum_diff(0, 0, (__be32 *)dataPnt, sizeof(__be32), 0);
>    }
>     return XDP_PASS;
> }
> char _license[] SEC("license") = "GPL";
>
> Prog section 'test' rejected: Permission denied (13)!
> - Type:         6
> - Instructions: 18 (0 over limit)
> - License:      GPL
>
> Verifier analysis:
>
> 0: (b7) r6 = 1
> 1: (61) r2 = *(u32 *)(r1 +4)
> 2: (61) r3 = *(u32 *)(r1 +0)
> 3: (3d) if r3 >= r2 goto pc+12
> R1=ctx(id=0,off=0,imm=0) R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=0,imm=0) R6=inv1 R10=fp0,call_-1
> 4: (1f) r2 -= r3
> 5: (18) r1 = 0x3fffffffc
> 7: (5f) r2 &= r1
> 8: (b7) r6 = 2
> 9: (b7) r1 = 17
> 10: (2d) if r1 > r2 goto pc+5
> R1=inv17 R2=inv(id=0,umin_value=17,umax_value=17179869180,var_off=(0x0; 0x3fffffffc)) R3=pkt(id=0,off=0,r=0,imm=0) R6=inv2 R10=fp0,call_-1
> 11: (b7) r1 = 0
> 12: (b7) r2 = 0
> 13: (b7) r4 = 4
> 14: (b7) r5 = 0
> 15: (85) call bpf_csum_diff#28
> invalid access to packet, off=0 size=4, R3(id=0,off=0,r=0)
> R3 offset is outside of the packet
>
> uname -sr
> Linux 5.0.0-1022-gke

You need to check pointer boundaries, try something like this:

SEC("test")
int intercept(struct xdp_md *ctx) {
    void * data = (void*)(long)ctx->data;
    void * data_end = (void*)(long)ctx->data_end;
    const int N = sizeof(__be32);

    if (data >= data_end)
        return XDP_ABORTED;

    if (data + N <= data_end) /* lets verifier to know that
data[0,...,N-1] is valid */
        bpf_csum_diff(0, 0, data, N, 0);

    return XDP_PASS;
}

RE: bpf_csum_diff - R3 offset is outside of the packet

[Francesco Ruta (fruta)]

Anton,

Thank you for your suggestion. Albeit that would work for the example, it doesn't seem to work for my original problem where the length of the buffer is computed dynamically.

Thank you again
Francesco

-----Original Message-----
From: Anton Protopopov <aspsk2@gmail.com> 
Sent: Saturday, December 7, 2019 7:16 AM
To: Francesco Ruta (fruta) <fruta@cisco.com>
Cc: xdp-newbies@vger.kernel.org
Subject: Re: bpf_csum_diff - R3 offset is outside of the packet

Hi Francesco,

пт, 6 дек. 2019 г. в 19:12, Francesco Ruta (fruta) <fruta@cisco.com>:
> Hello,
>
> I am trying to use bpf_csum_diff() to recompute a corrupted checksum in XDP; unfortunately, in this specific case, I am unable to just perform an incremental update.
> I understand that the verifier will reject code that uses dynamic length, but even the stripped-down test shown below is rejected.
> I tried directly in assembly adding extra range checks for the r3 register to placate the most phobic verifier to no avail.
> Is there any workaround -or any alternative?
> BR
> F.
>
> SEC("test")
> int intercept(struct xdp_md *ctx) {
>     void * data = (void*)(long)ctx->data;
>     void * dataEnd = (void*)(long)ctx->data_end;
>     void * dataPnt = data;
>     if (dataPnt >= dataEnd) {
>         return XDP_ABORTED;
>     }
>     __u32 dataLen = (__be32 *)dataEnd - (__be32 *)dataPnt;
>     if (dataLen > sizeof(__be32)){
>         (void)bpf_csum_diff(0, 0, (__be32 *)dataPnt, sizeof(__be32), 0);
>    }
>     return XDP_PASS;
> }
> char _license[] SEC("license") = "GPL";
>
> Prog section 'test' rejected: Permission denied (13)!
> - Type:         6
> - Instructions: 18 (0 over limit)
> - License:      GPL
>
> Verifier analysis:
>
> 0: (b7) r6 = 1
> 1: (61) r2 = *(u32 *)(r1 +4)
> 2: (61) r3 = *(u32 *)(r1 +0)
> 3: (3d) if r3 >= r2 goto pc+12
> R1=ctx(id=0,off=0,imm=0) R2=pkt_end(id=0,off=0,imm=0) 
> R3=pkt(id=0,off=0,r=0,imm=0) R6=inv1 R10=fp0,call_-1
> 4: (1f) r2 -= r3
> 5: (18) r1 = 0x3fffffffc
> 7: (5f) r2 &= r1
> 8: (b7) r6 = 2
> 9: (b7) r1 = 17
> 10: (2d) if r1 > r2 goto pc+5
> R1=inv17 
> R2=inv(id=0,umin_value=17,umax_value=17179869180,var_off=(0x0; 
> 0x3fffffffc)) R3=pkt(id=0,off=0,r=0,imm=0) R6=inv2 R10=fp0,call_-1
> 11: (b7) r1 = 0
> 12: (b7) r2 = 0
> 13: (b7) r4 = 4
> 14: (b7) r5 = 0
> 15: (85) call bpf_csum_diff#28
> invalid access to packet, off=0 size=4, R3(id=0,off=0,r=0)
> R3 offset is outside of the packet
>
> uname -sr
> Linux 5.0.0-1022-gke

You need to check pointer boundaries, try something like this:

SEC("test")
int intercept(struct xdp_md *ctx) {
    void * data = (void*)(long)ctx->data;
    void * data_end = (void*)(long)ctx->data_end;
    const int N = sizeof(__be32);

    if (data >= data_end)
        return XDP_ABORTED;

    if (data + N <= data_end) /* lets verifier to know that data[0,...,N-1] is valid */
        bpf_csum_diff(0, 0, data, N, 0);

    return XDP_PASS;
}