* [PATCH net 0/3] skb frag: kmap_atomic fixes
@ 2021-01-08 17:11 Willem de Bruijn
2021-01-08 17:11 ` [PATCH net 1/3] net: support kmap_local forced debugging in skb_frag_foreach Willem de Bruijn
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Willem de Bruijn @ 2021-01-08 17:11 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, Willem de Bruijn
From: Willem de Bruijn <willemb@google.com>
skb frags may be backed by highmem and/or compound pages. Various
code calls kmap_atomic to safely access highmem pages. But this
needs additional care for compound pages. Fix a few issues:
patch 1 expect kmap mappings with CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP
patch 2 fixes kmap_atomic + compound page support in skb_seq_read
patch 3 fixes kmap_atomic + compound page support in esp
Willem de Bruijn (3):
net: support kmap_local forced debugging in skb_frag_foreach
net: compound page support in skb_seq_read
esp: avoid unneeded kmap_atomic call
include/linux/skbuff.h | 3 ++-
net/core/skbuff.c | 28 +++++++++++++++++++++++-----
net/ipv4/esp4.c | 7 +------
net/ipv6/esp6.c | 7 +------
4 files changed, 27 insertions(+), 18 deletions(-)
--
2.30.0.284.gd98b1dd5eaa7-goog
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH net 1/3] net: support kmap_local forced debugging in skb_frag_foreach
2021-01-08 17:11 [PATCH net 0/3] skb frag: kmap_atomic fixes Willem de Bruijn
@ 2021-01-08 17:11 ` Willem de Bruijn
2021-01-08 17:11 ` [PATCH net 2/3] net: compound page support in skb_seq_read Willem de Bruijn
2021-01-08 17:11 ` [PATCH net 3/3] esp: avoid unneeded kmap_atomic call Willem de Bruijn
2 siblings, 0 replies; 6+ messages in thread
From: Willem de Bruijn @ 2021-01-08 17:11 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, Willem de Bruijn, Steven Rostedt, Linus Torvalds
From: Willem de Bruijn <willemb@google.com>
Skb frags may be backed by highmem and/or compound pages. Highmem
pages need kmap_atomic mappings to access. But kmap_atomic maps a
single page, not the entire compound page.
skb_foreach_page iterates over an skb frag, in one step in the common
case, page by page only if kmap_atomic must be called for each page.
The decision logic is captured in skb_frag_must_loop.
CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP extends kmap from highmem to all
pages, to increase code coverage.
Extend skb_frag_must_loop to this new condition.
Link: https://lore.kernel.org/linux-mm/20210106180132.41dc249d@gandalf.local.home/
Fixes: 0e91a0c6984c ("mm/highmem: Provide CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP")
Reported-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Willem de Bruijn <willemb@google.com>
Tested-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
include/linux/skbuff.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 333bcdc39635..c858adfb5a82 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -366,7 +366,7 @@ static inline void skb_frag_size_sub(skb_frag_t *frag, int delta)
static inline bool skb_frag_must_loop(struct page *p)
{
#if defined(CONFIG_HIGHMEM)
- if (PageHighMem(p))
+ if (IS_ENABLED(CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP) || PageHighMem(p))
return true;
#endif
return false;
--
2.30.0.284.gd98b1dd5eaa7-goog
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH net 2/3] net: compound page support in skb_seq_read
2021-01-08 17:11 [PATCH net 0/3] skb frag: kmap_atomic fixes Willem de Bruijn
2021-01-08 17:11 ` [PATCH net 1/3] net: support kmap_local forced debugging in skb_frag_foreach Willem de Bruijn
@ 2021-01-08 17:11 ` Willem de Bruijn
2021-01-08 23:30 ` Willem de Bruijn
2021-01-09 22:09 ` Willem de Bruijn
2021-01-08 17:11 ` [PATCH net 3/3] esp: avoid unneeded kmap_atomic call Willem de Bruijn
2 siblings, 2 replies; 6+ messages in thread
From: Willem de Bruijn @ 2021-01-08 17:11 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, Willem de Bruijn
From: Willem de Bruijn <willemb@google.com>
skb_seq_read iterates over an skb, returning pointer and length of
the next data range with each call.
It relies on kmap_atomic to access highmem pages when needed.
An skb frag may be backed by a compound page, but kmap_atomic maps
only a single page. There are not enough kmap slots to always map all
pages concurrently.
Instead, if kmap_atomic is needed, iterate over each page.
As this increases the number of calls, avoid this unless needed.
The necessary condition is captured in skb_frag_must_loop.
I tried to make the change as obvious as possible. It should be easy
to verify that nothing changes if skb_frag_must_loop returns false.
Tested:
On an x86 platform with
CONFIG_HIGHMEM=y
CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP=y
CONFIG_NETFILTER_XT_MATCH_STRING=y
Run
ip link set dev lo mtu 1500
iptables -A OUTPUT -m string --string 'badstring' -algo bm -j ACCEPT
dd if=/dev/urandom of=in bs=1M count=20
nc -l -p 8000 > /dev/null &
nc -w 1 -q 0 localhost 8000 < in
Signed-off-by: Willem de Bruijn <willemb@google.com>
---
include/linux/skbuff.h | 1 +
net/core/skbuff.c | 28 +++++++++++++++++++++++-----
2 files changed, 24 insertions(+), 5 deletions(-)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index c858adfb5a82..68ffd3f115c1 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -1203,6 +1203,7 @@ struct skb_seq_state {
struct sk_buff *root_skb;
struct sk_buff *cur_skb;
__u8 *frag_data;
+ __u16 frag_off;
};
void skb_prepare_seq_read(struct sk_buff *skb, unsigned int from,
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index f62cae3f75d8..4acf45154b17 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -3442,6 +3442,7 @@ void skb_prepare_seq_read(struct sk_buff *skb, unsigned int from,
st->root_skb = st->cur_skb = skb;
st->frag_idx = st->stepped_offset = 0;
st->frag_data = NULL;
+ st->frag_off = 0;
}
EXPORT_SYMBOL(skb_prepare_seq_read);
@@ -3496,14 +3497,27 @@ unsigned int skb_seq_read(unsigned int consumed, const u8 **data,
st->stepped_offset += skb_headlen(st->cur_skb);
while (st->frag_idx < skb_shinfo(st->cur_skb)->nr_frags) {
+ unsigned int pg_idx, pg_off, pg_sz;
+
frag = &skb_shinfo(st->cur_skb)->frags[st->frag_idx];
- block_limit = skb_frag_size(frag) + st->stepped_offset;
+ pg_idx = 0;
+ pg_off = skb_frag_off(frag);
+ pg_sz = skb_frag_size(frag);
+
+ if (skb_frag_must_loop(skb_frag_page(frag))) {
+ pg_idx = (pg_off + st->frag_off) >> PAGE_SHIFT;
+ pg_off = offset_in_page(pg_off + st->frag_off);
+ pg_sz = min_t(unsigned int, pg_sz - st->frag_off,
+ PAGE_SIZE - pg_off);
+ }
+
+ block_limit = pg_sz + st->stepped_offset;
if (abs_offset < block_limit) {
if (!st->frag_data)
- st->frag_data = kmap_atomic(skb_frag_page(frag));
+ st->frag_data = kmap_atomic(skb_frag_page(frag) + pg_idx);
- *data = (u8 *) st->frag_data + skb_frag_off(frag) +
+ *data = (u8 *)st->frag_data + pg_off +
(abs_offset - st->stepped_offset);
return block_limit - abs_offset;
@@ -3514,8 +3528,12 @@ unsigned int skb_seq_read(unsigned int consumed, const u8 **data,
st->frag_data = NULL;
}
- st->frag_idx++;
- st->stepped_offset += skb_frag_size(frag);
+ st->stepped_offset += pg_sz;
+ st->frag_off += pg_sz;
+ if (st->frag_off == skb_frag_size(frag)) {
+ st->frag_off = 0;
+ st->frag_idx++;
+ }
}
if (st->frag_data) {
--
2.30.0.284.gd98b1dd5eaa7-goog
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH net 3/3] esp: avoid unneeded kmap_atomic call
2021-01-08 17:11 [PATCH net 0/3] skb frag: kmap_atomic fixes Willem de Bruijn
2021-01-08 17:11 ` [PATCH net 1/3] net: support kmap_local forced debugging in skb_frag_foreach Willem de Bruijn
2021-01-08 17:11 ` [PATCH net 2/3] net: compound page support in skb_seq_read Willem de Bruijn
@ 2021-01-08 17:11 ` Willem de Bruijn
2 siblings, 0 replies; 6+ messages in thread
From: Willem de Bruijn @ 2021-01-08 17:11 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, Willem de Bruijn, Steffen Klassert
From: Willem de Bruijn <willemb@google.com>
esp(6)_output_head uses skb_page_frag_refill to allocate a buffer for
the esp trailer.
It accesses the page with kmap_atomic to handle highmem. But
skb_page_frag_refill can return compound pages, of which
kmap_atomic only maps the first underlying page.
skb_page_frag_refill does not return highmem, because flag
__GFP_HIGHMEM is not set. ESP uses it in the same manner as TCP.
That also does not call kmap_atomic, but directly uses page_address,
in skb_copy_to_page_nocache. Do the same for ESP.
This issue has become easier to trigger with recent kmap local
debugging feature CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP.
Fixes: cac2661c53f3 ("esp4: Avoid skb_cow_data whenever possible")
Fixes: 03e2a30f6a27 ("esp6: Avoid skb_cow_data whenever possible")
Signed-off-by: Willem de Bruijn <willemb@google.com>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
---
net/ipv4/esp4.c | 7 +------
net/ipv6/esp6.c | 7 +------
2 files changed, 2 insertions(+), 12 deletions(-)
diff --git a/net/ipv4/esp4.c b/net/ipv4/esp4.c
index 8b07f3a4f2db..a3271ec3e162 100644
--- a/net/ipv4/esp4.c
+++ b/net/ipv4/esp4.c
@@ -443,7 +443,6 @@ static int esp_output_encap(struct xfrm_state *x, struct sk_buff *skb,
int esp_output_head(struct xfrm_state *x, struct sk_buff *skb, struct esp_info *esp)
{
u8 *tail;
- u8 *vaddr;
int nfrags;
int esph_offset;
struct page *page;
@@ -485,14 +484,10 @@ int esp_output_head(struct xfrm_state *x, struct sk_buff *skb, struct esp_info *
page = pfrag->page;
get_page(page);
- vaddr = kmap_atomic(page);
-
- tail = vaddr + pfrag->offset;
+ tail = page_address(page) + pfrag->offset;
esp_output_fill_trailer(tail, esp->tfclen, esp->plen, esp->proto);
- kunmap_atomic(vaddr);
-
nfrags = skb_shinfo(skb)->nr_frags;
__skb_fill_page_desc(skb, nfrags, page, pfrag->offset,
diff --git a/net/ipv6/esp6.c b/net/ipv6/esp6.c
index 52c2f063529f..2b804fcebcc6 100644
--- a/net/ipv6/esp6.c
+++ b/net/ipv6/esp6.c
@@ -478,7 +478,6 @@ static int esp6_output_encap(struct xfrm_state *x, struct sk_buff *skb,
int esp6_output_head(struct xfrm_state *x, struct sk_buff *skb, struct esp_info *esp)
{
u8 *tail;
- u8 *vaddr;
int nfrags;
int esph_offset;
struct page *page;
@@ -519,14 +518,10 @@ int esp6_output_head(struct xfrm_state *x, struct sk_buff *skb, struct esp_info
page = pfrag->page;
get_page(page);
- vaddr = kmap_atomic(page);
-
- tail = vaddr + pfrag->offset;
+ tail = page_address(page) + pfrag->offset;
esp_output_fill_trailer(tail, esp->tfclen, esp->plen, esp->proto);
- kunmap_atomic(vaddr);
-
nfrags = skb_shinfo(skb)->nr_frags;
__skb_fill_page_desc(skb, nfrags, page, pfrag->offset,
--
2.30.0.284.gd98b1dd5eaa7-goog
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH net 2/3] net: compound page support in skb_seq_read
2021-01-08 17:11 ` [PATCH net 2/3] net: compound page support in skb_seq_read Willem de Bruijn
@ 2021-01-08 23:30 ` Willem de Bruijn
2021-01-09 22:09 ` Willem de Bruijn
1 sibling, 0 replies; 6+ messages in thread
From: Willem de Bruijn @ 2021-01-08 23:30 UTC (permalink / raw)
To: Network Development; +Cc: David Miller, Jakub Kicinski, Willem de Bruijn
On Fri, Jan 8, 2021 at 12:11 PM Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
>
> From: Willem de Bruijn <willemb@google.com>
>
> skb_seq_read iterates over an skb, returning pointer and length of
> the next data range with each call.
>
> It relies on kmap_atomic to access highmem pages when needed.
>
> An skb frag may be backed by a compound page, but kmap_atomic maps
> only a single page. There are not enough kmap slots to always map all
> pages concurrently.
>
> Instead, if kmap_atomic is needed, iterate over each page.
>
> As this increases the number of calls, avoid this unless needed.
> The necessary condition is captured in skb_frag_must_loop.
>
> I tried to make the change as obvious as possible. It should be easy
> to verify that nothing changes if skb_frag_must_loop returns false.
>
> Tested:
> On an x86 platform with
> CONFIG_HIGHMEM=y
> CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP=y
> CONFIG_NETFILTER_XT_MATCH_STRING=y
>
> Run
> ip link set dev lo mtu 1500
> iptables -A OUTPUT -m string --string 'badstring' -algo bm -j ACCEPT
> dd if=/dev/urandom of=in bs=1M count=20
> nc -l -p 8000 > /dev/null &
> nc -w 1 -q 0 localhost 8000 < in
>
> Signed-off-by: Willem de Bruijn <willemb@google.com>
I don't have a clear Fixes tag for this.
That was also true for commit c613c209c3f3 ("net: add
skb_frag_foreach_page and use with kmap_atomic"), which deals with the
same problem in a few other functions.
It goes back to when compound highmem pages may have appeared in the
skb frag. Possibly with vmsplice, around 2006. The skb_seq_read
interface itself was added in 2005.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH net 2/3] net: compound page support in skb_seq_read
2021-01-08 17:11 ` [PATCH net 2/3] net: compound page support in skb_seq_read Willem de Bruijn
2021-01-08 23:30 ` Willem de Bruijn
@ 2021-01-09 22:09 ` Willem de Bruijn
1 sibling, 0 replies; 6+ messages in thread
From: Willem de Bruijn @ 2021-01-09 22:09 UTC (permalink / raw)
To: Network Development; +Cc: David Miller, Jakub Kicinski, Willem de Bruijn
On Fri, Jan 8, 2021 at 12:11 PM Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
>
> From: Willem de Bruijn <willemb@google.com>
>
> skb_seq_read iterates over an skb, returning pointer and length of
> the next data range with each call.
>
> It relies on kmap_atomic to access highmem pages when needed.
>
> An skb frag may be backed by a compound page, but kmap_atomic maps
> only a single page. There are not enough kmap slots to always map all
> pages concurrently.
>
> Instead, if kmap_atomic is needed, iterate over each page.
>
> As this increases the number of calls, avoid this unless needed.
> The necessary condition is captured in skb_frag_must_loop.
>
> I tried to make the change as obvious as possible. It should be easy
> to verify that nothing changes if skb_frag_must_loop returns false.
>
> Tested:
> On an x86 platform with
> CONFIG_HIGHMEM=y
> CONFIG_DEBUG_KMAP_LOCAL_FORCE_MAP=y
> CONFIG_NETFILTER_XT_MATCH_STRING=y
>
> Run
> ip link set dev lo mtu 1500
> iptables -A OUTPUT -m string --string 'badstring' -algo bm -j ACCEPT
> dd if=/dev/urandom of=in bs=1M count=20
> nc -l -p 8000 > /dev/null &
> nc -w 1 -q 0 localhost 8000 < in
>
> Signed-off-by: Willem de Bruijn <willemb@google.com>
> ---
> include/linux/skbuff.h | 1 +
> net/core/skbuff.c | 28 +++++++++++++++++++++++-----
> 2 files changed, 24 insertions(+), 5 deletions(-)
>
> diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
> index c858adfb5a82..68ffd3f115c1 100644
> --- a/include/linux/skbuff.h
> +++ b/include/linux/skbuff.h
> @@ -1203,6 +1203,7 @@ struct skb_seq_state {
> struct sk_buff *root_skb;
> struct sk_buff *cur_skb;
> __u8 *frag_data;
> + __u16 frag_off;
frags can exceed 64k, so this needs to be __u32, like the other offsets.
I'll have to send a v2.
There's also something to be said for having a
BUILD_BUG_ON(sizeof(struct skb_seq_state) > sizeof(skb->cb));
as it's getting close. But I won't do that in this stable fix.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-01-09 22:10 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-08 17:11 [PATCH net 0/3] skb frag: kmap_atomic fixes Willem de Bruijn
2021-01-08 17:11 ` [PATCH net 1/3] net: support kmap_local forced debugging in skb_frag_foreach Willem de Bruijn
2021-01-08 17:11 ` [PATCH net 2/3] net: compound page support in skb_seq_read Willem de Bruijn
2021-01-08 23:30 ` Willem de Bruijn
2021-01-09 22:09 ` Willem de Bruijn
2021-01-08 17:11 ` [PATCH net 3/3] esp: avoid unneeded kmap_atomic call Willem de Bruijn
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).