WSL2-Linux-Kernel/include
Daniel Borkmann 92acdc58ab bpf, net: Rework cookie generator as per-cpu one
With its use in BPF, the cookie generator can be called very frequently
in particular when used out of cgroup v2 hooks (e.g. connect / sendmsg)
and attached to the root cgroup, for example, when used in v1/v2 mixed
environments. In particular, when there's a high churn on sockets in the
system there can be many parallel requests to the bpf_get_socket_cookie()
and bpf_get_netns_cookie() helpers which then cause contention on the
atomic counter.

As similarly done in f991bd2e14 ("fs: introduce a per-cpu last_ino
allocator"), add a small helper library that both can use for the 64 bit
counters. Given this can be called from different contexts, we also need
to deal with potential nested calls even though in practice they are
considered extremely rare. One idea as suggested by Eric Dumazet was
to use a reverse counter for this situation since we don't expect 64 bit
overflows anyways; that way, we can avoid bigger gaps in the 64 bit
counter space compared to just batch-wise increase. Even on machines
with small number of cores (e.g. 4) the cookie generation shrinks from
min/max/med/avg (ns) of 22/50/40/38.9 down to 10/35/14/17.3 when run
in parallel from multiple CPUs.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Link: https://lore.kernel.org/bpf/8a80b8d27d3c49f9a14e1d5213c19d8be87d1dc8.1601477936.git.daniel@iogearbox.net
2020-09-30 11:50:35 -07:00
..
acpi
asm-generic iomap: constify ioreadX() iomem argument (as in generic implementation) 2020-08-14 19:56:57 -07:00
clocksource include/: replace HTTP links with HTTPS ones 2020-08-12 10:57:59 -07:00
crypto mm, treewide: rename kzfree() to kfree_sensitive() 2020-08-07 11:33:22 -07:00
drm drm/i915: Fix sha_text population code 2020-09-02 10:48:11 +03:00
dt-bindings Here's some more updates that missed the last pull request because I 2020-08-12 12:19:49 -07:00
keys
kunit
kvm
linux bpf, net: Rework cookie generator as per-cpu one 2020-09-30 11:50:35 -07:00
math-emu treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
media media updates for v5.9-rc1 2020-08-07 13:00:53 -07:00
memory
misc
net bpf, net: Rework cookie generator as per-cpu one 2020-09-30 11:50:35 -07:00
pcmcia
ras
rdma RDMA 5.9 merge window pull request 2020-08-06 16:43:36 -07:00
scsi
soc Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-09-22 16:45:34 -07:00
sound ASoC: core: Do not cleanup uninitialized dais on soc_pcm_open failure 2020-09-07 12:40:24 +01:00
target
trace rxrpc: Fix a missing NULL-pointer check in a trace 2020-09-14 16:18:59 +01:00
uapi bpf: Add classid helper only based on skb->sk 2020-09-30 11:50:34 -07:00
vdso lib/vdso: Allow to add architecture-specific vdso data 2020-08-06 10:57:30 +02:00
video
xen xen: branch for v5.9-rc4 2020-09-06 09:59:27 -07:00