ftrace,bpf: Use single direct ops for bpf trampolines#10558
ftrace,bpf: Use single direct ops for bpf trampolines#10558kernel-patches-daemon-bpf[bot] wants to merge 10 commits intobpf-next_basefrom
Conversation
|
Upstream branch: f14cdb1 |
|
Upstream branch: f14cdb1 |
190cb7d to
f4e87ec
Compare
302988a to
a14913f
Compare
|
Upstream branch: 6006058 |
f4e87ec to
019b2bb
Compare
a14913f to
d5a1b3c
Compare
|
Upstream branch: 317a5df |
019b2bb to
081c367
Compare
d5a1b3c to
cd2ddbb
Compare
|
Upstream branch: ccaa6d2 |
081c367 to
dc066a3
Compare
cd2ddbb to
e461c3e
Compare
|
Upstream branch: ccaa6d2 |
dc066a3 to
b065d3c
Compare
e461c3e to
cbcc41d
Compare
|
Upstream branch: ccaa6d2 |
b065d3c to
7708f49
Compare
cbcc41d to
c63b4b8
Compare
|
Upstream branch: 1a8fa7f |
7708f49 to
03d5de0
Compare
c63b4b8 to
bc2b3f2
Compare
|
Upstream branch: 5714ca8 |
51f4424 to
a504524
Compare
|
Upstream branch: 5714ca8 |
a504524 to
b47143b
Compare
1a6c4b2 to
695292f
Compare
At the moment the we allow the jmp attach only for ftrace_ops that has FTRACE_OPS_FL_JMP set. This conflicts with following changes where we use single ftrace_ops object for all direct call sites, so all could be be attached via just call or jmp. We already limit the jmp attach support with config option and bit (LSB) set on the trampoline address. It turns out that's actually enough to limit the jmp attach for architecture and only for chosen addresses (with LSB bit set). Each user of register_ftrace_direct or modify_ftrace_direct can set the trampoline bit (LSB) to indicate it has to be attached by jmp. The bpf trampoline generation code uses trampoline flags to generate jmp-attach specific code and ftrace inner code uses the trampoline bit (LSB) to handle return from jmp attachment, so there's no harm to remove the FTRACE_OPS_FL_JMP bit. The fexit/fmodret performance stays the same (did not drop), current code: fentry : 77.904 ± 0.546M/s fexit : 62.430 ± 0.554M/s fmodret : 66.503 ± 0.902M/s with this change: fentry : 80.472 ± 0.061M/s fexit : 63.995 ± 0.127M/s fmodret : 67.362 ± 0.175M/s Fixes: 25e4e35 ("ftrace: Introduce FTRACE_OPS_FL_JMP") Signed-off-by: Jiri Olsa <jolsa@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org>
Make alloc_and_copy_ftrace_hash to copy also direct address for each hash entry. Signed-off-by: Jiri Olsa <jolsa@kernel.org>
We are going to use these functions in following changes. Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Adding update_ftrace_direct_add function that adds all entries (ip -> addr) provided in hash argument to direct ftrace ops and updates its attachments. The difference to current register_ftrace_direct is - hash argument that allows to register multiple ip -> direct entries at once - we can call update_ftrace_direct_add multiple times on the same ftrace_ops object, becase after first registration with register_ftrace_function_nolock, it uses ftrace_update_ops to update the ftrace_ops object This change will allow us to have simple ftrace_ops for all bpf direct interface users in following changes. Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Adding update_ftrace_direct_del function that removes all entries (ip -> addr) provided in hash argument to direct ftrace ops and updates its attachments. The difference to current unregister_ftrace_direct is - hash argument that allows to unregister multiple ip -> direct entries at once - we can call update_ftrace_direct_del multiple times on the same ftrace_ops object, becase we do not need to unregister all entries at once, we can do it gradualy with the help of ftrace_update_ops function This change will allow us to have simple ftrace_ops for all bpf direct interface users in following changes. Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Adding update_ftrace_direct_mod function that modifies all entries (ip -> direct) provided in hash argument to direct ftrace ops and updates its attachments. The difference to current modify_ftrace_direct is: - hash argument that allows to modify multiple ip -> direct entries at once This change will allow us to have simple ftrace_ops for all bpf direct interface users in following changes. Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Following changes need to lookup trampoline based on its ip address, adding hash table for that. Signed-off-by: Jiri Olsa <jolsa@kernel.org>
We are going to remove "ftrace_ops->private == bpf_trampoline" setup in following changes. Adding ip argument to ftrace_ops_func_t callback function, so we can use it to look up the trampoline. Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Using single ftrace_ops for direct calls update instead of allocating ftrace_ops object for each trampoline. With single ftrace_ops object we can use update_ftrace_direct_* api that allows multiple ip sites updates on single ftrace_ops object. Adding HAVE_SINGLE_FTRACE_DIRECT_OPS config option to be enabled on each arch that supports this. At the moment we can enable this only on x86 arch, because arm relies on ftrace_ops object representing just single trampoline image (stored in ftrace_ops::direct_call). Archs that do not support this will continue to use *_ftrace_direct api. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org>
|
Upstream branch: 9c1a352 |
b47143b to
ffdddba
Compare
f61cb76 to
91752c7
Compare
|
At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=1037389 irrelevant now. Closing PR. |
Pull request for series with
subject: ftrace,bpf: Use single direct ops for bpf trampolines
version: 1
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=1037389