Conversation
sloretz
force-pushed
the
sloretz__add_benchmark_tests
branch
from
60cbd2c to
1527a9e
Compare
fujitatomoya
left a comment
fujitatomoya
left a comment
There was a problem hiding this comment.
benchmark test would be really useful. Do we have any plan to integrate this kind of benchmark in CI/CD so that it can detect the performance degression?
There are benchmark jobs run for each release on |
|
@ros-pull-request-builder retest this please |
|
CI (build:
And one with the benchmark tests enabled (build:
|
|
Benchmark tests one more time with ros2/ci#674
|
|
CI LGTM! |
jacobperron
left a comment
jacobperron
left a comment
There was a problem hiding this comment.
One minor comment and a question. Otherwise, LGTM
rclpy/CMakeLists.txt
Outdated
| TIMEOUT 120 | ||
| WERROR ON | ||
| ) | ||
| # Skip benchmark tests by default |
There was a problem hiding this comment.
This comment looks like it's in the wrong place. Maybe at L209?
There was a problem hiding this comment.
Moved comment to correct spot in 5a0f651
| executor.spin_until_future_complete(result_fut) | ||
| assert GoalStatus.STATUS_SUCCEEDED == result_fut.result().status | ||
|
|
||
| benchmark(bm) |
There was a problem hiding this comment.
In the other tests we are asserting the result of benchmark, should we do that here?
There was a problem hiding this comment.
Looking at the assertions now I don't think they were adding value. I removed them in 76fd188
| executor.spin_until_future_complete(fut) | ||
| assert fut.result() # Raises if there was an error | ||
|
|
||
| benchmark(bm) |
There was a problem hiding this comment.
Same here, should we assert?
Signed-off-by: Shane Loretz <sloretz@osrfoundation.org>
Signed-off-by: Shane Loretz <sloretz@osrfoundation.org>
Signed-off-by: Shane Loretz <sloretz@osrfoundation.org>
sloretz
force-pushed
the
sloretz__add_benchmark_tests
branch
from
1527a9e to
76fd188
Compare
|
CI re-run
Benchmark job: |
|
@sloretz is this PR still relevant ? |
4 participants
This PR adds benchmarks that measure how quickly different types of entities are handled. This would be useful for testing performance improvements.
The benchmark tests are skipped unless
AMENT_RUN_PERFORMANCE_TESTSis set, which is the same behavior as asament_add_google_benchmark_test.WIP because this needs a new rosdep key
python3-pytest-benchmarkfor https://pypi.org/project/pytest-benchmark/Requires ros/rosdistro#33918