Add a benchmark to measure gc traversal #244
Conversation
| all_cycles = create_recursive_containers(n_levels) | ||
| for _ in range(loops): | ||
| gc.collect() | ||
| # Main loop to measure | ||
| t0 = pyperf.perf_counter() | ||
| collected = gc.collect() | ||
| total_time += pyperf.perf_counter() - t0 |
There was a problem hiding this comment.
Just to be sure I understand, the GC objects aren't cleaned up during gc.collect() because all_cycles is still bound in the locals. That's why we can re-use it in each loop. Is that right?
There was a problem hiding this comment.
Yeah, this test is just benchmarking the gc traversing all those objects again and again so no need to remove them
|
Why dont you create one benchmark as we know its supposed to be a workload, then things like traversal, collection are just metrics? The PyPy folks have a workload called gcbench we can reference. I stand to be corrected but I am not familiar with folks writing a specific workload for every metric they want to measure. Rather its applying some metrics on several workloads. Am I missing something here @pablogsal ? |
|
Thanks for you comments Joannah!
is very difficult to get a benchmark that is heavily exercising the GC only and that's not skewed by other things getting faster. For now, I added these benchmarks to ensure that we can keep track of several aspects of the -CPython- GC that we may want to track, such as how efficient traversal and removing cycles are.
These things matter for the runtime, so these benchmarks allow us to keep track of them. That's why I am adding them.
That's ok, we are not researching GCs, we are keeping track of optimizations in CPython and we want to keep track on how these things get faster for our specific GC, at least for the time being.
You mean this? https://github.com/mozillazg/pypy/blob/master/rpython/translator/goal/gcbench.py yes, I plan to add it after some modifications but notice that this benchmarks is not doing many things differently in the GC for the things we are adding here. Is just creating a tree and benchmark construction, which involves many other things than just the GC. |
I understand the motivation, I will give an example, treating traversal as just a metric, we should I am not too bent on this, its just appeared wierd to me at first glance, so your decision.
Yes, but I maybe biased, its usually sufficient for GC things for us, you can modify if you want. |
I think I understand your point of view, but these metrics/benchmarks are important for keeping track of potential improvements and we need to isolate them from other runtime improvements to ensure that what we think is getting faster gets faster. You can measure GC in many many ways, this is just more information, nothing else. Performance has plenty of "smaller" benchmarks like "list_unpack" and other things that are not measuring big end-to-end things. But thanks for your feedback, I promise to take into account for adding more benchmarks in the future.
When I measured this benchmark had quite a lot of GC activity, but also a lot of other things regarding the VM. The problem is that if the VM gets faster (but not the GC) so it will this benchmark so we won't know where the advantage is coming from. This is why we want to track this separately. |
No description provided.