Any docs on usage with wsgi apps?

[andybak]

i.e. how to get a log for an individual request?

[evanhempel]

So far its design is targeted toward profiling an entire application. You could run it against the entire WSGI app, but you'll probably find time waiting for a request would dominate the graph unless the server was extremely busy. You can work around that to some extent by filtering the output file to remove the stack frames that are waiting for a new request or filter to only include stack frames containing the method you're interested in.

grep -v waiting_method output.file > removed_waiting.out
or
grep function_name output.file > filtered_on_request.out

I think I can add an API to allow only collecting information for particular methods, but I'll need to put some thought into how best to do that... I'll update here when I have a proposal.

[cjgu]

It would be really awesome to be able to use it as a decorator to profile individual functions.

[evanhempel]

@andybak I've added code to allow filtering: python -m flamegraph -f function_name mywebapp-start.py should now work for profiling just the functions of interest. Let me know if this satisfies your usecase.

[evanhempel]

@cjgu I saw you made a few commits for prototype decorator support. How did that work? It seems like a useful feature. I have a concern with the code I saw so far that if we start and stop the thread for each function invocation it could end up being high overhead (on the other hand threads are lightweight so perhaps not). I'm interested to hear your thoughts.

[cjgu]

@evanhempel I tried it out on a flask app, decorating the entry point for a specific request handler, testing one request only by running curl against the endpoint. I was a bit disappointed in the frequency of the sampling however, getting out around 10 samples over a 500 ms period that the request took without sampling. From what I understand this setup should only start/stop the thread once right?

[evanhempel]

Strange. With an interval of 0.001s and a function runtime of 500ms if the profiling code had no overhead and the threads ran perfectly in parallel then we'd expect 500 samples. Now those ideal world assumptions are obviously not true, but I'd still expect more than 10. This needs some investigation, the reason for so few samples could be that the profiling code takes significant time, or that the context switching between threads is a comparatively large number of seconds...

I'm sure we can get those numbers better, but even so the flamegraph profiling method isn't geared toward short timeframes...

[evanhempel]

FYI, I stumbled across a python flamegraph project aimed Django, which maybe you'll find useful: https://github.com/23andme/djdt-flamegraph

I did notice that its using a different method to capture stack frames (signal module instead of using threads). It may be that that is a better profiling method, if so I'd consider switching this project to use that method as well. I haven't done any testing with that method yet, if you try it I'd be curious to hear what you find...