profiler - Which factors affect the speed of cpu tracing? -

when use yjp cpu-tracing profile on our own product, slow.

the product runs in 16 core machine 8gb heap, , use grinder run small load test (e.g. 10 grinder threads) have 7~10 steps during profiling. have script start product profiler, start profiling (using controller api) , start grinder emulate user operations. when operations finish, script tells profiler stop profiling , save snapshot.

during profiling, each step in grinder test, takes more 1 million ms finish. whole profiling takes more 10 hours 10 grinder threads, , each runs test 10 times. without profiler, finishes within 500 ms.

so... besides problems product profiled, there else affects performance of cpu tracing process itself?

last used yourkit (v7.5.11, pretty old, current version 12) had 2 cpu profiling settings: sampling , tracing, latter being faster , less accurate. since tracing supposed more accurate used myself , observed huge slowdown, in spite of statement slowdown "average". yet far less results: 2 seconds 10 minutes. code fragment of calculation engine, virtually no io, no waits on whatever, reading input, calculating , output result console - whole slowdown comes profiler, no external influences.

back question: option mentioned - samping vs tracing, affect performance, may try sampling.

now think of it: yourkit can setup such things automatically, making snapshots periodically or on low memory, profiling memory usage, object allocations, each of measures make profiling slowlier. perhaps should make online session instead of script controlled, see does.