mirror of
https://git.vectorsigma.ru/public/atlantis.git
synced 2026-07-29 18:38:33 +00:00
04f1d062d6
* Add benchmark for *PlanSuccess.Summary This is for taking a baseline and comparing with future improvements. I just did a test run and got the following results: $ go test -bench=. ./server/events/command goos: darwin goarch: arm64 pkg: github.com/runatlantis/atlantis/server/events/command BenchmarkPlanSuccess_Summary/empty_summary,_no_matches-10 82564 12415 ns/op 27352 B/op 114 allocs/op BenchmarkPlanSuccess_Summary/changes-10 167904 7148 ns/op 14309 B/op 69 allocs/op BenchmarkPlanSuccess_Summary/no_changes-10 92410 12984 ns/op 27705 B/op 114 allocs/op BenchmarkPlanSuccess_Summary/changes_outside_Terraform-10 89256 13505 ns/op 27882 B/op 117 allocs/op BenchmarkPlanSuccess_Summary/changes_and_changes_outside-10 159199 7527 ns/op 14493 B/op 72 allocs/op PASS ok github.com/runatlantis/atlantis/server/events/command 7.536s To have a better comparison I've ran all the benchmarks with 200,000 iterations, and got this: $ go test -bench=. ./server/events/command -benchtime=200000x goos: darwin goarch: arm64 pkg: github.com/runatlantis/atlantis/server/events/command BenchmarkPlanSuccess_Summary/changes_and_changes_outside-10 200000 7637 ns/op 14487 B/op 72 allocs/op BenchmarkPlanSuccess_Summary/empty_summary,_no_matches-10 200000 12379 ns/op 27352 B/op 114 allocs/op BenchmarkPlanSuccess_Summary/changes-10 200000 7134 ns/op 14312 B/op 69 allocs/op BenchmarkPlanSuccess_Summary/no_changes-10 200000 12960 ns/op 27692 B/op 114 allocs/op BenchmarkPlanSuccess_Summary/changes_outside_Terraform-10 200000 13731 ns/op 27862 B/op 117 allocs/op PASS ok github.com/runatlantis/atlantis/server/events/command 11.837s Signed-off-by: Leandro López (inkel) <leandro.lopez@grafana.com> * Compile summary regexps only once By initializing these regexes at program first run instead of every time the function is called we gain better resource usage and much faster results, as shown in this results with 200,000 iterations as in the previous commit: $ go test -bench=. ./server/events/command -benchtime=200000x goos: darwin goarch: arm64 pkg: github.com/runatlantis/atlantis/server/events/command BenchmarkPlanSuccess_Summary/empty_summary,_no_matches-10 200000 29.49 ns/op 0 B/op 0 allocs/op BenchmarkPlanSuccess_Summary/changes-10 200000 450.4 ns/op 0 B/op 0 allocs/op BenchmarkPlanSuccess_Summary/no_changes-10 200000 357.9 ns/op 0 B/op 0 allocs/op BenchmarkPlanSuccess_Summary/changes_outside_Terraform-10 200000 824.1 ns/op 193 B/op 3 allocs/op BenchmarkPlanSuccess_Summary/changes_and_changes_outside-10 200000 754.8 ns/op 177 B/op 3 allocs/op PASS ok github.com/runatlantis/atlantis/server/events/command 0.943s With 200,000 iterations the results were almost immediately and I couldn't observe the real performance, so I've removed the limitation and got the following: $ go test -bench=. ./server/events/command -benchtime=200000x goos: darwin goarch: arm64 pkg: github.com/runatlantis/atlantis/server/events/command BenchmarkPlanSuccess_Summary/empty_summary,_no_matches-10 200000 29.49 ns/op 0 B/op 0 allocs/op BenchmarkPlanSuccess_Summary/changes-10 200000 450.4 ns/op 0 B/op 0 allocs/op BenchmarkPlanSuccess_Summary/no_changes-10 200000 357.9 ns/op 0 B/op 0 allocs/op BenchmarkPlanSuccess_Summary/changes_outside_Terraform-10 200000 824.1 ns/op 193 B/op 3 allocs/op BenchmarkPlanSuccess_Summary/changes_and_changes_outside-10 200000 754.8 ns/op 177 B/op 3 allocs/op PASS ok github.com/runatlantis/atlantis/server/events/command 0.943s I've also compared the results of running the benchamrks with 200,000 iterations, 10 times each, and these are the results: $ benchstat orig.txt improv.txt name old time/op new time/op delta PlanSuccess_Summary/empty_summary,_no_matches-10 12.6µs ± 2% 0.0µs ± 1% -99.90% (p=0.000 n=10+9) PlanSuccess_Summary/changes-10 7.19µs ± 0% 0.36µs ± 2% -94.96% (p=0.000 n=8+9) PlanSuccess_Summary/no_changes-10 13.2µs ± 2% 0.4µs ± 0% -97.27% (p=0.000 n=10+9) PlanSuccess_Summary/changes_outside_Terraform-10 13.7µs ± 1% 0.8µs ± 0% -94.02% (p=0.000 n=9+9) PlanSuccess_Summary/changes_and_changes_outside-10 7.71µs ± 5% 0.75µs ± 0% -90.21% (p=0.000 n=10+8) name old alloc/op new alloc/op delta PlanSuccess_Summary/empty_summary,_no_matches-10 27.4kB ± 0% 0.0kB -100.00% (p=0.000 n=10+10) PlanSuccess_Summary/changes-10 14.3kB ± 0% 0.0kB -100.00% (p=0.000 n=9+10) PlanSuccess_Summary/no_changes-10 27.7kB ± 0% 0.0kB -100.00% (p=0.000 n=10+10) PlanSuccess_Summary/changes_outside_Terraform-10 27.9kB ± 0% 0.2kB ± 0% -99.31% (p=0.000 n=10+7) PlanSuccess_Summary/changes_and_changes_outside-10 14.5kB ± 0% 0.2kB ± 0% -98.78% (p=0.000 n=10+8) name old allocs/op new allocs/op delta PlanSuccess_Summary/empty_summary,_no_matches-10 114 ± 0% 0 -100.00% (p=0.000 n=10+10) PlanSuccess_Summary/changes-10 69.0 ± 0% 0.0 -100.00% (p=0.000 n=10+10) PlanSuccess_Summary/no_changes-10 114 ± 0% 0 -100.00% (p=0.000 n=10+10) PlanSuccess_Summary/changes_outside_Terraform-10 117 ± 0% 3 ± 0% -97.44% (p=0.000 n=10+10) PlanSuccess_Summary/changes_and_changes_outside-10 72.0 ± 0% 3.0 ± 0% -95.83% (p=0.000 n=10+10) Signed-off-by: Leandro López (inkel) <leandro.lopez@grafana.com> * Remove extra allocation There's no need for fmt.Sprintf for such a simple use. Signed-off-by: Leandro López (inkel) <leandro.lopez@grafana.com> * Add MarkdownRenderer benchmark for diff markdown format Signed-off-by: Leandro López (inkel) <leandro.lopez@grafana.com> * Compile markdown diff regexes once This change doesn't improve performance as dramatically as the summary regexes do, however, it does reduce the amount of resources used: $ benchstat markdowndiff.orig.txt markdowndiff.inkel.txt name old time/op new time/op delta RenderProjectResultsWithEnableDiffMarkdownFormat/single_successful_plan_with_diff_markdown_formatted/verbose_true-10 727µs ± 0% 717µs ± 2% -1.31% (p=0.002 n=10+10) RenderProjectResultsWithEnableDiffMarkdownFormat/single_successful_plan_with_diff_markdown_formatted/verbose_false-10 725µs ± 0% 715µs ± 1% -1.33% (p=0.000 n=9+10) name old alloc/op new alloc/op delta RenderProjectResultsWithEnableDiffMarkdownFormat/single_successful_plan_with_diff_markdown_formatted/verbose_true-10 171kB ± 0% 153kB ± 0% -10.60% (p=0.000 n=10+10) RenderProjectResultsWithEnableDiffMarkdownFormat/single_successful_plan_with_diff_markdown_formatted/verbose_false-10 171kB ± 0% 153kB ± 0% -10.60% (p=0.000 n=10+10) name old allocs/op new allocs/op delta RenderProjectResultsWithEnableDiffMarkdownFormat/single_successful_plan_with_diff_markdown_formatted/verbose_true-10 364 ± 0% 214 ± 0% -41.29% (p=0.000 n=10+9) RenderProjectResultsWithEnableDiffMarkdownFormat/single_successful_plan_with_diff_markdown_formatted/verbose_false-10 358 ± 0% 207 ± 0% -42.05% (p=0.000 n=10+10) Signed-off-by: Leandro López (inkel) <leandro.lopez@grafana.com> Signed-off-by: Leandro López (inkel) <leandro.lopez@grafana.com>