asdf was rewritten from scratch in Go and released on January 29th. No special attention was given to performance during the rewrite. The goal was feature parity in a more maintainable language. In this post I’m going to share some benchmarks I ran across many versions of asdf and show how the Go rewrite improved performance.
This is the second post I’ve made on
asdf performance. The benchmarks I
showed in that previous post revealed that the asdf current and asdf reshim
commands were extremely slow. asdf reshim took over 20 seconds to run, a big
problem when users run this command often, and when
custom shim templates
often run it automatically under the hood. asdf current is one of the main
commands users run to see what versions are current, so while it wasn’t quite
as slow, it was still a terrible user experience.
I ended up deciding to rewrite asdf in Go from scratch. Why I chose Go over the other languages I considered is a topic for another post. But I selected Go knowing that it wouldn’t be slower than the current implementation, and would allow me to make future performance optimizations as necessary. So how much of an improvement did the rewrite make?
Benchmarks
These benchmarks were performed on the same computer the benchmarks for the previous asdf performance post:
$ lshw -short
H/W path Device Class Description
======================================================
system 20DF0040US (LENOVO_MT_20DF_BU_Think_FM_ThinkPad E550)
/0 bus 20DF0040US
/0/3 memory 32KiB L1 cache
/0/4 processor Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz
/0/4/5 memory 32KiB L1 cache
/0/4/6 memory 256KiB L2 cache
/0/4/7 memory 4MiB L3 cache
/0/8 memory 8GiB System Memory
/0/8/0 memory 8GiB SODIMM DDR3 Synchronous 1600 MHz (0.6 ns)
/0/8/1 memory DIMM [empty]
/0/30 memory 128KiB BIOSI also only performed benchmarks for the new Go version (0.16.5 specifically). The Bash versions of asdf released after the last benchmarks I performed were skipped (0.11.x, 0.12.x, 0.13.x, 0.14.x, 0.15.x). There were no significant performance improvements made in those versions, so differences in performance between those versions and 0.10.0 would be minimal.
Like before I used hyperfine to run the benchmarks. I then used Gnuplot to generate the plots you see in this post.
Results
The asdf current is faster than it has ever been before. It is almost 5x
faster than the Bash implementation, but still takes about a third of a second
to run. It’s fast enough that users will tolerate it, but maybe not
fast enough to be used in a shell prompt yet.
From a performance perspective asdf exec is the most critical command asdf
provides. All asdf shims invoke it under the hood. Every time you run irb,
python, node, under the hood the shims for those commands are invoking
asdf exec. The latency here is a penalty you pay every time you invoke a
command that’s part of a tool managed by asdf. Thankfully it looks like the new
code is over 5x faster than the old Bash code. It runs in about 20
milliseconds. This is fast enough that for a single invocation users won’t
notice. And even recursive calls to asdf exec I expect performance to be
tolerable for most tasks. I think there is some work we could do to improve
this though, given that the Go code was written without any thought to
performance.
asdf reshim used to be the slowest subcommand asdf had. But now it’s almost
comparable to the others. Re-generation of all shims now takes just under one
second, over 20x faster than the older versions! This is a great improvement to
the user experience. The large improvement here is likely because this command
is mostly just reading and writing files to the file system. Generating shims
isn’t an overly complicated process, but it is iterative and work scales up as
the number of installed versions increases. I suspect the main thing slowing
down the Bash implementation was the fact that the Bash code had to run
numerous sub-processes to generate the shims. It seems this command is now fast
enough that no further optimizations are needed.
asdf where now runs in 5 milliseconds making it 10x faster than the Bash version.
asdf which now runs in 1 millisecond making it 8x faster than the Bash version.
Conclusion
The rewrite in Go significantly improved the performance of asdf, with
improvements ranging from 5-20x. Performance is no longer an issue for most
commands. asdf exec stands out as a command that would benefit from
optimization work in the future.