Stratus3D

Software Engineering, Web Development and 3D Design

Performance of Elixir's Access Behavior

In Erlang we have two ways of getting a value from a map. We can use map:get/2 or we can pattern match the value out of the list using the #{ key := Value } = Map syntax. In Elixir we have another option Access. With Access we can get values out of a map or a keyword/property list using a square bracket syntax:

map[:key]

What is interesting is that Access was originally implemented as a protocol and was converted to a behavior. Protocols are a way to facilitate polymorphism in Elixir and define a single interface for different data types. You can define a protocol and than implement it for all the built in data types and any custom data types you create. It’s specifically for data types. The protocol was replaced by a behavior because the protocol was too slow. Access is now implemented as a behavior and is much faster.

This got me thinking about the implementation. Is the Access behavior in Elixir just as fast as the map:get/2 function?

Benchmarking the Different Methods

The two implementations I am benchmarking look like this:

defmodule ElixirAccess do
  def get(map, key) do
    map[key]
  end
end

defmodule ElixirMapGet do
  def get(map, key) do
    Map.get(map, key)
  end
end

Building the Benchmark

I decided to benchmark each method by generating lists of a dozen random keys and then building a map containing 75% of those keys. Then I look up each of the generated keys in the map. I repeat the process 50000 times per method and take an average of the execution times. The benchmarking code is shown below:

defmodule ElixirAccessBenchmark do
  @impls [ElixirAccess, ElixirMapGet]
  @times 50000
  @num_keys 12
  # 75% of keys will be in the maps
  @num_keys_to_include 9

  def measure(module, map, keys) do
    {time, :ok} = :timer.tc(__MODULE__, :measure_impl, [module, map, keys])
    time
  end

  def run do
    Enum.each(@impls, fn(module) ->
      IO.puts("Benchmarking #{module}")
      times = run_impl(module)
      len = length(times)
      total = List.foldl(times, 0, fn(time, sum) ->
        time + sum
      end)
      average = total / len
      IO.puts("Average time for looking up all keys was #{inspect(average)} microseconds")
    end)
  end

  def run_impl(module) do
    Enum.map(1..@times, fn(_) ->
      list_of_keys = generate_keys()
      map = generate_map_with_some_keys(list_of_keys)
      measure(module, map, list_of_keys)
    end)
  end

  def measure_impl(module, map, keys) do
    Enum.each(keys, fn(key) ->
      _ = apply(module, :get, [map, key])
    end)
  end

  defp generate_keys do
    1..@num_keys
    |> Enum.to_list()
    |> Enum.map(fn(key) ->
      Integer.to_string(key)
    end)
  end

  defp generate_map_with_some_keys(keys) do
    keys
    |> Enum.take_random(@num_keys_to_include)
    |> Map.new(fn(key) ->
      {key, true}
    end)
  end
end

ElixirAccessBenchmark.run()

Results

Running the exs file containing this code produced the following output:

Benchmarking Elixir.ElixirAccess
Average time for looking up all keys was 1.0482 microseconds
Benchmarking Elixir.ElixirMapGet
Average time for looking up all keys was 1.0430 microseconds

I ran the script several times and consistently got similar numbers.

  • Access: 1.0482 microseconds average

  • Map.get/2: 1.0430 microseconds average

Conclusion

The Access behavior does have to do type checking at some point during execution to determine whether or not it’s dealing with a map or keyword list. That type checking isn’t free but it only makes the square bracket syntax 0.5% slower than Map.get/2. With such a minor small difference in execution time the square bracket syntax should not have a noticeable affect on performance when running real Elixir applications. Obviously the less type checking you can do the faster your app will be, but in cases like this it’s not something that will ever become a bottleneck, even when your app is pushed to the limits.

If you don’t care about performance you can safely use either of these methods to lookup values in a map. The Access behavior is only 0.5% slower, so even if you are concerned about performance there are probably other things in your codebase that are having a greater impact on the performance than the square bracket syntax.