Julia Equality Performance

Published: 2021-06-04
Discussion on: Acmion (on this website)

Equality in Julia is most often checked with either == or ===. The == operator performs casting and is overridable, while the === operator does not perform casting and can not be overloaded. In most common cases, your code will work just the same, regardless of the operator that you choose to use. As such, the question remains: Which of these two alternatives has better performance?

Evaluating the Performance

Let's evaluate the performance of both operators in different scenarios.

Constants

using Pkg
Pkg.add("BenchmarkTools")
using BenchmarkTools

display(@benchmark 1 == 1)
display(@benchmark 1 == 2)
display(@benchmark 1 === 1)
display(@benchmark 1 === 2)
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     0.025 ns (0.00% GC)
  median time:      0.027 ns (0.00% GC)
  mean time:        0.027 ns (0.00% GC)
  maximum time:     0.076 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     0.025 ns (0.00% GC)
  median time:      0.027 ns (0.00% GC)
  mean time:        0.027 ns (0.00% GC)
  maximum time:     0.061 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     0.025 ns (0.00% GC)
  median time:      0.027 ns (0.00% GC)
  mean time:        0.027 ns (0.00% GC)
  maximum time:     0.039 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     0.026 ns (0.00% GC)
  median time:      0.028 ns (0.00% GC)
  mean time:        0.028 ns (0.00% GC)
  maximum time:     0.049 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000

When comparing constants, both operators have the same performance. However, the question remains whether the Julia compiler is able to presolve these constant expressions. As such, we need to test a few more cases.

Random Values

using Pkg
Pkg.add("BenchmarkTools")
using BenchmarkTools

display(@benchmark rand() == rand())
display(@benchmark rand() === rand())

BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     14.413 ns (0.00% GC)
  median time:      15.406 ns (0.00% GC)
  mean time:        15.665 ns (0.00% GC)
  maximum time:     364.501 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     998
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     14.393 ns (0.00% GC)
  median time:      15.948 ns (0.00% GC)
  mean time:        16.211 ns (0.00% GC)
  maximum time:     391.441 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     998

Still the same performance.

Variables

using Pkg
Pkg.add("BenchmarkTools")
using BenchmarkTools

a = 1
b = 1
c = 2

display(@benchmark a == b)
display(@benchmark a == c)
display(@benchmark a === b)
display(@benchmark a === c)
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     26.303 ns (0.00% GC)
  median time:      26.380 ns (0.00% GC)
  mean time:        26.532 ns (0.00% GC)
  maximum time:     345.021 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     995
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     19.401 ns (0.00% GC)
  median time:      19.933 ns (0.00% GC)
  mean time:        20.196 ns (0.00% GC)
  maximum time:     369.852 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     996
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     5.192 ns (0.00% GC)
  median time:      5.211 ns (0.00% GC)
  mean time:        5.256 ns (0.00% GC)
  maximum time:     25.358 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     9.064 ns (0.00% GC)
  median time:      9.171 ns (0.00% GC)
  mean time:        9.373 ns (0.00% GC)
  maximum time:     345.561 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     999

This time we can see significant differences in performance, not just between operators but also compared to the constant case. Why is this so? Turns out that the @benchmark macro executes the provided code within a function and thus the access to each variable goes through the global scope. This seems to be causing a significant performance hit. Slightly surprising that the === operator performs better than the == (at least when taking into consideration the findings of the "Constant" section).

Within Functions

using Pkg
Pkg.add("BenchmarkTools")
using BenchmarkTools

function double_equals(a, b)
    return a == b
end

function triple_equals(a, b)
    return a === b
end

display(@benchmark double_equals(rand(), rand()))
display(@benchmark triple_equals(rand(), rand()))
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     18.592 ns (0.00% GC)
  median time:      19.856 ns (0.00% GC)
  mean time:        20.245 ns (0.00% GC)
  maximum time:     2.554 μs (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     998
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     14.325 ns (0.00% GC)
  median time:      15.347 ns (0.00% GC)
  mean time:        15.630 ns (0.00% GC)
  maximum time:     393.474 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     998

The === operator seems to have a slight edge here. However, running the code multiple times does give slight fluctuations in these times. Regardless, the === operator was always slightly faster.

Conclusions

The === operator does seem to have slightly better performance, especially if accessing variables from another scope. That said, the operators do not have the same function, even if in most cases they can be replaced with eachother without any adverse effects. Use === wherever applicable.

Discussion

Discuss this post here!

Post a Comment

Error! Incorrect value.
Note: Commenting is completely anonymous, thus, comments can not be edited.
Kennethvoino (2021-06-29 15:07:25 UTC)

Very useful

Reply to Comment

Error! Incorrect value.
Note: Commenting is completely anonymous, thus, comments can not be edited.

Report Comment

Error! Incorrect value.
Skipledek (2021-06-26 09:11:07 UTC)

Reply to Comment

Error! Incorrect value.
Note: Commenting is completely anonymous, thus, comments can not be edited.

Report Comment

Error! Incorrect value.