Under the Hood: How Julia Compiles

Under the Hood: How Julia Compiles#

This chapter is optional in the sense that you can write good Julia without it. I’ve put it near the end because by now you have the context to make it click — and because understanding what the compiler does turns most of this book’s advice from rules you memorize into consequences you can derive.

The pipeline#

When you call a Julia function, your code goes through four stages:

Source text
   ↓  parse
Abstract Syntax Tree (Expr)      ← macros run here
   ↓  lower
Lowered IR                        ← desugared: loops become gotos
   ↓  type inference + optimize
Typed IR                          ← the crucial stage
   ↓  codegen
LLVM IR
   ↓  LLVM
Native machine code

The remarkable thing is that you can inspect every stage from the REPL:

f(x) = x^2 + 1

@code_lowered f(3)      # the desugared form
@code_typed f(3)        # after type inference — the important one
@code_llvm f(3)         # LLVM IR
@code_native f(3)       # actual assembly for your CPU

I’d genuinely encourage you to run @code_llvm f(3) and then @code_llvm f(3.0) right now and compare them. Same source, two different programs. That’s the whole language in one demonstration.

Just-ahead-of-time compilation#

Julia’s model is sometimes called JAOT — just-ahead-of-time — and it’s worth distinguishing from the alternatives.

A static compiler (C, Rust) compiles everything before the program runs, needs all types declared, and produces one binary.

A JIT in the Java or JavaScript sense starts interpreting, watches which code is hot, and compiles that.

Julia does neither. It compiles whole functions, ahead of their execution, at the moment they’re first called with a given set of concrete argument types. There’s no interpretation phase and no warm-up profiling. The first call to f(3) compiles a complete Int64 version of f; the first call to f(3.0) compiles a complete Float64 version. Both are cached.

This is why:

  • Julia is fast from the first execution of a compiled method, not after a thousand iterations.

  • Julia has latency — that first call has to compile something.

  • Types have to be inferable, because compilation happens before the code runs and the compiler needs to commit to a layout.

Specialization: the source of the speed#

Let’s watch it happen.

add(a, b) = a + b

That looks dynamically typed. Now:

julia> @code_native add(1, 2)
    ...
    leaq  (%rdi,%rsi), %rax
    retq

One instruction. A single leaq that adds two 64-bit integers, then return. There is no type check, no method lookup, no boxing. It is exactly what a C compiler would emit for int add(int a, int b).

julia> @code_native add(1.0, 2.0)
    ...
    vaddsd  %xmm1, %xmm0, %xmm0
    retq

A different instruction — SSE floating-point addition — from the same source line.

This is specialization, and it’s the answer to “how can a dynamic language be fast?” The dynamism is resolved at compile time, once per type combination, rather than at runtime, once per call.

What breaks it#

Now you can derive the performance chapter yourself.

If the compiler can’t infer a concrete type, it cannot pick an instruction. Instead of leaq, it must emit a call into the runtime that inspects the value’s type tag, searches the method table for +, calls the result through a pointer, allocates a box for the result, and returns the box. That’s the difference between one instruction and roughly a hundred, plus a heap allocation.

So:

  • Untyped globals break it, because their type can change between now and the call.

  • Vector{Any} breaks it, because the element type is unknown.

  • Type-unstable functions break it downstream, because their callers can’t infer what came back.

  • Abstract struct fields break it, because the layout isn’t known.

Every performance rule in this book is a corollary of “let the compiler pick an instruction.”

Latency, and what’s been done about it#

The cost of this design is time-to-first-result. Historically this was Julia’s most-complained-about feature, memorably as “time to first plot” — loading a plotting package and drawing one figure could take 30 seconds while the compiler chewed through thousands of methods.

The fixes have come steadily:

  • Precompilation (long-standing) caches parsed and lowered code when a package is installed.

  • Native code caching (Julia 1.9) was the big one — precompiled machine code now persists to disk, not just the intermediate form. Package load times dropped dramatically.

  • Ongoing invalidation work — when you define a method that could change previously-compiled decisions, Julia must throw away the affected compiled code. A lot of engineering has gone into making these invalidations narrower.

  • --trim (1.12, experimental) goes the other way, removing code that can’t be reached, for deployment.

If you evaluated Julia before 1.9 and bounced off the latency, the experience today is a different one. It hasn’t vanished — a first call still compiles — but it’s no longer the defining frustration it was.

Note

What --trim is really about Compilation happens on demand, which means the compiler must be present at runtime, which means shipping Julia means shipping the compiler. That’s the 200 MB.

--trim inverts this for deployment: prove that everything reachable from your entry point can be statically resolved, compile it all ahead of time, and discard the rest. The 1.6 MB binary from the Julia team’s testing is a Julia program with no compiler inside it.

The requirement — no dynamic dispatch reachable from the entry point — is strict, and plenty of ordinary code doesn’t satisfy it yet. But note what it means: type stability, the thing that makes your code fast, is the same thing that makes it deployable. One discipline, two payoffs.

Why LLVM matters#

Julia generates LLVM IR and lets LLVM do the final optimization and code generation. This is the same backend that Clang, Rust, and Swift use.

Practically, this means Julia inherits decades of work it didn’t have to do: loop unrolling, auto-vectorization, constant folding, inlining, instruction scheduling, and code generation for x86, ARM, PowerPC, and GPU targets. When your @simd loop gets vectorized into AVX instructions, that’s LLVM, using the same optimizer that compiles the C library you’re competing with.

It also explains the shape of Julia’s performance profile: once your code is type stable, you are more or less running the output of a production C compiler, because you are.

Tools for looking#

Beyond the @code_* macros:

@code_warntype f(x)       # inference results, red = trouble
@time f(x)                # first call includes compile time; second doesn't
@snoopi_deep              # from SnoopCompile.jl — where compile time goes

SnoopCompile.jl is the specialist tool if you’re a package author fighting latency. It tells you which methods are being compiled and why, and helps you add precompilation directives that move that work to install time.

The bigger picture#

I want to close this chapter with the thing I find genuinely elegant about Julia’s design.

Most languages draw a hard line between “the language” and “the compiler’s business.” Julia mostly doesn’t. Int64 is a struct in the standard library. + is a generic function with hundreds of methods, and you can add another. Array is implemented in Julia on top of Memory. Macros run in the same language you write. Generated functions let you write code that writes code, at compile time, with the compiler’s own type information.

The result is that “using the language” and “extending the language” are the same activity. When you define Base.:+ for your Money type back in the dispatch chapter, you weren’t using a special hook for user types — you were doing the same thing the standard library does for Int64.

That’s why the ecosystem composes the way it does, and it’s the deepest reason I stayed with Julia after trying it.

Try it yourself#

  1. Run @code_native on add(1, 2) and add(1.0, 2.0) and compare the instructions.

  2. Run @code_typed on a type-stable function and a type-unstable one. Find where the Any appears.

  3. Time a function’s first and second call with @time. Note the compile time in the first.

  4. Write a function that adds two elements of a Vector{Any} and one that adds two elements of a Vector{Float64}, and compare @code_llvm for both. The size difference tells the whole story.

Next, the mindset chapter: what all of this means for how you actually write code day to day.