Metaprogramming

Metaprogramming#

Metaprogramming is code that writes code. Julia inherits this from Lisp, and it’s one of the features people either love immediately or find slightly alarming. Let me try to make it feel neither magical nor scary — just useful, in a small number of specific situations.

You have been using it since chapter one. @time, @view, @testset, @. — every one of those is a macro, transforming your code before it’s compiled. This chapter is about understanding what they do and, occasionally, writing your own.

Code is data#

The foundation is simple: in Julia, a piece of code is a value you can hold, inspect, and modify.

julia> ex = :(a + b * 2)
:(a + b * 2)

julia> typeof(ex)
Expr

The :( ... ) syntax makes an expression object rather than evaluating it. And that object has parts you can look at:

julia> ex.head
:call

julia> ex.args
3-element Vector{Any}:
  :+
  :a
  :(b * 2)

So a + b * 2 is really a tree: a call to +, with arguments a and (b * 2), and that second argument is itself a call to *. Every piece of Julia code has this structure. dump(ex) will show you the whole tree if you want to see it laid out.

You can build expressions programmatically and evaluate them:

julia> a, b = 10, 5
julia> eval(ex)
20

Warning

eval is almost never the right tool Now that I’ve shown you eval, let me tell you not to use it. It runs at global scope, it defeats the compiler’s ability to reason about your code, and reaching for it is usually a sign that a function or a dictionary would have solved the problem better.

If you find yourself building a string of code to eval, stop and ask what you’re really trying to do. Nine times out of ten the answer is “call a function with a parameter.”

Interpolation into expressions#

Inside a quoted expression, $ splices a value in:

julia> n = 3
julia> :(x + $n)
:(x + 3)

Note the difference: n would have stayed the symbol n, but $n inserted the number 3. This is the same $ you use in string interpolation, doing the analogous job.

For multi-line expressions, quote ... end is the block form of :( ):

ex = quote
    total = 0
    for x in data
        total += x
    end
    total
end

Macros#

A macro is a function that runs at parse time, takes expressions as input, and returns an expression to be compiled in its place. You invoke one with @.

Here’s the smallest useful example:

macro logged(expr)
    return quote
        println("Evaluating: ", $(string(expr)))
        $(esc(expr))
    end
end

julia> @logged 2 + 3 * 4
Evaluating: 2 + 3 * 4
14

Notice what a macro can do that a function cannot: it saw the source text of the expression, not just its value. A function receiving 2 + 3 * 4 gets 14 and has no way to know where that came from. That’s the entire reason macros exist, and it’s the test for whether you need one:

Do you need the source code of the argument, or its value? If the value, write a function. If the source, write a macro.

This is why @time is a macro (it must wrap the expression in timing calls before it runs), why @test is a macro (it prints the failing expression back to you), and why @. is a macro (it rewrites the syntax tree, adding dots).

esc and hygiene#

That esc() in the example is the one piece of macro-writing you have to understand.

Julia macros are hygienic: variables the macro introduces are automatically renamed so they can’t collide with variables in your code. That’s a good default — a macro that internally uses a variable called total shouldn’t clobber your total.

But hygiene also means that when the macro returns your expression, Julia would rename your variables too, and they’d stop referring to what you meant. esc() marks an expression as “this belongs to the caller, leave it alone.”

The rule of thumb: escape the user’s expressions, don’t escape anything you invented.

A macro worth writing#

Here’s one I actually use — asserting a condition with a readable failure message:

macro check(condition, message = "check failed")
    return quote
        if !$(esc(condition))
            error($message * ": " * $(string(condition)))
        end
    end
end

julia> balance = -50
julia> @check balance >= 0 "invalid ledger state"
ERROR: invalid ledger state: balance >= 0

The error message includes the source of the failing condition, which a plain function could never do.

Macros you should know from the standard library#

You’ll use these far more often than you’ll write your own:

@time expr           # time it and report allocations
@allocated expr      # just the bytes allocated
@show x              # print "x = value" — the best debugging tool in Julia
@assert cond         # runtime assertion
@view A[1:10]        # a slice without copying
@. expr              # broadcast everything in expr
@inbounds expr       # skip bounds checking (careful!)
@threads for ...     # parallelize a loop
@code_warntype f(x)  # show type inference results — the performance workhorse
@which f(x)          # which method does this call?
@edit f(x)           # open that method's source in your editor

@show deserves special mention. Instead of println("balance = ", balance), write @show balance and get balance = 1200.0. It’s a tiny thing that you’ll use a hundred times a day.

@which and @edit are the pair that make Julia’s ecosystem readable. Wondering how a package does something? @edit somefunc(x) drops you into the actual source, which is Julia, which you can read.

Generated functions#

One step beyond macros: a @generated function runs at compile time, sees the types of its arguments (not their values), and returns the code to compile for those types.

@generated function type_name(x)
    name = string(x)         # x here is the TYPE, not the value
    return :($name)
end

julia> type_name(3)
"Int64"

julia> type_name("hi")
"String"

The string was computed once, at compile time, for each type. At runtime there’s no work at all.

This is genuinely powerful for writing code that specializes on type structure — unrolling loops over a tuple’s fields, generating optimal code for a fixed-size array. It’s also a sharp tool with real constraints (a generated function can’t call functions that were defined after it, and can’t observe values). If you’re reaching for one in ordinary application code, you have almost certainly overshot.

When to use metaprogramming — and when not to#

I want to be direct, because this is the feature most likely to be misused by an enthusiastic beginner.

Good reasons to write a macro:

  • You need to see the source expression (for error messages, timing, logging, or tests).

  • You’re building a domain-specific language where the syntax genuinely helps — @model in a probabilistic programming package, @variable in an optimization package.

  • You’re generating a family of near-identical method definitions that would otherwise be dozens of lines of copy-paste.

Bad reasons:

  • To avoid writing a function. This is the big one. Most “I need a macro” instincts dissolve when you consider passing a function as an argument instead.

  • To make code shorter. Macros make code less readable to everyone who isn’t you, because they can’t be understood by reading normally — you have to know what the macro expands to.

  • For performance. Julia’s compiler already specializes your functions. A macro is not going to beat it.

Here’s my honest summary after years of writing Julia: you will read many macros and write very few. That’s the correct ratio. The ones in the standard library and in good packages are worth understanding thoroughly. Your own code is usually better off without them.

Debugging a macro#

When a macro misbehaves, @macroexpand shows you exactly what it produced:

julia> @macroexpand @check balance >= 0
quote
    if !(balance >= 0)
        error("check failed" * ": " * "balance >= 0")
    end
end

Being able to see the generated code takes almost all of the mystery out of macros. When something’s wrong, expand it and read.

Try it yourself#

  1. Build the expression :(2 * x + 1) by hand using Expr(:call, ...), then confirm it matches the quoted version.

  2. Write a macro @twice expr that evaluates its expression twice and returns the second result. Test it with something that prints, so you can see both runs.

  3. Use @macroexpand on @. a + b * c and read what the dot macro actually generates.

  4. Use @which sum([1,2,3]) to find the method, then @edit to read its source.

Solutions#

# 1.
e = Expr(:call, :+, Expr(:call, :*, 2, :x), 1)
e == :(2 * x + 1)      # true

# 2.
macro twice(expr)
    return quote
        $(esc(expr))
        $(esc(expr))
    end
end
@twice println("hello")     # prints twice

# 3.
@macroexpand @. a + b * c   # :(a .+ b .* c)

You now understand the layer of Julia that operates on code itself. Next we’ll come back down to earth and look at how Julia talks to the rest of the world — starting with Python.