# Metaprogramming Metaprogramming is code that writes code. Julia inherits this from Lisp, and it's one of the features people either love immediately or find slightly alarming. Let me try to make it feel neither magical nor scary — just useful, in a small number of specific situations. You have been using it since chapter one. `@time`, `@view`, `@testset`, `@.` — every one of those is a macro, transforming your code before it's compiled. This chapter is about understanding what they do and, occasionally, writing your own. ## Code is data The foundation is simple: in Julia, a piece of code is a **value** you can hold, inspect, and modify. ```{code-block} julia julia> ex = :(a + b * 2) :(a + b * 2) julia> typeof(ex) Expr ``` The `:( ... )` syntax makes an **expression object** rather than evaluating it. And that object has parts you can look at: ```{code-block} julia julia> ex.head :call julia> ex.args 3-element Vector{Any}: :+ :a :(b * 2) ``` So `a + b * 2` is really a tree: a call to `+`, with arguments `a` and `(b * 2)`, and that second argument is itself a call to `*`. Every piece of Julia code has this structure. `dump(ex)` will show you the whole tree if you want to see it laid out. You can build expressions programmatically and evaluate them: ```{code-block} julia julia> a, b = 10, 5 julia> eval(ex) 20 ``` :::{warning} `eval` is almost never the right tool Now that I've shown you `eval`, let me tell you not to use it. It runs at global scope, it defeats the compiler's ability to reason about your code, and reaching for it is usually a sign that a function or a dictionary would have solved the problem better. If you find yourself building a string of code to `eval`, stop and ask what you're really trying to do. Nine times out of ten the answer is "call a function with a parameter." ::: ## Interpolation into expressions Inside a quoted expression, `$` splices a *value* in: ```{code-block} julia julia> n = 3 julia> :(x + $n) :(x + 3) ``` Note the difference: `n` would have stayed the symbol `n`, but `$n` inserted the number 3. This is the same `$` you use in string interpolation, doing the analogous job. For multi-line expressions, `quote ... end` is the block form of `:( )`: ```{code-block} julia ex = quote total = 0 for x in data total += x end total end ``` ## Macros A **macro** is a function that runs at parse time, takes expressions as input, and returns an expression to be compiled in its place. You invoke one with `@`. Here's the smallest useful example: ```{code-block} julia macro logged(expr) return quote println("Evaluating: ", $(string(expr))) $(esc(expr)) end end ``` ```{code-block} julia julia> @logged 2 + 3 * 4 Evaluating: 2 + 3 * 4 14 ``` Notice what a macro can do that a function cannot: it saw the *source text* of the expression, not just its value. A function receiving `2 + 3 * 4` gets `14` and has no way to know where that came from. That's the entire reason macros exist, and it's the test for whether you need one: > **Do you need the source code of the argument, or its value?** If the value, write a function. If the source, write a macro. This is why `@time` is a macro (it must wrap the expression in timing calls before it runs), why `@test` is a macro (it prints the failing expression back to you), and why `@.` is a macro (it rewrites the syntax tree, adding dots). ### `esc` and hygiene That `esc()` in the example is the one piece of macro-writing you have to understand. Julia macros are **hygienic**: variables the macro introduces are automatically renamed so they can't collide with variables in your code. That's a good default — a macro that internally uses a variable called `total` shouldn't clobber *your* `total`. But hygiene also means that when the macro returns your expression, Julia would rename *your* variables too, and they'd stop referring to what you meant. `esc()` marks an expression as "this belongs to the caller, leave it alone." The rule of thumb: **escape the user's expressions, don't escape anything you invented.** ### A macro worth writing Here's one I actually use — asserting a condition with a readable failure message: ```{code-block} julia macro check(condition, message = "check failed") return quote if !$(esc(condition)) error($message * ": " * $(string(condition))) end end end ``` ```{code-block} julia julia> balance = -50 julia> @check balance >= 0 "invalid ledger state" ERROR: invalid ledger state: balance >= 0 ``` The error message includes the source of the failing condition, which a plain function could never do. ## Macros you should know from the standard library You'll use these far more often than you'll write your own: ```{code-block} julia @time expr # time it and report allocations @allocated expr # just the bytes allocated @show x # print "x = value" — the best debugging tool in Julia @assert cond # runtime assertion @view A[1:10] # a slice without copying @. expr # broadcast everything in expr @inbounds expr # skip bounds checking (careful!) @threads for ... # parallelize a loop @code_warntype f(x) # show type inference results — the performance workhorse @which f(x) # which method does this call? @edit f(x) # open that method's source in your editor ``` `@show` deserves special mention. Instead of `println("balance = ", balance)`, write `@show balance` and get `balance = 1200.0`. It's a tiny thing that you'll use a hundred times a day. `@which` and `@edit` are the pair that make Julia's ecosystem readable. Wondering how a package does something? `@edit somefunc(x)` drops you into the actual source, which is Julia, which you can read. ## Generated functions One step beyond macros: a `@generated` function runs at compile time, sees the *types* of its arguments (not their values), and returns the code to compile for those types. ```{code-block} julia @generated function type_name(x) name = string(x) # x here is the TYPE, not the value return :($name) end ``` ```{code-block} julia julia> type_name(3) "Int64" julia> type_name("hi") "String" ``` The string was computed once, at compile time, for each type. At runtime there's no work at all. This is genuinely powerful for writing code that specializes on type structure — unrolling loops over a tuple's fields, generating optimal code for a fixed-size array. It's also a sharp tool with real constraints (a generated function can't call functions that were defined after it, and can't observe values). If you're reaching for one in ordinary application code, you have almost certainly overshot. ## When to use metaprogramming — and when not to I want to be direct, because this is the feature most likely to be misused by an enthusiastic beginner. **Good reasons to write a macro:** - You need to see the source expression (for error messages, timing, logging, or tests). - You're building a domain-specific language where the syntax genuinely helps — `@model` in a probabilistic programming package, `@variable` in an optimization package. - You're generating a family of near-identical method definitions that would otherwise be dozens of lines of copy-paste. **Bad reasons:** - To avoid writing a function. This is the big one. Most "I need a macro" instincts dissolve when you consider passing a function as an argument instead. - To make code shorter. Macros make code *less* readable to everyone who isn't you, because they can't be understood by reading normally — you have to know what the macro expands to. - For performance. Julia's compiler already specializes your functions. A macro is not going to beat it. Here's my honest summary after years of writing Julia: **you will read many macros and write very few.** That's the correct ratio. The ones in the standard library and in good packages are worth understanding thoroughly. Your own code is usually better off without them. ## Debugging a macro When a macro misbehaves, `@macroexpand` shows you exactly what it produced: ```{code-block} julia julia> @macroexpand @check balance >= 0 quote if !(balance >= 0) error("check failed" * ": " * "balance >= 0") end end ``` Being able to see the generated code takes almost all of the mystery out of macros. When something's wrong, expand it and read. ## Try it yourself 1. Build the expression `:(2 * x + 1)` by hand using `Expr(:call, ...)`, then confirm it matches the quoted version. 2. Write a macro `@twice expr` that evaluates its expression twice and returns the second result. Test it with something that prints, so you can see both runs. 3. Use `@macroexpand` on `@. a + b * c` and read what the dot macro actually generates. 4. Use `@which sum([1,2,3])` to find the method, then `@edit` to read its source. ### Solutions ```{code-block} julia # 1. e = Expr(:call, :+, Expr(:call, :*, 2, :x), 1) e == :(2 * x + 1) # true # 2. macro twice(expr) return quote $(esc(expr)) $(esc(expr)) end end @twice println("hello") # prints twice # 3. @macroexpand @. a + b * c # :(a .+ b .* c) ``` You now understand the layer of Julia that operates on code itself. Next we'll come back down to earth and look at how Julia talks to the rest of the world — starting with Python.