Using Python from Julia

Using Python from Julia#

Let’s be honest about something. Julia’s ecosystem is excellent in the corners it cares about and thin elsewhere. Python’s ecosystem is enormous. If Julia forced you to give up scikit-learn, Hugging Face, boto3, and every other library you already depend on, adopting it would be a much harder sell.

It doesn’t. You can call Python from Julia, in-process, with automatic type conversion, and it’s genuinely pleasant. This chapter is about how — and, just as importantly, about when the boundary costs you something.

PythonCall.jl#

There are two packages for this. PyCall.jl is the older one, and you’ll see it in a lot of existing code. PythonCall.jl is the newer one, and it’s what I’d recommend for new work: it manages its own Python installation via CondaPkg.jl, so your Julia project’s Python dependencies are declared and reproducible just like the Julia ones.

pkg> add PythonCall

The first time you load it, it sets up an isolated Python environment inside your project. No system Python to conflict with, no “which pip did that install go to” confusion.

using PythonCall

np = pyimport("numpy")

a = np.array([1, 2, 3])
b = np.array([4, 5, 6])
np.dot(a, b)              # 32

That’s the whole idea. pyimport gives you a Python module, and you use it with normal Julia dot syntax.

Declaring Python dependencies#

Rather than shelling out to pip, declare them:

julia> using CondaPkg
julia> ] conda add scikit-learn pandas

This writes a CondaPkg.toml next to your Project.toml. Commit both, and a colleague running ] instantiate gets your Julia and Python dependencies at the right versions. This is the reproducibility story from the last chapter extended across the language boundary, and it’s the main reason I prefer PythonCall over PyCall.

Conversions#

Values crossing the boundary get converted:

using PythonCall

# Julia → Python
pylist([1, 2, 3])
pydict(Dict("a" => 1))
Py(3.14)

# Python → Julia
x = pyimport("math").pi
pyconvert(Float64, x)        # 3.141592653589793

PythonCall is deliberately explicit about conversion in the Python-to-Julia direction. That feels like extra typing until the first time an implicit conversion silently gives you a Vector{Any} and tanks your performance. Being asked “what type do you want this as?” is the package doing you a favour.

For NumPy arrays specifically, there’s a zero-copy path:

using PythonCall

np = pyimport("numpy")
py_arr = np.array([1.0, 2.0, 3.0])

jl_arr = pyconvert(Array, py_arr)     # Julia array sharing the same memory

No copy means a big NumPy array can be handed to Julia and worked on natively without paying to duplicate it. This matters when you’re passing training batches back and forth.

A realistic example#

Here’s the shape of what I actually do: use Python for a mature library, Julia for the computation.

using PythonCall
using Statistics

pd = pyimport("pandas")

# Let pandas do what pandas is good at: messy real-world I/O
df = pd.read_csv("ledger_export.csv")
amounts = pyconvert(Vector{Float64}, df["amount"].values)

# Then compute in Julia, where the loops are fast
function rolling_zscore(xs, window)
    out = similar(xs)
    for i in eachindex(xs)
        lo = max(firstindex(xs), i - window + 1)
        w = @view xs[lo:i]
        out[i] = (xs[i] - mean(w)) / (std(w) + eps())
    end
    return out
end

scores = rolling_zscore(amounts, 30)

Read that boundary carefully. Python reads the CSV — a task where pandas is battle-tested against every malformed file in existence. The data crosses once. Then the actual per-element computation happens in Julia, where a scalar loop compiles to machine code.

Plotting through Python#

If you have muscle memory for matplotlib, you can keep it:

using PythonCall
plt = pyimport("matplotlib.pyplot")

plt.plot([1, 2, 3], [4, 5, 6])
plt.xlabel("period")
plt.show()

Though I’d encourage you to try Plots.jl or Makie.jl at least once. Makie in particular does things — interactive, GPU-accelerated, million-point plots — that are hard to get elsewhere.

Going the other direction#

juliacall is PythonCall’s other half: a Python package that lets Python call Julia.

# In Python
from juliacall import Main as jl

jl.seval("f(x) = x^2 + 1")
jl.f(3)          # 10

This is the deployment path a lot of teams end up taking: keep the existing Python application, rewrite the numerically expensive kernel in Julia, and call into it. It’s the two-language problem again, but now you control both halves and they’re in the same process.

The cost of the boundary#

Now the important part, which the enthusiastic tutorials tend to skip.

Every crossing has overhead. A single call into Python costs on the order of microseconds — negligible once, ruinous a million times. This is the mistake to avoid:

# BAD — crosses the boundary once per element
total = 0.0
for i in 1:1_000_000
    total += pyconvert(Float64, py_array[i])
end

# GOOD — crosses once, then works natively
jl_array = pyconvert(Vector{Float64}, py_array)
total = sum(jl_array)

Same principle as everywhere in performance work: batch at the boundary. Move a lot of data across once, rather than a little data across many times.

You lose type inference. Anything coming out of Python is a Py until you convert it. If you pass a Py into your hot loop, the compiler can’t specialize and you’re back to dynamic dispatch on every operation. Convert to a concrete Julia type at the boundary, not deep inside your algorithm.

The GIL is still the GIL. Python’s global interpreter lock applies to the Python side. Julia’s multithreading doesn’t magically parallelize Python calls.

When to reach for Python, and when not to#

Reach for Python when:

  • The library is mature, complex, and has no Julia equivalent (deep learning model zoos, cloud SDKs, domain-specific scientific packages).

  • It’s I/O or setup code that runs once, where the boundary cost is irrelevant.

  • You’re incrementally migrating an existing Python codebase and want to move one piece at a time.

Stay in Julia when:

  • The computation is the point. Calling NumPy from Julia to do array math is round-tripping to avoid using the language you’re already in.

  • You’re in a hot loop. See above.

  • A good Julia package exists. Check juliahub.com first — the answer is more often yes than newcomers expect.

I use this bridge more than I expected to and less than I feared. It’s an escape hatch that means “no Julia package for X” is never a blocker, and that’s exactly what it should be.

Try it yourself#

  1. Install PythonCall, import math, and compute math.factorial(20). Convert the result to a Julia Int.

  2. Use CondaPkg to add a Python package, then look at the CondaPkg.toml it creates.

  3. Create a NumPy array of a million random floats, convert it to a Julia Vector{Float64}, and time sum on both sides with @time.

  4. Write a loop that calls a Python function a thousand times, and a version that passes the whole array once. Compare with @time and see the boundary cost for yourself.

Solutions#

# 1.
using PythonCall
m = pyimport("math")
pyconvert(Int, m.factorial(20))     # 2432902008176640000

# 3.
np = pyimport("numpy")
py_a = np.random.rand(1_000_000)
jl_a = pyconvert(Vector{Float64}, py_a)

@time pyconvert(Float64, np.sum(py_a))
@time sum(jl_a)

Exercise 4 is the one to actually run. Seeing the number yourself is worth more than my telling you the boundary is expensive.

Next we go one layer lower and talk to C — and to memory itself.