API — observing and analysing

Entry

One marked definition that the current run entered, as reported by entered.

count is always nothing. The default layer knows whether a definition was entered, never how often — a per-call counter costs 3.76x on eight threads and loses 40% of its increments to races unless it is atomic. Counting is record's job, and it is opt-in for that reason.

Probe

The one-field flag @experimental puts in a marked body, and the counter record reads.

Not part of the public surface — it is named rather than gensym'd only so that MyModule.__EXPERIMENTAL_API_ENTERED_energy__ is greppable when a query result surprises someone.

entered is the whole default layer: read on every call, written on the first. The remaining fields are untouched unless a recording is open.

detecting() -> Bool

Whether the summary at process exit is armed.

On by default, because a user who never asks is exactly the one who needs to be told. Set ENV["EXPERIMENTALAPI_SUMMARY"] = "0" before using ExperimentalAPI to silence it; the atexit hook is registered at load time, so a later assignment has no effect.

Detection itself is not a switch: the flag is in the definition and costs nothing to leave on. See overhead_when_detecting for the figure that justifies that sentence.

entered(m::Module) -> Vector{Entry}
entered() -> Vector{Entry}

The marked definitions this run has entered at least once.

Without an argument, every loaded module that carries marks. This is the question a docstring cannot answer: not "is this name experimental" but "did the number I am about to publish come out of code nobody has validated".

A definition that was never called is absent, not reported with a count of zero.

julia> MyPkg.energy(0.5);

julia> entered()
1-element Vector{ExperimentalAPI.Entry}:
 Entry(MyPkg.energy, "convergence not established below β ≈ 0.1")

Only definitions with a body are observed; see @experimental for which forms those are.

marked_modules() -> Vector{Module}

Every loaded module that carries at least one @experimental mark.

Found by walking the loaded packages rather than by a registry inside this package: a table here would be written while the marked package is precompiled, and so would be absent from its cache image. Same constraint that puts the marks themselves in the marked module.

overhead_when_detecting() -> Float64

The measured cost of the default layer, as a fraction of the unmarked body's run time.

0.03. Not computed at call time: it is the figure from the measurement that chose this mechanism — 10M calls of a realistic numeric body on Julia 1.12.2, minimum of 7–9 trials, giving 1.03x on one thread and 0.985x on eight. The eight-thread figure is below one because a flag written once and read thereafter stops dirtying its cache line.

It is reported rather than re-measured because a wall-clock measurement on a shared CI runner is a flake generator, and because a number that moves with the machine cannot be the thing a caller plans against. record reports its own overhead per run, which is the opposite case: that one depends on how often the marked code was entered.

probes(m::Module) -> Vector{Probe}
probes() -> Vector{Probe}

Every Probe a module carries — one per marked definition that has a body.

The recording layer arms and reads these; nothing else should need them. Exposed because record's cost model is only checkable by someone who can see how many probes there are.

summary_text() -> String
summary_text(es::AbstractVector{Entry}) -> String

What the exit summary prints, as a string. Empty when nothing marked was entered.

Carries the reason, not just the name: the name tells a reader which line to look at, the reason tells them whether the result they are holding is affected.

Attribution

One marked definition's share of a profile buffer, as reported by attribute.

samples is a sample count, never a call count: inclusive is the fraction of samples with this definition anywhere on the stack, exclusive the fraction with it on top.

Hit

One marked definition a record block entered, and what it cost.

count is exact. paths is a bounded sample: a backtrace costs microseconds, so the recorder stops looking once it has seen enough, and a path that only occurs after the budget is spent is absent. inclusive counts time in anything this definition called; exclusive counts only time in the definition itself, which is what separates a marked wrapper over settled code from a marked kernel.

Record

What record observed: a Vector-like of Hit, plus what the run was.

Indexing, iteration and == are the Hit vector's, so record(f) == [] reads the way it looks. The extra properties are why it is a type and not a plain vector: an empty Vector{Hit} cannot tell "touched nothing" from "recording was off", and those mean opposite things.

TimingBackend

How record gets inclusive/exclusive time. The one implementation lives in this package's Profile extension; without it, timing is missing rather than zero.

assert_clean(f; throw = true) -> Bool

Run f and assert it entered nothing marked.

The gate. true when the run touched no marked definition; otherwise it throws, naming every mark it went through and why — or returns false if throw = false, which is what a caller doing its own reporting wants.

assert_clean() do
    publish(compute(model))
end

attribute(data) -> Vector{Attribution}

Attribute an existing profile buffer to marked definitions, after the fact.

using Profile
Profile.@profile long_run()
attribute(Profile.fetch())

The twelve-hour-run case: a job that was already profiled must not have to be run again to learn which of its time went through unvalidated code. What comes back is samples, not calls — which is why it is an Attribution and not a Hit. A sampling profiler cannot count entries, and a field called count holding a sample total would read as a measurement it did not make.

experimental_fraction(r::Record) -> Union{Float64,Missing}

The share of the recorded run spent inside marked code, in [0, 1].

missing when no TimingBackend was loaded — a run whose time was never attributed has no fraction, and reporting 0.0 would say the opposite of what is known. Derived from the inclusive times, so it is time and not calls: one entry into a marked kernel that runs for a minute matters more than a million into a marked accessor.

merge_records(rs) -> Record

Combine records — from separate processes, separate workers, or separate blocks — into one.

Counts add, paths and callers union, elapsed times add. Order-independent and associative: workers finish in whatever order they finish in, and provenance must not depend on that.

read_record(path::AbstractString) -> Record

Read back a record written by write_record.

The method field of every Hit comes back nothing: a Method is not a thing a file can carry, and reconstructing one would mean claiming the code in this process is the code that produced the record.

record(f; paths = true, timing = false, with_profile = false, rethrow = true) -> Record

Run f and report which marked definitions it entered, how often, and by which paths.

r = record() do
    simulate(model; steps = 10_000)
end
isempty(r) || @warn "used unvalidated code" [(h.name, h.count) for h in r]

A definition that was never entered is absent, not reported with a count of zero: the record says what happened, and enumerating what did not is experimental's job.

Nests: an inner record reports its own block, and the outer one still counts each entry once. Counts are exact under threads — per-thread counters sized by Threads.maxthreadid(), not nthreads(), because the interactive pool means a task can have a thread id above the worker count.

recording() -> Bool

Whether a record block is open.

false by default and outside record, which is the whole cost argument: detection is on always and free, counting happens only where somebody asked for it.

timing_backend() -> TimingBackend

The loaded timing backend. NoTiming() until Profile is loaded, after which record reports seconds instead of missing.

write_record(path::AbstractString, r::Record) -> String

Write a record to TOML. Returns path.

Evidence, not a printout: what a run went through belongs next to the result it produced, and it has to be readable by something that is not this package — a year later the package may not resolve. See stamp for the same idea aimed at a result file rather than at a record.

@entered expr

Evaluate expr, print which marked definitions it went through, and return its value.

The question entered answers about a whole process, asked about one call:

julia> ExperimentalAPI.@entered sweep(model; βs = 0.05:0.05:2.0)
┌ @entered sweep(model; βs = 0.05:0.05:2.0)   at sweep.jl:42
│   MyPkg.energy       ×10000 — convergence not established below β ≈ 0.1
│   MyPkg.correlator   ×  500 — edge cases at zero separation untested
└ 15 of 17 observable marked definitions were not entered
0.42713…

The value of expr comes back, so this drops into existing code the way @time does. The last line is the one that makes a clean answer mean something:

julia> ExperimentalAPI.@entered publish(result)
┌ @entered publish(result)   at sweep.jl:57
└ entered nothing marked — 17 observable marked definitions were loaded

"Entered nothing" and "nothing is marked anywhere" are different states, and a report that could not tell them apart would be worth nothing on a package that has no marks yet.

What it is, exactly

record(() -> expr; paths = false, timing = false), plus the report. It asks which and how often — the cheap question, and the one that needs neither a backtrace nor a sampler. Call record directly for call paths, for inclusive/exclusive time (never both — see the measurement in its docstring), and for the Record as data. This returns the value of expr, not the record.

See also entered for the whole-process question, record for the full instrument, and reach for the same question asked without running anything.

Reach

What reach found.

There is deliberately no verdict field. A stored verdict makes :clean with a non-empty unresolved representable, and that state is the single thing this analysis must never report. verdict derives it instead, the way isbreaking derives its answer from a Diff.

Reached

One marked definition the analysis proved reachable from the entry point.

method is the method that was resolved — nothing when the dependency is not a call, which is what a marked const is. path is the chain of names from the entry point down to it, so the reader learns which of their own code to distrust rather than only that something is wrong.

Unresolved

One call site the analysis could not pin to a method.

"Cannot tell" is only actionable if the reader can go and look, which is what file/line are for. candidates is the difference between "some call here is dynamic" and "this call could go to k(::KB), which is marked".

combine(a::Symbol, b::Symbol) -> Symbol

Fold two verdicts into one: :clean < :unknown < :depends.

Needed because reach on a module folds one answer per public entry point into one answer for the module. Commutative and associative, so the order the entry points come back in cannot change the result.

dependents(m::Module, name::Symbol; kwargs...) -> Vector{Symbol}

The public names of m whose call graph reaches name.

Propagation read backwards, which is the direction the question is actually asked in: a mark gets deleted because somebody looked at the definition, not at who reaches it. Compare reach(m; ignore = [name]) to see what removing it would change.

isclean(r::Reach) -> Bool

Whether verdict is :clean. false for :unknown as well as for :depends — the predicate answers "may I rely on this", and the honest non-answer is not a yes.

reach(f, types::Type{<:Tuple}; maxdepth = 32, ignore = Symbol[]) -> Reach
reach(m::Module; kwargs...) -> Reach

Report whether calling f with types can reach anything declared @experimental — including through callers that never name it.

r = reach(analyse, Tuple{Model,Float64})
verdict(r) === :clean || error("depends on: ", [x.mark.name for x in r.reached])

The module form folds every public entry point of m into one answer and reports which of them are affected in affected_entries: function-by-function does not scale to a package.

ignore names marks to treat as absent, which answers "what would removing this mark change?" without removing it. maxdepth bounds the walk and maxcandidates bounds how many methods a single ambiguous call site is willing to check; hitting either bound is reported as :unknown, never as :clean.

What it can and cannot resolve

A more specific unmarked method shadowing a marked one is resolved as what actually runs: Int goes to more_specific(::Int) and is clean, while UInt8 falls through to the marked ::Integer and is not.

reach_script(path::AbstractString; kwargs...) -> Reach

The same analysis for a script — a file that produces a figure rather than a package.

Top-level declarations that cannot live inside a function (using, import, module, const, type definitions) are evaluated in a scratch module, because the analysis has to resolve the names the script uses; everything else is analysed as one thunk. So this loads the script's dependencies, and a script whose top level has side effects will have them.

verdict(r::Reach) -> Symbol

:depends, :unknown or :clean, derived from what reach found.

• :depends — a marked definition is reachable. Proved, not suspected.
• :clean — the whole call graph was resolved and nothing marked is in it.
• :unknown — at least one call site could not be pinned to a method, and nothing marked was proved reachable through the rest.

:depends wins over :unknown: an unresolvable call elsewhere does not make a proved dependency less proved. :unknown wins over :clean, which is the whole point.