API

ExperimentalAPI

Say, at the definition site, that a public name is not settled yet — and turn that into a check something runs.

Visibility is already a language feature: export and public decide who is allowed to call a name. Stability is the orthogonal question — whether the name is finished — and today the only place to answer it is a sentence in a docstring that no tool reads and no CI job verifies.

using ExperimentalAPI

@experimental "reads Test's internal result tree; not dogfooded in CI yet" function render_test_report(records)
    # ...
end

That mark is three things at once, and only the third is the reason to have it:

• a declaration — the reason travels with the name, in the source, where the author is;
• a query — experimental hands a tool the list, stable hands it the complement;
• a check — audit reports every public name that is neither documented nor declared, so "document it or admit it is unfinished" becomes a test that fails.

The check is the point. A marker that is only ever written is a claim; a marker something compares against the public surface is a contract.

The three questions this answers

What this is not

• Not visibility. public (Julia 1.11) already decides who may call a name. A name can be public and experimental, or public and settled; those are independent axes.
• Not deprecation. @deprecate points the other way — a settled name on its way out.
• Not type stability. Unrelated axis, different tooling.
• Not a runtime wrapper. @experimental emits the definition unchanged plus one push! at load time. Calls are untouched.
• Not a documentation generator. The prose belongs to the author; this only ever checks whether prose exists.

Scope of the check

audit compares names(M) — exported and public names — against two accounts: a docstring, or a mark. It sees names, not signatures and not prose quality. A public name with a docstring reading "TODO" is accounted for; a settled name whose method signature changed under it is invisible here. See compare for the same limit on the release side.

Audit

What audit found. Every field is a sorted Vector{Symbol}, and every name in surface appears in exactly one of foreign, documented, declared, unaccounted — except that a name may be both documented and declared, in which case it is counted as documented.

unaccounted is the one a test asserts is empty. dangling needs no external oracle to be wrong: it is the module contradicting itself.

Diff

The result of compare: how a module's public surface moved between two snapshots.

isbreaking is the one-bit reading of the first two rows.

Mark

One @experimental declaration.

reason is the field the whole design exists for: why a name is not settled is knowledge only the author has, so it travels with the mark rather than being reconstructed by a reader later. file/line point at the declaration, which is the definition site in the attached form.

See experimental to read them back, and mark to look one up by name.

audit(m::Module) -> Audit

List the public names of m that are neither documented nor declared experimental.

julia> audit(Pinax).unaccounted
15-element Vector{Symbol}:
 :completeness_overview
 :dump_test_report
 ⋮

Two accounts are accepted, and the choice between them is the author's: write the docstring, or say @experimental and why. What is not accepted is saying nothing — which is the state a public name is in by default, and the state no reader can distinguish from a settled one.

Also reports dangling: marks on names that are not public. That check needs no reference implementation to be right, because the module is disagreeing with itself.

What this cannot see

• Signatures. A name that stays present while its arguments change is invisible here.
• Prose quality. A docstring exists or it does not; isdocumented reads no further.
• Methods on other packages' functions. They are not in names(m) and never will be.
• Names public only inside an extension, which is a separate module.

See test_surface to run this as a test, and snapshot to carry the result into a release decision.

compare(old::AbstractDict, new::Union{Module,AbstractDict}) -> Diff

Say mechanically what moved between two snapshots — and therefore whether the change is breaking.

d = compare(read_snapshot("api.toml"), MyPackage)
isbreaking(d) && error("breaking: $(d.removed_stable) removed, $(d.demoted) demoted")

A removed name that was declared @experimental in old lands in removed_experimental and does not make the change breaking. That is the whole contract: the mark was the notice, and it was given in the source, at the definition, before the removal.

demoted — a name that was stable and is now marked experimental — counts as breaking. Retroactively withdrawing a promise is a change to what callers were told, not a correction of it.

experimental(m::Module) -> Vector{Mark}

Every name in m declared @experimental, sorted by name.

This is the query the marker exists to make possible — a tool asks the module, rather than a human reading docstrings. A module with no marks answers with an empty vector; it never errors for not having opted in.

for mk in experimental(MyPackage)
    println(mk.name, " — ", mk.reason)
end

The result is a fresh vector; mutating it does not change the module. Note that a mark does not imply the name is public — see audit's dangling.

isbreaking(d::Diff) -> Bool

Whether d removes a settled name or demotes one — the two moves that break callers who were told the truth.

This is the release gate: a true here means the version bump is major (or, under Julia's 0.x convention, a minor bump). Because compare reads names and not signatures, false means no breakage of this kind was found, not nothing broke.

isdocumented(m::Module, name::Symbol) -> Bool

Whether name has a docstring.

A re-exported name counts: the lookup follows the binding to the module the name actually comes from, so a package that puts a dependency's name on its own surface is not asked to re-document it. audit still reports those separately as foreign, because who owns a name and who documented it are different questions.

This answers whether prose exists, never whether it is any good. A docstring reading "TODO" is documented as far as this package is concerned.

isexperimental(m::Module, name::Symbol) -> Bool

Whether name in m is declared @experimental.

The one-bit form of mark, for a caller that only needs the verdict — for instance the mechanical statement that changing this name is not breaking. Does not consult docstrings and does not consult visibility: it answers only whether a mark exists.

mark(m::Module, name::Symbol) -> Union{Mark,Nothing}

The Mark on name in m, or nothing if it carries none.

This is how the reason gets to whoever needs it — an error message, a docs page, a release checklist:

mk = mark(MyPackage, :render_report)
mk === nothing || @warn "not settled" mk.reason mk.tracking

read_snapshot(path::AbstractString) -> Dict{String,Any}

Read a TOML file written by write_snapshot.

snapshot(m::Module) -> Dict{String,Any}

Write down what m currently promises: its stable names, and its experimental ones with their reasons.

The result is plain Dict/String/Vector data, so TOML.print accepts it as-is (write_snapshot does that). Take one at each release; hand the old one and the new module to compare.

module = "MyPackage"
version = "0.4.2"
stable = ["adapt", "measure"]

[experimental.render_report]
reason = "reads Test's internal result tree"
tracking = "https://github.com/org/MyPackage.jl/issues/12"

stable(m::Module) -> Vector{Symbol}

The public surface of m minus the names declared @experimental — the names whose removal or renaming is a breaking change.

The covenant, in other words. snapshot writes this down so two releases can be compared; isbreaking is what reads the comparison.

surface(m::Module) -> Vector{Symbol}

The public surface of m: every name it exports or declares public, sorted, excluding the module's own name.

This is names(m), given a name — since Julia 1.11 that list is precisely "exported or public", which is the surface a stability claim has to be made about. Names reachable only as M.internal_thing are not here, and are nobody's promise.

test_surface(m::Module; skip = Symbol[], outputlevel::Int = 0) -> Audit

Assert, as a @testset, that every public name of m is either documented or declared @experimental — and that every mark applies to a name that is actually public.

Available once Test is loaded (it lives in a package extension, so ExperimentalAPI itself never pulls Test into a runtime dependency). Put it in runtests.jl:

using MyPackage, ExperimentalAPI, Test

ExperimentalAPI.test_surface(MyPackage)

skip is for adopting this on a package that already has a backlog: the listed names are allowed to be unaccounted for. A stale entry fails the test — a name in skip that has since been documented, declared, or removed is reported, so the list can only shrink.

Returns the Audit on the normal return path whether the testset passed or not. outputlevel ≥ 1 also prints it.

write_snapshot(path::AbstractString, m::Module) -> String

snapshot m and write it to path as TOML. Returns path.

Committing this file at each release is what gives the next release something to compare against; a repository with no committed snapshot can be told what is experimental now, but not what changed.

@experimental "reason" definition
@experimental "reason" name₁ name₂ …
@experimental "reason" since=v"0.4.0" tracking="…" definition

Declare that a public name is not settled, and say why.

The reason is required and comes first. Everything after it is either one definition — which is emitted unchanged, so this costs nothing at run time — or a list of names already defined elsewhere:

# attached to the definition
@experimental "signature will be wrapped once the write-back refactor settles" function ingest(config; doc, kwargs...)
    # ...
end

# declared for names defined in an included file
@experimental "reads Test's internal result tree; not dogfooded in CI" render_test_report dump_test_report load_test_dump

Optional since= and tracking= come between the reason and the subject. tracking is what turns a mark into something a reader can act on: the issue or PR where the shape is being decided.

Attaches to function, short-form f(x) = …, struct, mutable struct, abstract type, primitive type, macro (recorded as Symbol("@name")), const, and plain assignment. Anything else — a module (which Julia requires as a direct top-level statement, so it cannot be wrapped), a definition produced by another macro, a qualified Base.foo(…) method — is rejected with a message pointing at the name-list form, because guessing which name such an expression defines is exactly the kind of silence this package exists to remove.

Marking is not visibility: a marked name still has to be exported or declared public to be part of the surface. A mark on a name that is neither is reported by audit as dangling.

See also experimental, audit, stable.