Internals of JET.jl

bail_out_toplevel_call(interp::JETInterpreter, ...)

An overload for abstract_call_gf_by_type(interp::JETInterpreter, ...), which keeps inference on non-concrete call sites in a toplevel frame created by virtual_process.

bail_out_call(interp::JETInterpreter, ...)

With this overload, abstract_call_gf_by_type(interp::JETInterpreter, ...) doesn't bail out inference even after the current return type grows up to Any and collects as much error points as possible. Of course this slows down inference performance, but hoopefully it stays to be "practical" speed since the number of matching methods are limited beforehand.

add_call_backedges!(interp::JETInterpreter, ...)

An overload for abstract_call_gf_by_type(interp::JETInterpreter, ...), which always add backedges (even if a new method can't refine the return type grew up to Any). This is because a new method definition always has a potential to change the JET analysis result.

const_prop_entry_heuristic(interp::JETInterpreter, @nospecialize(rettype), sv::InferenceState, edgecycle::Bool)

An overload for abstract_call_method_with_const_args(interp::JETInterpreter, ...), which forces constant prop' even if the inference result can't be improved anymore, e.g. when rettype is already Const; this is because constant prop' can still produce more accurate analysis by throwing away false positive error reports by cutting off the unreachable control flow.

analyze_task_parallel_code!(interp::JETInterpreter, @nospecialize(f), argtypes::Vector{Any}, sv::InferenceState)

Adds special cased analysis pass for task parallelism (xref: https://github.com/aviatesk/JET.jl/issues/114). In Julia's task parallelism implementation, parallel code is represented as closure and it's wrapped in a Task object. NativeInterpreter doesn't run type inference nor optimization on the body of those closures when compiling code that creates parallel tasks, but JET will try to run additional analysis pass by recurring into the closures.

is_from_same_frame(parent_linfo::MethodInstance, current_linfo::MethodInstance) ->
    (report::InferenceErrorReport) -> Bool

Returns a function that checks if a given InferenceErrorReport is generated from current_linfo. It also checks current_linfo is a "lineage" of parent_linfo (i.e. entered from it).

This function is supposed to be used to filter out reports from analysis on current_linfo without using constants when entering into the constant analysis. As such, this function assumes that when a report should be filtered out, the first elment of its virtual stack frame st is for parent_linfo and the second element of that is for current_linfo.

Example: Assume linfo2 will produce a report for some reason. In the example analysis below, report2 will be filtered out on re-entering into linfo3 with constants (i.e. linfo3′). Note that report1 is still kept there because of the lineage check.

entry
└─ linfo1
   ├─ linfo2 (report1: linfo2)
   ├─ linfo3 (report1: linfo1->linfo2, report2: linfo3->linfo2)
   │  └─ linfo2 (report1: linfo2, report2: linfo2)
   └─ linfo3′ (report1: linfo1->linfo2, ~~report2: linfo1->linfo3->linfo2~~)

mutable struct AbstractGlobal
    t::Any     # analyzed type
    iscd::Bool # whether this abstract global variable is declarared as constant or not
end

Wraps a global variable whose type is analyzed by abtract interpretation. AbstractGlobal object will be actually evaluated into the context module, and a later analysis may refer to its type or alter it on another assignment.

virtual_process(s::AbstractString,
                filename::AbstractString,
                actualmod::Module,
                interp::JETInterpreter,
                config::ToplevelConfig,
                virtualmod::Module,
                ) -> res::VirtualProcessResult

Simulates Julia's toplevel execution and collects error points, and finally returns res::VirtualProcessResult

• res.included_files::Set{String}: files that have been analyzed
• res.toplevel_error_reports::Vector{ToplevelErrorReport}: toplevel errors found during the text parsing or partial (actual) interpretation; these reports are "critical" and should have precedence over inference_error_reports
• res.inference_error_reports::Vector{InferenceErrorReport}: possible error reports found by JETInterpreter
• res.toplevel_signatures: signatures of methods defined within the analyzed files
• res.actual2virtual::Pair{Module, Module}: keeps actual and virtual module

This function first parses s::AbstractString into toplevelex::Expr and then iterate the following steps on each code block (blk) of toplevelex:

1. if blk is a :module expression, recusively enters analysis into an newly defined virtual module
2. lowers blk into :thunk expression lwr (macros are also expanded in this step)
3. replaces self-references of the original root module (i.e. actualmod) with that of virtualmod: see fix_self_references
4. ConcreteInterpreter partially interprets some statements in lwr that should not be abstracted away (e.g. a :method definition); see also partially_interpret!
5. finally, JETInterpreter analyzes the remaining statements by abstract interpretation

ConcreteInterpreter

The trait to inject code into JuliaInterpreter's interpretation process; JET.jl overloads:

• JuliaInterpreter.step_expr! to add error report pass for module usage expressions and support package analysis
• JuliaInterpreter.evaluate_call_recurse! to special case include calls
• JuliaInterpreter.handle_err to wrap an error happened during interpretation into ActualErrorWrapped

partially_interpret!(interp::ConcreteInterpreter, mod::Module, src::CodeInfo)

Partially interprets statements in src using JuliaInterpreter.jl:

• concretizes "toplevel definitions", i.e. :method, :struct_type, :abstract_type and :primitive_type expressions and their dependencies
• concretizes user-specified toplevel code (see ToplevelConfig)
• directly evaluates module usage expressions and report error of invalid module usages (TODO: enter into the loaded module and keep JET analysis)
• special-cases include calls so that top-level analysis recursively enters the included file

@invoke f(arg::T, ...; kwargs...)

Provides a convenient way to call invoke; @invoke f(arg1::T1, arg2::T2; kwargs...) will be expanded into invoke(f, Tuple{T1,T2}, arg1, arg2; kwargs...). When an argument's type annotation is omitted, it's specified as Any argument, e.g. @invoke f(arg1::T, arg2) will be expanded into invoke(f, Tuple{T,Any}, arg1, arg2).

This could be used to call down to NativeInterpreter's abstract interpretation method of f while passing JETInterpreter so that subsequent calls of abstract interpretation functions overloaded against JETInterpreter can be called from the native method of f; e.g. calls down to NativeInterpreter's abstract_call_gf_by_type method:

@invoke abstract_call_gf_by_type(interp::AbstractInterpreter, f, argtypes::Vector{Any}, atype, sv::InferenceState,
                                 max_methods::Int)

@invokelatest f(args...; kwargs...)

Provides a convenient way to call Base.invokelatest. @invokelatest f(args...; kwargs...) will simply be expanded into Base.invokelatest(f, args...; kwargs...).

@withmixedhash (mutable) struct T
    fields ...
end

Defines struct T while automatically defining its Base.hash(::T, ::UInt) method which mixes hashes of all of T's fields (and also corresponding Base.:(==)(::T, ::T) method).

This macro is supposed to abstract the following kind of pattern:

https://github.com/aviatesk/julia/blob/999973df2850d6b2e0bd4bcf03ef90a14217b63c/base/pkgid.jl#L3-L25

struct PkgId
    uuid::Union{UUID,Nothing}
    name::String
end

==(a::PkgId, b::PkgId) = a.uuid == b.uuid && a.name == b.name

function hash(pkg::PkgId, h::UInt)
    h += 0xc9f248583a0ca36c % UInt
    h = hash(pkg.uuid, h)
    h = hash(pkg.name, h)
    return h
end

with @withmixedhash

@withmixedhash struct PkgId
    uuid::Union{UUID,Nothing}
    name::String
end

See also: EGAL_TYPES

@jetconfigurable function config_func(args...; configurations...)
    ...
end

This macro asserts that there's no configuration naming conflict across the @jetconfigurable functions so that a configuration for a @jetconfigurable function doesn't affect the other @jetconfigurable functions. This macro also adds a dummy splat keyword arguments (jetconfigs...) to the function definition so that any configuration of other @jetconfigurable functions can be passed on to it.