AbstractAnalyzer Framework

JET offers an infrastructure to implement a "plugin" code analyzer. Actually, JET's default error analyzer is one specific instance of such a plugin analyzer built on top of the framework.

In this documentation we will try to elaborate the framework APIs and showcase example analyzers.

Warning

The APIs described in this page is very experimental and subject to changes. And this documentation is also very WIP.

Interfaces

JET.AbstractAnalyzerType
abstract type AbstractAnalyzer <: AbstractInterpreter end

An interface type of analyzers that are built on top of JET's analyzer framework.

When a new type NewAnalyzer implements the AbstractAnalyzer interface, it should be declared as subtype of AbstractAnalyzer, and is expected to the following interfaces:


  1. JETInterface.AnalyzerState(analyzer::NewAnalyzer) -> AnalyzerState:
    Returns the AnalyzerState for analyzer::NewAnalyzer.

  1. JETInterface.AbstractAnalyzer(analyzer::NewAnalyzer, state::AnalyzerState) -> NewAnalyzer:
    Constructs an new NewAnalyzer instance in the middle of JET's top-level analysis or abstract interpretation, given the previous analyzer::NewAnalyzer and state::AnalyzerState.

  1. JETInterface.ReportPass(analyzer::NewAnalyzer) -> ReportPass:
    Returns ReportPass used for analyzer::NewAnalyzer.

  1. JETInterface.AnalysisCache(analyzer::NewAnalyzer) -> analysis_cache::AnalysisCache:
    Returns code cache used for analyzer::NewAnalyzer.

See also AnalyzerState, ReportPass and AnalysisCache.

Example

JET.jl defines its default error analyzer JETAnalyzer <: AbstractAnalyzer as the following (modified a bit for the sake of simplicity):

# the default error analyzer for JET.jl
struct JETAnalyzer{RP<:ReportPass} <: AbstractAnalyzer
    state::AnalyzerState
    analysis_cache::AnalysisCache
    report_pass::RP
end

# AbstractAnalyzer API requirements
JETInterface.AnalyzerState(analyzer::JETAnalyzer) = analyzer.state
JETInterface.AbstractAnalyzer(analyzer::JETAnalyzer, state::AnalyzerState) = JETAnalyzer(ReportPass(analyzer), state)
JETInterface.ReportPass(analyzer::JETAnalyzer) = analyzer.report_pass
JETInterface.AnalysisCache(analyzer::JETAnalyzer) = analyzer.analysis_cache
source
JET.AnalyzerStateType
mutable struct AnalyzerState
    ...
end

The mutable object that holds various states that are consumed by all AbstractAnalyzers.


JETInterface.AnalyzerState(analyzer::AbstractAnalyzer) -> AnalyzerState

If NewAnalyzer implements the AbstractAnalyzer interface, NewAnalyzer should implement this AnalyzerState(analyzer::NewAnalyzer) -> AnalyzerState interface.

A new AnalyzerState is supposed to be constructed using the general configurations passed as keyword arguments jetconfigs of the NewAnalyzer(; jetconfigs...) constructor, and the constructed AnalyzerState is usually kept within NewAnalyzer itself:

function NewAnalyzer(world::UInt=Base.get_world_counter(); jetconfigs...)
    ...
    state = AnalyzerState(world; jetconfigs...)
    return NewAnalyzer(..., state)
end
JETInterface.AnalyzerState(analyzer::NewAnalyzer) = analyzer.state
source
JET.ReportPassType
abstract type ReportPass end

An interface type that represents AbstractAnalyzer's report pass. analyzer::AbstractAnalyzer injects report passes using the (::ReportPass)(::Type{InferenceErrorReport}, ::AbstractAnalyzer, state, ...) interface, which provides a flexible and efficient layer to configure the analysis done by AbstractAnalyzer.


JETInterface.ReportPass(analyzer::AbstractAnalyzer) -> ReportPass

If NewAnalyzer implements the AbstractAnalyzer interface, NewAnalyzer should implement this ReportPass(analyzer::NewAnalyzer) -> ReportPass interface.

ReportPass allows NewAnalyzer to provide a very flexible configuration layer for NewAnalyzer's analysis; an user can define their own ReportPass to control how NewAnalyzer collects report errors while still using the analysis routine implemented by NewAnalyzer.

Example

For example, JETAnalyzer accepts a custom ReportPass passed as part of the general configurations (see the documentation of AbstractAnalyzer for an example implementation). And we can setup a custom report pass IgnoreAllExceptGlobalUndefVar, that ignores all the reports that are otherwise collected by JETAnalyzer except UndefVarErrorReport:

# custom report pass that ignores all the reports except `UndefVarErrorReport`
struct IgnoreAllExceptGlobalUndefVar <: ReportPass end

# ignores all the reports analyzed by `JETAnalyzer`
(::IgnoreAllExceptGlobalUndefVar)(::Type{<:InferenceErrorReport}, @nospecialize(_...)) = return

# forward to `BasicPass` to collect `UndefVarErrorReport`
function (::IgnoreAllExceptGlobalUndefVar)(::Type{UndefVarErrorReport}, @nospecialize(args...))
    BasicPass()(UndefVarErrorReport, args...)
end

no_method_error()    = 1 + "1"
undef_global_error() = undefvar
report_call(; report_pass=IgnoreAllExceptGlobalUndefVar()) do
    if rand(Bool)
        return no_method_error()    # "no matching method found" error report won't be reported here
    else
        return undef_global_error() # "`undefvar` is not defined" error report will be reported
    end
end
source
JET.AnalysisCacheMethod
AnalysisCache(analyzer::AbstractAnalyzer) -> analysis_cache::AnalysisCache

Returns AnalysisCache for this analyzer::AbstractAnalyzer. AbstractAnalyzer instances can share the same cache if they perform the same analysis, otherwise their cache should be separated.

source
JET.valid_configurationsFunction
JETInterface.valid_configurations(analyzer::AbstractAnalyzer) -> names or nothing

Returns a set of names that are valid as a configuration for analyzer. names should be an iterator of Symbol. No validations are performed if nothing is returned.

source
JET.aggregation_policyFunction
JETInterface.aggregation_policy(analyzer::AbstractAnalyzer)

Defines how analyzer aggregates InferenceErrorReports. Defaults to default_aggregation_policy.


default_aggregation_policy(report::InferenceErrorReport) -> DefaultReportIdentity

Returns the default identity of report::InferenceErrorReport, where DefaultReportIdentity aggregates reports based on "error location" of each report. DefaultReportIdentity aggregates InferenceErrorReports aggressively in a sense that it ignores the identity of error point's MethodInstance, under the assumption that errors are identical as far as they're collected at the same file and line.

source
JET.VSCode.vscode_diagnostics_orderFunction
vscode_diagnostics_order(analyzer::AbstractAnalyzer) -> Bool

If true (default) a diagnostic will be reported at entry site. Otherwise it's reported at error point.

source
JET.InferenceErrorReportMethod
InferenceErrorReport

In order for Report <: InferenceErrorReport to implement the interface, it should satisfy the following requirements:

Report <: InferenceErrorReport is supposed to be constructed using the following constructor

Report(::AbstractAnalyzer, state, spec_args...) -> Report

where state can be either of:

  • state::Tuple{Union{Core.Compiler.InferenceState, Core.Compiler.OptimizationState}, Int64}: a state with the current program counter specified
  • state::InferenceState: a state with the current program counter set to state.currpc
  • state::InferenceResult: a state with the current program counter unknown
  • state::MethodInstance: a state with the current program counter unknown

See also: @jetreport, VirtualStackTrace, VirtualFrame

source
JET.copy_reportFunction
JETInterface.copy_report(orig::Report) where Report<:InferenceErrorReport -> new::Report

Returns new new::Report, that should be identical to the original orig::Report, except that new.vst is copied from orig.vst so that the further modification on orig.vst that may happen in later abstract interpretation doesn't affect new.vst.

source
JET.report_colorFunction
JETInterface.report_color(::Report) where Report<:InferenceErrorReport -> Symbol

Configures the color for Report (defaults to :red).

source
JET.analyze_and_report_call!Function
analyze_and_report_call!(analyzer::AbstractAnalyzer, f, [types]; jetconfigs...) -> JETCallResult
analyze_and_report_call!(analyzer::AbstractAnalyzer, tt::Type{<:Tuple}; jetconfigs...) -> JETCallResult
analyze_and_report_call!(analyzer::AbstractAnalyzer, mi::MethodInstance; jetconfigs...) -> JETCallResult

A generic entry point to analyze a function call with AbstractAnalyzer. Finally returns the analysis result as JETCallResult. Note that this is intended to be used by developers of AbstractAnalyzer only. General users should use high-level entry points like report_call and report_opt.

source
JET.call_test_exFunction
call_test_ex(funcname::Symbol, testname::Symbol, ex0, __module__, __source__)

An internal utility function to implement a @test_call-like macro. See the implementation of @test_call.

source
JET.func_testFunction
func_test(func, testname::Symbol, args...; jetconfigs...)

An internal utility function to implement a test_call-like function. See the implementation of test_call.

source
JET.analyze_and_report_file!Function
analyze_and_report_file!(analyzer::AbstractAnalyzer, filename::AbstractString; jetconfigs...) -> JETToplevelResult

A generic entry point to analyze a file with AbstractAnalyzer. Finally returns the analysis result as JETToplevelResult. Note that this is intended to be used by developers of AbstractAnalyzer only. General users should use high-level entry points like report_file.

source
JET.analyze_and_report_package!Function
analyze_and_report_package!(analyzer::AbstractAnalyzer,
                            package::Union{AbstractString,Module,Nothing} = nothing;
                            jetconfigs...) -> JETToplevelResult

A generic entry point to analyze a package with AbstractAnalyzer. Finally returns the analysis result as JETToplevelResult. Note that this is intended to be used by developers of AbstractAnalyzer only. General users should use high-level entry points like report_package.

source
JET.analyze_and_report_text!Function
analyze_and_report_text!(analyzer::AbstractAnalyzer, text::AbstractString,
                         filename::AbstractString = "top-level";
                         jetconfigs...) -> JETToplevelResult

A generic entry point to analyze a top-level code with AbstractAnalyzer. Finally returns the analysis result as JETToplevelResult. Note that this is intended to be used by developers of AbstractAnalyzer only. General users should use high-level entry points like report_text.

source
JET.@jetreportMacro
@jetreport struct NewReport <: InferenceErrorReport
    ...
end

A utility macro to define InferenceErrorReport. It can be very tedious to manually satisfy the InferenceErrorReport interfaces. JET internally uses this @jetreport utility macro, which takes a struct definition of InferenceErrorReport without the required fields specified, and automatically defines the struct as well as constructor definitions. If the report NewReport <: InferenceErrorReport is defined using @jetreport, then NewReport just needs to implement the print_report_message interface.

For example, JETAnalyzer's MethodErrorReport is defined as follows:

@jetreport struct MethodErrorReport <: InferenceErrorReport
    @nospecialize t # ::Union{Type, Vector{Type}}
    union_split::Int
end
function print_report_message(io::IO, (; t, union_split)::MethodErrorReport)
    print(io, "no matching method found for ")
    if union_split == 0
        print_callsig(io, t)
    else
        ts = t::Vector{Any}
        nts = length(ts)
        for i = 1:nts
            print_callsig(io, ts[i])
            i == nts || print(io, ", ")
        end
        print(io, " (", nts, '/', union_split, " union split)")
    end
end

and constructed as like MethodErrorReport(sv::InferenceState, atype::Any, 0).

source

Examples