AbstractAnalyzer Framework

JETInterface

This baremodule exports names that form the APIs of AbstractAnalyzer Framework. using JET.JETInterface loads all names that are necessary to define a plugin analysis.

abstract type AbstractAnalyzer <: AbstractInterpreter end

An interface type of analyzers that are built on top of JET's analyzer framework.

When a new type NewAnalyzer implements the AbstractAnalyzer interface, it should be declared as subtype of AbstractAnalyzer, and is expected to implement the following interfaces:

Required interfaces

1. JETInterface.AnalyzerState(analyzer::NewAnalyzer) -> AnalyzerState: Returns the AnalyzerState for analyzer::NewAnalyzer.

2. JETInterface.AbstractAnalyzer(analyzer::NewAnalyzer, state::AnalyzerState) -> NewAnalyzer: Constructs a new NewAnalyzer instance in the middle of JET's top-level analysis or abstract interpretation, given the previous analyzer::NewAnalyzer and state::AnalyzerState.

3. JETInterface.ReportPass(analyzer::NewAnalyzer) -> ReportPass: Returns ReportPass used for analyzer::NewAnalyzer.

4. JETInterface.AnalysisToken(analyzer::NewAnalyzer) -> AnalysisToken: Returns a unique AnalysisToken object used for analyzer::NewAnalyzer.

See also AnalyzerState, ReportPass and AnalysisToken.

Example

JET.jl defines its default error analyzer JETAnalyzer <: AbstractAnalyzer as the following (modified a bit for the sake of simplicity):

# the default error analyzer for JET.jl
struct JETAnalyzer{RP<:ReportPass} <: AbstractAnalyzer
    state::AnalyzerState
    report_pass::RP
end

# AbstractAnalyzer API requirements
JETInterface.AnalyzerState(analyzer::JETAnalyzer) = analyzer.state
JETInterface.AbstractAnalyzer(analyzer::JETAnalyzer, state::AnalyzerState) = JETAnalyzer(ReportPass(analyzer), state)
JETInterface.ReportPass(analyzer::JETAnalyzer) = analyzer.report_pass
let global_analysis_token = AnalysisToken()
    JETInterface.AnalysisToken(analyzer::JETAnalyzer) = global_analysis_token
end

mutable struct AnalyzerState
    ...
end

The mutable object that holds various states that are consumed by all AbstractAnalyzers.

JETInterface.AnalyzerState(analyzer::AbstractAnalyzer) -> AnalyzerState

If NewAnalyzer implements the AbstractAnalyzer interface, NewAnalyzer should implement this AnalyzerState(analyzer::NewAnalyzer) -> AnalyzerState interface.

A new AnalyzerState is supposed to be constructed using the general configurations passed as keyword arguments jetconfigs of the NewAnalyzer(; jetconfigs...) constructor, and the constructed AnalyzerState is usually kept within NewAnalyzer itself:

function NewAnalyzer(world::UInt=Base.get_world_counter(); jetconfigs...)
    ...
    state = AnalyzerState(world; jetconfigs...)
    return NewAnalyzer(..., state)
end
JETInterface.AnalyzerState(analyzer::NewAnalyzer) = analyzer.state

JETInterface.AnalysisToken(analyzer::AbstractAnalyzer) -> AnalysisToken

Returns AnalysisToken for the given analyzer::AbstractAnalyzer. AbstractAnalyzer instances can share the same cache if they perform the same analysis, otherwise their cache should be separated.

If NewAnalyzer implements the AbstractAnalyzer interface, it must implement this function to return a consistent token for instances that should share the same cache.

abstract type ReportPass end

An interface type that represents AbstractAnalyzer's report pass. analyzer::AbstractAnalyzer injects report passes using the (::ReportPass)(::Type{InferenceErrorReport}, ::AbstractAnalyzer, state, ...) interface, which provides a flexible and efficient layer to configure the analysis done by AbstractAnalyzer.

JETInterface.ReportPass(analyzer::AbstractAnalyzer) -> ReportPass

If NewAnalyzer implements the AbstractAnalyzer interface, NewAnalyzer should implement this ReportPass(analyzer::NewAnalyzer) -> ReportPass interface.

ReportPass allows NewAnalyzer to provide a very flexible configuration layer for NewAnalyzer's analysis; an user can define their own ReportPass to control how NewAnalyzer collects report errors while still using the analysis routine implemented by NewAnalyzer.

Example

For example, JETAnalyzer accepts a custom ReportPass passed as part of the general configurations (see the documentation of AbstractAnalyzer for an example implementation). And we can setup a custom report pass IgnoreAllExceptGlobalUndefVar, that ignores all the reports that are otherwise collected by JETAnalyzer except UndefVarErrorReport:

# custom report pass that ignores all the reports except `UndefVarErrorReport`
struct IgnoreAllExceptGlobalUndefVar <: ReportPass end

# ignores all the reports analyzed by `JETAnalyzer`
(::IgnoreAllExceptGlobalUndefVar)(::Type{<:InferenceErrorReport}, @nospecialize(_...)) = return

# forward to `BasicPass` to collect `UndefVarErrorReport`
function (::IgnoreAllExceptGlobalUndefVar)(::Type{UndefVarErrorReport}, @nospecialize(args...))
    BasicPass()(UndefVarErrorReport, args...)
end

no_method_error()    = 1 + "1"
undef_global_error() = undefvar
report_call(; report_pass=IgnoreAllExceptGlobalUndefVar()) do
    if rand(Bool)
        return no_method_error()    # "no matching method found" error report won't be reported here
    else
        return undef_global_error() # "`undefvar` is not defined" error report will be reported
    end
end

abstract type ToplevelAbstractAnalyzer <: AbstractAnalyzer end

A specialized interface type for analyzers that perform top-level analysis of Julia code.

ToplevelAbstractAnalyzer extends AbstractAnalyzer to provide clear separation between analyzers that support top-level analysis and those that don't, offering several key architectural benefits:

• Type Safety: Only analyzers that explicitly extend ToplevelAbstractAnalyzer can be used with JET's virtual_process system, making it clear at compile time which analyzers support top-level analysis capabilities.
• Responsibility Separation: Analyzers like OptAnalyzer that don't perform top-level analysis are no longer required to handle top-level specific code paths, reducing complexity and improving performance.

Top-Level Analysis Capabilities

ToplevelAbstractAnalyzer provides specialized functionality for analyzing top-level Julia constructs such as:

• Global variable assignments
• Constant declarations (const statements)
• Module definitions and imports
• Method definitions at the top level
• Package-level code execution

This analyzer type is used in virtual_process system to analyze Julia code as it would be executed at the top level, handling both concrete interpretation of some statements and abstract interpretation of others.

Usage

ToplevelAbstractAnalyzer is typically used through JET's virtual process system:

# Create a concrete interpreter with a toplevel analyzer
interp = JETConcreteInterpreter(JETAnalyzer(...))
analyzer = ToplevelAbstractAnalyzer(interp)

# Analyze top-level code
result = analyze_and_report_text!(interp, "x = 1; y = x + 1")

Implementation Requirements

Concrete subtypes of ToplevelAbstractAnalyzer must implement all the interfaces required by AbstractAnalyzer, and will automatically inherit the specialized top-level analysis behaviors provided by this type.

See Also

• AbstractAnalyzer: The base analyzer interface
• JETAnalyzer: JET's default error analyzer that implements this interface
• virtual_process: The main function that uses this analyzer type
• ConcreteInterpreter: The concrete interpreter that works with this analyzer

JETInterface.valid_configurations(analyzer::AbstractAnalyzer) -> names or nothing

Returns a set of names that are valid as a configuration for analyzer. names should be an iterator of Symbol. No validations are performed if nothing is returned.

JETInterface.aggregation_policy(analyzer::AbstractAnalyzer)

Defines how analyzer aggregates InferenceErrorReports. This policy determines how duplicate or similar reports are identified and grouped. Defaults to default_aggregation_policy.

default_aggregation_policy(report::InferenceErrorReport) -> DefaultReportIdentity

Returns the default identity of report::InferenceErrorReport using DefaultReportIdentity, which aggregates reports based on their "error location".

DefaultReportIdentity aggregates InferenceErrorReports by creating an identity based on:

1. The report type
2. The signature of the method where the error was found
3. The file and line number where the error occurred

This approach ignores the specific MethodInstance identity, allowing errors to be aggregated if they occur at the same file and line, under the assumption that errors at the same location are likely duplicates even if in different method specializations.

vscode_diagnostics_order(analyzer::AbstractAnalyzer) -> Bool

If true (default) a diagnostic will be reported at entry site. Otherwise it's reported at error point.

InferenceErrorReport

In order for Report <: InferenceErrorReport to implement the interface, it should satisfy the following requirements:

• Required fields
  Report should have the following fields, which explains where and how this error is reported:

  • vst::VirtualStackTrace: a virtual stack trace of the error
  • sig::Signature: a signature of the error point

  Note that Report can have additional fields other than vst and sig to explain why this error is reported (mostly used for print_report_message).

• Required overloads
  

  • JETInterface.copy_report(report::Report) -> new::Report
  • JETInterface.print_report_message(io::IO, report::Report)

• Optional overloads
  

  • JETInterface.print_signature(::Report) -> Bool
  • JETInterface.report_color(::Report) -> Symbol

Report <: InferenceErrorReport is supposed to be constructed using the following constructor

Report(::AbstractAnalyzer, state, spec_args...) -> Report

where state can be either of:

• state::Tuple{Union{Compiler.InferenceState, Compiler.OptimizationState}, Int64}: a state with the current program counter specified
• state::InferenceState: a state with the current program counter set to state.currpc
• state::InferenceResult: a state with the current program counter unknown
• state::MethodInstance: a state with the current program counter unknown

See also: @jetreport, VirtualStackTrace, VirtualFrame

ToplevelErrorReport()

In order for Report <: ToplevelErrorReport to implement the interface, it should satisfy the following requirements:

• Required fields
  Report should have the following fields:

  • file::String: the filename of this error
  • line::Int: the line number of this error

• Required overloads
  

  • JETInterface.print_report(io::IO, report::Report)

JETInterface.copy_report(orig::Report) where Report<:InferenceErrorReport -> new::Report

Returns new new::Report, that should be identical to the original orig::Report, except that new.vst is copied from orig.vst so that the further modification on orig.vst that may happen in later abstract interpretation doesn't affect new.vst.

print_report(io::IO, report::ToplevelErrorReport)

Prints a report of the top-level error report to the given io.

JETInterface.print_report_message(io::IO, report::Report) where Report<:InferenceErrorReport

Prints to io and describes why report is reported.

JETInterface.print_signature(::Report) where Report<:InferenceErrorReport -> Bool

Configures whether or not to print the report signature when printing Report (defaults to true).

JETInterface.report_color(::Report) where Report<:InferenceErrorReport -> Symbol

Configures the color for Report (defaults to :red).

analyze_and_report_call!(analyzer::AbstractAnalyzer, f, [types]; jetconfigs...) -> JETCallResult
analyze_and_report_call!(analyzer::AbstractAnalyzer, tt::Type{<:Tuple}; jetconfigs...) -> JETCallResult
analyze_and_report_call!(analyzer::AbstractAnalyzer, mi::MethodInstance; jetconfigs...) -> JETCallResult

A generic entry point to analyze a function call with AbstractAnalyzer. Finally returns the analysis result as JETCallResult. Note that this is intended to be used by developers of AbstractAnalyzer only. General users should use high-level entry points like report_call and report_opt.

call_test_ex(funcname::Symbol, testname::Symbol, ex0, __module__, __source__)

An internal utility function to implement a @test_call-like macro. See the implementation of @test_call.

func_test(func, testname::Symbol, args...; jetconfigs...)

An internal utility function to implement a test_call-like function. See the implementation of test_call.

analyze_and_report_file!(interp::ConcreteInterpreter, filename::AbstractString; jetconfigs...) -> JETToplevelResult

A generic entry point to analyze a file with interp::ConcreteInterpreter. Finally returns the analysis result as JETToplevelResult. Note that this is intended to be used by developers of AbstractAnalyzer and ConcreteInterpreter only. General users should use high-level entry points like report_file.

analyze_and_report_package!(interp::ConcreteInterpreter,
                            package::Union{AbstractString,Module,Nothing} = nothing;
                            jetconfigs...) -> JETToplevelResult

A generic entry point to analyze a package with interp::ConcreteInterpreter. Finally returns the analysis result as JETToplevelResult. Note that this is intended to be used by developers of AbstractAnalyzer and ConcreteInterpreter only. General users should use high-level entry points like report_package.

analyze_and_report_text!(interp::ConcreteInterpreter, text::AbstractString,
                         filename::AbstractString = "top-level";
                         jetconfigs...) -> JETToplevelResult

A generic entry point to analyze a top-level code with interp::ConcreteInterpreter. Finally returns the analysis result as JETToplevelResult. Note that this is intended to be used by developers of AbstractAnalyzer and ConcreteInterpreter only. General users should use high-level entry points like report_text.

add_new_report!(analyzer::AbstractAnalyzer, result::InferenceResult, report::InferenceErrorReport)

Adds a new error report to the analyzer's collection for a specific inference result.

This function associates an InferenceErrorReport with its corresponding result::InferenceResult in the analyzer's internal storage. The report becomes part of the analysis results that can be retrieved later using get_reports(analyzer, result).

Reports are stored in the order they are added, which can be important for maintaining the logical sequence of errors discovered during analysis.

@jetreport struct NewReport <: InferenceErrorReport
    ...
end

A utility macro to define InferenceErrorReport. It can be very tedious to manually satisfy the InferenceErrorReport interfaces. JET internally uses this @jetreport utility macro, which takes a struct definition of InferenceErrorReport without the required fields specified, and automatically defines the struct as well as constructor definitions. If the report NewReport <: InferenceErrorReport is defined using @jetreport, then NewReport just needs to implement the print_report_message interface.

For example, JETAnalyzer's MethodErrorReport is defined as follows:

@jetreport struct MethodErrorReport <: InferenceErrorReport
    @nospecialize t # ::Union{Type, Vector{Type}}
    union_split::Int
end
function print_report_message(io::IO, (; t, union_split)::MethodErrorReport)
    print(io, "no matching method found for ")
    if union_split == 0
        print_callsig(io, t)
    else
        ts = t::Vector{Any}
        nts = length(ts)
        for i = 1:nts
            print_callsig(io, ts[i])
            i == nts || print(io, ", ")
        end
        print(io, " (", nts, '/', union_split, " union split)")
    end
end

and constructed as like MethodErrorReport(sv::InferenceState, atype::Any, 0).