Skip to content

Commit

Permalink
reflection: allow names to return using-ed names
Browse files Browse the repository at this point in the history 
This commit makes it possible for `names` to return `using`-ed names
as well:
```julia
julia> using Base: @assume_effects

julia> Symbol("@assume_effects") in names(@__MODULE__; usings=true)
true
```

Currently, to find all names available in a module `A`, the following
steps are needed:
1. Use `names(A; all=true, imported=true)` to get the names defined by
   `A` and the names explicitly `import`ed by `A`.
2. Use `jl_module_usings(A)` to get the list of modules `A` has
   `using`-ed and then use `names()` to get the names `export`ed by
   those modules.

This method is implemented in e.g. REPL completions, but it has a
problem: it could not get the names explicitly `using`-ed by
`using B: ...` (#36529, #40356, JuliaDebug/Infiltrator.jl#106, etc.).

This commit adds a new keyword argument `usings::Bool=false` to
`names(A; ...)`, which, when `usings=true` is specified, returns all
names introduced by `using` in `A`.
In other words, `usings=true` not only returns explicitly `using`-ed
names but also incorporates step 2 above into the implementation of
`names`.

By using this new option, we can now use
`names(A; all=true, imported=true, usings=true)` to know all names
available in `A`, without implementing the two-fold steps on application
side.
As example application, this new feature will be used to simplify and
enhance the implementation of REPL completions.

- fixes #36529

Co-authored-by: Nathan Daly <[email protected]>
Co-authored-by: Sebastian Pfitzner <[email protected]>
  • Loading branch information
@aviatesk @NHDaly @pfitzseb
3 people committed May 30, 2024
1 parent 6a10d03 commit 9f2e685
Show file tree
Hide file tree
Showing 4 changed files with 167 additions and 31 deletions.
5 changes: 4 additions & 1 deletion NEWS.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ Julia v1.12 Release Notes
New language features
---------------------

- A new keyword argument `usings::Bool` has been added to `names`. By using this, we can now
find all the names available in module `A` by `names(A; all=true, imported=true, usings=true)`. ([#54609])

Language changes
----------------

Expand All @@ -17,7 +20,7 @@ Language changes
may pave the way for inference to be able to intelligently re-use the old
results, once the new method is deleted. ([#53415])

- Macro expansion will no longer eargerly recurse into into `Expr(:toplevel)`
- Macro expansion will no longer eagerly recurse into into `Expr(:toplevel)`
expressions returned from macros. Instead, macro expansion of `:toplevel`
expressions will be delayed until evaluation time. This allows a later
expression within a given `:toplevel` expression to make use of macros
Expand Down
19 changes: 12 additions & 7 deletions base/reflection.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,28 +77,33 @@ function fullname(m::Module)
end

"""
names(x::Module; all::Bool = false, imported::Bool = false)
names(x::Module; all::Bool=false, imported::Bool=false, usings::Bool=false) -> Vector{Symbol}
Get a vector of the public names of a `Module`, excluding deprecated names.
If `all` is true, then the list also includes non-public names defined in the module,
deprecated names, and compiler-generated names.
If `imported` is true, then names explicitly imported from other modules
are also included. Names are returned in sorted order.
are also included.
If `usings` is true, then names explicitly imported via `using` are also included.
Names are returned in sorted order.
As a special case, all names defined in `Main` are considered \"public\",
since it is not idiomatic to explicitly mark names from `Main` as public.
!!! note
`sym ∈ names(SomeModule)` does *not* imply `isdefined(SomeModule, sym)`.
`names` will return symbols marked with `public` or `export`, even if
`names` may return symbols marked with `public` or `export`, even if
they are not defined in the module.
!!! warning
`names` may return duplicate names. The duplication happens, e.g. if an `import`ed name
conflicts with an already existing identifier.
See also: [`Base.isexported`](@ref), [`Base.ispublic`](@ref), [`Base.@locals`](@ref), [`@__MODULE__`](@ref).
"""
names(m::Module; all::Bool = false, imported::Bool = false) =
sort!(unsorted_names(m; all, imported))
unsorted_names(m::Module; all::Bool = false, imported::Bool = false) =
ccall(:jl_module_names, Array{Symbol,1}, (Any, Cint, Cint), m, all, imported)
names(m::Module; kwargs...) = sort!(unsorted_names(m; kwargs...))
unsorted_names(m::Module; all::Bool=false, imported::Bool=false, usings::Bool=false) =
ccall(:jl_module_names, Array{Symbol,1}, (Any, Cint, Cint, Cint), m, all, imported, usings)

"""
isexported(m::Module, s::Symbol) -> Bool
Expand Down
52 changes: 43 additions & 9 deletions src/module.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -991,10 +991,19 @@ JL_DLLEXPORT jl_value_t *jl_module_usings(jl_module_t *m)
return (jl_value_t*)a;
}

JL_DLLEXPORT jl_value_t *jl_module_names(jl_module_t *m, int all, int imported)
uint8_t _binding_is_from_explicit_using(jl_binding_t *b) {
jl_binding_t *owner = jl_atomic_load_relaxed(&b->owner);
return (owner != NULL && owner != b && !b->imported);
}

void _append_symbol_to_bindings_array(jl_array_t* a, jl_sym_t *name) {
jl_array_grow_end(a, 1);
//XXX: change to jl_arrayset if array storage allocation for Array{Symbols,1} changes:
jl_array_ptr_set(a, jl_array_dim0(a)-1, (jl_value_t*)name);
}

void append_module_names(jl_array_t* a, jl_module_t *m, int all, int imported, int usings)
{
jl_array_t *a = jl_alloc_array_1d(jl_array_symbol_type, 0);
JL_GC_PUSH1(&a);
jl_svec_t *table = jl_atomic_load_relaxed(&m->bindings);
for (size_t i = 0; i < jl_svec_len(table); i++) {
jl_binding_t *b = (jl_binding_t*)jl_svecref(table, i);
Expand All @@ -1003,16 +1012,41 @@ JL_DLLEXPORT jl_value_t *jl_module_names(jl_module_t *m, int all, int imported)
jl_sym_t *asname = b->globalref->name;
int hidden = jl_symbol_name(asname)[0]=='#';
int main_public = (m == jl_main_module && !(asname == jl_eval_sym || asname == jl_include_sym));
if ((b->publicp ||
if (((b->publicp) ||
(imported && b->imported) ||
(usings && _binding_is_from_explicit_using(b)) ||
(jl_atomic_load_relaxed(&b->owner) == b && !b->imported && (all || main_public))) &&
(all || (!b->deprecated && !hidden))) {
jl_array_grow_end(a, 1);
// n.b. change to jl_arrayset if array storage allocation for Array{Symbols,1} changes:
jl_array_ptr_set(a, jl_array_dim0(a)-1, (jl_value_t*)asname);
}
(all || (!b->deprecated && !hidden)))
_append_symbol_to_bindings_array(a, asname);
}
}

void append_exported_names(jl_array_t* a, jl_module_t *m, int all)
{
jl_svec_t *table = jl_atomic_load_relaxed(&m->bindings);
for (size_t i = 0; i < jl_svec_len(table); i++) {
jl_binding_t *b = (jl_binding_t*)jl_svecref(table, i);
if ((void*)b == jl_nothing)
break;
if (b->exportp && (all || !b->deprecated))
_append_symbol_to_bindings_array(a, b->globalref->name);
table = jl_atomic_load_relaxed(&m->bindings);
}
}

JL_DLLEXPORT jl_value_t *jl_module_names(jl_module_t *m, int all, int imported, int usings)
{
jl_array_t *a = jl_alloc_array_1d(jl_array_symbol_type, 0);
JL_GC_PUSH1(&a);
append_module_names(a, m, all, imported, usings);
if (usings) {
// If `usings` is specified, traverse the list of `using`-ed modules and incorporate
// the names exported by those modules into the list.
for(int i=(int)m->usings.len-1; i >= 0; --i) {
jl_module_t *usinged = module_usings_getidx(m, i);
append_exported_names(a, usinged, all);
}
}
JL_GC_POP();
return (jl_value_t*)a;
}
Expand Down
122 changes: 108 additions & 14 deletions test/reflection.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -125,11 +125,18 @@ not_const = 1
# For curmod_*
include("testenv.jl")

module TestMod36529
x36529 = 0
y36529 = 1
export y36529
end

module TestMod7648
using Test
import Base.convert
import ..curmod_name, ..curmod
export a9475, foo9475, c7648, foo7648, foo7648_nomethods, Foo7648
using ..TestMod36529: x36529 # doesn't import TestMod36529 or y36529, even though it's exported
export a9475, c7648, f9475, foo7648, foo7648_nomethods, Foo7648

const c7648 = 8
d7648 = 9
Expand All @@ -142,10 +149,11 @@ module TestModSub9475
using Test
using ..TestMod7648
import ..curmod_name
export a9475, foo9475
export a9475, f9475, f54609
a9475 = 5
b9475 = 7
foo9475(x) = x
f9475(x) = x
f54609(x) = x
let
@test Base.binding_module(@__MODULE__, :a9475) == @__MODULE__
@test Base.binding_module(@__MODULE__, :c7648) == TestMod7648
Expand All @@ -169,18 +177,104 @@ let
@test Base.binding_module(TestMod7648, :d7648) == TestMod7648
@test Base.binding_module(TestMod7648, :a9475) == TestMod7648.TestModSub9475
@test Base.binding_module(TestMod7648.TestModSub9475, :b9475) == TestMod7648.TestModSub9475
@test Set(names(TestMod7648))==Set([:TestMod7648, :a9475, :foo9475, :c7648, :foo7648, :foo7648_nomethods, :Foo7648])
@test Set(names(TestMod7648, all = true)) == Set([:TestMod7648, :TestModSub9475, :a9475, :foo9475, :c7648, :d7648, :f7648,
:foo7648, Symbol("#foo7648"), :foo7648_nomethods, Symbol("#foo7648_nomethods"),
:Foo7648, :eval, Symbol("#eval"), :include, Symbol("#include")])
@test Set(names(TestMod7648, all = true, imported = true)) == Set([:TestMod7648, :TestModSub9475, :a9475, :foo9475, :c7648, :d7648, :f7648,
:foo7648, Symbol("#foo7648"), :foo7648_nomethods, Symbol("#foo7648_nomethods"),
:Foo7648, :eval, Symbol("#eval"), :include, Symbol("#include"),
:convert, :curmod_name, :curmod])
defaultset = Set(Symbol[:Foo7648, :TestMod7648, :a9475, :c7648, :f9475, :foo7648, :foo7648_nomethods])
allset = defaultset ∪ Set(Symbol[
Symbol("#eval"), Symbol("#foo7648"), Symbol("#foo7648_nomethods"), Symbol("#include"),
:TestModSub9475, :d7648, :eval, :f7648, :include])
imported = Set(Symbol[:convert, :curmod_name, :curmod])
usings_from_Test = Set(Symbol[
Symbol("@inferred"), Symbol("@test"), Symbol("@test_broken"), Symbol("@test_deprecated"),
Symbol("@test_logs"), Symbol("@test_nowarn"), Symbol("@test_skip"), Symbol("@test_throws"),
Symbol("@test_warn"), Symbol("@testset"), :GenericArray, :GenericDict, :GenericOrder,
:GenericSet, :GenericString, :LogRecord, :Test, :TestLogger, :TestSetException,
:detect_ambiguities, :detect_unbound_args])
usings_from_Base = delete!(Set(names(Module(); usings=true)), :anonymous) # the name of the anonymous module itself
usings = Set(Symbol[:x36529, :TestModSub9475, :f54609]) ∪ usings_from_Test ∪ usings_from_Base
@test Set(names(TestMod7648)) == defaultset
@test Set(names(TestMod7648, all=true)) == allset
@test Set(names(TestMod7648, all=true, imported=true)) == allset ∪ imported
@test Set(names(TestMod7648, usings=true)) == defaultset ∪ usings
@test Set(names(TestMod7648, all=true, usings=true)) == allset ∪ usings
@test isconst(TestMod7648, :c7648)
@test !isconst(TestMod7648, :d7648)
end

# tests for `names(...; usings=true)`

baremodule Test54609Simple
module Inner
export exported
global exported::Int = 1
global unexported::Int = 0
end
using Base: @assume_effects
using .Inner
end
let usings = names(Test54609Simple; usings=true)
@test Symbol("@assume_effects") ∈ usings
@test :Base ∉ usings
@test :exported ∈ usings
@test :unexported ∉ usings
end # baremodule Test54609Simple

baremodule _Test54609Complex
export exported_new
using Base: @deprecate_binding
global exported_new = nothing
@deprecate_binding exported_old exported_new
end # baremodule _Test54609Complex
baremodule Test54609Complex
using .._Test54609Complex
end # baremodule Test54609Complex
let usings = names(Test54609Complex; usings=true)
@test :exported_new ∈ usings
@test :exported_old ∉ usings
@test :_Test54609Complex ∈ usings # should include the `using`ed module itself
usings_all = names(Test54609Complex; usings=true, all=true)
@test :exported_new ∈ usings_all
@test :exported_old ∈ usings_all # deprecated names should be included with `all=true`
end

module TestMod54609
module M1
const m1_x = 1
export m1_x
end
module M2
const m2_x = 1
export m2_x
end
module A
module B
f(x) = 1
secret = 1
module Inner2 end
end
module C
x = 1
y = 2
export y
end
using .B: f
using .C
using ..M1
import ..M2
end
end # module TestMod54609
let defaultset = Set((:A,))
imported = Set((:M2,))
usings_from_Base = delete!(Set(names(Module(); usings=true)), :anonymous) # the name of the anonymous module itself
usings = Set((:A, :f, :C, :y, :M1, :m1_x)) ∪ usings_from_Base
allset = Set((:A, :B, :C, :eval, :include, Symbol("#eval"), Symbol("#include")))
@test Set(names(TestMod54609.A)) == defaultset
@test Set(names(TestMod54609.A, imported=true)) == defaultset ∪ imported
@test Set(names(TestMod54609.A, usings=true)) == defaultset ∪ usings
@test Set(names(TestMod54609.A, all=true)) == allset
@test Set(names(TestMod54609.A, all=true, usings=true)) == allset ∪ usings
@test Set(names(TestMod54609.A, imported=true, usings=true)) == defaultset ∪ imported ∪ usings
@test Set(names(TestMod54609.A, all=true, imported=true, usings=true)) == allset ∪ imported ∪ usings
end

let
using .TestMod7648
@test Base.binding_module(@__MODULE__, :a9475) == TestMod7648.TestModSub9475
Expand All @@ -189,10 +283,10 @@ let
@test parentmodule(foo7648, (Any,)) == TestMod7648
@test parentmodule(foo7648) == TestMod7648
@test parentmodule(foo7648_nomethods) == TestMod7648
@test parentmodule(foo9475, (Any,)) == TestMod7648.TestModSub9475
@test parentmodule(foo9475) == TestMod7648.TestModSub9475
@test parentmodule(f9475, (Any,)) == TestMod7648.TestModSub9475
@test parentmodule(f9475) == TestMod7648.TestModSub9475
@test parentmodule(Foo7648) == TestMod7648
@test parentmodule(first(methods(foo9475))) == TestMod7648.TestModSub9475
@test parentmodule(first(methods(f9475))) == TestMod7648.TestModSub9475
@test parentmodule(first(methods(foo7648))) == TestMod7648
@test nameof(Foo7648) === :Foo7648
@test basename(functionloc(foo7648, (Any,))[1]) == "reflection.jl"
Expand Down

0 comments on commit 9f2e685

Please sign in to comment.