|
| 1 | +# GeneralizedGenerated |
| 2 | + |
| 3 | +[](https://JuliaStaging.github.io/GeneralizedGenerated.jl/stable) |
| 4 | +[](https://JuliaStaging.github.io/GeneralizedGenerated.jl/dev) |
| 5 | +[](https://travis-ci.com/JuliaStaging/GeneralizedGenerated.jl) |
| 6 | +[](https://codecov.io/gh/JuliaStaging/GeneralizedGenerated.jl) |
| 7 | +[](https://doi.org/10.5281/zenodo.3596233) |
| 8 | + |
| 9 | +GeneralizedGenerated enables the generalized generated functions. Specifically, **it supports closure constructions in generated functions**. |
| 10 | + |
| 11 | +Besides, some utility stuffs relevant to GeneralizedGenerated's implementation are exported, |
| 12 | +which **allows you to keep `eval` and `invokelastest`** away from Julia |
| 13 | +metaprogramming. |
| 14 | + |
| 15 | +## Notes about Usage: |
| 16 | + |
| 17 | +`GeneralizedGenerated.jl` has issues about latency and extensive memory consumptions, and is sometimes likely to trigger segfault bugs when generated functions get enormous([#45](https://github.com/JuliaStaging/GeneralizedGenerated.jl/issues/45), [#59](https://github.com/JuliaStaging/GeneralizedGenerated.jl/issues/59)). This suggests that you should avoid your expressions from being too large. |
| 18 | + |
| 19 | +In terms of **use cases where no closure is needed**, you'd better use [RuntimeGeneratedFunctions.jl](https://github.com/SciML/RuntimeGeneratedFunctions.jl), which has better scalability than `GeneralizedGenerated.jl`. |
| 20 | + |
| 21 | +P.S: |
| 22 | +- You should also re-check if closures are really necessary in your code. |
| 23 | +- If you use `mk_function` or similar stuffs in a non-global loop, but only call those generated functions once, you might re-think if your design can be refined to avoid this. |
| 24 | + |
| 25 | +## Background: World Age Problem |
| 26 | + |
| 27 | +See an explanation [here](https://discourse.julialang.org/t/world-age-problem-explanation/9714/4). |
| 28 | + |
| 29 | +```julia |
| 30 | +julia> module WorldAgeProblemRaisedHere! |
| 31 | + do_this!(one_ary_fn_ast::Expr, arg) = begin |
| 32 | + eval(one_ary_fn_ast)(arg) |
| 33 | + end |
| 34 | + res = do_this!(:(x -> x + 1), 2) |
| 35 | + @info res |
| 36 | + end |
| 37 | +ERROR: MethodError: no method matching (::getfield(Main.WorldAgeProblemRaisedHere!, Symbol("##1#2")))(::Int64) |
| 38 | +The applicable method may be too new: running in world age 26095, while current world is 26096. |
| 39 | + |
| 40 | +julia> module WorldAgeProblemSolvedHere! |
| 41 | + |
| 42 | + do_this!(one_ary_fn_ast::Expr, arg) = begin |
| 43 | + runtime_eval(one_ary_fn_ast)(arg) |
| 44 | + end |
| 45 | + res = do_this!(:(x -> x + 1), 2) |
| 46 | + @info res |
| 47 | + end |
| 48 | +[ Info: 3 |
| 49 | +Main.WorldAgeProblemSolvedHere! |
| 50 | +``` |
| 51 | +
|
| 52 | +## Support Closures in Generated Functions |
| 53 | +
|
| 54 | +```julia |
| 55 | + |
| 56 | + |
| 57 | +@gg function f(x) |
| 58 | + quote |
| 59 | + a -> x + a |
| 60 | + end |
| 61 | +end |
| 62 | + |
| 63 | +f(1)(2) # => 3 |
| 64 | + |
| 65 | +@gg function h(x, c) |
| 66 | + quote |
| 67 | + d = x + 10 |
| 68 | + function g(x, y=c) |
| 69 | + x + y + d |
| 70 | + end |
| 71 | + end |
| 72 | +end |
| 73 | + |
| 74 | +h(1, 2)(1) # => 14 |
| 75 | +``` |
| 76 | +
|
| 77 | +Note there're some restrictions to the generalized generated functions yet: |
| 78 | +
|
| 79 | +- Multiple dispatch is not allowed, and `f(x) = ...` is equivalent to `f = x -> ...`. This will never gets supported for it needs a thorough implementation of multiple dispatch in GG. |
| 80 | +- Comprehensions for generated functions are not implemented yet. It won't cost a long time for being supported. |
| 81 | +
|
| 82 | +The evaluation module can be specified in this way: |
| 83 | +
|
| 84 | +```julia |
| 85 | +julia> module S |
| 86 | + run(y) = y + 1 |
| 87 | + end |
| 88 | +Main.S |
| 89 | + |
| 90 | +julia> @gg g(m::Module, y) = @under_global :m :(run(y)); |
| 91 | +# the global variable `run` is from the local variable `m` |
| 92 | +# <=> |
| 93 | +# @gg g(m::Module, y) = :($(:m).run(y)); |
| 94 | + |
| 95 | +julia> g(S, 1) |
| 96 | +2 |
| 97 | +``` |
| 98 | +
|
| 99 | +Of course you can use structures to imitate modules: |
| 100 | +
|
| 101 | +```julia |
| 102 | +julia> struct S |
| 103 | + run :: Function |
| 104 | + end |
| 105 | +Main.S |
| 106 | + |
| 107 | +julia> @gg function g(m::S, y) |
| 108 | + @under_global :m quote |
| 109 | + run(y) |
| 110 | + end |
| 111 | + end; |
| 112 | +# <=> |
| 113 | +# @gg function g(m::S, y) |
| 114 | +# :($(:m).run(y)) |
| 115 | +# end; |
| 116 | + |
| 117 | +julia> g(S(x -> x + 1), 1) |
| 118 | +2 |
| 119 | + |
| 120 | +julia> const pseudo_module = S(x -> x + 1); |
| 121 | +julia> @gg function g(y) |
| 122 | + @under_global pseudo_module quote |
| 123 | + run(y) |
| 124 | + end |
| 125 | + end |
| 126 | +# <=> |
| 127 | +# @gg function g(y) |
| 128 | +# :($(pseudo_module).run(y)) |
| 129 | +# end |
| 130 | +julia> g(1) |
| 131 | +2 |
| 132 | +``` |
| 133 | +
|
| 134 | +julia> @generated function g() |
| 135 | + Module = Main |
| 136 | + mk_expr(Module, :( (x -> x)(1))) |
| 137 | +end |
| 138 | +
|
| 139 | +## No `eval`/`invokelatest`! |
| 140 | +
|
| 141 | +```julia |
| 142 | +# do something almost equivalent to `eval` |
| 143 | +# without introducing the world age problem! |
| 144 | + |
| 145 | +f = mk_function(:((x, y) -> x + y)) |
| 146 | +f(1, 2) |
| 147 | +# => 3 |
| 148 | + |
| 149 | +f = mk_function([:x, :y]#= args =#, []#= kwargs =#, :(x + y)) |
| 150 | +f(1, 2) |
| 151 | +# => 3 |
| 152 | + |
| 153 | + |
| 154 | +module GoodGame |
| 155 | + xxx = 10 |
| 156 | +end |
| 157 | +# Specify global module |
| 158 | +f = mk_function(GoodGame, :(function () xxx end)) |
| 159 | +f() |
| 160 | +# => 10 |
| 161 | +``` |
| 162 | +
|
| 163 | +The function created by `mk_function` always has the signature `f(args…; kwargs…) = ...` if you need to use the function in a context where it will be passed multiple arguments, use the following pattern |
| 164 | +
|
| 165 | +```julia |
| 166 | +f = mk_function(:((x, y) -> x + y)) |
| 167 | + |
| 168 | +function F(g, pairs) |
| 169 | + map(pairs) do (x,y) |
| 170 | + g(x,y) |
| 171 | + end |
| 172 | +end |
| 173 | + |
| 174 | +pairs = zip(1:10,2:11) |
| 175 | +F((x,y)->f(x,y), pairs) |
| 176 | +#= |
| 177 | +=> |
| 178 | +10-element Array{Int64,1}: |
| 179 | + 3 |
| 180 | + 5 |
| 181 | + 7 |
| 182 | + 9 |
| 183 | + 11 |
| 184 | + 13 |
| 185 | + 15 |
| 186 | + 17 |
| 187 | + 19 |
| 188 | + 21 |
| 189 | +=# |
| 190 | +``` |
| 191 | +
|
| 192 | +Tips |
| 193 | +============== |
| 194 | +
|
| 195 | +Note, `mk_function` just accepts a function-like AST, to eval more kinds of |
| 196 | +ASTs, use `runtime_eval`: |
| 197 | +
|
| 198 | +```julia |
| 199 | +a = 0 |
| 200 | +runtime_eval(:(a + 1)) == 1 # true |
| 201 | + |
| 202 | +module GoodGameOnceAgain |
| 203 | + a = 2 |
| 204 | +end |
| 205 | +runtime_eval(GoodGameOnceAgain, :(a + 3)) == 5 |
| 206 | +``` |
| 207 | +
|
| 208 | +# Known Bugs |
| 209 | +
|
| 210 | +1. Type annotations. |
| 211 | +
|
| 212 | + Type annotations for cell variables (variables shared to any inner functions of the current scope) do not work. You might consider changing your generated code from |
| 213 | +
|
| 214 | + ```julia |
| 215 | + a :: t = b |
| 216 | + # when 'a' is cell, |
| 217 | + # the closure-converted code 'a.contents :: t = b' fails due to the Julia syntax |
| 218 | + ``` |
| 219 | +
|
| 220 | + to |
| 221 | +
|
| 222 | + ```julia |
| 223 | + a = b :: t |
| 224 | + ``` |
| 225 | +
|
| 226 | +2. Precompilation |
| 227 | +
|
| 228 | + GG is designed for purely runtime generated functions, and currently has difficulties in precompiling a GG function. |
| 229 | +
|
| 230 | + When developing a package, please do not define a GG function in the top level! |
0 commit comments