Discuss how to approach macro-heavy libraries

[wende]

In Elixir a lot of libraries rely on macros and while these used inside functions can be easily mimicked and wrapped, when it comes to those as top scope definitions (right inside modules f.i) it starts to become quite unwieldy

One of the simplest examples is how to write a test. Let's say we've got a standard ExUnit.Case macro.

defmodule MyTest do
  use ExUnit.Case

  test "I want to test something" do 
    assert 1 == 1
  end
end

Right now there is no way to express that in Elchemy.
And while it is possible to do some tricks like

module MyTest do

{- ex 
  use ExUnit.Case
  test "my_test" do
     assert execute_test()
  end
-}

executeTest : Bool
executeTest = 
  1 == 1

end

While this is correct code that would work i'd say it's quite far from what we would call an elegant solution.
This issue's purpose is to discuss all the possibilities and come out with something most that solves most problems without causing too much over-complication

[wende]

I'm currently working on a proof of concept implementation of a Plugin approach to this.

Entire idea is that instead of adding macros to modify Elchemy AST (which would just end up being a type-safety hazardous mess) you get a toolset to create new modules manipulating Elixir AST.

So that from user-point of view calling

module SomeTestSuite exposing (suite)
import Elchemy.ExUnitPlugin

suite = 
  -- This will create a module using ExUnit.Case
  ExUnitPlugin.suite {someField = "mycontext"} -- <- this is a starting context passed further
  -- This will tell it to define a setup inside
  |> ExUnitPlugin.setup (\ {someField} -> {ctx = someField} )
  -- This will tell it to define a test inside
  |> ExUnitPlugin.test "1 + 1 equals 2" (\context -> 1 + 1 == 2 )

Which could then be used normally wherever desired

defmodule MyProjectTest do
  use ExUnit.Case

   SomeTestSuite.suite()
end

Which is already proven to work

---

Under the hood however it needs to create a module and INJECT an anonymous function into it's body.
So our example would actually resolve to (AST manipulation apart):

defmodule SomeTestSuite do

  # It's actually using `Module.create` but using defmodule for better readability here
  def suite() do 
    defmodule ExUnitPlugin.H(ContentHash) do
      use ExUnit Case 
      
      test "2 + 2 equals 1", context do
        assert unquote(f).()
      end
    end
  end

The entire problem however, is how to smuggle the anonymous function inside the created module.
We can't unquote it because anonymous functions can't be unquoted. I obviously want to avoid polluting the global state by setting some Processes/Agents/Whatever by all costs.

---

Reading on Elixir forum I found this (which I wasn't aware of before)

@OvermindDL1
An anonymous function is actually a public function in the module that the anonymous function is defined in, and a call to it 'essentially' just {ModuleName, #, [args]} (a bit different in reality, but this gets the point across).

But I wasn't able to find a way to call such a function.

@OvermindDL1 Can you elaborate on how that works and if it's possible to call an anonymous function just by having it's value passed (somehow) to the nested module?

[wende]

Ok. Found :erlang.fun_info looks like exactly what I need

:erlang.fun_info(Test.test)
[
  pid: #PID<0.98.0>,
  module: Test,
  new_index: 0,
  new_uniq: <<64, 160, 30, 242, 89, 33, 1, 230, 243, 33, 207, 54, 132, 229, 36,
    142>>,
  index: 0,
  uniq: 33882359,
  name: :"-test/0-fun-0-",
  arity: 1,
  env: [],
  type: :local
]

[wende]

Hmm. It might be no use after all. The functions are obviously local so I don't think it's possible to pass them further

[OvermindDL1]

Heh, you can pass them around just fine. An anonymous function is just, essentially internally, a tuple of the module name, the function index in the module function array, and a set of bindings to pass in to it as it's environment (which become part of its called arguments).

To 'pass it in', if you don't need an environment, is easy, you can just remotely call it, otherwise maybe unwrap the environment in to it.

However, I'm not quite understanding what you are trying to do, let me read in more detail...

Are you talking about the |> ExUnitPlugin.test "1 + 1 equals 2" (\context -> 1 + 1 == 2 ) bit needing to generate assert unquote(f).()? If so then you could do it if the f is the AST of the function wrapped in an anonymous function?

[OvermindDL1]

And don't forget, even macro-heavy libraries are still just only function calls, everything is function calls. :-)

If you approach it more like Lisp instead of metaprogramming like in other languages, then that is much closer to how Elixir really is. :-)

[wende]

@OvermindDL1 Obviously. But I want to give a [relatively] simple tool for people from outside to be able to write wrappers for common libraries like they would normally do it.
With macros:

        Test name body ->
            Application (atom "test")
                [ Value (String name)
                , Variable "context"
                , Do
                    -- A Quotation here is a place we will paste our function in
                    [ dotApplication Quotation [ Variable "context" ] 
                    ]
                ]
                |> unquote body

Instead of having to understand all of the internals of how ExUnit works and just do all of the macro work manually.

[wende]

What I would want to do is to be able to build AST "around" anonymous functions.

So for instance with a simpler example than ExUnit.
Let's say we want to define a module implementing a bevahiour.

So that would resolve to something like

defmodule MyCallingModule do
  def behaviour_implementation(f_to_call) do
    defmodule Whatever do
      @behaviour MyBehaviour
      
      def some_callback() do
        # Here somehow we need to call any function inside our parent, but since in Elm a global function and a local function are virtually the same I need to be able to pass an anonymous function here
        f_to_call.()
      end
    end
  end
end

[wende]

@OvermindDL1 What do you mean remotely call it?

Can you give an example of how to serialize (into AST) an anonymous function and to be able to execute it after rebuilding from serialized information about it?

[OvermindDL1]

Can you give an example of how to serialize (into AST) an anonymous function and to be able to execute it after rebuilding from serialized information about it?

What format is the anonymous function in, is it the AST of it, like fn x -> x end or so?

If you have no environment that needs to be included then just pack it as, say, {modulename, functionname, [args...]} and then apply that where needed?