before_all and after_all macro evaluation out of order

[mpcarter]

Was following the guidance from the documentation on use of after_all config with a specific environment. e.g.

after_all:
  - "@IF(@this_env = 'prod', @grant_schema_usage())"

But I got errors or inconsistent behavior instead. It appears that the order of evaluation of macros is happening from the inside to out.
Here is a very simple example

@macro()
def simple_macro(evaluator):
    return ["select true as b", "select false as b"]

after_all:
  - "@IF(@this_env = 'prod', @simple_macro())"

When I run a plan in my dev environment, the expectation would be that nothing would happen. Instead, SQLMesh executes the second statement

sqlmesh.core.engine_adapter.base - INFO - Executing SQL: SELECT FALSE AS b (base.py:2113)

If I change the macro to this

@macro()
def simple_macro(evaluator):
    return ["select true as b", "select false as b", "select null as b"]

I instead get an error regardless of the environment

2025-03-21 08:28:45,787 - MainThread - sqlmesh.core.context - INFO - Plan application failed. (context.py:1459)
Traceback (most recent call last):
  File "...\sqlmesh\Lib\site-packages\sqlmesh\core\macros.py", line 205, in send
    return call_macro(func, self.dialect, self._path, self, *args, **kwargs)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\sqlmesh\Lib\site-packages\sqlmesh\core\macros.py", line 1270, in call_macro
    bound = sig.bind(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\python312\current\Lib\inspect.py", line 3277, in bind
    return self._bind(args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\python312\current\Lib\inspect.py", line 3196, in _bind
    raise TypeError('too many positional arguments') from None
TypeError: too many positional arguments

...

sqlmesh.utils.errors.ConfigError: Failed to resolve macros for
@IF(@this_env <> 'prod', @simple_macro())
Error trying to eval macro. at '.'
Error: Failed to resolve macros for
@IF(@this_env <> 'prod', @simple_macro())
Error trying to eval macro. at '.'

What appears to be occurring is that SQLMesh is first evaluating the inner most macro, @simple_macro() and injecting that into the arguments for the @IF macro.

So first example evaluated like this

@IF(@this_env = 'prod', @simple_macro())
@IF('dev' = 'prod', "select true as b", "select false as b")
"select false as b"

And second example like this

@IF(@this_env = 'prod', @simple_macro())
@IF('dev' = 'prod', "select true as b", "select false as b", "select null as b")
TypeError: too many positional arguments

I would expect it to work like this

@IF(@this_env = 'prod', @simple_macro())
@IF('dev' = 'prod', @simple_macro())
<do nothing>

[mpcarter]

As a workaround, you can alter the config and macro to push the environment check into the macro itself.

@macro()
def simple_macro(evaluator):
    if evaluator._environment_naming_info.name == 'prod':
        return ["select true as b", "select false as b", "select null as b"]

after_all:
  - "@simple_macro()"

[georgesittas]

I believe this is behaving as expected; @IF is intentionally not short-circuiting, i.e. its arguments need to get rendered first. For example, you could use macros for the condition itself, so that's one reason.

Closing as a non-issue for now, but happy to continue discussing. As you pointed out, the solution is to push the logic into the macro itself.

cc @tobymao