Allow returning native python objects in minijinja-py

[fortzi]

Description

Python's implementation of Jinja2 allows returning native objects, as opposed to always stringifying the result. This is useful if you are using Jinja outside the context of creating text files, like an intermediate step where users may use templates to define values that will then be used elsewhere.

Example in Jinja2

from jinja2 import Environment
from jinja2.nativetypes import NativeTemplate

env = Environment()
env.template_class = NativeTemplate

value = env.from_string('{{ 1 }}').render()
assert isinstance(value, int)

Suggested behavior in minijinja-py

from minijinja import Environment

env = Environment(native=True)
value = env.render_str('{{ 1 }}')

assert isinstance(value, int)

[mitsuhiko]

Matching the behavior of the NativeTemplate with minijinja is incredibly complex and most of how the native template support in Jinja2 works is pretty wonky. Now there might be ways to get this going with minijinja, but the complexity of this is very high and the question is if this is worthwhile.

I would love to better understand under which circumstances this is necessary and useful.

[fortzi]

First, thanks for the quick reply and for maintaining this library!

In systems where Jinja is used widely, users use system-provided data objects to write templates, and often, intermediate variables are repeated in many templates. Instead of repeating the definition of a complicated variable in many templates, we render those separately and feed them into where they're used.
The intermediate values can be inspected and thought of individually, which is invaluable in debugging the final result in retrospect, especially in large systems where there are many many templates, and Minijinja is more useful due to its superior performance.
Moreover, using them helps maintaining a cleaner namespace and separate concerns. While namespaces, {% import %} and {% inlcude %} exist in Jinja, users benefit from using precalculated intermediate variables, which greatly reduces complexity their mental load. This is especially true for non-developers, which I assume are a great portion of the users who write Jinja templates.

This account describes my use case, but I believe that this is the case for every system where users manage a great number of templates.

I admit that I initially thought that it's going to be as easy as "just don't stringify the result!", but I understand that it's probably much more complicated.

[mitsuhiko]

Could you provide a motivating example with a few templates that demonstrates this? In particular I wonder if you just care about the single value response {{ x }} or also the "parse the rendered result back" case of [{% for x in foo %}{{ x }},{% endfor %}] etc.

[fortzi]

Sure:

Say you want to use the expected delivery date for some order that a customer has made

Without intermediate variables

Example 1

{%- set fulfillments = (customer_orders or []) | selectattr('id', 'eq', latest_customer_order_id) -%}
{%- set estimated_delivery_dates = fulfillments | map(attribute='estimated_delivery_at') | unique | select | list -%}
{%- if estimated_delivery_dates | length == 1 -%}
        {%- set estimated_delivery_date = estimated_delivery_dates[0] | with_tzinfo | format_date(...) -%}
{%- else -%}
    {%- set estimated_delivery_date = 'next week' -%}
{%- endif -%}

Hi {{ CustomerName }},

Thank you for ordering ... !
Your order is due to arrive by {{ estimated_delivery_date }}.
...

Example 2

{%- set fulfillments = (customer_orders or []) | selectattr('id', 'eq', latest_customer_order_id) -%}
{%- set estimated_delivery_dates = fulfillments | map(attribute='estimated_delivery_at') | unique | select | list -%}
{%- if estimated_delivery_dates | length == 1 -%}
        {%- set estimated_delivery_date = estimated_delivery_dates[0] | with_tzinfo | format_date(...) -%}
{%- else -%}
    {%- set estimated_delivery_date = 'next week' -%}
{%- endif -%}

Hi {{ CustomerName }},

We're sorry to hear that your order is being delayed.
According to our courier, your order should arrive by {{ estimated_delivery_date }}.
...

With intermediate variable estimated_delivery_date

{%- set fulfillments = (customer_orders or []) | selectattr('id', 'eq', latest_customer_order_id) -%}
{%- set estimated_delivery_dates = fulfillments | map(attribute='estimated_delivery_at') | unique | select | list -%}

{%- if estimated_delivery_dates | length == 1 -%}
    {{ estimated_delivery_dates[0] | with_tzinfo | format_date(...) }}
{%- else -%}
    next week
{%- endif -%}

Example 1

Hi {{ CustomerName }},

Thank you for ordering ... !
Your order is due to arrive by {{ estimated_delivery_date }}.
...

Example 2

Hi {{ CustomerName }},

We're sorry to hear that your order is being delayed.
According to our courier, your order should arrive by {{ estimated_delivery_date }}.
...

In this example, the variable's type is list[str], but it can similarly be a more complex object like list[FulfillmentInfo], etc.

I'll add that while it's sometimes possible to parse the render's output, it can be surprisingly costly operation if the returned object is big.
If implementing this feature for Minijinja means parsing on Minijinja's side, then this endeavour may be futile. It will likely only be meaningful if parsing can be skipped by using whatever object is at hand before stringifying it.

[fortzi]

I'd like to add a use case here: this feature will make it possible to use {% set %}...{% endset %} blocks to create non-string variables, just like in the single line version of set {% set X = ... %}.