Python

The pytexl package runs the Texl engine in Python programs. It renders templates from strings, loads documents from local files and over HTTP, and lets templates call your Python functions, async ones included.

Installing the package

The package is not published on PyPI yet. Install it from a clone of the repository. You need Rust 1.85 or later and Python 3.10 or later:

pip install ./pytexl

pip builds the native module with maturin. To build a wheel instead, run maturin build --release -m pytexl/Cargo.toml; the wheel is written to target/wheels.

Quick start

import asyncio
import pytexl

engine = pytexl.Engine({"user": {"name": "Ada"}})
text = asyncio.run(engine.render("Hello <% user.name %>!"))
print(text)                                    # Hello Ada!

The engine is asynchronous: render and load are coroutines, because a tag may wait for a document being loaded or for an async function. In a program that is not asynchronous, run them with asyncio.run. In asyncio code, await them:

async def main():
    engine = pytexl.Engine()
    print(await engine.render("2 + 2 = <% 2 + 2 %>"))   # 2 + 2 = 4

asyncio.run(main())

Loading documents

engine.load reads a document, renders it, and returns its rendering, source and data:

from pathlib import Path

doc = await engine.load(Path("templates/report.texl"))
if doc:
    Path("report.md").write_text(doc.rendering)

A path is resolved against the current working directory. A string is a URL, resolved against the engine's base URL, which is the current working directory by default: engine.load("templates/report.texl") reads the same file. Full URLs work too:

doc = await engine.load("https://example.com/templates/report.texl")

The engine reads documents with loaders, chosen by the scheme of the URL:

Inside a document, imports are resolved against the URL of the document:

In a document fetched over HTTP, relative paths and absolute paths resolve against its URL in the same way, so "/lib/format.texl" in https://example.com/docs/report.texl fetches https://example.com/lib/format.texl.

Templates rendered with engine.render, or with a context created without a URL, have no URL of their own: their imports resolve against the root path /, so "lib/format.texl" and "/lib/format.texl" both read the local file /lib/format.texl. Use absolute paths there, load the template as a document, or give the context the URL of a document (see Contexts).

A document that cannot be read counts as a failed load: load gives None, and texl.import gives Nothing. A response with an HTTP error status is a failed load too.

Each engine loads and renders each document once, and reuses the result for later imports. Create a new engine to see changes to the documents.

Your own loaders

A loader is a function that takes the URL of a document and returns its source, or an awaitable of it. Register it for a scheme with add_loader:

templates = {"mem://greeting": "Hello <% reader %>!"}

engine = pytexl.Engine({"reader": "Ada"})
engine.add_loader("mem", lambda uri: templates[uri])

doc = await engine.load("mem://greeting")
print(doc.rendering)                           # Hello Ada!

If the loader raises an exception, or returns something other than a string, the document cannot be loaded. async loaders work the same way:

async def load_from_db(uri):
    row = await db.fetch_one("SELECT source FROM templates WHERE uri = ?", uri)
    return row["source"]

engine.add_loader("db", load_from_db)

set_default_loader replaces the loader of the URLs without a scheme. The engine normalizes those paths before calling it: they are absolute and percent-encoded, such as /dir/my%20doc.texl. This loader serves the files of one folder only:

root = Path("site").resolve()
engine.set_default_loader(lambda path: (root / path.lstrip("/")).read_text())

Globals and context values

The values you pass to the engine and to contexts become Swan values:

Values that come back to Python, from context.get, from document data and as function arguments, are converted the other way:

A tuple with one item is that item, and an empty tuple is Nothing, as in Swan.

Functions

A Python function becomes a Swan function. A Swan call with several arguments passes them as separate arguments, and a call with () passes none:

engine = pytexl.Engine({
    "shout": lambda s: s.upper() + "!",
    "add": lambda a, b: a + b,
    "answer": lambda: 42,
})
await engine.render('<% shout "hi" %> <% add(2, 3) %> <% answer() %>')   # 'HI! 5 42'

The return value is converted to a Swan value. If the function raises an exception, the result is Nothing. An async function, or a function that returns an awaitable, gives a Swan promise, and the tag waits for it:

async def weather(city):
    await asyncio.sleep(0.1)                   # a network request, in a real program
    return "sunny"

engine = pytexl.Engine({"weather": weather})
await engine.render('Today in Rome: <% weather "rome" %>')   # 'Today in Rome: sunny'

A coroutine object passed as a value is awaited once, and every template sees its result.

Swan functions and promises

The functions that a template defines come back to Python as Function objects, which you call like Python functions. A document can be a library:

doc = await engine.load("lib/format.texl")     # defines plural = (count, word) -> ...
doc.data["plural"](3, "apple")                 # '3 apples'

Swan promises come back as Promise objects, which you await:

context = engine.create_context({"weather": weather})
await context.render('<% forecast = weather "rome" %>')
await context.get("forecast")                  # 'sunny'

Undefined values

An invalid operation in Swan does not raise an exception: it produces an undefined value, which records what went wrong. It comes back to Python as an Undefined object, with the same details as undef.inspect:

context = engine.create_context({"price": "12.5"})    # a string, by mistake
await context.render("<% total = price * 4 %>")
total = context.get("total")
total                  # <pytexl.Undefined BinaryOperation op='product' lhv='12.5' rhv=4 pos=15>
total.op, total.lhv    # ('product', '12.5')
bool(total)            # False

None always means Nothing: a name that is not bound, or an empty tuple. To tell the two apart, test isinstance(value, pytexl.Undefined).

Contexts

engine.render uses a fresh scope each time, so the names a template binds are lost when it ends. A context keeps them: it is a scope that you render several templates in, and that you can read and write from Python.

context = engine.create_context({"price": 12.5})
await context.render("<% total = price * 4 %>")
context.get("total")                           # 50
await context.render("Total: <% total %>")     # 'Total: 50'

A context can also stand for a document: pass its URL or path as uri, and the imports of the templates rendered in the context resolve against it. Use it to render a document whose source you already have, without reading it again:

context = engine.create_context(uri=Path("templates/report.texl"))
rendering = await context.render(source)   # imports resolve in templates/

Event loops and threads

An engine runs its work on the asyncio event loop of the code that awaits it. It can outlive the loop: you can call asyncio.run several times with the same engine. Await Promise objects in a running loop.

The engine is single-threaded. Use an engine, its contexts, and the promises and functions that come from them on the thread that created the engine: the methods of an engine or a context raise RuntimeError on another thread. Each thread can have engines of its own. The loaders read files and fetch URLs in worker threads, so they do not block the event loop.

API reference

The docstrings of the package document the same API, in Google style. Run help(pytexl) to read them.

Engine

An engine renders templates and loads documents. Its globals, documents and engine-wide names (bound with bind) are shared by everything it renders.

Engine(globals, base_url)

Engine(globals: Mapping[str, Any] | None = None,
       base_url: str | os.PathLike[str] | None = None)

Creates an engine with the file and HTTP loaders.

Raises ValueError if base_url is not an absolute URL.

engine = pytexl.Engine({"site": {"name": "My site"}}, base_url=Path("templates"))

engine.base_url

base_url: str

Returns the base URL of the engine, normalized. It is read-only.

pytexl.Engine(base_url="https://Example.com/a/../b/").base_url   # 'https://example.com/b/'

engine.set(name, value)

def set(name: str, value: Any) -> None

Binds a global, visible to every template and document of the engine, including contexts created earlier. A global with the same name is replaced.

engine.set("today", datetime.date.today().isoformat())

engine.add_loader(scheme, loader)

def add_loader(scheme: str, loader: Loader) -> None

Registers the loader of the URLs with a scheme, replacing any previous loader of that scheme, the built-in ones included.

engine.add_loader("mem", lambda uri: templates[uri])

engine.set_default_loader(loader)

def set_default_loader(loader: Loader) -> None

Registers the loader of the URLs without a scheme, replacing loaders.load_file. The loader receives absolute, percent-encoded paths, such as /dir/my%20doc.texl.

engine.set_default_loader(lambda path: (root / path.lstrip("/")).read_text())

engine.load(uri)

async def load(uri: str | os.PathLike[str]) -> Document | None

Loads and renders the document at uri.

Returns the document, or None if it cannot be loaded. Raises ValueError if uri cannot be resolved to a URL. The result is cached: later imports of the same URL reuse it.

doc = await engine.load("templates/report.texl")
print(doc.rendering if doc else "not found")

engine.render(template)

async def render(template: str) -> str

Renders a template in a fresh scope, and returns the rendered text. The names the template binds with = or : are discarded afterwards.

await engine.render("2 + 2 = <% 2 + 2 %>")    # '2 + 2 = 4'

engine.create_context(vars, uri)

def create_context(
    vars: Mapping[str, Any] | None = None,
    uri: str | os.PathLike[str] | None = None,
) -> Context

Creates a context: a scope that keeps its names across renders.

With a uri, imports in the context resolve against it, as in a document loaded with load; nothing is read from it. Without one, they resolve against the root path /, as in render. Raises ValueError if uri cannot be resolved to a URL.

context = engine.create_context({"user": "Ada"})
report = engine.create_context(uri=Path("templates/report.texl"))

Context

A scope of an engine, created with engine.create_context. Templates rendered in a context share its names, and so do get and set. The context also sees the globals of its engine.

context.get(name)

def get(name: str) -> Any

Returns the value of a name, converted to a Python value. The name is looked up in the context, then in the engine globals and the built-ins. A name that is not bound gives None.

await context.render("<% x = (1, 2) %>")
context.get("x")                               # (1, 2)

context.set(name, value)

def set(name: str, value: Any) -> None

Binds a name in the context, replacing any previous value.

context.set("items", ["a", "b"])

context.names()

def names() -> list[str]

Returns the names bound in the context, sorted. Engine globals and built-ins are not included. The list always includes texl, the namespace that holds texl.import.

engine.create_context({"b": 1, "a": 2}).names()   # ['a', 'b', 'texl']

context.render(template)

async def render(template: str) -> str

Renders a template in the context, and returns the rendered text. The names the template binds stay in the context.

await context.render("<% count = 1 %>")
await context.render("Count: <% count %>")     # 'Count: 1'

Document

@dataclass(frozen=True)
class Document:
    uri: str
    source: str
    data: dict[str, Any]
    rendering: str

A loaded document, as returned by engine.load.

doc = await engine.load("settings.texl")
print(doc.data["title"], doc.rendering)

Promise

class Promise:
    done: bool
    def __await__(self) -> Generator[Any, None, Any]

A Swan promise. Awaiting it returns its value, converted to a Python value. Promises never fail: a promise whose value would be undefined resolves to None instead, as in Swan. The done attribute is True once the promise has a value. Await a promise in a running event loop.

await context.render("<% p = promise.create() %>")
p = context.get("p")                           # <pytexl.Promise pending>
await context.render("<% promise.resolve p 42 %>")
await p                                        # 42

Function

class Function:
    def __call__(self, *args: Any) -> Any

A Swan function. Calling it with no argument passes Nothing, with one argument passes that argument, and with several arguments passes them as a tuple. The result is converted to a Python value. A call never raises: an invalid call returns an Undefined.

await context.render("<% double = x -> x * 2 %>")
double = context.get("double")
double(21)             # 42
double("a")            # <pytexl.Undefined BinaryOperation op='product' lhv='a' rhv=2 pos=17>

Undefined

class Undefined:
    type: str
    op: str | None
    val: Any
    lhv: Any
    rhv: Any
    pos: int | None
    expected: str | None
    token: str | None
    def inspect(self) -> dict[str, Any]

A Swan undefined value, the result of an invalid operation. Its attributes are the members of undef.inspect: type names the kind of error, and the other attributes depend on it. The attributes that do not apply are None.

The operands are converted to Python values, so an operand that is itself undefined is an Undefined too. pos is the position of the error in the source of the tag, in bytes; built-in functions report 0. inspect() returns type and the attributes that apply, as a dict.

An Undefined is falsy. Two Undefined objects are equal when they have the same type and attributes, whatever their position. Passed back to Swan, an Undefined is the original undefined value.

await context.render('<% err = text.size 1 %>')
err = context.get("err")
err.type               # 'UnaryOperation'
err.inspect()          # {'type': 'UnaryOperation', 'op': 'text.size', 'val': 1, 'pos': 0}

Loader

Loader = Callable[[str], str | Awaitable[str]]

The type of a document loader: a function that takes a URL and returns the source of the document, or an awaitable of it. It raises an exception if the document cannot be read.

loaders

The module pytexl.loaders holds the loaders that every engine starts with. You can register them again, for example after replacing one.

loaders.load_file(uri)

async def load_file(uri: str) -> str

Reads a document from the local file system, as UTF-8, in a worker thread. It serves file:// URLs and paths without a scheme, which are absolute paths. Raises ValueError if uri is not a local file URL or path, and OSError if the file cannot be read.

await loaders.load_file("file:///home/ada/report.texl")

loaders.load_http(uri, timeout)

async def load_http(uri: str, *, timeout: float = 30.0) -> str

Fetches a document over HTTP or HTTPS, in a worker thread, with urllib.request. Redirects are followed, and the body is decoded with the charset of the response, UTF-8 by default. Raises urllib.error.URLError if the request fails or the response has an error status, and ValueError if uri is not an HTTP or HTTPS URL.

engine.add_loader("https", functools.partial(loaders.load_http, timeout=5))

loaders.file_path(uri)

def file_path(uri: str) -> pathlib.Path

Returns the local path of a file:// URL or of an absolute path, with percent-encoded characters decoded. The query and the fragment are ignored. Raises ValueError if uri has another scheme or names a file on another host.

loaders.file_path("file:///my%20dir/doc.texl?v=1")   # PosixPath('/my dir/doc.texl')