Release Announcement: Wove 2.0.0, Beautiful python async. Now featuring remote execution envs

• Release page: https://curvedinf.github.io/wove/version-history/2.0.0.html
• Github: https://github.com/curvedinf/wove
• Docs: https://curvedinf.github.io/wove/
• Install: uv add wove or pip install wove

Hi all, The latest Wove release has an exciting new feature: the ability to transport local python callables to your remote task systems like Celery and AWS Batch as an integral part of any inline wove async orchestration. This feature allows you to use the same magical implicit async orchestration as before, but also automatically export locally defined tasks to network worker environments. For example rendering a PDF on a celery worker via function defined in a django view:

# celery_worker.py
import os

from celery import Celery
from wove.integrations.worker import run

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")

app = Celery("project")
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()


@app.task(name="billing.wove_task", queue="wove")
def wove_task(payload):
    # Magically deserialize a task and execute it
    return run(payload)

and

# views.py
import httpx
import wove
from django.http import JsonResponse
from wove import weave
from .models import Customer, Invoice


# To be remotely executed on celery
def render_invoice_pdf(invoice, customer):
    return {
        "url": f"https://cdn.example.com/invoices/{invoice['id']}.pdf",
        "customer": customer["name"],
    }

# This config can also be in wove_config.py. It defines the remote env.
wove.config(
    default_environment="local",
    environments={
        "local": {"executor": "local"},
        "reports": {
            "executor": "celery",
            "executor_config": {
                "broker_url": "redis://redis:6379/0",
                "result_backend": "redis://redis:6379/1",
                "task_name": "billing.wove_task",
                "queue": "wove",
                "callback_host": "0.0.0.0",
                "callback_port": 9010,
                "callback_url": "http://web:9010/wove/events/shared-secret",
                "callback_token": "shared-secret",
            },
            "delivery_timeout": 120.0,
        },
    },
)

# A django view with wove async orchestration of a remote task
async def invoice_page(request, invoice_id):
    async with httpx.AsyncClient(timeout=5.0) as client:
        async with weave(invoice_id=invoice_id) as w:
            # First gather some needed data in parallel
            @w.do
            async def invoice(invoice_id):
                return await Invoice.objects.values(
                    "id",
                    "customer_id",
                    "total",
                    "status",
                ).aget(id=invoice_id)

            @w.do
            async def fraud_score(invoice_id):
                response = await client.get(
                    f"https://risk.internal/invoices/{invoice_id}/score"
                )
                response.raise_for_status()
                return response.json()

            @w.do
            async def customer(invoice):
                return await Customer.objects.values(
                    "id",
                    "name",
                    "email",
                ).aget(id=invoice["customer_id"])

            # Using the collected data, render a PDF on the celery worker
            @w.do(
                environment="reports",
                retries=2,
                delivery_timeout=120.0,
                delivery_idempotency_key="invoice-pdf:{invoice_id}",
            )
            def pdf(invoice, customer, invoice_id):
                # This task, including the called function, get sent to the remote env for execution.
                return render_invoice_pdf(invoice, customer)
            
            # Return context for the response
            @w.do
            def response(invoice, customer, fraud_score, pdf):
                return {
                    "invoice": invoice,
                    "customer": customer,
                    "fraud_score": fraud_score,
                    "pdf": pdf,
                }
    
    # As usual, the with block ending executes the graph immediately
    return JsonResponse(w.result.final)

What is Wove For?

Wove is for running high latency async tasks like web requests and database queries concurrently in the same way as asyncio, but with a drastically improved user experience.

Improvements compared to asyncio include:

• Reads Top-to-Bottom: The workflow is declared in the order it runs, inline with the code that needs the result.
• Implicit Parallelism: Parallelism and execution order are implicit based on function and parameter naming.
• Sync or Async: Mix async def and def freely without restructuring the call site around one concurrency style.
• Normal Python Data: Task outputs behave like normal Python values without making you manage shared mutable state.
• Automatic Scheduling: Wove builds a dependency graph from your task signatures and runs independent tasks concurrently as soon as possible.
• Automatic Detachment: Run inline workflows outside the current request, command, or worker when waiting would be the wrong user experience.
• Remote Task Environments: Keep quick work local while sending selected long-running or infrastructure-heavy tasks to your worker service, queue, workflow engine, cluster, or scheduler.
• Extensibility: Define parallelized workflow templates that can be overridden inline.
• High Visibility: Wove includes debugging tools that allow you to identify where exceptions and deadlocks occur across parallel tasks, and inspect inputs and outputs at each stage of execution.
• Minimal Boilerplate: Get started with just the with weave() as w: context manager and the @w.do decorator.
• Fast: Wove has low overhead and internally uses asyncio, so performance is comparable to using threading or asyncio directly.
• Free Threading Compatible: Running a modern GIL-less Python? Build true multithreading without changing the workflow shape.
• Zero Required Dependencies: Core Wove installs without third-party packages. Serialization, networking, and backend libraries are only needed when a workflow opts into features that use them.

New for 2.0.0

• Remote task environments: let tasks stay local by default while selected tasks run on remote environments with automatic orchestration
• Wove config: use wove.config(...) to set file-wide reusable settings or wove_config.py for project-wide settings.
• Backend adapters: built-in integrations for Celery, Temporal, Ray, RQ, Taskiq, ARQ, Dask, Kubernetes Jobs, AWS Batch, and Slurm.
• Executor layer: public executor interface, local, stdio, HTTP/HTTPS, gRPC, and WebSocket executors for custom task execution.
• merge(...) upgrade: sync or async callers can use sync or async callables, with w.do(...) -style options including remote environments.