Async-Only Consumer Constraint Creates Ecosystem Incompatibility

Issue 90 Edition 2026-03-31 4 min read

General

Sources: 1 • Confidence: High • Updated: 2026-04-01 03:38

Key takeaways

LLM plugins can define models in both synchronous and asynchronous forms.
The llm-all-models-async plugin converts synchronous models into asynchronous models using a thread pool.
This sync-to-async conversion required an additional LLM plugin hook mechanism that shipped in LLM 0.30.
Async model variants are usually used for API-backed models, while sync variants more often run the model directly inside the plugin.
Version 0.1 of the llm-all-models-async plugin has been released.

LLM plugins can define models in both synchronous and asynchronous forms.
The llm-mrchatterbox plugin is synchronous only.
Datasette can only use asynchronous models, which blocks use of sync-only plugins for features such as datasette-enrichments-llm.

The llm-all-models-async plugin converts synchronous models into asynchronous models using a thread pool.
Version 0.1 of the llm-all-models-async plugin has been released.

This sync-to-async conversion required an additional LLM plugin hook mechanism that shipped in LLM 0.30.

Async model variants are usually used for API-backed models, while sync variants more often run the model directly inside the plugin.

What are the concurrency, latency, and resource-usage characteristics of the thread-pool conversion under realistic load (e.g., thread exhaustion behavior, throughput limits, queuing effects)?
Which specific LLM 0.30 hook(s) are required by the adapter, and what is the minimal supported LLM version for llm-all-models-async?
Does the sync-to-async conversion preserve error semantics, cancellation behavior, and streaming/token-yielding behavior (if applicable) expected by async consumers?
Which sync-only plugins beyond llm-mrchatterbox are intended to be supported, and are any model classes excluded from conversion (by design or limitation)?
Does Datasette integration (e.g., datasette-enrichments-llm) work end-to-end when using a converted sync model via llm-all-models-async?

Async only consumers can pressure the LLM plugin ecosystem toward async interfaces, creating a compliance layer opportunity via adapters such as llm-all-models-async.
LLM 0.30 adds hook mechanisms enabling cross cutting interface adaptation, implying increasing platform extensibility that could support more middleware style plugins.
Thread pool based sync to async bridging can expand the usable plugin set for async only integrations such as Datasette, potentially increasing overall ecosystem interoperability.

Evidence that Datasette integrations work end to end using converted sync models via llm-all-models-async.
Published benchmarks or reports on latency, throughput, and resource usage under load for the thread pool conversion, including behavior under thread exhaustion.
Clear documentation of required LLM 0.30 hooks and minimal supported LLM version, plus a list of sync only plugins successfully supported.

Thread pool conversion shows unacceptable concurrency limits or queuing delays under realistic load, preventing practical use in async only consumers.
Converted models fail to preserve expected async behavior such as error semantics, cancellation, or streaming token yielding, breaking consumers.
Adapter requires newer platform changes beyond LLM 0.30 or proves incompatible with key consumers, limiting interoperability impact.