Quick Start
Define some tasks, create a pool, call the tasks like async functions, let the pool close itself. That is the whole loop, and it takes a few minutes.
Introduction
Section titled “Introduction”If you have used workers before: Knitting is workers with a function-call API and much less overhead. If you haven’t, the terms below are all the vocabulary you need.
Quick definitions
Section titled “Quick definitions”- Task: a function the workers can run. An exported function already counts
as one; wrap it in
task({ f })when you want options like timeouts or aborts. call.*(): runs a task on the pool and returns aPromisewith the result.isMain:trueon the host,falseinside a worker. Workers re-import your module, so anything that should happen once — creating the pool, starting a server — belongs behind this check.createPool(): starts the workers and hands back a typedcallobject and ashutdown. It is disposable too, which is what letsusingclose it.- Host ↔ Worker: the host is the process that creates the pool. The workers are the threads (or processes) that run the tasks.
Examples
Section titled “Examples”Four snippets, roughly in the order you will need them.
import { createPool, isMain } from "knitting";
// A task is just an exported function the workers can run.export const greet = (name: string) => `hello ${name}`;
if (isMain) { // `using` shuts the pool down automatically when this block ends. using pool = createPool({ threads: 1 })({ greet });
console.log(await pool.call.greet("knitting")); // hello knitting}import { createPool, isMain } from "knitting";
export const hello = () => "hello ";export const world = (prefix: string) => `${prefix}world!`;
if (isMain) { using pool = createPool({ threads: 2 })({ hello, world });
// call.hello() returns a promise; Knitting resolves it before world runs. const lines = await Promise.all( Array.from({ length: 3 }, () => pool.call.world(pool.call.hello())), );
console.log(lines.join(" ")); // hello world! hello world! hello world!}import { createPool, isMain } from "knitting";
// Several tasks share one pool. Calls are promises, so you can chain them.export const double = (n: number) => n * 2;export const square = (n: number) => n * n;
if (isMain) { using pool = createPool({ threads: 2 })({ double, square });
const results = await Promise.all( [1, 2, 3, 4, 5].map(async (n) => pool.call.square(await pool.call.double(n))), );
console.log(results); // [4, 16, 36, 64, 100]}import { createPool, isMain, task } from "knitting";
// Wrap a function with task() when you want options like a timeout.// This call is too slow, so it falls back to the default instead of hanging.export const slow = task({ timeout: { time: 100, default: "timed out" }, f: async (name: string) => { await new Promise((resolve) => setTimeout(resolve, 1_000)); return `hello ${name}`; },});
if (isMain) { using pool = createPool({ threads: 1 })({ slow });
console.log(await pool.call.slow("knitting")); // timed out}Build it step by step
Section titled “Build it step by step”-
Import what you need:
import { createPool, isMain } from "knitting"; -
Export your tasks at module scope. Workers find them by name, so they have to be reachable from the top level of the file:
export const square = (n: number) => n * n;export const greet = (name: string) => `hello ${name}`; -
Create the pool behind
isMain. Workers re-import this module, and without the guard every one of them would try to start a pool of its own:if (isMain) {using pool = createPool({ threads: 2 })({ square, greet });}usingcloses the pool when the block ends, so there is nothing to clean up. -
Call the tasks. They hand back ordinary promises, so
Promise.allbatches them:if (isMain) {using pool = createPool({ threads: 2 })({ square, greet });const [n, message] = await Promise.all([pool.call.square(8),pool.call.greet("knitting"),]);console.log({ n, message }); // { n: 64, message: "hello knitting" }} -
Shut down when you are done.
With
usingthat already happened at the end of the block: the workers stop and the process can exit. Callshutdown()yourself when you want to pick the moment, or when your runtime has nousing:const pool = createPool({ threads: 2 })({ square, greet });try {console.log(await pool.call.square(8));} finally {await pool.shutdown();}
A task can make its own pool
Section titled “A task can make its own pool”One task in a short script? Skip the separate createPool and let the task
carry its own:
import { isMain, task } from "knitting";
export const double = task({ f: (n: number) => n * 2,}).createPool({ threads: 2 });
if (isMain) { try { console.log(await double.call(21)); // 42 } finally { await double.shutdown(); }}Good habits
Section titled “Good habits”None of this is required. All of it saves you an afternoon later.
Keep tasks in their own module(s)
Section titled “Keep tasks in their own module(s)”Workers import that module and nothing around it, so they start fast and stay out of the way of your app code.
- package.json
- deno.json
Directorysrc
Directoryknitting
- database.ts
- img_parsing.ts
- jwt.ts
Directoryapp/
- …
Directorypages/
- …
Promise inputs are awaited on the host
Section titled “Promise inputs are awaited on the host”call.*() accepts Promise<supported> inputs. Knitting resolves them on the
host before dispatch, so unresolved promise state never crosses the thread
boundary. Request handlers get this for free — hand the call a body you have
not read yet:
app.post("/validate", async (c) => { const result = await pool.call.validate(c.req.text()); return c.json(result);});If that promise rejects, the call rejects on the host and the worker never runs.
Pick the cheapest payload that fits
Section titled “Pick the cheapest payload that fits”Smaller and flatter travels faster. Numbers, booleans and short strings first,
then typed arrays and Buffer, then compact JSON. When you have bytes plus some
metadata to describe them, use Envelope. See
Supported payloads.
Start strict, open up later
Section titled “Start strict, open up later”Worker permissions start restricted. Sensitive paths like .env, .git,
~/.ssh and /etc are blocked, and node_modules is deny-write. Open up only
what your tasks actually touch. See Permissions.
Footguns
Section titled “Footguns”Almost everything here follows from one fact: workers re-import the module that defines your tasks.
- One argument per task.
pool.call.add(a, b)won’t work — pass a tuple or object:pool.call.add([a, b])for([a, b]) => a + b. - Guard host code with
isMain. Without it, pool creation (and any other host-only code) re-runs inside every worker. - Top-level imports run in every worker.
importis hoisted, so it executes before anyisMaincheck. Keep tasks in their own lean module so workers don’t load your whole server framework. - Export your tasks. An unexported
task()/importTask()is invisible to the worker loader, so the call just hangs — no handler is ever registered. importTasktargets are plain functions, nottask()wrappers (that throws aTypeError). Puttimeout/abortSignaloptions on theimportTaskcall instead.- Worker
console.*is silent by default in strict mode. Passpermission: { console: true }to surface worker logs. - Can’t tell what the pool is doing? Pass
debug: truetocreatePool, or setKNITTING_DEBUG=*. Setup, import and lifecycle diagnostics go to stderr, each line tagged with its worker and a millisecond timer. If that is too much, name the parts you care about:debug: { host: true, imports: true }. Turned off, it costs nothing. - Only supported payloads cross the boundary.
Map,Set, class instances, and functions are rejected — see Payloads. - Dynamic payloads cap at ~8 MiB by default; raise
payload.maxPayloadBytes(andpayload.payloadMaxByteLength) for larger ones.
Where to go next
Section titled “Where to go next”- Defining tasks —
task(),importTask(), timeouts, and aborts. - Creating pools — threads, balancers, and shutdown.
- Payloads — what crosses the boundary, and
Envelope. - Performance and the inliner — when to let the host run some work too.
Two things worth knowing about before you need them. Workers can each run as a
separate process, inside a bwrap sandbox or a
container, when threads are not enough isolation. And large buffers can skip the
copy entirely with ProcessSharedBuffer, which puts
the bytes in shared memory instead of sending them.