AI — Complete Reference

The AI fleet routes work to a set of GPU hosts. You ask for an artifact and declare what the work is for; the broker decides where it runs, loads it if needed, and hands back a URL to call. This document covers every supported feature and configuration.

Overview

Three moving parts:

• Broker — the control plane. Owns scheduling, the catalog, and a data-plane proxy. Reachable on the LAN at http://192.168.1.13:8080.
• Agents — one per GPU host. Poll desired state, launch/stop services, report health + nvidia-smi.
• Client — ai-fleet-client, a pip package wrapping the broker API.

The unit of work is a ticket. You POST a request, the broker queues it, schedules a host, loads the model if it isn't running, and marks the ticket ready with a proxy URL. All your data-plane calls go through that URL — the real backend address stays hidden.

Quick start

import ai_fleet_client as sb, httpx

url = sb.request("qwen3.6-35b-a3b-heretic-v2-q4",
                 environment="production", urgency="interactive")

r = httpx.post(f"{url}/v1/chat/completions", json={
    "messages": [{"role": "user", "content": "Hello"}]})
print(r.json()["choices"][0]["message"]["content"])

Install & configuration

pip install ai-fleet-client --extra-index-url https://ai.wtray.com/wheels/

Environment variables read by the client:

Artifacts

An artifact is the concrete thing you ask for — a model id with its quantisation (qwen3-vl-32b-q8 and a Q4 of the same model are different artifacts), a ComfyUI workflow, or a service name (kokoro, faster-whisper, pyannote).

A run-profile is a tested way to run one artifact on a specific host/GPU with specific settings. One artifact has many profiles, and profiles of the same artifact can differ — notably in context length. Enumerate the catalog at runtime with sb.fleet() rather than hard-coding.

Workload class

Every request and session declares three independent axes. They drive scheduling, not model selection. environment and urgency are mandatory.

Rank is lexicographic: rank = environment×100 + urgency. A running service is committed at the highest rank actively using it and can only be evicted by a strictly higher rank — or unloaded once nothing needs it.

Requirements

Satisfy-or-queue hints. The broker only considers profiles that meet them, and returns 400 if no profile anywhere can. Unknown keys are ignored (forward-compatible).

The same artifact can offer different context windows

An artifact id names a model and a quantisation — not a guaranteed context length. The same id can be tested at different windows on different hardware, because a smaller card cannot hold a larger KV cache. Today:

Two ways to deal with it, and you should use one of them if window size matters:

# 1. Demand a floor — profiles below it are filtered out entirely
url = sb.request("qwen3-4b-q4", requirements={"min_context": 16384})

# 2. Or observe what you actually got
url = sb.request("qwen3-4b-q4")
print(sb.last_placement)
# {'host': 'ai-9', 'service': 'qwen3-4b-q4__gpu1', 'context': 4096, ...}

Ready tickets carry host, service and context, and ai-fleet-client records the last one in sb.last_placement and logs it. Everything else about a shared artifact id is held identical across hosts — same weights (verified by file size), same sampling flags, same chat template. Differences that remain are performance-only (--no-mmap on ai-150 vs mmap on ai-151, which changes paging, not output).

url = sb.request("darkidol-qwen3.8-27b-q4", environment="production", urgency="interactive",
                 requirements={"min_context": 32768})

Placement

Constrain how a profile is laid out on the host. Pass placement= (and/or gpu= for a specific index).

gpu=1 is equivalent to placement="gpu1" and matches only single-GPU variants on that index.

Sessions

A session declares the full set of artifacts a feature will use, so the broker can plan placement across the whole fleet at once, pre-warm them, and hold them for the session's lifetime — instead of scheduling each call blind to what comes next.

REQS = [
  {"artifact":"qwen3.6-35b-a3b-heretic-v2-q4", "environment":"production","urgency":"interactive","keep_alive":120},
  {"artifact":"darkidol-qwen3.8-27b-q4",       "environment":"production","urgency":"interactive","keep_alive":120},
  {"artifact":"qwen3-embedding-4b-q8", "environment":"production","urgency":"interactive","keep_alive":120,
   "placement":"cpu"},   # frees a GPU for the VLM
]
with sb.session(REQS, priority=5.0) as s:
    text  = s.request("qwen3.6-35b-a3b-heretic-v2-q4")
    vlm   = s.request("darkidol-qwen3.8-27b-q4")
    embed = s.request("qwen3-embedding-4b-q8", placement="cpu")
    # run the whole loop in here — all three stay warm

Each entry accepts artifact, environment, urgency, keep_alive (all required per entry), plus optional requirements, placement, gpu_index. requires may also be a plain list of artifact-id strings.

• s.request(artifact, …) — resolve against the warm plan; defaults to the class the artifact was declared with. Accepts the same overrides as sb.request.
• s.plan() — the broker's current placement plan (per-artifact host options and whether each is hot / planned / unavailable).
• s.close() — release (also on with-block exit).

Session semantics

Verify a session is actually in use: sb.status()["sessions"] — if that is 0 while your app is running, you are making isolated request() calls and the broker is scheduling each one blind to the next.

Agent & operator auth

Since 2026-08-29, /agent/* and the operator endpoints (/pin, /unpin, /force-config, /restart, /admin/enable) require the shared key: header X-Agent-Key matching ~/ai/secrets/agent.key on the broker box. Agents read it from agent_key.txt on their share (deployed by deploy.sh). Operator example: curl -H "X-Agent-Key: $(cat ~/ai/secrets/agent.key)" …. If the broker-side key file is removed, endpoints revert to LAN-trust (open) — enforcement follows the file.

Usage statistics

/stats (browser) and /stats.json (tooling) show usage aggregates aimed at spotting churn live: per-GPU heavy moves, power-guard blocks (rising count = the scheduler is repeatedly being asked to do the thing that crashed hosts on 2026-08-29), A→B→A alternations, per-artifact wait percentiles, and per-requester sessionless share + advisory-flagged pairs. Aggregates only; email-shaped requesters are masked; counters reset on broker restart. Same edge password as the dashboard.

Current defaults & retired services

Retirement is a gate, not a deletion: a retired artifact keeps its entry in manifests/artifacts.json with "disabled": true, is projected nowhere, and comes back by removing the flag. Clients must move to the replacement id; requests for a retired id queue until it is re-enabled.

Artifact catalog

The host RAM budget is measured, not declared. ram_mib_budget in manifests/hosts/<host>.json gates how much CPU-placed work may co-reside. It is now a starting point: the reconciler derives it from the host’s own idle readings (total − worst idle − 2,048 MiB), applies it through the same catalog rebuild the GPU budgets use, and logs every move. Whatever the source, a budget may never claim the last 2,048 MiB of a machine — ai-151 declared 33,427 MiB against 32,689 physical, which can only be met by swapping, and a host that swaps slows every service on it at once. Unlike the GPU budgets, RAM budgets are applied, not just logged: the blast radius is one host’s CPU co-residency rather than the fleet’s busiest lane losing its fastest card.

Live as of 2026-09-08 (uncensored-defaults cutover). Authoritative source is sb.fleet(). This table is generated from /fleet, not hand-maintained.

Co-residency & VRAM capacity

This is the most important operational section on this page.

Hardware budgets

The large-model conflict

The big text and vision models each want a whole card. Since the 2026-09-08 cutover the live lanes are qwen3.6-35b-a3b-heretic-v2-q4 (director) and darkidol-qwen3.8-27b-q4 (vision + narration); unseen-gemma4-26b-nsfw-q4 (roleplay) is a 26B MoE in the same footprint class as the old Mistral profile.

Declared budgets for the new ids are the placeholders they were registered with (file size plus KV and compute at ctx 32768, rounded up); the reconciler's vram reconcile log lines are the measurements to tune them from. Peak card usage sampled during the A/B runs: UNSEEN ~19,100 MiB on the 5090; DarkIdol ~22,000 on the 5090 at 32k ctx and ~19,600 on a 3090 at 16k.

The working three-model layout

A text + vision + embedding app still runs fully resident only with embeddings pushed to CPU:

ai-150 / gpu0  ->  qwen3.6-35b-a3b-heretic-v2-q4   # 26,000 of 30,000 MiB (5090)
ai-150 / cpu   ->  qwen3-embedding-4b-q8            # system RAM, no VRAM
ai-151 / gpu0  ->  darkidol-qwen3.8-27b-q4          # 21,000 of 22,000 MiB (3090, ctx 16k)
ai-150 / gpu1  ->  unseen-gemma4-26b-nsfw-q4        # 21,000 of 22,000 MiB (3090), roleplay

Without placement="cpu" on the embeddings, the broker may park them on ai-151/gpu0, which blocks the vision model entirely and forces a swap on every loop turn. The retired ids (qwen3-30b-a3b-q4, qwen3-vl-32b-q4, mistral-small-3.2-24b-q4, qwen3-8b-q4) stay in the catalog behind disabled: true; see Current defaults & retired services.

The layout the fleet holds

Since 2026-09-21 no host is pinned. Every card is free for whichever request needs it, so the roleplay lane, the vision lanes, the ComfyUI engines, speech and embeddings are all placed on demand. A pin is now a short-lived tool for benchmarks, never a standing layout: the old pinned layout kept the character orchestrator's image lanes warm, but it also left no card that could hold unseen-gemma4-26b-nsfw-q4, so production roleplay could not be placed at all.

The price is cold starts: a model that is not resident loads on the first request that needs it. Measured on an empty fleet (2026-09-21, four lanes requested together): roleplay 26B about 100 s, ComfyUI engine about 60 s, speech about 20 s, vision about 15 s, embeddings on CPU about 35 s. Hold anything interactive with a session or a longer keep_alive; idle hosts drain to empty after idle_timeout_sec (30 min). GET /status shows each host's pinned flag and current_config.

Client API — core

request()

sb.request(artifact, *, environment, urgency, keep_alive=30.0, requirements=None,
           priority=0.0, placement=None, gpu=None, requester=None,
           broker=None, timeout_sec=300.0) -> str

Blocks until a healthy endpoint exists; returns the proxy URL. Raises RequestTimeout on timeout, AIError on failure/cancel, UpgradeRequired if the client is too old.

session() / Session

sb.session(requires, *, priority=0.0, requester=None,
           broker=None, ttl_sec=None) -> Session
sb.subscribe(...)   # alias — a session IS a subscription

ttl_sec overrides the session lifetime, clamped to 30–3600 s (never rejected). Default is 600 s. Session.ttl_sec reports what the broker actually granted, which may differ from what you asked for.

Session.advisories lists requirements that some host cannot hold at the same time. Each entry is {artifacts: [a, b], conflict_on: [hosts], coresident_on: [hosts]}. Alternating between a flagged pair makes that host unload and reload gigabytes on every switch — the broker cannot avoid it, only report it. Each conflict is also logged once at session open.

comfy_run()

sb.comfy_run(artifact, params=None, *, environment, urgency,
             keep_alive=600.0, timeout_sec=600.0,
             poll_interval=3.0, requester=None,
             broker=None) -> list[dict]

Runs a ComfyUI workflow artifact end to end and returns its outputs. comfy_run() does the five steps a caller would otherwise repeat — resolve the graph, get a proxy, submit, poll, fetch — with the cold-start retry included.

outs = sb.comfy_run("comfyui-foley-stableaudio",
                    {"prompt": "birds chirping at dawn",
                     "seconds": 15.0, "seconds_total": 15.0},
                    environment="production", urgency="background")
open("birds.mp3", "wb").write(outs[0]["data"])

Returns [{filename, subfolder, type, kind, node, data}] — kind is ComfyUI's output bucket (audio, images, …) and data is bytes. The graph is fetched from the broker rather than hardcoded, so a workflow revision cannot silently break callers, and unknown params are rejected rather than ignored.

Introspection

Cancellation

sb.cancel(ticket, *, broker=None)         # cancel a QUEUED ticket
sb.cancel_ticket(ticket, *, broker=None)  # release a DISPATCHED ticket

Client API — speech

Text to speech

sb.tts_say(text, *, character_voice="af_bella", language="en", speed=1.0,
           chunk_min_chars=50, chunk_max_chars=300, inter_chunk_gap_ms=80,
           pronunciation_overrides=None, output_path=None, bundle=None,
           environment="production", urgency="realtime", artifact=None,
           priority=0.0, timeout_sec=300.0) -> bytes | str

sb.tts_say_stream(text, ...)  # generator of (chunk_text, wav_bytes)

Output is 24 kHz mono float32 RIFF/WAVE. tts_say buffers into one WAV (bundling on by default); tts_say_stream yields per chunk so you can start playback immediately. Chunks are silence-trimmed, with inter_chunk_gap_ms of breath prepended after the first.

Speech to text

sb.transcribe(audio, *, language="en", initial_prompt=None, model=None,
              word_timestamps=True, hotwords=None, no_speech_threshold=None,
              low_confidence_threshold=None, suppress_hallucinations=True,
              hallucination_blocklist=None, ...) -> dict

audio accepts bytes, a path, or a file-like object.

async with sb.transcribe_stream(language="en", hotwords=["Karrthûn"],
                                silence_ms=700, min_speech_ms=100,
                                max_utterance_ms=30000,
                                partial_interval_ms=300) as s:
    await s.send(pcm_int16_16k_mono)
    async for ev in s.events(): ...

Event types: ready, vad_start, partial, final, vad_stop, error. Send raw int16 16 kHz mono PCM; any frame size works.

TTS markup

tts_say* accept inline SSML-style markup:

Supported language codes: en, en-us, en-gb, es, fr, hi, it, pt, pt-br, ja, zh. pronunciation_overrides={"gaol":"jail"} applies a dict globally. sb.split_sentences(text) exposes the sentence splitter.

Client API — admin

Benchmarking exact hardware

A testing feature, switched off by default. It pins a request to one specific GPU so the same work can be measured on different hardware.

url = sb.request("qwen3-4b-q4", requirements={
    "gpu": "ai-9:gpu1",                # exact card: <host>:<placement>
    "bench_code": "bench-...",         # ask the operator
})
print(sb.last_placement)               # confirm what actually served it

One field, not two. Neither half identifies hardware alone — a host does not say which GPU, and an index does not say which machine:

A bare host ("ai-9") or bare index ("gpu1") returns 400 rather than being quietly ignored — a mistyped benchmark that runs on the wrong hardware is worse than one that fails.

This is not a scheduling control. A pin removes the broker's ability to route around a busy or failing card, which is the entire reason the scheduler exists. Used for normal traffic it would undo load spreading, the warm-host preference and every anti-thrash guarantee. Ask for a code, run the comparison, expect it to be revoked.

Without a valid code the broker returns 403. The gate fails closed: no codes issued — the normal state — means nobody can pin, and an empty or corrupt codes file means the same. Accepted pins log as BENCH hardware pin from <requester>, refusals as REJECTED hardware pin.

gpu_index and placement="gpuN" still work but need the same code, because they are hardware targeting by another name. Semantic placements need no code — "cpu", "all", "single" and "mixed" name a shape rather than a card and mean the same thing on every host.

simulate_load rides the same gate and answers a different question: not "how fast is this card" but "where would this land if the fleet were busy" — requirements={"simulate_load": {"ai-150": 5}, "bench_code": "..."}.

How placement is decided

When a queued request has no running worker, the broker picks an owning host by sorting candidates on this key — lowest wins:

(evict, warmth, taken, in_flight, strength, host)

Busy workers (2026-09-08). A service whose /health JSON says "busy": true is skipped by dispatch, breaks the continuity rule (a host whose only workers for the capability are busy does not claim the request), and counts as demand for idle-unload, eviction and the autoscaler's drain. This is how a second lora-train-sdxl request lands on another card while the first is training instead of queuing behind it: in-flight accounting only sees the millisecond POST /train, not the ten-minute job. Any service can opt in by reporting busy.

Among already-running replicas, pick_worker_for sorts by (worker_load, strength, host, service).

Aging. A queued request's weight is priority + age_factor × wait_seconds (age_factor = 1.0), so waiting work climbs and cannot starve forever. Standing (session) demand does not age — it is steady pull at its priority, not a starvation climb.

Swapping. A host only swaps to a new config when the aged weight of the work the new config serves exceeds what's currently served plus a swap_penalty of 30.

Warmth, TTLs & eviction

Autoscaling

The broker horizontally scales replicable services under sustained load. On by default; opt out per-artifact with "autoscale": false.

Conservative by design: only a real, persistent backlog on free hardware triggers a replica, and autoscaling never evicts live work.

Orchestrators

Two kinds of thing answer a request on this fleet:

service        input -> output
orchestrator   input -> internal review -> [regeneration] -> output

A service is one model or one ComfyUI lane on one GPU, launched by a host's agent: it returns what it produced, and whether that was any good is the caller's problem. An orchestrator owns the outcome: it requests services through the broker in an order it knows, reviews what comes back — by measurement, by a vision model, by a decision model where the call is a judgement — regenerates what fails, records why, and returns a result the caller can use without looking. The arrow shape is the simple case, not a rule: a real orchestrator reviews at every stage, regenerates the source when a defect is inherited, and asks the fleet to decide when the rule has exceptions.

Orchestrators run on the broker box, hold no GPU, and are broker clients like any project. They are catalog artifacts of kind: "orchestrator", so a client finds them the way it finds a model — sb.request("character-orchestrator") returns the endpoint — but they carry an endpoint, not run-profiles: no agent launches them. Code: orchestrators/.

character-orchestrator — a visual-novel character, background and shot from one request (client: renai)

Base http://192.168.1.13:8092, LAN only, JSON over HTTP, CORS open; every route below is on it, and every job is asynchronous (a POST returns {job, state}, GET /jobs/<job> follows it whatever kind of work it is, and GET /characters/<job> answers the same). Three things it makes:

• Characters. POST /characters with an identity line, a required height (5'2" or 157 cm), a wardrobe, poses and expressions, and optionally an engine profile. Her first render is her reference sheet, made from those words alone - a new character has no images to reference: front, back and side views plus two head panels, composed on a ruled height backdrop with every full-body view scaled so the top of her head sits at her stated height. The sheet is her root. base.png is its front view with her default outfit put on it, and every later frame is an edit of that, so her proportions and her face are settled before any clothes exist. The sheet's body is a featureless figure-drawing mannequin only when the spec declares "adult": true; every mannequin view is asked whether it shows explicit detail or reads as a child, and either answer refuses it. Output is a directory of frames plus character.json: flats, body-plus-face layers, camera-shot rectangles, illuminant multipliers, her sheet, her painted references, and every decision with its reason. GET /characters/<id>/blocks is the character as one block per stage, the sheet leading, each with its images, the gates' answers and what is missing; POST /characters/<id>/redo re-runs one block, POST /characters/<id>/add adds one piece to her spec and renders only that.
• Backgrounds. POST /backgrounds renders a place with nobody in it, at 1824×1248 so a crop still holds its detail, and measures its ground: each spot the request names is probed by adding one adult of the standard height inside a band mask there, measuring the figure the model draws and discarding the probe — twice, on two seeds, and the spot is believed only when the two agree (see measuring a place's ground); two or more spots also give a ground-plane equation, so a character can be placed between them. A posture is measured the same way, once per spot and pose, and cached on the background. A place that already exists is asked for more without a prompt: meant records what it should have, make builds the weathers, lightings and dressings it lacks, and spots finds and measures a spot again. GET /backgrounds, GET /backgrounds/<id>, and GET /backgrounds/<id>/blocks — the place as one block per stage (place, ground, weathers, lightings, variations, shots; atmospheres is kept beside lightings as its old name for a release), each carrying what exists, what was rejected, what the place is meant to have, what is missing and what is available, so a client shows a gap instead of inferring it from file names.
• Shots. POST /shots is a picture of a background with people in it: a cast is placed spot by spot, far to near, each character scaled so that she is her stated height there, injected inside her own silhouette with room for a contact shadow, composited onto the background and relit, then measured from the spot's ground row to the top of her head. A shot may name an atmosphere — a weather (rain), a lighting (dusk) or both (rain@dusk) — and any number of dressings (all of them links built on the background, made once and reused) and a camera — wide, waist or close, a crop of the finished picture around whoever it is on. Every framing is cut from that one picture and listed on the manifest as framings; camera only says which one the shot leads with. The same cast shot again in the same place supersedes the earlier shot — the old picture moves to evidence/ with a version record and the manifest's version counts up — while a shot given an explicit id is a shot of its own. GET /shots, GET /shots/<id>, GET /shots?background=<id>.

Its library of poses, expressions, props, hair styles, weathers, lightings, blink frames and dressings is versioned data a client grows over /library/<kind>: an edit makes a new version, old versions are kept, and every job records the versions it rendered with. Soft dressing is fleet-rendered and client-chosen: things that touch the place (birds, litter, a parked car, figures in the distance) are baked as variants of the background, and things that do not (a pigeon, a cat, a cup) are rendered as transparent cutouts the client places at runtime (POST /dressings/layers, /files/dressings/). /files/ serves every output read-only and POST /verdicts writes a person's judgement into the corpus the vision gate is scored against. Every service can also be driven by a plain-language conversation (/talk): a local LLM collects the values, relative words move a number by its field's step, exact values the person gives are kept, and nothing runs until POST /talk/<conversation>/run; the contract is published at GET /talk/services. A place is the root of a chain of links, one kind of operation each: a weather (<id>~rain.png) adds material over the whole frame and is made by the unmasked edit lane at low denoise, because a ControlNet holds edges and rain has none in a dry place; a dressing (+birds) adds a thing inside a region found from its own words; a lighting (@dusk) changes how the surfaces already there look, held in place by the picture's own edges. Each link names its parent and is cached, so one weather serves every lighting built over it, and weather comes before lighting because an overcast wet park lit at dusk is not a dry park lit at dusk with rain added afterwards. How much of the place each link held is measured and kept on its version, and a shot taken in a weather that moved the ground says so. The older scene path — plates with placeholders painted into them — was removed on 2026-09-11; a place is a background and a picture with people in it is a shot. A character's blocks say what each stage is meant to hold and what is missing, hair is a stage of its own, and POST /characters/<id>/add adds one piece to her spec and renders only that. It renders through comfyui-sprite-base-flat, comfyui-sprite-qwen-edit and -ref, and comfyui-scene-plate / -inject / -inject-masked / -edit / -edit-masked / -relight; a vision model gates every stage (CHAR_VLM, default darkidol-qwen3.8-27b-q4) and a director model makes judgement calls (CHAR_DIRECTOR). Contract and fields: orchestrators/character/README.md. The vision gate's accuracy is measured against a labelled corpus (/mnt/ai-data/renai/labels/, tools/vlm_labels.py rescore) and a different vision model is adopted only if it scores better there.

A place and the links built on it

A base is rendered in flat overcast light, because every link is built on it and a lighting pass only partly redraws shadows — whatever light a base is born with is nearly permanent, and 0.69 to 0.89 of one place's midday shadow pattern survived into every hour of it. The flat light is asked for positively and the things that would spoil it are forbidden in the negative prompt: written as “no cast shadows” in the positive prompt it was ignored on every seed, moved to the negative it held on every seed, and saying nothing about light at all was no better than asking wrongly, because the model then supplies its own idea of the subject. Neither names a fixture — whether a light is switched on belongs to the hour, but the engine cannot know whether a place has a street lamp, a neon sign or a screen, so both say “artificial light”.

The plate is then asked whether that light arrived: is direct sunlight visible (hard-edged shadows, sunlit patches), and is any artificial light source switched on. A base that fails is re-rolled, because it is the root everything inherits. A base's light can also be redone without redrawing the place — POST /backgrounds {"remake": ["light"]} relights it flat while a canny map of its own edges holds the geometry, so the ground, the spots and their probes stay valid. It cannot rescue a base whose shadows are already hard: a hard shadow is an edge, so the lock that holds the geometry holds the shadows too.

Which lane draws a weather is a property of the library entry (meta.lane), not of the kind. Weather that is only the state of the sky — clear, cloudy, overcast — is drawn by the locked relight lane, because a flat sky has almost no edges for a ControlNet to hold and the lane may repaint it while every fixture stays put. Weather that puts material on the surfaces of the place — rain, snow, fog — is drawn by the unmasked edit lane at low denoise, which is the dial between “holds the place” and “free to change it”.

Every link is measured and gated. structure_kept records how much of the parent survived and goes on the version; a lighting is held to 0.80 and rolled again — loosening the ControlNet each time — before the light-field fallback is tried at all. The weather gate is asked of the pair, the place and the weather side by side: does the right picture fail to show the change, and is anything in it drawn wrong. That shape matters — measured against the label corpus, the gates that work on this service are numbers and side-by-side pairs, while single-picture yes/no questions miss most of what a person calls bad.

What that leaves, stated plainly because a client should not discover it by asking: a lighting must not claim what its weather forbids, so @noon (“shadows short and directly beneath things”) cannot be satisfied under overcast or rain, which say nothing casts a hard shadow. snow falls but does not lie. cloudy and fog are refused more often than not. A refused link is simply absent, and the block's missing says so.

Talk — driving any service in plain language

Every service above can be driven by a conversation instead of a hand-built request body. A local LLM on the fleet reads the message against the service's field list and the target's current values and names the operations it collected; the orchestrator computes every value from those operations, and nothing runs until the run call. No Claude is involved at any point.

The contract is machine-readable and needs no POST to discover: GET /talk/services returns every service, each one's fields and target_keys, and a contract object giving the exact request and reply shape. It is a GET, so a read-only client can fetch it and build against it rather than guessing. The shape is repeated here so it is greppable, but GET /talk/services is the authority — if the two ever disagree, the endpoint is right.

GET  /talk/services              every service, its fields, its target_keys, and `contract`
POST /talk                       a message; returns the conversation and its reply
GET  /talk/<conversation>        transcript, current values, run and supersedes (survives a refresh)
POST /talk/<conversation>/run    202 {job, state, supersedes, calls}
GET  /jobs/<job>                 follow the work the run started

POST /talk — request body:

{"service":      "a name from GET /talk/services",
 "target":       {...},          the keys in that service's target_keys
                                 background: {background, stage}
                                 shot:       {background, shot, stage}
                                 character:  {character, block, names, add}
 "conversation": null,           null on the first message, then the id from the reply
 "message":      "what the person wrote",
 "pinned":       {field: value}} sent on every message and on run; the model never changes one

POST /talk — reply (GET /talk/<conversation> returns the same, minus reply):

{"conversation": "c-a7b4d9cb",
 "reply":        "one or two sentences, or a question",
 "fields":       {field: {value, source, said, was, clamped}},
                                 source is said | relative | current | default
                                 `was` and `said` are what it changed and why
 "missing":      [],             required fields with no value yet
 "ready":        true,           nothing missing and nothing asked
 "run":          [{"method": "POST", "path": "/backgrounds", "body": {...}}],
                                 the exact calls the run will make
 "supersedes":   ["a.png", ...], pictures that become earlier versions when it runs
 "messages":     [...],          the transcript
 "pinned":       {...}, "runs": [...]}

run and supersedes are what a client shows beside the button: this is what will happen, and this is what it will set aside. Both are on the GET as well as on the turn that produced them, so a page refresh does not lose them. A conversation is continued by posting again with the same conversation id.

Worked example — changing the words of a background that already exists:

POST /talk
{"service": "background.make", "target": {"background": "park-bench-bg"},
 "message": "make the path narrower and put a low stone wall along the left side"}

-> {"conversation": "c-a7b4d9cb",
    "reply": "prompt changed (make the path narrower ...). This sets aside 18 pictures -
              park-bench-bg.png, park-bench-bg@dawn.png, ... and 12 more; they move to
              earlier versions. I narrowed the path and added a low stone wall ...",
    "fields": {"prompt": {"value": "<the new words>", "source": "relative",
                          "said": "make the path narrower ...", "was": "<the old words>"},
               "spots":  {"value": {...}, "source": "current"}},
    "run": [{"method": "POST", "path": "/backgrounds",
             "body": {"id": "park-bench-bg", "prompt": "...", "remake": ["place"], ...}}],
    "supersedes": ["park-bench-bg.png", "park-bench-bg@dawn.png", ...],
    "ready": true, "missing": []}

Note the remake: a place that already exists is not redrawn by a change of words unless the call says so, so a conversation that edits its prompt asks for the redraw — and a redraw takes the ground, the spots and every weather, lighting, dressing and shot built on those pixels, which is what supersedes is naming before you commit.

The run body carries keys the field list does not publish, because it is the call as the orchestrator will make it rather than a list of collected values. GET /talk/services returns these under contract.reply.run_body_keys; they are repeated here so a grep finds them:

remake   what must be built again although it already exists.
         THIS IS WHAT MAKES A RUN DESTRUCTIVE.
         ["place"] redraws a background, and its ground, its spots and every
                   weather, lighting, dressing and shot go with it
         ["light"] redoes only its light and keeps the ground
         absent    the call can only ADD; it cannot replace anything
make     the links to build on a place that exists: weathers, lightings,
         dressings, and `on` for the node to build them over
meant    what the place is declared to have, so a client can show what is missing

A client that can read remake can tell a draft from a replacement without guessing, and supersedes then names exactly what the replacement will set aside — computed from the id the run will WRITE to, so pinning a new id supersedes nothing.

Words to values: a stated number is set as stated; slightly moves a numeric field by one of its steps, more or less by two, a lot by four; double and half scale it; as many as you can and the least take its max and min; put it back takes the previous value. Everything clamps to the field's range, and a clamped value says clamped: true.

Errors: 404 unknown service or conversation; 409 a run with values still missing; a message the model cannot place returns a question in reply with ready: false rather than a guess.

Drafts — working beside a thing, then taking its place

A conversation can build beside the thing it is about instead of replacing it, so a person can look at the result before anything downstream is touched. A draft is not a new kind of file: it is another object of the same kind with a derived id — park-bench-bg-draft for park-bench-bg — which is what makes it non-destructive by construction. It writes somewhere else, so nothing built on the original can be touched, and it is a first-class object while it exists: its own blocks, its own /files/ urls, its own measured ground. That last point is the reason for the design rather than a candidate slot: whether a new place is usable at all is decided by its ground, and a ground is invisible in a picture.

POST /talk/<conversation>/run  {"draft": true}
  -> {"job": "...", "state": "queued", "supersedes": [],
      "draft": {"id":      "park-bench-bg-draft",
                "kind":    "background",
                "of":      "park-bench-bg",
                "blocks":  "/backgrounds/park-bench-bg-draft/blocks",
                "files":   "/files/backgrounds/",
                "promote": "/backgrounds/park-bench-bg/promote"}}

Read the draft object rather than deriving the id: the naming rule may change, and a client that derives it would break silently. A draft exists only once its job succeeds — a build that fails puts its leavings in evidence, so nothing is left in the working directory claiming to be an object that GET would answer 404 for.

Aiming at the parent or at the draft means different things, and both are intended. A run aimed at <id> builds from the original plus the conversation's words. A run aimed at <id>-draft builds from the draft, and that is how you iterate: each rebuild leaves a version on the draft carrying the instruction that caused it, the seed, the job and the conversation, plus the picture it replaced — so the edits behind a draft can be read back and an earlier one returned to. The history lives here, not in a browser.

POST /<backgrounds|shots|characters|dressings>/<id>/promote
     {"from": "<draft id>", "force": false}
  -> {"promoted": "park-bench-bg", "from": "park-bench-bg-draft",
      "kind": "background", "log": [...]}

Promote is the only moment anything is superseded. Everything the target had is set aside — it was drawn from pixels that are being replaced — and nothing is rebuilt: a link is made again when something next asks for it. The draft's own records come with it, because its ground was measured in ITS pixels and they are the only ones true of the picture being promoted. A draft that is not fit to take the thing's place is refused with the reason (a place whose spots never measured cannot be stood in), and force: true overrides that.

Nothing here deletes: a superseded picture is renamed, recorded in the version ledger, and moved to the evidence tree, where it is still served at /files/evidence/.... That is what makes a promote reversible and an earlier version openable.

Seasons — a place across the year

A location is drawn once, on its own lane (Flux), from the words a person wrote plus the profile's look and a flat overcast light. Everything after that is a modifier put onto that picture by the image editor, which holds the place: the same furniture, the same camera. Seasons, weathers and hours all go the same way, so a season of a location is that location, not another drawing of a similar one.

Each modifier carries a strength in the library — how far it may repaint what it is put on. A season changes what lies on a place and must not rebuild it (0.8); a weather or an hour changes how the whole picture reads (1.0). Two tiers are available: a draft (4 steps) picks a seed, and a final (20 steps) is what gets judged — a draft predicts the layout at its seed, not the look.

A location knows whether it is a room, asked of its picture twice in different words and kept on its record. In a room a modifier edits only the view through the windows and the light coming in: applied whole, winter replaced a cafe's street with open fields and rain put puddles on a classroom floor. An interior is drawn with its view through the windows from the start, so every node of it shows the same street. No modifier may add trees, branches or poles, and none may add fog or haze unless its words ask for it.

The chain as a graph

A place is a graph, and the fleet answers in nodes. A client must never work the tree out by parsing ~ @ + !: the naming rule belongs to the fleet and has already changed once.

GET /backgrounds/<id>/graph        -> {background, nodes, edges}
GET /backgrounds/graph/blank       ?concept=<words> | ?conversation=<c>
GET /talk/new                      every kind that can be made from nothing
GET /backgrounds/graph/contract    the machine-readable shape

Each node carries id, kind (place · ground · season · weather · lighting · dressing · attempt), label, state, picture (a /files/ url or null), parent, why when it failed, facts, the talk service and target that define it, talking (conversations aimed at it), and branches — what can still be made FROM it.

The five states have real sources: empty from disk, defined from a talk conversation that is ready and unrun, running from the job queue, built from a picture, failed with a reason. Four of the five have no picture, and "never asked for" is not "tried and failed". ground never has a picture — it carries its numbers, because it is the link that fails most and the only failure invisible in a picture.

A failure is a branch, not a stop. Every rejected try is kept as a node of kind attempt with its picture and its reason, so the canvas shows what was tried and why it died. A reason is classified — a measurement turns a dial, prose about the picture rewords — and is verified before it changes anything: the model's claim goes back as a factual question, because its judgements fabricate while its facts hold. A reword puts the fault in the NEGATIVE prompt and a restatement of what should happen in the positive, never the fault itself.

Taking a thing back out

An object is retired with a DELETE on its own url. Everything it owns — its picture, its record, its weather and lighting links, its version ledger — moves to the evidence tree under a .retired-<stamp> name, the id answers 404 and it leaves the listings. This is the same thing promote does to what a draft replaces, so nothing is deleted and a retirement can be undone by hand from evidence.

DELETE /<backgrounds|shots|characters|dressings>/<id>     {"force": false}
  -> {"retired": "talk-test-bench", "kind": "background", "files": 6, "log": [...]}

A background refuses while a shot stands in it — a shot names its place in its own id and could not be built again once the place is gone — and says which shots; force: true overrides that. DELETE /characters/<job id> still cancels a queued job, and a job id is matched first, so the two cannot collide.

How failure teaches a modifier

A modifier — a season, weather, lighting or dressing — is drawn from its words in the library, and those words are what a failure changes. A lesson is written as a new version of the modifier, so every later build inherits it and one call rolls it back. Three things teach:

• a bad verdict with a reason on a picture (POST /verdicts, or the verdict talk service) — checked in the background, once per modifier however many of its pictures were named;
• a season build whose attempt fails with a confirmed fault — once per build;
• a weather or lighting that fails every rung of its control ladder, from its last try's reason.

A lesson goes only to the modifier that made the picture: the last mark in its id (p!spring~snow@dusk was made by dusk, from p!spring~snow). The fault must be confirmed in the picture and absent from the picture it was drawn from — litter a location already has is not spring's doing. A location's own plate was made by no modifier and teaches nothing. The rewrite may not introduce a person, a species, or anything belonging to another layer — a season may not name the sky or the hour, a weather may not name the hour, a lighting may name both — and a rewrite that does is asked once more with those words named, then refused. When the words could not have caused the fault, nothing is written.

POST /verdicts   {"verdict": "bad", "images": ["/files/backgrounds/park-bench-bg!spring.png"],
                  "why": "petals on the ground and no tree that could shed them"}
  -> {"judged": 1, "no_row": [], "lessons": {"checking": ["park-bench-bg!spring.png"], "see": "GET /lessons"}}

GET /lessons
  -> {"lessons": [{"modifier": "spring", "kind": "seasons", "taught": true, "version": 3,
                   "words": "in spring: new pale-green leaves only half out, ...",
                   "fault": "...", "picture": "park-bench-bg!spring.png", "by": "operator", "at": "..."}]}

A location's own plate is made by no modifier, so a fault on it teaches the location's words, versioned in its record (prompt_version, prompt_history). The rewrite may not name a weather, the sky's condition, the hour or a person, has any season taken out, and must keep every phrase of the old words the fault was not about. A lesson on a draft redraws it once; a live location is never replaced by a lesson — promote decides that. Slop claims do not teach: checked against the picture they come back reversed.

A row with taught: false carries why: not confirmed in the picture, already in the parent, unrelated to the words, or refused for naming what the modifier may not.

Measuring a place's ground

A spot is declared in words, found in the picture with a labelled grid — six columns by four rows, then its row asked again of a full-width strip around that row under four finer rows, because the model reads columns well and rows poorly — and then rolled until two measurements agree: a person of the standard height is drawn standing there and measured, and the spot is believed only once two rolls land within 25% of each other. Up to four rolls are spent and the largest agreeing set wins, so an ordinary spot costs two renders and an unlucky one costs three.

Both halves of that are what the numbers required. One roll is not a measurement: on the same picture, same cell, different seed, one spot came back 50 px and then 245 px, and another 385 px then 147 px. Both numbers are plausible alone, so no floor and no plausibility threshold can separate them, and the small one was cached as that place's ground — a character placed there would have been drawn about a third the height of the bench. But two rolls are not enough either: rolled 14 times a good spot ran 552–728 px and another, 8 times, 348–578 px, so two rolls drawn at random disagree by more than 25% in 18% and 25% of pairs. Demanding that the first two agree refused about one build in five of places that measure perfectly well. Across every possible draw from those rolls — 364 and 56 of them — a third roll settled it every time.

A spot that still has no two rolls in agreement fails, naming every figure it drew, rather than caching one of them. A place whose spots cannot be measured refuses to calibrate instead of quietly calibrating wrongly. The labelled-grid locator is not the source of the variance: ten locates of the same spot on the same picture landed in the same cell ten times out of ten.

A limit worth knowing. The probe does not check that the place has ground in it: a close-up of a bench against a wall, with no floor in frame and nowhere a 170 cm person could stand, still measured repeatably — 14 rolls inside 1.32×. On a place that does have ground the figure's size is the place's and not the mask's (bands of 436, 686 and 936 px gave figures of 475, 494 and 489 px at the same foot row), but nothing yet tells the two cases apart.

What it knows so a caller need not: the edit order (dress, pose, prop, hair, expression); that a base is never accepted without passing the gates; that expressions are consistent across a wardrobe because each later one is rendered with a head crop of the first as reference; that a spot in a place is declared in words and then found in the picture, and carries the pose those words ask for, so nobody stands in the middle of a path at a bench's row; that a shot's pose is rendered as the character's own sprite first, so her proportions are hers — except for a posture the sprite pipeline draws badly, where the place's own measured probe figure stands in and she is painted over it; that a character drawn into a place is drawn from a painted edit of her sprite — the sprite's details in the scene's style — and composited by her own silhouette onto the background, so every character in a place shares its pixels exactly; that scale is measured rather than declared, in the picture where the model has perspective, and never asked of a client; that her pose and expression are judged against her placeholder and her sprite's face on side-by-side pairs, and an expression that drifts is repaired by a masked edit of her face; that a figure is relit to the place's own light and the remaining gap is gated against how mixed that light is (a lamp beside a night sky allows more than a flat afternoon); that an object an atmosphere or a dressing introduces is judged by the director model — a soft element is accepted, a hard fixture that does not belong is re-rolled; and that an expression pass fills the face out 5–13% in mean width, which no setting changes.

Broker HTTP API

Base: http://192.168.1.13:8080. Use the client unless you have a reason not to.

Data plane

Sessions

Introspection & control

Clients advertise their version via X-AI-Client; below the floor the broker answers 426.

Host telemetry

Per-host fields on GET /status (and, where noted, GET /fleet). All are nulled when a host's agent is down, so a dead box never reports frozen numbers as current.

Manifest schema

Fleet catalog: ~/ai/manifests/artifacts.json — the source of truth, projected into per-host configs. Host budgets: ~/ai/manifests/hosts/<host>.json. The broker hot-reloads on mtime change; no restart needed.

Artifact

ComfyUI workflow artifacts

A comfyui-workflow artifact stores an API-format graph plus a params map of {name: {node, input, default?, required?}}. Clients call POST /workflow/{aid}/render to get the resolved graph, request the artifact to obtain a ComfyUI proxy URL, then POST the graph to {proxy}/prompt. Each param targets exactly one node input, so a value used by several nodes is fixed in the stored graph. requires_nodes / requires_files gate the artifact to hosts that actually carry them — a wrong path silently de-lists the whole workflow.

Two traps. The RemBG nodes are installed but have no onnxruntime backend; the node blocks fetching u2net with no outbound DNS and wedges ComfyUI's single-threaded executor, after which every job queues forever and looks like a hung sampler — recover with POST /restart. No lane uses those nodes. And /object_info stalls the server (a custom node calls the HuggingFace API inside that handler); query /object_info/{NodeName} instead.

LoRA training (lora-train-sdxl)

A trainer-kind artifact: an HTTP sidecar on ai-150's 5090 that trains an SDXL LoRA (kohya format, rank 16 on the UNet attention projections) from images a client sends inline, and writes the file straight into ComfyUI's models/loras. Request it like any artifact and talk to the proxy URL:

url = sb.request("lora-train-sdxl", environment="development", urgency="background", keep_alive=120)
r = httpx.post(f"{url}/train", json={
    "name": "renai_alice",              # -> models/loras/renai_alice.safetensors
    "trigger": "alice_rn",              # token prepended to every caption; use it in prompts
    "images": [{"b64": "...", "caption": "1girl, solo, brown hair, twintails, school uniform"}, ...],
    "steps": 800, "rank": 16, "resolution": 1024, "lr": 2e-4,   # all optional (defaults shown)
    "text_encoder_lr": 1e-4,             # LoRA on both CLIP text encoders too (0 = UNet only)
    "subject": "character",              # or "scene": lighting variance is then reported, not dropped
    "identity_tags": "brown hair, twintails, beige cardigan, ...",   # stored in the LoRA, served by GET /loras meta
    "palette_lock": True, "canonical": 0,  # match every image's foreground tint/lighting to the first (reference) image
}, timeout=120).json()                   # {"job_id", "lora_name", "state", "queue_position"}
job = httpx.get(f"{url}/jobs/{r['job_id']}").json()   # state queued|running|done|failed, step/total, loss, eta_sec
loras = httpx.get(f"{broker}/loras").json()           # {"hosts", "ready_everywhere", "meta": {name: {trigger, identity_tags, subject}}}

Every submission is scored by a two-pass dataset check: an instant histogram pass (foreground, haze, colour signature against the set, brightness, warmth) and, before training, the fleet's own vision lane describing each image in a fixed schema and comparing it with the set's majority (style, hair, outfit; for scenes room type and lighting), so a rejected image comes with a reason in words. POST /dataset-check on the trainer runs both passes without training; outliers are dropped for subject: character and reported for subject: scene. Prompt the LoRA as trigger + identity_tags from /loras meta; never add colour words for other items that can bleed ("red hair ribbon" turned brown hair crimson). Sprite lanes declare an output_check and the client re-rolls the seed when LayerDiffuse returns an opaque frame. Location LoRAs: caption lighting and time of day so the trigger owns the room, not the light. A single base render amplified through the reference lane yields a style of room, not one room (operator-verified 2026-09-08): exact-room fidelity across angles needs a multi-view source — 6-10 views of the same room supplied by the client (its own scene blockout, or repeated inpainting from one view) — trained with subject: scene. comfyui-anime-view is the fleet-side view amplifier for building that set from one base render.

Three placements: ai-150's 5090 (24,000 MiB) and both 3090s (20,000 MiB each, roughly half the step rate; ai-150's second card listens on :11521), so three jobs train in parallel and a game's 7-10 LoRAs (characters plus locations) batch through in an hour or two. Each trainer runs its own jobs serially. Requesting one evicts whatever large LLM holds that card (the roleplay default relocates) for the job's duration. Keep keep_alive short: a running job protects itself (the trainer reports busy, and a busy worker is never unloaded, evicted or drained), while a long keep_alive only pins an idle trainer to the card afterwards — measured today: a 40-minute keep_alive held a finished trainer on a 3090 that a client's model then could not use. When the job is done the LoRA exists on ai-150; the broker box's ai-lora-mirror service copies new *.safetensors between the hosts' models/loras every minute, and GET /loras lists it under ready_everywhere once every ComfyUI host has it — from then on comfyui-anime-lora and comfyui-anime-lora-ref accept it as lora_name on any host. Locations train on the same pipeline (one base render of the room, amplified into angles through comfyui-anime-ref, then trained). For scenery keep the adapter light: weight 0.3 with weight_type "prompt is more important" (or 0.15-0.2 standard) — the lane's character default of 0.6 burns rooms into neon oversaturation, and a LoRA trained on that set learns the burn. Namespace the names, e.g. renai-<game>-char-<id> and renai-<game>-loc-<id> (letters, digits, - and _ survive sanitising; anything else becomes _). Caption only what varies between images (expression, pose, background), never the identity traits the trigger should own — fully described images leave nothing for the trigger to learn (measured: same 800 steps, wrong girl vs. the right one). Triggers must be non-words (rnchr_a1x, not alice). Its ready probe (POST /selftest) runs real kernels on the GPU in the training dtype (bf16, or fp16 with a grad scaler where bf16 GEMMs fail, as on ai-151), so a broken torch build fails the probe rather than the first job. Sidecar source: sidecars/lora_train/.

Audio generation (music & sound effects)

Distinct from speech. TTS/STT are their own artifacts (kokoro, faster-whisper); generated music and sound effects run as ComfyUI workflows on the burst tier. Two local paths, installed on ai-150 and ai-151 and generation-tested on both, and a third foley model registered to compare against the first:

Request them by these artifact ids:

import ai_fleet_client as sb, httpx

# 1. resolve the stored graph with your params
g = httpx.post("http://192.168.1.13:8080/workflow/comfyui-foley-stableaudio/render",
               json={"params": {"prompt": "birds chirping at dawn, wind in leaves",
                                "seconds": 15.0, "seconds_total": 15.0}}).json()["graph"]

# 2. get a ComfyUI proxy URL — background so it queues behind live work
url = sb.request("comfyui-foley-stableaudio", environment="production",
                 urgency="background", keep_alive=600)

# 3. submit, then poll /history/{id} and fetch the file from /view
pid = httpx.post(f"{url}/prompt", json={"prompt": g}).json()["prompt_id"]

VRAM figures are measured, not estimated. At ~3–5 GiB the models are not the placement constraint — the comfyui-engine profile is. Engine profiles exist only on ai-150 gpu0/gpu1 and ai-151 gpu0. ai-9 has ComfyUI and an ACE-Step checkpoint but no engine profile, so it cannot serve audio until one is added, and its GTX 1080 Ti (Pascal) will be materially slower at diffusion. ai-151 gpu1 runs the speech stack (~7500 of 10000 MiB) and also has no engine profile. ai-151 additionally carries ace_step_1.5_turbo_aio, which ai-150 does not.

Choose by duration, not by type. Stable Audio for short one-shots; ACE-Step for long beds. Nodes are native — CheckpointLoaderSimple + CLIPLoader{type: stable_audio} + ConditioningStableAudio + EmptyLatentAudio + VAEDecodeAudio. Always give Stable Audio a negative prompt of music, melody, singing, instruments, speech or it drifts into music.

Run-profile

Operations

Troubleshooting

Requests time out repeatedly

Almost always model thrash: two models that can't co-reside are being requested alternately, and each turn pays a 15–60 s swap against a client timeout that's too short.

• Check sb.status()["sessions"] — if 0, the app isn't using sessions.
• Verify the working set can co-reside (capacity tables). Push small models to placement="cpu".
• Raise timeout_sec above cold-load time — a 32B VLM takes 53 s+; 60 s is too tight.

"Upstream unreachable" from a ready ticket

The proxy URL outlived the model (15 min TTL vs 30 s keep_alive). Re-request, and hold the model with a session or a longer keep_alive.

Ticket says ready but the proxy returns 410

A ticket whose proxy TTL has lapsed reports failed with proxy ticket expired; request a new ticket, and the proxy route rejects it with 410 ticket expired. Treat failed as "request again", not as a fatal error.

The broker stops scheduling entirely (reconciler wedged)

Check broker.log for reconciler heartbeat stale ... — restarting repeating with a non-advancing tick number. That means a tick is blocking longer than the 30 s watchdog, so the reconciler is killed and restarted before it ever completes. A mounted-share walk on the reconciler thread wedges it; if you add work to a tick, never let it touch a mount synchronously — a degraded share can block for over a minute and turn a slow tick into a permanent restart loop.

Everything queues and nothing dispatches

Check for a stray pin (sb.status() → pinned) and the global enable flag. A pinned host serves only its pinned config.

A model won't load

Check the per-service log on the host share. After 2 consecutive failures to reach healthy, the circuit breaker benches it for 10 minutes. Common causes: OOM (VRAM under-declared), missing model file, bad split.

Silently smaller context than expected

You landed on a profile with a smaller tested window. Pass requirements={"min_context": N} to make it fail loudly instead.

History

This reference describes the fleet as it is. Every change, with its measurements and reasons, is in CHANGELOG.md in the repository.