Generate

When to use generate#

Choice, noul and score pick between answers you wrote in advance. Some actions need text you could not have listed ahead of time: a name read off a document, a value for a form field, a short reply. Generate writes that text from the same kind of state.

If the answer is one of a known set, use a decision primitive instead. You get probabilities you can threshold, which generate does not give you. Generate is for the part that really has to be written.

Request#

Generate has its own endpoint, POST /v1/generate, and asks for one piece of text per call. The full reference is in Generate: POST /v1/generate.

Example#

An accounts-payable team receives invoices as emails and PDFs, and needs the same four fields from each one for its payment system. The fields are always the same, but the values are different on every invoice, so they can't be listed as options. That is a job for generate with a shape.

• state is the invoice text. It could equally be text taken from a PDF, or an image of the scan.
• instructions says what to do in a few words. The shape carries the detail.
• shape is the JSON Schema the payment system expects. pattern pins the invoice id to INV- plus four digits and the date to ISO format. total_eur is a number, not a string with a currency sign. required means no field can be skipped.
• max_tokens caps the output. 120 is plenty for four short fields.

{
  "state": "Invoice INV-2291 from Nordic Steel AB, total EUR 18,440.00, due 2026-10-30.",
  "instructions": "Extract the invoice fields.",
  "shape": {
    "type": "object",
    "properties": {
      "invoice_id": { "type": "string", "pattern": "^INV-[0-9]{4}$" },
      "vendor":     { "type": "string" },
      "total_eur":  { "type": "number" },
      "due_date":   { "type": "string", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$" }
    },
    "required": ["invoice_id", "vendor", "total_eur", "due_date"]
  },
  "max_tokens": 120
}

Wity is held to the shape while it writes, so the output always parses and always matches the schema. There is no retry loop, and no "please answer in JSON" in the prompt. The response gives the output twice:

{
  "model": "wity-1",
  "text": "{\"invoice_id\": \"INV-2291\", \"vendor\": \"Nordic Steel AB\", \"total_eur\": 18440.0, \"due_date\": \"2026-10-30\"}",
  "value": { "invoice_id": "INV-2291", "vendor": "Nordic Steel AB", "total_eur": 18440.0, "due_date": "2026-10-30" },
  "finish_reason": "stop",
  "usage": { "input_tokens": 121, "output_tokens": 53 },
  "metadata": { "elapsed_ms": 1350.2 }
}

• value is the parsed object, ready to hand to the payment system. text is the same thing as a raw JSON string.
• finish_reason is stop when Wity finished on its own. It is length when it hit max_tokens. In that case value is null, and you should raise the limit or send the item to a person.
• usage.input_tokens is what you pay for. output_tokens is shown for reference and not billed.

Free text#

Leave out shape and you get plain text in text. This suits a message a person will read. Say how long it should be and who it is for. Wity writes only what you ask for, with no preamble such as "Sure, here's a reply".

Here a support agent's tool drafts the first line of a reply. The customer has said what they want, so the draft only needs to confirm it:

{ "state": "Customer: my order #4471 arrived with a cracked screen. I want a replacement, not a refund.",
  "instructions": "Write the one-sentence reply the agent sends to the customer.",
  "max_tokens": 80 }
 
→ "We're sorry your order #4471 arrived with a cracked screen, and we'll send you a replacement right away."

The reply names the order, acknowledges the problem and promises what the customer asked for, in one sentence as instructed. The agent reads it, edits it if needed, and sends it.

Decide, then generate#

In a real workflow, generate rarely runs alone. The usual pattern is to decide first with a choice or noul, then generate only when the decision calls for text, and check the text before acting on it. Here is the whole loop for the same accounts-payable team. Their inbox also gets questions like this one:

Answering takes one look at the ledger, and an assistant can do it, as long as the reply never states a date the ledger doesn't have. First, two small helpers, one for each endpoint:

import os, requests
 
API = "https://wity-proxy-production-2c33.up.railway.app"
H = {"Authorization": f"Bearer {os.environ['WITY_API_KEY']}"}
 
def decide(state, questions):
    r = requests.post(f"{API}/v1/systemone", headers=H, timeout=60,
                      json={"state": state, "questions": questions, "reasoning": "auto"})
    r.raise_for_status()
    return r.json()["answers"]
 
def generate(state, instructions, shape=None, max_tokens=128):
    body = {"state": state, "instructions": instructions, "max_tokens": max_tokens}
    if shape:
        body["shape"] = shape
    r = requests.post(f"{API}/v1/generate", headers=H, json=body, timeout=60)
    r.raise_for_status()
    return r.json()

1. Decide what the email is

The state holds the email and the relevant part of the ledger. A choice question sorts the email into one of the inbox's four kinds. Nothing is written yet: if this turned out to be a bank change, the flow would go somewhere else entirely (see the checklist on the Noul page).

state = {
    "email": email,
    "ledger": {
        "INV-2291": {"status": "approved", "scheduled_payment": "2026-11-04", "amount_eur": 18440.00},
        "INV-2310": {"status": "awaiting_approval", "scheduled_payment": None, "amount_eur": 2150.00},
    },
}
 
kind = decide(state, {"kind": {
    "type": "choice",
    "instructions": "What is this email?",
    "criteria": {
        "new_invoice": "Sends us an invoice to pay",
        "payment_query": "Asks when or whether an invoice we owe will be paid",
        "bank_change": "Asks us to pay to different bank details",
        "other": "Anything else",
    },
}})["kind"]
# kind["choice"] == "payment_query", probability 0.96

2. Generate the reply

It is a payment question, so now generate writes the answer from the same state. The instructions say to use only what is in the ledger. The shape asks for two things: the reply, and which invoice it is about. The second field is not for the supplier. It gives your code something to check the reply against.

reply = generate(
    state,
    "Write the reply to the supplier about the invoice they ask about. "
    "Use only the status and dates in the ledger. Two sentences, friendly, no sign-off.",
    shape={
        "type": "object",
        "properties": {
            "invoice_id": {"type": "string", "pattern": "^INV-[0-9]{4}$"},
            "reply":      {"type": "string", "maxLength": 400},
        },
        "required": ["invoice_id", "reply"],
    },
    max_tokens=200,
)

{
  "value": {
    "invoice_id": "INV-2291",
    "reply": "Hi Lars, INV-2291 has been approved and is scheduled for payment on 4 November 2026. Sorry for the wait, and thanks for checking in."
  },
  "finish_reason": "stop"
}

The reply takes the approval status and the scheduled date from the ledger and turns them into a plain sentence. It ignores INV-2310, which the supplier did not ask about.

3. Check, then act

Before anything is sent, the code checks three things. The decision was confident. The reply was not cut off. The invoice it talks about really exists in the ledger. If any check fails, a person takes over, so a wrong answer never goes out on its own.

if kind["probabilities"]["payment_query"] < 0.8:
    route_to_person()                                   # not sure what the email is
elif reply["finish_reason"] == "length":
    route_to_person()                                   # cut off: don't send half a reply
elif reply["value"]["invoice_id"] not in state["ledger"]:
    route_to_person()                                   # talks about an invoice we don't have
else:
    reply_to_sender(reply["value"]["reply"])

All three pass here, so Lars gets his answer in seconds. A reply about an invoice that doesn't exist would have gone to a person instead. That check is only possible because the output has a shape.

Shapes#

shape is a JSON Schema with "type": "object" at the top. These are the parts that do the most work:

• pattern pins a string to a regular expression: a 4-digit code, an invoice id, an ISO date. Use it for anything your code will parse or look up.
• enum limits a string to fixed values; maxLength caps its length. Use maxLength on any text a person will read.
• number, integer, boolean, nested objects and arrays all work, and required makes sure every field is present.
• Add a field your code can check, like invoice_id above, next to any text that will be sent or acted on.

Writing good instructions#

• Say what to write, how long, and for whom: "two sentences, to the supplier, no sign-off".
• Say where the facts must come from ("only the dates in the ledger"), and put those facts in the state.
• Let the shape carry the format. There is no need to describe JSON in the instructions.
• Set max_tokens with room to spare, and treat finish_reason: "length" as a failure, not an answer.

With an image

Add image (a data URL) to read text or details off a photo, scan or screenshot. The invoice example above works the same way on a photographed invoice, and pattern keeps each field in the format your system expects. See Images.

What generate is not#

• Not a decision: it has no probabilities. If the answer is one of a known set, use Choice, Noul or Score instead, and get a probability you can threshold.
• Not for long writing: output is capped at 512 tokens. It is meant for the text inside an action.