Tasks for a local LLM
A small service in front of the LLM running on my own GPU: other apps call a named task and get back a decision, a JSON document or text, without knowing about prompts or models.
Several of my apps need a bit of judgement from a language model: sort this, pull a date out of that. Rather than each one carrying its own prompt, they all go through this service, called camel. It sits in front of Ollama (the software that runs the model) on one GPU at home.
A task is a stored prompt plus the shape of the answer: a choice from a list, a JSON document that follows a schema, or free text. The model is held to that shape, so even a small model can't ramble, and the calling app always gets something it can read. The app only sends its data and a token. The prompt and the model stay in camel, and every change to a task is kept, so I can go back to an older version.
An AI assistant can also write and tune tasks through camel's MCP server (MCP is the protocol AI assistants use to call outside tools). It tries a task on real samples and adjusts it until the answers are right. I read the answers myself, not just whether they're well formed.