058362b87f
Initial skeleton for the deterministic compound-AI pipeline that turns `tolkien deploy <name>` into a valinor scaffold. v0 wires one classifier (`db_kind`) end-to-end against the Ollama-on-radagast backend so we can iterate prompts with confidence before adding the rest. - pyproject.toml: uv project, Python 3.13, httpx/pydantic/typer/pyyaml. - src/tolkien/llm.py: single-turn /api/chat wrapper. JSON-mode + temp 0 + 30m keep-alive. Pydantic schema validation on parse. - src/tolkien/cli.py: `tolkien classify db-kind --doc <file>` runs the classifier and prints JSON. `tolkien deploy <name>` is stubbed. - src/tolkien/prompts/db_kind.md: tight system prompt with 1-shot example and explicit "return ONLY JSON" guard. - src/tolkien/schemas.py: DbKindResult pydantic model. - eval/cases/mealie.yml + run_eval.py: regression harness. Currently one case (mealie); failures print field-level diffs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
37 lines
898 B
YAML
37 lines
898 B
YAML
# Eval case for the `db_kind` classifier.
|
|
# The pipeline's correctness on real apps is gated on this kind of fixture —
|
|
# re-run `eval/run_eval.py` after every prompt change.
|
|
|
|
name: mealie
|
|
input: |
|
|
App: mealie (recipe manager). Excerpt from official docker-compose docs:
|
|
|
|
environment:
|
|
DB_ENGINE: postgres
|
|
POSTGRES_USER: mealie
|
|
POSTGRES_PASSWORD: change-me
|
|
POSTGRES_SERVER: postgres
|
|
POSTGRES_PORT: 5432
|
|
POSTGRES_DB: mealie
|
|
ALLOW_SIGNUP: "true"
|
|
BASE_URL: https://mealie.example.com
|
|
OPENAI_API_KEY: sk-... # optional, enables recipe parsing via OpenAI
|
|
TZ: Europe/Amsterdam
|
|
|
|
expected:
|
|
needs_postgres: true
|
|
db_env_vars:
|
|
- DB_ENGINE
|
|
- POSTGRES_USER
|
|
- POSTGRES_PASSWORD
|
|
- POSTGRES_SERVER
|
|
- POSTGRES_PORT
|
|
- POSTGRES_DB
|
|
secrets:
|
|
- POSTGRES_PASSWORD
|
|
- OPENAI_API_KEY
|
|
config:
|
|
- ALLOW_SIGNUP
|
|
- BASE_URL
|
|
- TZ
|