clan-rag-flm/TODO.md

TODO

TUI improvements

• Add scrolling to the response view
• Press up arrow to recall / edit the previous message
• Add cancel / interrupt for in-progress requests
• Better help text (keybinds, available commands)

Inference backend

• Move from pure-FLM to lemonade as the single inference layer. Standardize on lemonade (nix-amd-ai's lemond) as the one backend clan-rag talks to: for our deployment it still routes to FastFlowLM on the NPU, but going through lemonade lets the same module serve GPU/CPU setups (llama.cpp) without a code path per backend. Foundation exists on branch feat/backend-abstraction (defer to lemond when hardware.amd-npu.enableLemonade is on; standalone FLM otherwise) — the goal is to make lemonade the default/only path and drop the bespoke FLM-serving unit. Caveats to carry over: embedModel must be a lemonade registry recipe name (not the FLM default embeddinggemma-300m), and switching embed model changes vector dims → MeiliSearch embedder config + a full reindex.

NixOS module

• Wire up nix-amd-ai services more completely so the module is self-contained: the current nixosModule imports nix-amd-ai.nixosModules.default and sets hardware.amd-npu.enable / enableFastFlowLM, but whether that is enough to bring up the NPU drivers and FastFlowLM without additional config elsewhere is untested. Goal: someone should be able to add only services.clan-rag.enable = true (plus the flake input) and have everything work, including XRT kernel modules, firmware, and the FLM service.