#8 — Investigate ways to embed LLM core into Facade
Repo: Twill-AI/facade State: closed | Status: done Assignee: Unassigned
Created: 2024-05-14 · Updated: 2025-09-15
Description
Task:
Embed “LlmEngine” component into Facade to communicate with front-end app via async websocket commands. Need to provide following integrations from Facade side (see “C3 v2” diagram):
- bi-directional communication with async WebSocket (simplest in LlmEngine but auth-ed in Facade);
- call external (LLM, etc.) API, so there are no CPU or RAM heavy logic inside but need to provide credentials in unified with LlmEngine way;
- [TenantData] call Python classes in the application for the tenant information:
- store Chats memory via LangChain’s PostgresChatMessageHistory;
- [in future] interact with per-tenant Vector database through LangChain’s SupabaseVectorStore;
- [TwillData] interact with shared-across-tenants Vector database through LangChain’s SupabaseVectorStore.
LlmEngine repo should be able to work separately with mocking some parts of integrations and have some “in memory” or mocked replacements for PostgresChatMessageHistory, SupabaseVectorStore and .
Options:
- Use separate repo and import into Facade as one module.
- Keep in Facade but have an ability to copy into Jupiter Notebooks for experiments/tuning/etc.
AC:
- Prepare table of options with ways to handle different use cases and pros/cons of each approach.
Notes
Add implementation notes, blockers, and context here
Related
Add wikilinks to related people, meetings, or other tickets