#292 — LlmEngine improve initialization time

Repo: Twill-AI/twill-llm-engine State: open | Status: open Assignee: meliascosta

Created: 2025-05-12 · Updated: 2025-05-12

Description

Caused by https://twill-network.slack.com/archives/C078KS0R05B/p1747044975615369

Startup time for BusinessData and Facade are drastically different - BD starts in a couple of seconds while Facade in dozens of seconds.

If measure modules initialization time for Facade with adding PYTHONPROFILEIMPORTTIME=1 into “.env” file then Python interpreter would print details in a view of a table

import time: self [us] | cumulative | imported package
import time:       302 |        302 |   _io
import time:        63 |         63 |   marshal
import time:       354 |        354 |   posix
import time:       538 |       1255 | _frozen_importlib_external

For Facade file with these logs is 300+ KB in size. PYTHONPROFILEIMPORTTIME_parser.py.txt (rename with removing “.txt” suffix) may parse it and at 2025/05/12 it produces:

Top 10 slowest packages by self import time (self [us]):
Package                                   self [us]  cumulative [us]
----------------------------------------------------------------------
twill_llm_engine.nlp.tables_kg              3509022          6711830
langcodes.data_dicts                        1362471          1362471
hpack.huffman_table                          948747           948747
fastapi.openapi.models                       746001          1415874
twill_llm_engine.nlp.llms                    493242          4133320
rich._emoji_codes                

## Notes

_Add implementation notes, blockers, and context here_

## Related

_Add wikilinks to related people, meetings, or other tickets_