#178 — BUG: Parallel tool calls fail due to asynpg interface error

Repo: Twill-AI/twill-llm-engine State: closed | Status: done Assignee: meliascosta

Created: 2024-10-24 · Updated: 2024-10-29

Description

This happened after we moved the ASQDatabase object outside of llm-engine

Steps to reproduce:

1. Use a prompt that will produce parallel tool calls >>> “Who are my top 5 vendors?” usually works
2. Second tool call fails with error:

File "/home/mec/toptal/twill/repos/twill-llm-engine/twill_llm_engine/utils/database.py", line 196, in get_entities
    results = await self.db.run_query(SQL_GET_VENDORS)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


  File "/home/mec/.cache/pypoetry/virtualenvs/twill-llm-engine-uD5CZwxz-py3.12/lib/python3.12/site-packages/shared/multitenancy/database.py", line 76, in run_query
    return await self._connection.fetch(query)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


  File "/home/mec/.cache/pypoetry/virtualenvs/twill-llm-engine-uD5CZwxz-py3.12/lib/python3.12/site-packages/asyncpg/connection.py", line 691, in fetch
    return await self._execute(
           ^^^^^^^^^^^^^^^^^^^^


  File "/home/mec/.cache/pypoetry/virtualenvs/twill-llm-engine-uD5CZwxz-py3.12/lib/python3.12/site-packages/asyncpg/connection.py", line 1794, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^


  File "/home/mec/.cache/pypoetry/virtualenvs/twill-llm-engine-uD5CZwxz-py3.12/lib/python3.12/site-packages/asyncpg/connection.py", line 1892, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^


  File "/home/mec/.cache/pypoetry/virtualen

## Notes

_Add implementation notes, blockers, and context here_

## Related

_Add wikilinks to related people, meetings, or other tickets_