Twill Brain

#75 Widget data cache nightly updating

2 min read

#75 — Widget data cache nightly updating

Repo: Twill-AI/facade State: closed | Status: done Assignee: Unassigned

Created: 2024-08-16 · Updated: 2025-03-24

Description

Fastest to implement approach, eventually need to implement Twill-AI/python-shared#38

Idea is to blindly (i.e. without any BusinessData income/insights) update “values” in all dashboard’s widgets each night.

AC:

  • Cron trigger in Facade runs through all tenants, all widgets with attached dashboard (similar to run_regular_sync), executes “sql_query” in relevant BusinessData storage, updates “values_rows” and “values_updated_at” if widget values_updated_at + from_seconds(values_update_interval_sec) <= now. If values_update_interval_sec = -1 then widget data shouldn’t be updated.
  • Each night run logs number of “updated/total” widgets per tenant and duration of handling for each tenant.

Implementation details

Not covers all widgets updates because of https://github.com/Twill-AI/twill-llm-engine/issues/162. If we detect cases when data from SQL query doesn’t match widget headers then warning log “can’t match data for widget so updating nothing” is produced and widget is skipped. https://github.com/Twill-AI/facade/issues/141 should cover these cases.

As of now, we will implement nightly updates for basic widgets that don’t need transformations or forecasting added to them. @meliascosta will mark those with values_update_interval_sec = -1. Besides that, we will also check that the columns of the query result set and the contents of

Notes

Add implementation notes, blockers, and context here

Add wikilinks to related people, meetings, or other tickets

Graph View