Twill Brain

#541 [Bug]: merchant_router.get_dashboard tests are flaky and leaky

2 min read

#541 — [Bug]: merchant_router.get_dashboard tests are flaky and leaky

Repo: Twill-AI/facade State: closed | Status: done Assignee: meliascosta

Created: 2025-08-29 · Updated: 2025-09-18

Description

Steps to reproduce

Caused by https://github.com/Twill-AI/facade/actions/runs/17247303279/job/49082681117 and https://github.com/Twill-AI/facade/actions/runs/17318186798/job/49165240630?pr=540

Tests in app/tests/api/test_merchants_router.py for “get_dashboard” are not stable - they fail on different dates. Additionally they are checking states quite far from reality:

  • Generate not-like-real KPI entry ID-s (like 25cb9232-45ff-4b4a-99b9-b7c7aeea5b81_2025-08-28_monthly_Monthly Volume instead of fcabc994-b84b-4a00-b4b1-6efea9731035_2025-08-01_monthly_$ Volume).
  • “Arrange” part doesn’t generate “shouldn’t be returned” values (extra months, extra merchants, extra partners, etc.) so we are not checking filters in (quite complex) queries of get_dashboard endpoint handler.
  • “Assert” part makes “patchy” checks for existence of a field or its type instead of checking the whole response at once (aka assert response.json() == { ... }).

Plus “Arrange” part is not efficient in terms of db communication - tests make multiple “insert” queries for the same table instead of single “bulk insert” or make pair of “insert” and next “update” queries instead of direct “insert” with right values.

Expected behavior

Tests are stable and makes real-world checks. See corrections of 2 tests (doesn’t include most of fixes for “make real-world checks”):

Notes

Add implementation notes, blockers, and context here

Add wikilinks to related people, meetings, or other tickets

Graph View