Analytics Framework
Date: 2025-04-23 · Duration: 80.88999938964844 min Organizer: Nader Attendees: Ahmad, Martin, Mike, Nader
Summary
-
Communication Options for Analytics: Discussed aligning Python analytics, LLMs, and visualization tools; 3 approaches presented by Martin: (1A) in-thread functions, (1B) separate service with MCP protocol, (2) LLM-generated code in sandboxed environments. Security was a concern for LLM-generated code; sandboxing to limit data access.
-
Implementation Trade-offs: Emphasized expressivity vs. reliability; Ahmad favored preset functions for control. Martin proposed hybrid model with ‘experimental’ button for free-form testing. Explored Azure sandbox capabilities, demonstrated Gemini in Colab for data analysis.
-
Arbitrary Code Testing Limitations: Martin’s demo using Gemini on Azure cost data showed million percent MAPE error; Nader noted time-consuming nature of LLM prompting. Team agreed that while arbitrary code has development potential, it’s not suitable for production use.
-
Analytics & Visualization Framework: Decided analytics code should intake/output tables; visualization outputs as JSON for eCharts. Martin recommended decoupling data analysis from visualization code. Next steps include creating eCharts rendering components and handling styles in front-end.
-
Next Steps: Ahmad to calculate KPIs & update GitHub; use Colab/Gemini for KPI analysis. Modular approach with predefined Python code configurable by LLM. Gemini identified as optimal analytics tool for LLM integration.
Action Items
- Ahmad Abd Calculate KPIs and update status on GitHub (01:06:42) Try using Colab/Gemini for KPI calculation (01:07:04) Create output formats compatible with eCharts visualization (01:00:09)
Martin Elias Costa Check with Raymond about abstracting visualization component for Ahmad to use (01:01:02) Define JSON structure for visualization output (59:30)
Raymond Create front-end component that renders config with eCharts (01:00:47)