Refactor Tour Coach: LLM Powered Mission-Aware Service

This document outlines the plan to refactor the Tour Coach service, transforming it into a mission-aware system driven by Large Language Models (LLMs) and supported by structured data persistence. The goal is to provide users with personalized coaching, workout plans, and progress updates directly tied to their specific missions, enhancing their training experience and optimizing their performance.

Overview: Embracing LLMs for Smarter Coaching

The current Tour Coach service relies on rule-based logic, which can be inflexible and lack the nuanced understanding needed to provide truly personalized coaching. This refactoring aims to replace these rigid rules with the power of LLMs, enabling the service to make more intelligent decisions based on a user's mission goals, current progress, and real-time data. By integrating LLMs, the Tour Coach can dynamically adapt to changing circumstances, provide tailored recommendations, and ultimately guide users towards achieving their objectives with greater precision.

The refactored service will leverage gpt-4o-mini for daily coaching decisions, utilizing the more robust gpt-4o for critical event days where richer reasoning is required. All LLM outputs will adhere to a strict JSON schema defined in schemas/coach_output.json, ensuring consistency and predictability. A comprehensive error-handling mechanism will be implemented to gracefully manage any parsing errors, API failures, or unexpected LLM responses, ensuring a smooth and reliable user experience.

Database Schema: Structuring Data for Enhanced Performance

To support the LLM-driven decision-making process and ensure data integrity, a new database schema will be implemented. The centerpiece of this schema is the event_progress table, which stores detailed information about a user's progress during each stage of their mission. This table captures key metrics such as stage number, stage type, ride mode, points earned, bonuses achieved, completion timestamp, and associated activity ID. By storing this information in a structured format, the service can easily access and analyze a user's past performance, providing valuable context for future coaching decisions.

CREATE TABLE event_progress (
 id UUID PRIMARY KEY,
 mission_id UUID REFERENCES missions(id),
 stage_number INT,
 stage_type TEXT,
 ride_mode TEXT,
 points INT,
 bonuses JSONB,
 completed_at TIMESTAMP,
 activity_id TEXT
);

The event_progress table is designed to be flexible and extensible, allowing for the addition of new metrics and data points as the service evolves. The use of JSONB for storing bonuses enables the storage of complex and variable data structures, providing maximum flexibility in capturing different types of bonus opportunities.

LLM Integration Strategy: A Symphony of Models and Schemas

The integration of LLMs is at the heart of this refactoring effort. The Tour Coach service will strategically utilize different LLMs based on the specific task and context. For routine daily coaching decisions, the efficient gpt-4o-mini model will be employed, striking a balance between performance and cost. However, for critical event days, where more nuanced reasoning is required, the more powerful gpt-4o model will be utilized.

To ensure consistency and predictability in LLM outputs, all responses will be validated against a predefined JSON schema (schemas/coach_output.json). This schema specifies the expected format and data types of the LLM's output, ensuring that the service can reliably process and interpret the information. Any parsing errors encountered during schema validation will be meticulously logged, and the request will be retried once. If the error persists, an administrative notification will be triggered, alerting the development team to investigate the issue.

Expected Coaching Output: A Glimpse into the Future of Personalized Training

The refactored Tour Coach service will provide users with a comprehensive and personalized coaching experience. The expected coaching output, delivered in JSON format, will include a range of insights and recommendations designed to optimize a user's training and performance. This output will encompass elements such as recommended actions, reasons for those actions, intensity recommendations, flags for potential issues (e.g., fatigue warning), confidence levels, recommended ride modes, rationales for ride mode selection, expected points, bonus opportunities, strategic notes, and detailed workout plans.

{
 "action": "maintain",
 "reason": "You slept well but TSB is -9...",
 "intensity_recommendation": "moderate",
 "flags": ["fatigue_warning"],
 "confidence": 0.85,
 "recommended_ride_mode": "gc",
 "mode_rationale": "Conserve energy for tomorrow's mountain stage",
 "expected_points": 5,
 "bonus_opportunities": ["consecutive_gc"],
 "strategic_notes": "Focus on staying with the peloton",
 "workout_plan": {
 "type": "Endurance",
 "duration_minutes": 90,
 "zones": {"Z2": 60, "Z3": 30}
 }
}

The coaching output is designed to be actionable and informative, providing users with clear guidance on how to optimize their training and achieve their mission goals. The inclusion of confidence levels and rationales helps users understand the reasoning behind the recommendations, fostering trust and engagement.

Service Architecture Changes: A Leaner, Meaner Coaching Machine

The refactoring of the Tour Coach service will involve significant changes to its architecture. All rule-based fallbacks, which were a source of inflexibility and complexity, will be removed. The service entrypoint will be modified to accept a mission_id parameter, providing the necessary context for personalized coaching. Metrics will be fetched from connected data sources, ensuring that the LLM has access to the most up-to-date information. All coaching outputs will be persisted to a new mission_runs table, providing a historical record of coaching decisions. Activity data will be linked to event_progress records, enabling the analysis of performance trends. A robust JSON schema validation mechanism with retry logic will be implemented to ensure the integrity of LLM outputs.

Tasks: Charting the Course to Success

The refactoring of the Tour Coach service will be a multi-faceted effort, involving a range of tasks. These tasks include:

• Refactoring the service entrypoint to accept mission_id.
• Implementing the EventProgress table and service layer.
• Integrating LLM decision-making with JSON schema validation.
• Adding comprehensive error handling and retry logic.
• Creating data persistence for all coaching outputs.
• Removing legacy rule-based decision paths.
• Creating a CLI/worker job interface.
• Adding unit tests with mocked LLM responses.
• Creating integration tests with real data flow.

Definition of Done: Crossing the Finish Line

The refactoring of the Tour Coach service will be considered complete when the following criteria are met:

• The CLI/worker job run_mission_day(user_id, mission_id) completes successfully.
• All outputs are written to the mission_run and event_progress tables.
• Unit and integration tests pass with mocked APIs.
• JSON schema validation catches malformed LLM responses.
• Error handling gracefully manages API failures.

User Outcome: A Personalized Coaching Experience

The refactored Tour Coach service will provide users with a personalized and engaging coaching experience. Each daily run will yield a coherent coaching note, workout plan, and points update directly tied to their mission. These updates will be visible moments after the job finishes, providing users with timely and actionable guidance. By leveraging the power of LLMs and structured data persistence, the Tour Coach service will empower users to achieve their training goals with greater efficiency and effectiveness.

In conclusion, this refactoring of the Tour Coach service marks a significant step towards personalized, data-driven training. By integrating LLMs, implementing a robust database schema, and streamlining the service architecture, we are creating a powerful tool that will empower users to achieve their athletic aspirations.

For further information on LLMs and their applications, visit OpenAI.