AI Engineering Buildcamp: from RAG to Agents Cohort 3

Capstone Preparation 3 Statistics

Distribution of scores and reported study time for this homework.

Submissions

9

Median total score

6

Average total score

6

Score distribution

All values are points.

Questions score

Min
6
Median
6.0
Max
6
Q1
6.0
Avg
6.0
Q3
6.0

Learning in public score

Min
-
Median
0.0
Max
-
Q1
0.0
Avg
0.0
Q3
0.0

Total score

Min
6
Median
6.0
Max
6
Q1
6.0
Avg
6.0
Q3
6.0

Time distribution

All values are hours reported by students.

Lectures

Min
2.0
Median
4.5
Max
8.0
Q1
2.8
Avg
4.8
Q3
6.5

Homework

Min
1.0
Median
3.0
Max
4.0
Q1
1.8
Avg
2.8
Q3
4.0

Question breakdown

Correctness and answer distribution per question.

1. What tools will your agent need? List the tools you plan to implement for your agent (e.g., search, fetch/read, write/update, external APIs).

9 / 9 correct (100.0%)

Answer Count
Short list of tools for Data Assistant Agent: retrieve_context Description: Finds relevant warehouse metadata plus similar previously verified NL→SQL pairs before any SQL drafting. When to call: First step for every user question. Inputs: user_question Returns: schema_context, verified_query_examples, retrieval_confidence draft_sql_candidate Description: Generates a candidate SQL query (and short rationale for analyst) using retrieved context. When to call: After context retrieval, for new or partially matched questions. Inputs: user_question, schema_context, verified_query_examples Returns: sql_candidate, reasoning_summary, generation_confidence review_gate_and_queue Description: Decides whether SQL is safe and already trusted enough to run now, otherwise stores it for daily analyst review. When to call: After SQL drafting, always. Inputs: user_question, sql_candidate, confidences, guardrail_policy Returns: decision run_now_or_queue, review_ticket_id_optional, analyst_payload run_verified_query Description: Executes only approved or previously verified read-only SQL against DuckDB and returns structured results. When to call: Only if review gate returns run_now. Inputs: sql_verified, row_limit, timeout_seconds Returns: rows, row_count, column_names, execution_status, execution_error_optional compose_whatsapp_answer Description: Converts results into short end-user text without exposing SQL, and handles pending-review responses when needed. When to call: Final step for every request. Inputs: user_question, query_results_or_pending_status, trust_summary Returns: user_message_text 1
My ATS Gap Analyser agent uses four tools: extract_job_requirements(job_description: str) → dict — Parses a job description and returns structured requirements including job title, required skills, nice-to-have skills, experience years, and keywords score_cv(cv_text: str, requirements: dict) → dict — Compares CV text against extracted requirements and returns a match score (0-100), matched keywords, missing keywords, and a summary suggest_improvements(cv_text: str, missing_keywords: List[str]) → dict — Searches the ATS best practices knowledge base using RAG and returns 4-5 specific actionable CV improvement suggestions generate_cover_letter(cv_text: str, job_description: str, match_score: int) → str — Generates a tailored cover letter based on the CV, job description, and match score All tools are defined as methods on an ATSTools class. Tool definitions are auto-generated using a custom get_instance_tools() helper that introspects the class methods, signatures, and docstrings — similar to Alexey's toyaikit.get_instance_methods pattern. 1
1. process_video_and_extract_concepts(video_url: str) -> str: Downloads the YouTube transcript, chunks/indexes it, and returns the core high-level concepts taught in the video. 2. search_video_transcript(search_query: str) -> str: Performs a lexical search over the video's transcript to find specific explanations. 3. evaluate_user_answer(question: str, user_answer: str, reference_context: str) -> str: Uses the strict GapFinder rubric to grade a user's answer. 1
For Applied ML Teaching Copilot, I planned the following tools: 1. search_course_materials This tool searched the indexed Applied Machine Learning course materials and returned the most relevant records for a user query. Inputs: query: str, num_results: int = 5. Returns: a list of records with id, module, lesson, topic, source_type, and a short content snippet. Status: essential. 2. get_course_material This tool retrieved the full content of a specific course-material record by id. Input: material_id: str. Returns: the full course-material record or an error dictionary if the id was not found. Status: essential. 3. add_course_note This future tool would add a new instructor-provided note to the knowledge base. Inputs: module, lesson, topic, content, source_type. Returns: confirmation and the new note id. Status: future / nice to have. 4. generate_study_guide This future tool would generate a short study guide from retrieved course material. Inputs: topic and level. Returns: key ideas, examples, common mistakes, and follow-up questions. Status: future / nice to have. 1
1. fetch_content — Essential Fetches the full text of a web article, YouTube transcript, or local file. Returns structured content the agent can either summarize or pass to add_to_knowledge_base. Inputs: source: str (URL or file path) Returns: {title, text, url, content_type, estimated_read_minutes} Serves: "Add this URL", "Summarize this article", "Add this PDF" 2. add_to_knowledge_base — Essential Saves a processed document to resources.json and rebuilds the search index. Always called after fetch_content, never instead of it. Inputs: title, text, url, topic, type, difficulty Returns: {success, id} Serves: "Add this to my knowledge base" 3. search_knowledge_base — Essential (you already have this — formalize it) Your existing search(), promoted to a tool the agent can call with parameters. Inputs: query: str, filters?: {exclude_completed, difficulty, type}, num_results?: int Returns: list of matching documents (with completed status injected from progress state) Serves: study plans, "what's next", time-constrained sessions 4. update_progress — Essential Records a progress event to a persistent file (data/progress.json). Three event types: completed, time_logged, concept_mastered. Inputs: resource_id: str, event: "completed"|"time_logged"|"concept_mastered", metadata?: {minutes, concept} Returns: {success} Serves: "I finished the Karpathy series", "I spent 3 hours on transformers" 5. get_progress — Essential Reads the full progress state. The agent should call this before generating any study plan, so it doesn't re-recommend things you've already done. Inputs: topic?: str (optional filter) Returns: {completed: [...], time_by_topic: {...}, concepts_mastered: [...], not_started: [...]} Serves: "Show me my progress", "create a study plan", "what should I focus on?" 1
search_sections - calls Elasticsearch, returns matching contract sections with highlights. Claude uses this to find relevant contract language given a query read_pdf - ingest a new CBA pdf - ability to download from API or ingest locally correct_timing_rule — human says "that rule is wrong, the trigger is receipt not occurrence." Writes the correction back to your JSONL. This is the human-in-the-loop intervention you mentioned. add_local_config — write local-specific settings: timezone, custom deadline language, contract version overrides, info on managers, corporate groups, contact for companies - MCP server compute_deadline — trigger date + offset + unit → actual date, respecting working days + federal holidays MCP for local config - Claude grounds in the local context GoogleCalendar/ical/outlook? 1
The agent needs five tools that cover the retrieve → expand → synthesize loop: 1. search_items — keyword search over the locally ingested AI news DB (HN, Reddit, arXiv, GitHub, RSS). Filters: query, content_type, source_id, min_score. 2. get_item_detail — fetch the full record (title, summary, URL, score, source) for a given item ID. 3. fetch_url — fetch and extract readable content from an arbitrary URL, so the agent can follow links inside items (HN → blog post, arXiv → abstract). 4. web_search — external web search (Exa/Tavily) for questions the local DB can't answer. 5. generate_digest — synthesize a curated digest from a set of retrieved items. 1
search_docs web_search fetch_page 1
Tools for the AI Diet Coach Agent: 1. search_recipes — Full-text search over the recipe database by goal, ingredient, cuisine, or mood. Input: query: str Returns: up to 5 matching recipe objects Essential — this is the core RAG search tool from Week 1. 2. filter_by_max_cook_time — Returns every recipe that can be prepared within a given number of minutes. Input: max_minutes: int Returns: list of recipe objects sorted by cooking time Essential — handles "I only have X minutes" queries without guessing. 3. filter_by_category — Returns all recipes in a specific food category (e.g. Chicken, Seafood, Vegetarian). Input: category: str Returns: list of recipe objects in that category Essential — covers dietary-restriction and protein-type queries. 4. get_recipe_details — Retrieves the full ingredients list and step-by-step instructions for a specific recipe by name. Input: name: str Returns: full recipe object, or an error message if not found Essential — lets the agent drill into a recipe when the user asks "how do I make X?" I considered a build_meal_plan tool but rejected it — once the agent has recipes from the tools above, the LLM can reason through a weekly plan itself, so a dedicated planning tool would just duplicate logic. 1

2. Choose one tool and implement it. Describe what the tool does, the function signature (inputs and outputs), and any challenges you faced.

9 / 9 correct (100.0%)

Answer Count
Tool: extract_job_requirements This tool parses a raw job description and extracts structured requirements so the agent can make objective comparisons against a CV. Function signature: def extract_job_requirements(self, job_description: str) -> dict Input: raw job description text as a string Output: a dict with these fields: json{ "job_title": "Data Analyst", "required_skills": ["SQL", "Python", "pandas", "Power BI"], "nice_to_have_skills": ["dbt", "Azure", "AWS"], "experience_years": 3, "keywords": ["Data Analyst", "Logistics", "SQL", "Python"] } Implementation: uses Groq llama-3.3-70b-versatile with response_format={"type": "json_object"} and temperature=0. A Pydantic model JobRequirements defines the schema which is passed to the LLM in the system prompt to enforce structure. Challenge: the LLM occasionally merged required and nice-to-have skills into a single list. Fixed by being explicit in the prompt that these are two separate categories with different weights. 1
def evaluate_user_answer(question: str, user_answer: str, reference_context: str) -> str: """ Uses the strict GapFinder rubric to grade a user's answer. """ print(f"Evaluating user answer...") prompt = f""" Evaluate the user's answers and provide a markdown-formatted response with: - What they understood well - What they misunderstood or missed - What to revisit <QUESTION> {question} </QUESTION> <CONTEXT> {reference_context} </CONTEXT> <USER_ANSWERS> {user_answer} </USER_ANSWERS> """ print(f"question: {question}") print(f"reference context: {reference_context}") print(f"user answer: {user_answer}") response = client.chat.completions.create( model=MODEL, messages=[ {"role": "system", "content": "You are an expert tutor grading a student's answer."}, {"role": "user", "content": prompt} ], temperature=0.0 ) return response.choices[0].message.content I'm not sure yet when the agent should generate the gap report. Should the agent generate the report after the first response? Or should the agent remind the user at the beginning of the conversation to let them know if they want the gap report? I'd also like to add that the agent should point out specific parts of the video so the user can review them again. That would be another feature that hasn't been implemented yet. 1
Tool implemented: search_recipes What it does: Performs a full-text search over the recipe database using TF-IDF scoring. The agent calls this tool whenever the user describes a dietary goal, ingredient, cuisine type, or mood — for example "high protein dinner" or "quick Asian meal". It returns the top 5 most relevant recipes so the agent can reason over them and make personalized recommendations. Function signature: def search_recipes(query: str) -> list: return index.search(query, num_results=5) Input: query: str — a natural-language search string Returns: a list of up to 5 recipe objects, each containing name, category, area, ingredients, instructions, and cooking_time_minutes Tool schema (OpenAI function-calling format): { "type": "function", "name": "search_recipes", "description": ( "Search the recipe database for meals that match a query. " "Use this when the user describes a goal, ingredient, cuisine type, " "or any free-text request like 'high protein' or 'low calorie pasta'." ), "parameters": { "type": "object", "properties": { "query": { "type": "string", "description": "Natural-language search query, e.g. 'high protein chicken'" } }, "required": ["query"] } } Challenges: The main challenge was writing a precise description field. The agent uses this description to decide when to call the tool, so vague wording like "searches recipes" caused the agent to call it for every query including time-constrained ones where filter_by_max_cook_time was the better choice. Adding concrete examples ("'high protein'", "'low calorie pasta'") to the description steered the agent toward the right tool for the right query. 1
I implemented search_course_materials. Function signature: search_course_materials(query: str, num_results: int = 5) -> list[dict] This tool searched the Applied Machine Learning course-material knowledge base using a minsearch index. It searched over topic and content, while keeping module, lesson, source_type, and id as metadata fields. It returned compact results with id, module, lesson, topic, source_type, and content_snippet so the agent could decide whether to fetch the full material afterward. I also implemented get_course_material(material_id: str) -> dict as a companion fetch tool. The main challenge was converting the Week 1 fixed RAG search function into an agent-friendly tool. I also had to add grounding behavior so the agent did not answer from general model knowledge when the course materials were insufficient. 1
I chose fetch_content because it's the entry point for everything. Without it, the knowledge base stays static. Tool: fetch_content What it does fetch_content is the agent's ingestion and reading tool. Given any source — a YouTube URL, a web article URL, or a local file path — it fetches the full text content and returns it in a consistent format the agent can pass to other tools or use to generate a summary. It serves two purposes: ingesting new resources into the knowledge base, and on-demand summarisation of content without saving it. Keeping these two actions separate (fetch first, save only if confirmed) lets the agent summarise something the user is curious about without permanently modifying their knowledge base. Function signature Input: a string that is either a URL (YouTube or web article) or an absolute local file path Output: a dict with consistent keys regardless of source type: { "title": str, "text": str, # full transcript / article text / file contents "url": str, "content_type": str, # "video" | "article" | "file" "estimated_read_minutes": int } Challenges The main challenge was a breaking API change in youtube-transcript-api. My code was written for the documented pattern: YouTubeTranscriptApi.get_transcript(video_id) # v0.x — broken in v1.x When I ran it, I got AttributeError: type object 'YouTubeTranscriptApi' has no attribute 'get_transcript'. The library had dropped its class method in v1.x and replaced it with an instance method, and the transcript snippets changed from plain dicts (t["text"]) to dataclass objects (snippet.text). Nothing in the error message pointed to the version change — I had to inspect dir(YouTubeTranscriptApi) to discover that fetch was now the only public method, then probe the return type to find the new attribute interface. 1
search_sections tool queries Elasticsearch for contract sections matching a phrase or concept. Takes query (string), source_file (optional, scopes to one contract), and phrase_match (bool, for exact contract language). Returns scored section hits with highlighted fragments showing where the match appears in the contract text. The challenge was the two-field mapping — using a stemmed english analyzer for recall but a separate text.exact sub-field with no stemming for highlights, so returned snippets show verbatim contract language rather than stemmed tokens. But the even bigger challenge is creating a data structure for this incredibly complex web of data! Fun challenge! I could see wider applications of my tool in terms of being able to analyze versions of contract articles against drafts and ratified versions. Officers have to present and agree on language with membership on contracts so it is often important to keep track of proposed language and when it is agreed upon and when it is legally in effect. 1
What it does Queries the local items table and returns a ranked, paginated list. Performs a case-insensitive substring match on title and summary, with optional filters on content type, source, and minimum relevance score. Results are ordered by relevance_score DESC, then published_at DESC, then id DESC for stable ordering. Function signature (app/agent/tools.py:16) async def search_items( db: AsyncSession, *, query: str | None = None, content_type: ContentType | None = None, source_id: int | None = None, min_score: float | None = None, page: int = 1, limit: int = 20, ) -> ItemList Inputs - db: active async SQLAlchemy session - query: free-text query matched against title and summary (optional) - content_type: enum filter — PAPER, NEWS, REPO, etc. (optional) - source_id: restrict to one source (optional) - min_score: relevance floor 0.0–1.0 (optional) - page, limit: pagination, validated (page ≥ 1, 1 ≤ limit ≤ 100) Output ItemList Pydantic model: { items: list[ItemOut], total: int, page: int, limit: int } — already used by the HTTP layer, so the agent and API share one schema. Files - app/agent/tools.py — implementation - app/agent/__init__.py — package marker - tests/test_agent_tools.py — 5 tests (query match, content-type filter, min-score filter, no-filter ordering, invalid-limit validation), all passing Challenges 1. Function-callable, not HTTP-bound. The existing /api/items route mixes business logic with FastAPI dependency injection. An agent tool needs to be a pure async function that takes a session — so I lifted the query logic out of the route into app.agent.tools rather than calling the route through HTTP. 2. LIKE-injection safety. User queries flow into a SQL ILIKE. I escaped \, %, and _ with an explicit escape character so a query like 100% doesn't behave as a wildcard. 3. Schema serialization across the async boundary. Returning ORM Item objects works inside the route (FastAPI auto-serializes), but a tool consumed by an LLM loop needs JSON-safe output. I converted to ItemOut via model_validate before returning so the result is immediately serializable to the tool-call response. 4. Stable ordering for paginated agent calls. If the agent paginates, ties on relevance_score (common at 0.0) caused items to shuffle between calls. Added id DESC as the final tiebreaker. 5. Validation belongs in the tool, not the framework. FastAPI's Query(ge=1, le=100) enforced bounds at the HTTP layer for free; in a plain function I had to add explicit ValueErrors so a misbehaving LLM gets a clear error instead of nonsense pagination. 1

3. Create your agent with the tools you defined. Write down the instructions/prompt you gave your agent. If tested, share an example interaction (what you asked, tools used, answer given).

9 / 9 correct (100.0%)

Answer Count
Query 1 — Study plan (3 iterations) Iteration 1: get_progress({}) — saw 0 completed, got full not-started list Iteration 2: search_knowledge_base({"query": "AI engineering", "exclude_completed": true}) — returned 5 relevant resources Iteration 3: Final answer with a 5-hour plan across 4 resources, ended with the right prompt ("Let me know when you're done") Query 2 — Summarize a URL (2 iterations) Iteration 1: fetch_content({"source": "https://simonwillison.net/..."}) — fetched the live article Iteration 2: Specific summary — dual LLMs, confused deputy attacks, data exfiltration, the proposed solution — not generic Query 3 — Edge case: quantum computing (2 iterations) Iteration 1: search_knowledge_base({"query": "quantum computing", "exclude_completed": true}) — returned [] Iteration 2: Gracefully said "nothing in your knowledge base on this" rather than hallucinating resources 1
request USER: I ant to use this video to leran more about tokenization an embeddings: "https://www.youtube.com/watch?v=wjZofJX0v4M" response TOOL CALL: process_video_and_extract_concepts {"video_url":"https://www.youtube.com/watch?v=wjZofJX0v4M"} request TOOL RETURN: process_video_and_extract_concepts response The video covers several core concepts related to tokenization and embeddings, which are key components in understanding how models like GPT function: 1. **Generative Pretrained Transformer (GPT)** 2. **Transformer Architecture** 3. **Tokenization and Embeddings** 4. **Attention Mechanism** 5. **Probability Distribution and Sampling** Do you have any specific questions about these topics, or would you like to focus on a particular concept? If you're unsure, I can suggest some questions to guide your understanding. request USER: Ask me a question about Tokenization and Embeddings response Great choice! Here are some questions related to Tokenization and Embeddings: 1. **Coverage Question**: What is the process of tokenization, and why is it essential in natural language processing? 2. **Explain in Your Own Words**: Describe what embeddings are and how they represent words or tokens in a high-dimensional space. 3. **Application Question**: How would you use tokenization and embeddings to improve the performance of a text classification model? Feel free to answer any or all of these questions! request USER: Tokenization is the process of breaking down text into smaller units called tokens, which can be words, subwords, or characters. In natural language processing (NLP), these tokens are the basic pieces that models analyze and learn from. It is important because computers cannot directly understand raw text, so tokenization provides a structured way to represent language as data. Good tokenization can improve model performance by preserving meaning and handling variations in language more effectively. response TOOL CALL: search_video_transcript {"search_query":"tokenization"} request TOOL RETURN: search_video_transcript response TOOL CALL: evaluate_user_answer {"question":"What is the process of tokenization, and why is it essential in natural language processing?","user_answer":"Tokenization is the process of breaking down text into smaller units called tokens, which can be words, subwords, or characters. In natural language processing (NLP), these tokens are the basic pieces that models analyze and learn from. It is important because computers cannot directly understand raw text, so tokenization provides a structured way to represent language as data. Good tokenization can improve model performance by preserving meaning and handling variations in language more effectively.","reference_context":""} request TOOL RETURN: evaluate_user_answer response Your response about tokenization is solid! You correctly defined the process and highlighted its importance in natural language processing. Here's what you did well and where you can improve: ### What You Understood Well: - You accurately described tokenization as breaking down text into smaller units (tokens) and noted its essential role in structuring language for model analysis. - You emphasized that good tokenization improves model performance, which shows a good grasp of its significance. ### Areas for Improvement: - **Depth of Explanation**: It would be beneficial to elaborate further on different tokenization techniques (such as word, character, and subword tokenization) and their specific applications. - **Challenges in Tokenization**: Consider discussing the challenges that may arise during tokenization, like handling punctuation, variations in language, and how these challenges can impact NLP tasks. ### What to Revisit: 1. **Tokenization Techniques**: Investigate the various methods of tokenization and their use cases in NLP for a more comprehensive understanding. 2. **Challenges and Considerations**: Learn about the common challenges in tokenization and strategies to address them. 3. **Real-World Applications**: Explore how tokenization is applied in tasks like sentiment analysis and machine translation to grasp its practical significance further. Would you like to attempt another question on embeddings, or do you want to dive deeper into tokenization techniques? 1
Agent instructions: You are an ATS Gap Analyser assistant. You help job seekers understand why their CV may not be passing ATS screening and what to fix. When a user provides both a CV and a job description, always follow this sequence: 1. Call extract_job_requirements on the job description 2. Call score_cv with the CV and extracted requirements 3. Call suggest_improvements with the CV and missing keywords 4. Call generate_cover_letter with the CV, job description, and match score 5. Present a clear summary with the score, gaps, suggestions, and cover letter User input: "Please analyse my CV against this job description and tell me how well it matches, what the gaps are, what I should fix, and write me a cover letter." Calling tool: extract_job_requirements Calling tool: score_cv Calling tool: suggest_improvements Calling tool: generate_cover_letter Your CV matches the job description by 70%. The gaps in your CV include: - Missing required skills: numpy, Tableau - Missing nice to have skills: dbt, Azure, AWS, logistics, supply chain knowledge To improve your CV, consider: - Add numpy to the Skills section to demonstrate proficiency in numerical computing - Include Tableau in the Skills section and describe its application in your roles - Insert dbt in the Skills section with an example of data transformation work - Incorporate Azure or AWS with an example of cloud platform experience - Emphasize logistics and supply chain in your Skills section with relevant examples Cover letter: [tailored cover letter generated] 1
Agent instructions: You are an AI diet coach helping users reach their weight-loss goals through personalized meal recommendations. You have access to a database of 201 recipes. Use your tools to answer every question - never invent recipes that are not in the database. When to call each tool: - search_recipes: when the user describes a goal, ingredient, cuisine, or mood. Always start here for general queries. - filter_by_max_cook_time: when the user mentions a time limit. - filter_by_category: when the user specifies a protein or food type. - get_recipe_details: when the user wants the full recipe for a specific dish. How to respond: - For each recommended recipe include: name, category, cooking time, key ingredients, and one sentence explaining why it fits the user's goal. - Prefer lower-calorie options and lean proteins for weight-loss goals. - If no suitable recipes are found, say so honestly and give a general tip. - Be concise and supportive. Example interaction: Question: "I only have 15 minutes tonight. What vegetarian meals can I make?" Tools called: filter_by_max_cook_time({"max_minutes": 15}) filter_by_category({"category": "Vegetarian"}) Final answer: Here are some vegetarian meals you can prepare in 15 minutes or less: Beetroot Latkes - Vegetarian | 15 min | Beetroot, Egg, Plain Flour, Greek Yogurt, Mint. Quick to prepare and low in calories. Chinese Tomato Egg Stir Fry - Vegetarian | 8 min | Plum Tomatoes, Eggs, Sesame Seed Oil, Spring Onions. Simple, fresh ingredients with protein from the eggs. Would you like the detailed recipe for either of these? The agent called two different tools in sequence, one for the time constraint and one for the dietary restriction, and combined their results into a single answer. 1
I created a custom OpenAI tool-calling agent using the OpenAI Responses API. The agent used two implemented tools: search_course_materials and get_course_material. Agent instructions: The agent was instructed to act as an Applied Machine Learning Teaching Copilot for instructors and students. It had to search the course materials when asked about a topic, fetch full material when needed, answer only from retrieved course materials, cite material ids, explain clearly, and say when the current knowledge base was insufficient. Example interaction: User question: When should I use MAE instead of MSE in a regression problem? Tools used: 1. search_course_materials({"query": "MAE vs MSE regression", "num_results": 5}) 2. get_course_material({"material_id": "aml-001"}) Final answer: The agent answered that MAE was preferable when I wanted an interpretable metric in the target units and less sensitivity to outliers, while MSE was preferable when large errors were especially costly because it penalized large mistakes more strongly. Source: aml-001 I also tested a decision-tree study guide query and an out-of-scope CNN query. For the CNN query, the agent correctly stated that the current course materials were insufficient instead of inventing unsupported content. 1
Agent instructions/prompt: You are a grievance deadline assistant for union representatives. You have tools to list available contracts, search contract language by keyword or phrase, retrieve structured timing rules already extracted from a contract article, and compute exact deadline dates from a trigger date and timing rule. Before computing any deadline, search the contract for holiday and working day definitions — do not assume federal holidays apply. The contract's own language is the only authority on what counts as a non-working day. Always tell the rep which article and exact contract language each deadline comes from, and what holiday schedule was or was not applied and why. If a rule is ambiguous, ask the rep to clarify before computing. Example interaction: Rep: I work under the Verizon contract. A grievance occurred on May 1st 2026. What are my deadlines? Agent called: list_contracts → confirmed dol_5_VERIZON_DELAWARE_INC_VERIZON_SERVICES_CORP.txt is indexed Agent called: search_sections with query "holiday working day business day definition" scoped to the Verizon contract → found no explicit holiday definitions in the grievance procedure Agent called: search_sections with query "grievance procedure steps deadline" → found Article 13, Procedure for Adjusting Grievances Agent called: compute_deadline with trigger_date="2026-05-01", offset=30, unit="calendar_days" → returned May 31, 2026 (Sunday) with a weekend warning Agent called: compute_deadline with trigger_date="2026-05-01", offset=10, unit="calendar_days" → returned May 11, 2026 (Monday) for discipline grievances Agent responded: The Verizon contract uses calendar days for all grievance deadlines and defines no non-working holidays, so no holiday schedule was applied. Standard grievances must be filed within 30 calendar days (by May 31 — a Sunday, so file by May 29 to be safe). Discipline grievances use an expedited 10-day track directly to the third-tier supervisor, making the deadline May 11. Arbitration notice is due 30 calendar days after the company's answer, a rolling clock that starts when the response is received. 1
Architecture The agent is a tool-calling loop that uses the OpenAI-compatible chat completions API (pointed at Kimi K2.5, matching the project's existing app/services/kimi.py pattern). On each turn: 1. Send the conversation + tool schemas to the model. 2. If the model returns tool_calls, dispatch each to the local Python implementation, append results as role: tool messages, and loop. 3. If the model returns plain content, that's the final answer. 4. Hard cap at max_turns=5 to prevent runaway loops. Files - app/agent/agent.py — system prompt, tool schemas, dispatch, loop - app/agent/tools.py — search_items (from question 2) - tests/test_agent.py — 3 tests with the LLM client mocked, all passing System Prompt / Instructions You are AI Tracker, a research assistant for a personal AI news aggregator. The user follows AI/ML and wants concise, source-grounded answers. The local corpus contains items ingested from Hacker News, Reddit (r/MachineLearning, r/LocalLLaMA), arXiv (cs.AI, cs.CL), GitHub trending, and AI-lab RSS feeds (OpenAI, Anthropic, Google AI, Hugging Face). Operating rules: 1. Always call `search_items` before answering any factual question about recent AI news, papers, or repos. Do not answer from prior knowledge alone. 2. Prefer specific, narrow queries. If the first search is too broad, refine with `content_type` (paper/news/repo/discussion) or `min_score`. 3. Cite items by title and URL. Never fabricate titles, authors, or URLs. 4. If `search_items` returns zero results, say so plainly and suggest a broader query — do not invent items. 5. Keep answers tight: a one-line summary, then a bulleted list of the most relevant items (title, source, one-sentence why-it-matters, URL). 6. If the user asks something the local corpus cannot answer (e.g. "what happened today on Twitter"), say the corpus does not cover it. The prompt's design choices: - Rule 1 forces grounding — without it Kimi will happily answer from training data and skip the tool. - Rule 3 is the anti-hallucination guardrail; titles and URLs are the bits the model is most tempted to invent. - Rule 4 handles the empty-result case explicitly, which is the most common silent failure mode in RAG agents. - Rule 6 acknowledges scope — the corpus is finite and admitting that beats fabricating. Tool Schema (presented to the model) search_items(query?, content_type?, source_id?, min_score?, page=1, limit=10) — all parameters optional, content_type constrained to the ContentType enum so the model can't invent values like "tweet". Example Interaction (from the test suite) User: "Any new MoE papers?" Turn 1 — model issues tool call: { "name": "search_items", "arguments": {"query": "mixture of experts"} } Tool result (returned to model): { "items": [{ "id": 1, "title": "Mixture-of-Experts scaling laws", "summary": "MoE routing improves compute-optimal training.", "url": "https://arxiv.org/abs/2401.00001", "content_type": "paper", "relevance_score": 0.95 }], "total": 1, "page": 1, "limit": 10 } Turn 2 — final answer: ▎ "Found 1 paper on MoE scaling. See arxiv.org/abs/2401.00001." Tool calls trace: [{name: "search_items", arguments: {query: "mixture of experts"}, ok: true}] 1

Calculated: 11 May 2026, 14:55