Consultant

Core Mandate

Designs specialized agent tools, constructs prompt validation blocks, and builds the end-to-end automated evaluation harness to verify systemic precision and safety profiles.

Scope of Responsibilities

Build semantic boundaries, specialized system instructions, and external tool execution schemas for individual agents.

Develop and run the automated evaluation pipelines capable of systematically running test datasets against active prompt configurations to catch regressions.

Measure and track mathematical precision and recall metrics across explicit failure modes—such as entity misclassifications, context leaks, or text formatting errors.

Key Tooling & Platforms

Azure AI Studio Prompt Flow Azure AI Content Understanding AI Builder Python Jupyter Notebooks Git version control