Copilot in Power BI, Microsoft Fabric data agents, Copilot Studio and lakehouse tools such as Databricks Genie all promise the same thing: ask a question in plain language and get a trustworthy answer. In practice, the quality of that answer depends far less on the AI model than on the data it is grounded in.
When an agent gives a wrong number, the root cause is almost always familiar: ambiguous names, duplicated logic, missing context or security that lives somewhere the agent cannot see. The good news is that the fixes are equally familiar to any strong analytics team.
A readiness checklist
- Clean star schemas. Facts and dimensions with clear grain, one-to-many relationships and no hidden many-to-many traps. Agents reason over structure; give them a simple one.
- Business-friendly names. “Net Revenue” rather than “Amt_2”. Hide technical columns and helper tables that nobody should query.
- Descriptions and synonyms. Describe what each table, column and measure means, and add the words people actually use: sales, revenue, bookings.
- One certified definition per KPI. If three measures could plausibly answer “what was revenue last quarter?”, the agent will eventually pick the wrong one. Consolidate and certify.
- Security enforced in the model. When row-level security lives in the semantic model, agents inherit it automatically. If it lives only in report filters, they do not.
- Known gaps documented. Tell users—and the agent’s instructions—what the data does not cover, such as months that are not yet closed.
Measure before you scale
Before rolling out an agent, write down 30 to 50 real questions from the people who will use it, together with the correct answers. Run the set after every change to the model or the agent’s instructions, and track accuracy over time. This evaluation set is the single most useful artifact in an AI analytics project: it turns “it seems to work” into a number.
Log the questions people actually ask and the ones that fail. They tell you which synonyms, measures and descriptions to add next.
Start narrow
Pick one domain—revenue cycle, sales performance or supply chain—one audience and one well-governed model. Prove accuracy there, then expand. Broad, unfocused agents over dozens of models are where trust is lost.
Connect agents to tools deliberately
Standards such as the Model Context Protocol (MCP) make it straightforward to connect agents to enterprise systems: semantic models, ticketing, documents and workflows. Treat each connection like any other integration. Grant the least privilege required, keep humans in the loop for actions that change data or send messages, and log every call.
Respect privacy by design
Keep personal and health information out of prompts unless it is genuinely needed, keep data in the regions your obligations require, and make sure your AI services are covered by the same agreements and controls as the rest of your data platform.
The bottom line
Organizations with well-modelled, well-documented and well-secured data will get value from AI agents quickly. Everyone else will get confident-sounding wrong answers. The work to close that gap is not exotic—and it improves your human-facing analytics at the same time.