Making an Org Agent-Ready @Meta
The standards and governance an organization needs before it can trust its agents with decisions.
Agents made every analyst in our org faster almost overnight. They also exposed a quieter problem: an agent is only as reliable as what it reads, and what our org had written down was scattered, duplicated, and often out of date. Two people could ask the same question and get two different numbers, each delivered with full confidence. At Meta, I traced where this came from, wrote a paper proposing a set of standards and a governance framework to fix it, and presented it to our 50+ person data science org. Nothing was mandated, and teams adopted it anyway: a shared metric spec repository, Skills built to its guidelines, and agents that clean up after other agents.
Challenges
What I did
- 1ChallengesThe same metric was defined differently in different places, and agents could not tell which version was true.
- 1What I didOne source of truth for every definition, set once and linked everywhere else.
- 2ChallengesHow any given number was calculated was buried in layers of queries, so neither people nor agents could check it.
- 2What I didA clear, owned spec for every number that drives a decision.
- 3ChallengesDomain knowledge was stuck in individuals' personal notes, out of reach of everyone else's agents.
- 3What I didA shared hierarchy, so what one person learns reaches the whole team.
- 4ChallengesWhen definitions changed, the old versions stayed behind and agents kept using them.
- 4What I didNamed owners and a change process for everything agents rely on.
- Role
- Author of the framework; set the standards the org built to
- Audience
- A 50+ person data science org supporting Meta's Ads Sales
- Scope
- Metric definitions, shared context, Skills, and their governance
- Outcome
- Adopted by teams without a mandate: specs for 25+ metrics and data fields, and Claude Skills built to its guidelines across the org
A note on the examples. The paper and everything built from it are internal to Meta, so the examples on this page are synthetic. The framework is the one I proposed.
The bottleneck isn't the model
Most AI transformations stall on context, not capability. The models are good enough. What holds them back is what they are given to read. A person searching the wiki skips the outdated page on the fourth screen of results. An agent finds it by keyword and trusts it as much as anything else.
That makes everything an agent can reach a production input, as consequential as code or data. My paper's central argument was that context should be managed the same way: as an asset with owners, standards, and a lifecycle, not as documentation that gets written once and forgotten.
One answer to every question
Every number that influences a decision gets a single definition that people and agents can both read: what it measures, where it comes from, and who owns it. Those numbers get the strictest governance in the framework, because they are the ones leaders act on.
Here is what the alternative looks like. Three people ask their agents for last quarter's revenue.
"What was revenue last quarter?"
Nobody in that picture did anything wrong. The dashboard reports bookings because that is its job, the notebook predates a cleanup, and the wiki page was right last year. Their agents found different definitions and reported answers up to $14.8M apart, the highest being 21% above the governed number.
The payoff of a single definition is not just accuracy, it is trust. Without one, a wrong answer looks exactly like a right one. With one, the agent either gets it right or says it can't answer.
Test the definition, not every answer
Most teams make agents trustworthy by grading their output: write a set of questions, run the agent, and have people or another model score the answers. Those evaluations are expensive to build, never finished, and only as good as their coverage, because every new question is a new chance to be wrong.
A spec changes what needs testing. When the agent's job is to pick the right metric rather than to work out the calculation, the calculation can be tested once, the way software is, with a known input and a known answer, and every question that lands on that metric inherits the result. What's left to evaluate narrows to one question: did the agent pick the right metric? And when no spec fits, it says so instead of improvising.
Well-kept context doesn't make evaluation disappear. It turns most of it into ordinary tests that pass or fail, and leaves few enough judgment calls to check properly.
Knowledge that belongs to the team
The most valuable context in an org starts in one person's head: which data to trust, which exception everyone forgets. Left in personal notes, it makes one person's agent smarter, leaves everyone else's guessing, and leaves the company when they do.
The framework organizes context in levels that mirror the org chart. Teams can tailor how work gets done, but definitions are set once and never overridden lower down. Useful knowledge gets promoted upward, so the organization gets smarter, not just the individuals in it.
- Preferences cascade downEach level can tailor how work gets done. The most specific level wins.
- Definitions never doWhat a number means is set once, at the top, and linked from below rather than restated.
- Knowledge is promoted upWhen one person's note proves useful to a second, it moves up a level.
Skills as shared infrastructure
Skills package repeatable work, like a causal analysis or a weekly business review, so an agent does it the same way every time. Treated as one-off scripts, they multiply and drift. Treated as shared infrastructure, they compound.
The paper asked for Skills to be built the way good software is: each does one job and builds on the ones beneath it, each has an owner, each is tested before it's shared, and all of them sit in one catalog where anyone can find them. A Skill nobody can find gets rebuilt by the next team that needs it.
Governance proportional to the stakes
Governance is what keeps all of this true after launch day, and it only works if it is light enough that people follow it. So the framework scales process to the stakes rather than applying the same rules to everything.
- Numbers that drive decisionsOwnerA named metric ownerReviewEvery change, before it shipsWhen it changesVersioned; every old copy updated or removed
- Shared SkillsOwnerThe author or owning teamReviewTested before sharing, re-tested on changeWhen it changesNew version, catalog updated
- Team contextOwnerThe teamReviewPeriodic, plus automated staleness checksWhen it changesEdited in place; links to definitions, never restates them
- Personal notesOwnerEach personReviewNone until promotedWhen it changesWhatever proves useful moves up a level
When a definition changes, outdated versions are updated or removed, not just flagged, because an agent that never reads the warning will use the old number anyway. And because people miss things, automated checks look for the stale copies they leave behind.
What came of it
I presented the paper to the full data science org, more than 50 people. Nothing was mandated, and nobody had to adopt it. Teams put it to work on their own.
- A metric spec repository became the home for decision-grade definitions, with specs for more than 25 metrics and data fields, and existing queries were rewritten on top of those specs instead of carrying their own logic.
- Teams across the org designed their Claude Skills to the paper's guidelines.
- A Skill discovery dashboard gave everyone one place to find, share, and reuse them.
- Several repositories adopted autonomous cleanup jobs, in which agents find redundant documentation and remove it before another agent can read it.
The last one is the clearest sign the idea landed. Once teams saw stale context as a production risk, they put agents to work keeping it clean.
How I'll roll it out on my next team
- 1Define the numbers that drive decisionsStart with the metrics leadership reviews every week. Give each one a spec and an owner, and point every agent at it.
- 2Give knowledge a homeStand up the company, org, team, and individual levels, and have them link to the definitions instead of restating them.
- 3Turn proven work into shared SkillsCatalog the Skills people already rely on, give each an owner, and test them before they're shared.
- 4Automate the upkeepAdd the checks and cleanup agents that keep everything true without a standing meeting.
Each step pays off on its own, and each makes the next one easier.
About the figures
The paper and the repositories it led to are internal to Meta, so every figure here was built for this page. The revenue example is computed from a synthetic quarter of a few thousand ad accounts, and the three answers differ only in which definition each agent found.