The Agentic AI Analysis console validated in the study.
How a moderated usability and pilot-validation study proved a UN-grounded AI agent could cut research time in half — and revealed why, in high-stakes institutions, trust is the real deliverable.
UN offices on the front lines of peace, security and political affairs were synthesizing decades of documentation across six to seven sources per question. Transcend built an AI agent grounded in official UN documentation; I designed and ran the validation study that decided whether senior practitioners would actually trust it.
When a political-affairs officer writes a one-page briefing on a peace process, decades of resolutions, envoy reports, press statements and field consultations sit scattered across disconnected sources — each demanding its own search, its own verification, its own judgment call about what to trust. Coordination and validation consume hours where every hour, and every error, carries consequence.
Answering a single question meant stitching together six or seven separate systems by hand.
In institutions where mistakes are measured in consequence, general-purpose AI tools were unsuitable for sensitive work.
Practitioners carried over distrust from consumer LLMs — they would not act on an answer they could not trace.
Transcend piloted the agent with 15 users, then ran a moderated study with a subset of senior UN professionals across political affairs, mediation and knowledge management. The scenarios came straight from real practice.
The aim was never simply to confirm the platform functioned. It was to determine whether it earned the confidence of the practitioners whose judgment it exists to support — the only test that matters in a setting like this.
Grounding the system in official documentation, paired with visible in-text citations, moved participants from reflexive skepticism toward conditional confidence. They trusted this system in a way they did not trust general-purpose tools for institutional work.
The Agentic AI Analysis console validated in the study.
Two scores came in just under the 4.0 target — and neither reflected a weakness in the agent. Both traced to one addressable cause: the pilot corpus surfaced too few sources per answer, and too narrow a range. Participants said, without prompting, that they would extend more trust the moment that changed.
Where citation depth was thin, even accurate answers drew scrutiny. Where the system was honest about the limits of its own coverage, users responded with greater — not lesser — confidence.— Synthesis across sessions
Surface how many sources support a claim and how diverse they are; make tracing an answer back to its logic effortless. Credibility is a design problem, not a model problem.
60% never noticed the data-filtering controls — they defaulted to refining answers through the prompt box, the behavior consumer LLMs trained into them. Controls that fight that instinct simply go unused.— Behavioral finding
Stop fighting the prompt-first instinct. Solve data transparency and source selection inside the interaction model people already have, not beside it.
The pilot met its feasibility threshold and advanced into a second phase with an evidence-led roadmap: surface source coverage explicitly, deepen citations, align the interaction model with how people actually work, expand the corpus's recency, and scale to support up to 300 users. The UN's willingness to pilot this makes it a first mover among multilateral institutions — and a model for how AI can responsibly support diplomatic and peacebuilding work.
Make the breadth and depth of evidence behind every answer visible up front.
Reshape the interaction model around how practitioners actually research and write.
Expand from a 15-person pilot to institutional deployment, corpus recency included.
Design grounded in domain expertise, validated with the experts doing the real work, and engineered so that trust, traceability and human judgment stay at the center of the build — not bolted on after the fact. In environments that cannot afford to be wrong, that method is the product.
I help teams validate high-stakes AI with the experts who'll actually use it — so trust is designed in, not assumed.