Tracking AI Agent Tool Usage in APIM

You can't manage what you can't see.

When you expose a dozen backend tools to an army of AI agents, you need to know exactly who is calling what. If a rogue agent gets stuck in a loop and hammers your incident management API, you need to spot it instantly.

Azure API Management gives you the telemetry. You just have to extract it from the traffic stream.

Extracting Caller Identity

Before you can emit a metric, you need to know who the caller is.

You start by validating the Entra JWT. Crack the token open at the gateway edge and extract the caller's context—specifically the Object ID (OID) or Application ID. Store these as context variables.

The Inbound Policy Pipeline

Order of operations is everything here.

You cannot emit a metric for an unauthorized call. You cannot rate-limit a user whose token you haven't validated. Your inbound policy pipeline must follow a strict, logical sequence. Validate. Throttle. Record.

<include-fragment fragment-id="validate-entra-token" />
<include-fragment fragment-id="rate-limit-per-subscription" />
<include-fragment fragment-id="quota-per-subscription" />
<include-fragment fragment-id="emit-tool-call-metric" />
 

Dashboarding in Azure Monitor

Once you configure the emit-tool-call-metric fragment, the data flows directly into Azure Monitor.

You now have a custom metric dimensioned by caller ID and tool name. You can build dashboards to track cross-tenant usage, set up alerts for sudden spikes in specific function calls, and definitively prove the ROI of your internal MCP servers.

track tool usage. Add at the end:

What this approach doesn't cover

The pattern above gets you usage counts and basic dimensions — enough for capacity planning, chargeback, and spotting outliers. What it doesn't get you is the audit story your compliance team is going to ask for:

• The full request and response payload for every tool call, not just the count
• Argument-level redaction before the data is logged (regex deny patterns evaluated before the upstream call)
• Per-identity policy decisions logged alongside the call (allowed, denied, why)
• A consistent audit schema across multiple MCP servers, including third-party ones

You can build all of that in APIM policy. I've done it. It's a lot of XML, Liquid templates, and Application Insights queries to maintain — and you'll end up reinventing the same patterns each project.

The MCP Audit & Compliance Gateway does this out of the box as an Azure Managed Application — deploys into your subscription, data stays in your environment, free tier up to 10,000 invocations/month on the Azure Marketplace. If you want metrics, stay in APIM. If you want a full audit trail your auditors will sign off on, that's what it's for.

See the code here https://github.com/jackweldonweb/apim-mcp-terraform, check license for usage information.

Building MCP governance on Azure? The MCP Audit & Compliance Gateway gives you full per-call audit, per-identity policy, and argument masking — deployed as an Azure Managed App into your own subscription. Free tier on the Azure Marketplace.