Azure service usage audit: proving each one does real work
In the Well-Architected Framework, a "workload" is your whole solution; in the startup program docs, it is a single Azure service you actively use. Chasing that difference, I asked a blunt question of our own setup: is every service we added actually doing work, or just switched on? The metrics said they all work—even a load balancer that is almost silent turned out to be insurance that earns its keep during failures.
Same word, why a different unit?
The milestone asks a startup to keep using several services, for real, for 60 days (Day 0). To check we were honoring the "for real" part, I had a Claude Code subagent compare our setup against the Well-Architected docs.
The two documents use the same word at different scales. Well-Architected calls a workload the collection of resources, code, models and data that together deliver a business outcome. By that measure our solution is roughly media processing, speech transcription, search, web and GPU inference.
The startup docs count each unique service you use in your solution. So we decided to use Well-Architected as a mirror rather than a counting rule: does each service do what its service guide says it is for?
The reviewer found most services matched their guide and flagged a few. One was keeping a low-priority GPU node always at one instance. Another was API Management on the Developer tier. The third was a load balancer with almost no traffic. The GPU node was settled quickly. This subscription can't get dedicated GPU quota, so a multi-region low-priority cluster backing always-on inference is our actual design. For the rest, we went to the numbers.
Did the metrics show anything that was just switched on?
Instead of answering from gut feeling, we pulled two days of metrics.
- API Management: 4,741 requests one day, 5,266 the next. Transcript correction, admin analysis and search embeddings all go through the gateway.
- AI Search: 1,992 queries, then 148. The first day was high because we were filling an index.
- Container Registry: 18–222 image pulls a day. Billing is a tier fee plus storage, so pulls don't add cost.
- Front Door: 1,000–6,500 requests a day, covering admin pages, a report page I check daily, and this blog.
The only clear outlier was the GPU-side load balancer: a few MB a day, roughly health-probe traffic. My first reaction was "maybe this should go." Then I looked again at what it is for, and quiet turned out to be correct.
- If the Redis-based routing is unavailable and we fall back to the app's own routing, it still spreads requests across two nodes in the same region. Without it, one node takes everything and queues grow.
- If the backend fails to connect directly to the node it picked, the request is retried once through the load balancer as a backup path.
The lesson for me: don't judge a failover service by its everyday traffic. We wrote its role into our solution doc and kept it.
Why an API Management gateway, and where did search costs come from?
Honestly, I thought API Management was overkill for a team our size. Putting it in front of Azure OpenAI during the milestone period changed my mind.
Two reasons. First, model keys are no longer scattered across job code and machines; callers only know the gateway key. Second, a daily token cap (1 million tokens) stops runaways. We once had a correction job loop on the same work for hours because of a bug; now that kind of thing stops at the gateway.
If the gateway itself goes down, work stops, so on 5xx, 401 or a dropped connection we retry once against the direct endpoint. A quota rejection (403) stops on purpose. The downsides: the guide says Developer tier isn't meant for production, and provisioning took a long time.
Search uses Cohere embeddings (embed-v-4-0) from Foundry to build vectors for the AI Search index, then reorders query results with the Cohere reranker (rerank v5 pro). Both are sold directly by Azure, so they bill against credits. That is also when we made a rule: no Marketplace serverless deployments, because those don't draw on credits.
We call the embedding model from our own code instead of AI Search's built-in vectorizer link, mainly to keep the billing path certain. The trade-off was cost. Of roughly $6 in Foundry model spend on Nov 19, reranking and embeddings took more than half. That came from filling the index for the first time. Once we're back to 10–20 searches a day it should shrink (an estimate).
Why didn't we follow the Cosmos DB autoscale advice?
Both the reviewer and Advisor recommended Cosmos DB autoscale. We kept fixed 600 RU/s with 429 retries; the retry design is in Day 3.
The numbers decided it. We see about 70 429s a day, and retries finish all of them. Autoscale bills each hour at that hour's peak, and in the last 72 hours, 66 briefly hit 600. Switching could have doubled or tripled the daily cost (an estimate). The gain would only be "a bit faster during bursts."
Correction: the earlier post's ~$33 a day was a "normal day" estimate excluding one-off jobs. Our recent actual billed average is about $45, which includes new fixed costs like the gateway and one-off usage like the index fill.
One feeling from this stretch: having moved the work I used to conduct from Claude windows on my PC into scheduled Functions, metrics like these are now my only eyes. It's convenient, but because I can't watch it run, "it's probably fine" without a metric makes me uneasy. Next step is metric alerts on Cosmos 429s, gateway 5xx and the log cap.
The checklist we used to find services that are merely switched on:
- Pull two days of one "unit of work" metric per service (requests, queries, pulls).
- Separately check whether the bill is a fixed fee or usage-based.
- If traffic is low, ask what the service does during a failure first.
- Follow advice only if errors are actually left unresolved today.
Also today
- We enabled dead-lettering on two Event Grid subscriptions. Events that fail repeatedly now land in Blob storage instead of silently disappearing, which makes debugging easier.
- We asked support one last time about dedicated AML T4 quota for failover. The answer was that new GPU allocations for sponsored subscriptions are hard for now, with Foundry models suggested as the alternative. So GPU work is designed around AML low-priority clusters, Container Apps serverless GPU and Foundry only.
- We capped Log Analytics to stay within the free allowance and attached the GPU cluster's node events and utilization logs instead. Along the way we found most of our log volume was one model library repeating the same warning.