nest.codes blog
Day 2 · · 4 min read

Azure Container Apps Serverless GPU: Real T4 ASR Cost

$0.263 an hour. That was the serverless T4 line on the Azure Container Apps price sheet, and my AI coding assistant turned it into "about three cents per hour of audio." This post answers a simple question: what does it actually cost to run an open speech-recognition model on Container Apps serverless GPU? Short answer: the vCPU and RAM charges outweighed the GPU itself, and once we right-sized the job it landed at roughly $0.78 an hour.

Why we put a GPU on Container Apps

Our speech transcription currently runs through a managed API, MAI-Transcribe-2. We wanted to transcribe the same audio with an open model as well, so we could compare quality and cost side by side. The models were Qwen3-ASR 1.7B plus the 0.6B forced aligner for timestamps.

Left to ourselves, we would have dropped this onto the Azure ML low-priority cluster we already run. The milestone period (rules in Day 0) gave us a reason to try a GPU we don't have to provision as a VM. Honestly, it's not a service we would have picked first.

Attach a serverless GPU workload profile to a Consumption environment, and a single T4 is allocated only while a Container Apps Job is running. In eastus it attached immediately, with no quota request. For what it's worth, the Korea regions don't offer Container Apps GPUs yet.

It wasn't $0.263 an hour

The first run was a single audio file of a little over ten minutes. It worked, but most of the wall-clock time was the image pull: 622 seconds to pull an 8.9 GB image with the models baked in, against 66 seconds of actual transcription.

Looking at the numbers, I asked: "This is on-demand. Why is it so cheap?" It wasn't. Serverless GPU bills GPU-seconds plus vCPU-seconds plus RAM GiB-seconds. We had launched at the profile's maximum, 8 vCPU and 56 GiB, which comes to about $1.56 an hour. That's roughly twice an on-demand T4 VM (about $0.75).

Transcription keeps the GPU busy and leaves the CPU mostly idle, so we dropped to 4 vCPU and 16 GiB. Speed didn't change, and the price halved to about $0.78 an hour. The size in a serverless profile's name is only a ceiling. You have to pick CPU and RAM per job yourself.

How much of the GPU were we using?

The next question was whether we were using 100% of the GPU. We hadn't measured, so we didn't know. We made the job log utilization every five seconds and changed three things:

We reran on a batch of audio files totalling about 66 minutes. After warm-up, GPU utilization was 72–94% per file and 77% on average. Post-warm-up throughput was around 45x realtime, which works out to about $0.017 per hour of audio. That's roughly a sixth of MAI's current promotional rate ($0.10 per hour). On this sample, though, MAI still produced better transcripts, especially word spacing.

The remaining bottleneck is startup. Every run pulls the image fresh, which costs about eight minutes. So instead of launching many short runs, we batch once a day. A scheduled Azure Function starts the job daily, and the job stops after 88 minutes, finishing whatever file it's on. In effect, the cost cap is enforced as a time cap. Next up is moving inference to vLLM to squeeze more throughput out of the T4's VRAM.

What I think after using it

Renting a GPU by the second was honestly better than I expected. No node pools, no driver installs, no shutdown scripts. On the other hand, it isn't cheap. It's on-demand with no eviction risk, so it costs several times what a low-priority T4 does (a little over $0.1 an hour). Cold starts with a large image still hurt.

My verdict: it fits batch GPU work that runs an hour or two a day well. For inference that runs 24/7, I still think VMs or a low-priority cluster win.

This job now runs on its own in Azure, and I see the results in a report email. It's convenient, but not knowing whether it actually ran until the mail arrives still makes me a little nervous.

When you read a serverless GPU price sheet, don't stop at the GPU line. Add GPU, vCPU, and RAM together.

Also today

#azure#azure-container-apps#serverless-gpu#speech-to-text#azure-functions
All posts in this series
  1. Prologue · Azure Credits via NVIDIA Inception: Why We Moved
  2. Day 1 · Azure PostgreSQL Migration: Quota Isn't SKU Availability
  3. Day 2 · Azure Container Apps Serverless GPU: Real T4 ASR Cost
  4. Day 3 · Cosmos DB 429 at 1% Average RU: Retry, Don't Scale
  5. Day 4 · Cosmos DB for Transcript JSON: One Container, Not Two
  6. Day 5 · Container Apps job cold start: 8 min to 1.8 with Blob
  7. Days 6–7 · Azure service usage audit: proving each one does real work