About this lab
You are an MLOps engineer at a fast growing online retailer. Every day the support and product teams generate tens of thousands of customer reviews that must be classified by sentiment and topic before they reach the analytics dashboards. Calling Gemini one review at a time through an online endpoint is too slow and too costly at this volume. Your team needs a way to process the entire backlog reliably and cheaply overnight.
In this lab you will build an end to end batch inference pipeline on Vertex AI. You will turn a dataset of customer reviews into a JSONL request file, upload it to Cloud Storage, and submit a Gemini batch prediction job with the Google Gen AI SDK. You will poll the job to completion, read the results back from Cloud Storage, parse the structured output each review produced, and aggregate the sentiment and topic counts. By the end you will know when batch inference beats online prediction and how to run it as a repeatable production job.
Lab details
- Level: Intermediate
- Cloud: GCP
- Domain: AI Engineering
- Duration: 75 minutes
What you will learn
- Explain when batch inference is a better choice than online prediction for Gemini workloads
- Build a JSONL request file that asks Gemini for structured output on each record
- Submit and monitor a Gemini batch prediction job on Vertex AI with the Google Gen AI SDK
- Read batch results back from Cloud Storage and parse the structured responses
- Aggregate model outputs into business ready summary metrics
- Clean up Cloud Storage artifacts to control cost
Concepts covered
- Batch inference
- Structured output
- Cloud Storage I/O
- Job orchestration and polling
- Cost optimization
Technologies used
- Vertex AI Batch Prediction
- Gemini 2.5 Flash
- Google Gen AI SDK (google-genai)
- Google Cloud Storage
- Vertex AI Workbench
Prerequisites
- Basic Python knowledge
- Understanding of JSON
- A Google Cloud account with billing enabled and the Vertex AI API available
Topics
- Gemini
- Vertex AI
- Batch Prediction
- GenAI
- MLOps
- GCP
