CASE STUDY 2 OF 8

Packaging Data Extraction with Multi-Agent AI

Consumer & retail dataAzure AI FoundryAzure OpenAIDocument Intelligence

A global consumer and retail data provider whose product catalogue holds hundreds of millions of products. Working alongside another vendor, our team turns photos of product packaging into validated, structured product records.

Target outcomes
80–90%

cut in analyst coding time per product

20–30×

more products coded per analyst

98%+

accuracy on safety-critical fields

Weeks

to add a new language or market, not months

Challenges

  • Several minutes of manual coding per product; large batches take weeks
  • Each new market needs native-language analysts and months to set up
  • Allergen or nutrition errors carry safety and regulatory risk

Solution

A multi-agent extraction pipeline on Azure AI Foundry, with people reviewing exceptions only.

  • Document Intelligence reads text and nutrition tables from photos taken at several angles
  • A multimodal Azure OpenAI extractor builds the record; an independent verifier re-checks critical fields
  • Taxonomy agent and Azure AI Search map attributes and check the existing catalogue
  • Rules engine validates nutrition arithmetic, allergen consistency and duplicates
  • Low-confidence or safety-critical fields go to reviewers; approved records keep full provenance

Review effort scales with risk

Safety-critical fields get the strictest checks, while descriptive fields are automated more fully.

Built to expand

Language-independent taxonomy mapping and accuracy tracked by field, category and language make a new market a configuration task.

How we deliver it

From Discovery to Handover

0. Discovery & baseline

2–3 weeks

Timed baseline for coding, field risk tiers, gold set design, taxonomy and catalogue integration review

1. MVP (one category, one market)

10–12 weeks

Extract, verify, validate and review pipeline, evaluation harness, product master integration

2. Categories & languages

8 weeks

More categories and languages, taxonomy agent, throughput tuning

3. Scale & client taxonomies

Ongoing

New markets in waves, client-specific taxonomy agents, operations handover

Tech stack

Built on Proven Platforms

  • AI platform: Azure AI Foundry, Azure OpenAI Service (multimodal), Azure AI Document Intelligence
  • Retrieval & validation: Azure AI Search, rules-based validation engine
  • Pipeline & compute: Azure Kubernetes Service, Service Bus, Blob Storage, Cosmos DB
  • Integration: Azure API Management, product master and taxonomy APIs
  • Security: Microsoft Entra ID, Azure Key Vault, private endpoints
  • MLOps & DevOps: Foundry evaluations, gold-set regression tests, GitHub Actions / Azure DevOps
Where this applies. Any organisation that turns physical or visual material into structured data at volume: product labels, specification sheets, invoices, certificates and regulatory documents.

Reference scenario: the client profile is a composite based on conditions common to this kind of organisation. All figures are delivery targets with a defined way to measure them, not results reported by a named client.

Want Results Like These?

Tell us where your data, reporting or AI efforts are stuck and we will map out a practical next step together.