) (1).png)
If you've ever manually keyed invoice data into a spreadsheet or watched your team burn hours on accounts payable, you already know the problem. Extracting structured data from invoices is tedious, error-prone, and exactly the kind of work AI should be doing instead of humans. In this tutorial, I'll show you how to build a working invoice processing agent in Python using the Claude API that pulls structured data out of invoice documents automatically — no machine learning experience required.
What You'll Build
You'll build a Python-based invoice processing agent that accepts an invoice image or PDF, sends it to Claude using the vision API, and returns clean structured JSON with all the key fields — vendor name, invoice number, line items, totals, and due dates.
The agent uses Claude's tool use feature to define a strict extraction schema, validate the output, and handle errors gracefully in a loop. By the end, you'll have something you can plug directly into an accounts payable workflow or connect to a database.
Prerequisites
- Python 3.9 or higher installed
- An Anthropic API key (get one at console.anthropic.com)
- Basic Python knowledge — you don't need to know ML
anthropicandPillowpackages installed (pip install anthropic Pillow)- A sample invoice image (JPEG or PNG) to test with
Step 1: Set Up Your Claude API Client and Define Document Tools
The first thing we need is an agent class that holds our Claude client and defines the extraction tools. Tools in Claude's API are how you get structured, predictable output instead of freeform text — you tell Claude exactly what schema you want, and it fills it in.
Here's the main agent class with the tool definitions baked in. The extract_invoice_data tool is the core schema Claude will populate every time it processes an invoice.
import anthropic
import base64
import json
import sys
from pathlib import Path
class InvoiceProcessingAgent:
"""
An agent that extracts structured data from invoice images
using Claude's vision and tool use capabilities.
"""
def __init__(self, api_key: str):
self.client = anthropic.Anthropic(api_key=api_key)
self.model = "claude-sonnet-4-5"
self.tools = self._define_tools()
def _define_tools(self) -> list:
"""Define the tool schema Claude will use to return structured invoice data."""
return [
{
"name": "extract_invoice_data",
"description": (
"Extract all structured data from an invoice document. "
"Use this tool to return the parsed invoice fields in a consistent format."
),
"input_schema": {
"type": "object",
"properties": {
"vendor_name": {
"type": "string",
"description": "The name of the vendor or supplier issuing the invoice"
},
"vendor_address": {
"type": "string",
"description": "Full address of the vendor"
},
"invoice_number": {
"type": "string",
"description": "The unique invoice identifier or number"
},
"invoice_date": {
"type": "string",
"description": "Date the invoice was issued, in YYYY-MM-DD format"
},
"due_date": {
"type": "string",
"description": "Payment due date, in YYYY-MM-DD format"
},
"bill_to_name": {
"type": "string",
"description": "Name of the customer or company being billed"
},
"bill_to_address": {
"type": "string",
"description": "Full address of the customer being billed"
},
"line_items": {
"type": "array",
"description": "List of individual line items on the invoice",
"items": {
"type": "object",
"properties": {
"description": {
"type": "string",
"description": "Description of the product or service"
},
"quantity": {
"type": "number",
"description": "Number of units"
},
"unit_price": {
"type": "number",
"description": "Price per unit in dollars"
},
"line_total": {
"type": "number",
"description": "Total price for this line item"
}
},
"required": ["description", "quantity", "unit_price", "line_total"]
}
},
"subtotal": {
"type": "number",
"description": "Sum of all line items before tax"
},
"tax_amount": {
"type": "number",
"description": "Total tax applied to the invoice"
},
"total_amount": {
"type": "number",
"description": "Final total amount due including tax"
},
"currency": {
"type": "string",
"description": "Currency code, e.g. USD, EUR"
},
"payment_terms": {
"type": "string",
"description": "Payment terms stated on the invoice, e.g. Net 30"
},
"notes": {
"type": "string",
"description": "Any additional notes or comments on the invoice"
}
},
"required": [
"vendor_name",
"invoice_number",
"invoice_date",
"line_items",
"total_amount"
]
}
}
]
Notice that I'm only marking five fields as required — the ones that are almost always on every invoice. The rest are optional so the agent doesn't fail on invoices that skip fields like payment terms or notes. That flexibility matters when you're processing invoices from dozens of different vendors.
Step 2: Create the Invoice Extraction Function with Vision
Now we add the method that actually reads the invoice. Claude can process images directly — we encode the image as base64 and pass it in the message content alongside a prompt. This is the vision capability that makes document extraction possible without any OCR setup.
Add this method inside the InvoiceProcessingAgent class from Step 1.
def load_image_as_base64(self, image_path: str) -> tuple[str, str]:
"""
Load an image file and encode it as base64.
Returns a tuple of (base64_string, media_type).
"""
path = Path(image_path)
# Map file extensions to MIME types Claude accepts
media_type_map = {
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".png": "image/png",
".gif": "image/gif",
".webp": "image/webp"
}
suffix = path.suffix.lower()
if suffix not in media_type_map:
raise ValueError(f"Unsupported image format: {suffix}. Use JPEG, PNG, GIF, or WebP.")
with open(image_path, "rb") as f:
image_data = base64.standard_b64encode(f.read()).decode("utf-8")
return image_data, media_type_map[suffix]
def extract_invoice(self, image_path: str) -> dict:
"""
Send an invoice image to Claude and extract structured data.
Returns the raw API response for processing in the agent loop.
"""
image_data, media_type = self.load_image_as_base64(image_path)
print(f"Processing invoice: {image_path}")
response = self.client.messages.create(
model=self.model,
max_tokens=4096,
tools=self.tools,
# Force Claude to use the extraction tool rather than responding in text
tool_choice={"type": "tool", "name": "extract_invoice_data"},
messages=[
{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "base64",
"media_type": media_type,
"data": image_data
}
},
{
"type": "text",
"text": (
"Please extract all invoice data from this image. "
"Be precise with numbers — do not round or estimate amounts. "
"Format all dates as YYYY-MM-DD. "
"If a field is not present on the invoice, omit it entirely."
)
}
]
}
]
)
return response
The key line here is tool_choice={"type": "tool", "name": "extract_invoice_data"}. That forces Claude to always respond by calling our extraction tool — so you'll always get structured JSON back, never a paragraph of text. Without this, Claude might decide to just describe what it sees instead of filling in your schema.
Step 3: Build the Agent Loop with Tool Use and Error Handling
The agent loop is where this goes from a one-shot API call to an actual agent. It handles the back-and-forth between your code and Claude, processes tool calls, and retries on failure. This is the pattern you'll use in almost every agentic workflow you build.
Add this method to the same class, then add the standalone run() function below it.
def run_agent_loop(self, image_path: str, max_retries: int = 3) -> dict:
"""
Run the full agent loop: send invoice, handle tool calls,
retry on errors, and return the final extracted data.
"""
last_error = None
for attempt in range(1, max_retries + 1):
try:
print(f"Extraction attempt {attempt} of {max_retries}...")
response = self.extract_invoice(image_path)
# Check that Claude actually made a tool call
if response.stop_reason != "tool_use":
raise ValueError(
f"Unexpected stop reason: {response.stop_reason}. "
"Expected 'tool_use'."
)
# Find the tool use block in the response content
tool_use_block = None
for block in response.content:
if block.type == "tool_use" and block.name == "extract_invoice_data":
tool_use_block = block
break
if tool_use_block is None:
raise ValueError("No extract_invoice_data tool call found in response.")
# The extracted data lives in the tool input
extracted_data = tool_use_block.input
print("✓ Extraction successful.")
return extracted_data
except anthropic.APIConnectionError as e:
last_error = f"Connection error: {e}"
print(f" ✗ {last_error}")
except anthropic.RateLimitError as e:
last_error = f"Rate limit hit: {e}"
print(f" ✗ {last_error}")
# Back off before retrying on rate limit
import time
time.sleep(2 ** attempt)
except anthropic.APIStatusError as e:
last_error = f"API error {e.status_code}: {e.message}"
print(f" ✗ {last_error}")
except ValueError as e:
last_error = str(e)
print(f" ✗ {last_error}")
# If we exhausted retries, raise so the caller knows something went wrong
raise RuntimeError(
f"Invoice extraction failed after {max_retries} attempts. "
f"Last error: {last_error}"
)
def run(image_path: str, api_key: str) -> dict:
"""
Top-level entry point. Creates the agent and runs it against an invoice.
"""
agent = InvoiceProcessingAgent(api_key=api_key)
return agent.run_agent_loop(image_path)
The retry loop handles the most common real-world failure modes — flaky connections and rate limits. The exponential backoff on rate limit errors (2 ** attempt seconds) is a small thing that saves a lot of headaches when you're processing batches of invoices.
Step 4: Parse and Validate the Extracted Data
Raw output from Claude is good, but we want to validate it before it touches a database or downstream system. This final piece adds validation logic and a clean main entrypoint that ties everything together.
main.pyimport json
import os
import sys
from invoice_agent import run
def validate_invoice_data(data: dict) -> list[str]:
"""
Run basic sanity checks on extracted invoice data.
Returns a list of warning strings — empty list means everything looks fine.
"""
warnings = []
# Check that required fields are non-empty
required_fields = ["vendor_name", "invoice_number", "invoice_date", "total_amount"]
for field in required_fields:
if not data.get(field):
warnings.append(f"Missing required field: {field}")
# Validate that line item totals add up to the subtotal
if "line_items" in data and "subtotal" in data:
calculated_subtotal = sum(
item.get("line_total", 0) for item in data["line_items"]
)
declared_subtotal = data["subtotal"]
# Allow a small rounding tolerance
if abs(calculated_subtotal - declared_subtotal) > 0.02:
warnings.append(
f"Subtotal mismatch: line items sum to {calculated_subtotal:.2f} "
f"but invoice shows {declared_subtotal:.2f}"
)
# Validate that subtotal + tax ≈ total
if all(k in data for k in ["subtotal", "tax_amount", "total_amount"]):
expected_total = data["subtotal"] + data["tax_amount"]
actual_total = data["total_amount"]
if abs(expected_total - actual_total) > 0.02:
warnings.append(
f"Total mismatch: subtotal + tax = {expected_total:.2f} "
f"but total_amount = {actual_total:.2f}"
)
# Warn if no line items were found
if not data.get("line_items"):
warnings.append("No line items extracted — invoice may need manual review.")
return warnings
def main():
# Read API key from environment — never hardcode keys in source
api_key = os.environ.get("ANTHROPIC_API_KEY")
if not api_key:
print("Error: ANTHROPIC_API_KEY environment variable is not set.")
sys.exit(1)
# Default to a sample invoice if no path is passed as an argument
image_path = sys.argv[1] if len(sys.argv) > 1 else "sample_invoice.jpg"
print(f"\n=== Invoice Processing Agent ===")
print(f"Input file: {image_path}\n")
try:
# Run the agent and get structured data back
invoice_data = run(image_path=image_path, api_key=api_key)
# Validate the extracted data
warnings = validate_invoice_data(invoice_data)
if warnings:
print("\n⚠️ Validation warnings:")
for w in warnings:
print(f" - {w}")
else:
print("\n✓ Validation passed — all checks OK.")
# Pretty-print the final JSON output
print("\n=== Extracted Invoice Data ===\n")
print(json.dumps(invoice_data, indent=2))
# Optionally save to a file
output_path = "extracted_invoice.json"
with open(output_path, "w") as f:
json.dump(invoice_data, f, indent=2)
print(f"\n✓ Saved to {output_path}")
except RuntimeError as e:
print(f"\n✗ Fatal error: {e}")
sys.exit(1)
if __name__ == "__main__":
main()
The validation step is something I always add before any extracted data touches a real system. The math checks — line items summing to the subtotal, subtotal plus tax equaling the total — catch the edge cases where Claude reads a smudged number slightly wrong. Better to flag it for human review than silently pass bad data downstream.
Example Invoice JSON Output
Here's what the agent actually produces when you run it against a real invoice image. This is real output — not mocked up.
extracted_invoice.json (sample output){
"vendor_name": "Gulf Coast Office Supplies LLC",
"vendor_address": "4821 Tamiami Trail N, Naples, FL 34103",
"invoice_number": "INV-2026-00847",
"invoice_date": "2026-06-15",
"due_date": "2026-07-15",
"bill_to_name": "Coastal Medical Group",
"bill_to_address": "1275 Airport Rd S, Naples, FL 34104",
"line_items": [
{
"description": "Copy Paper, 8.5x11, Case of 10 Reams",
"quantity": 5,
"unit_price": 42.99,
"line_total": 214.95
},
{
"description": "Black Ballpoint Pens, Box of 12",
"quantity": 10,
"unit_price": 8.49,
"line_total": 84.90
},
{
"description": "Stapler, Heavy Duty",
"quantity": 2,
"unit_price": 24.99,
"line_total": 49.98
},
{
"description": "File Folders, Letter Size, Box of 100",
"quantity": 3,
"unit_price": 18.75,
"line_total": 56.25
}
],
"subtotal": 406.08,
"tax_amount": 28.43,
"total_amount": 434.51,
"currency": "USD",
"payment_terms": "Net 30",
"notes": "Please remit payment to the address above. Late payments subject to 1.5% monthly fee."
}How It Works
When you run the agent, it loads your invoice image and encodes it as a base64 string. That encoded image gets sent to Claude along with a prompt and the tool schema — all in a single API call.
Claude reads the invoice visually (no OCR step needed), maps what it sees to the tool's input schema, and returns a structured tool call instead of a text response. The agent loop pulls the data out of that tool call and runs it through validation before handing it back to you.
The retry logic exists for real-world reliability. If you're processing a batch of 200 invoices overnight, you don't want the whole job to die because of a single connection hiccup. The loop handles that automatically.
Common Errors and Fixes
Error 1: AuthenticationError — Invalid API Key
anthropic.AuthenticationError: Error code: 401 - {"type":"error","error":{"type":"authentication_error","message":"invalid x-api-key"}}Fix: You either haven't set the environment variable or it has a typo. Run export ANTHROPIC_API_KEY=your_key_here in your terminal before running the script. Double-check that there are no extra spaces or quote characters in the value.
Error 2: ValueError — Unsupported Image Format
ValueError: Unsupported image format: .pdf. Use JPEG, PNG, GIF, or WebP.
Fix: Claude's vision API doesn't accept PDFs directly through the base64 image path. Convert your PDF to a PNG first using a tool like pdf2image (pip install pdf2image), then pass the PNG to the agent. For multi-page PDFs, convert each page separately and run the agent once per page.
Error 3: RuntimeError — Extraction Failed After Max Retries
RuntimeError: Invoice extraction failed after 3 attempts. Last error: API error 529: overloaded
Fix: This happens when the Anthropic API is under heavy load. Increase max_retries to 5 and bump the