) (1).png)
What You'll Build
You're going to build a multi-agent quality inspection system that analyzes product images, detects manufacturing defects, and generates structured defect reports — all in about 150 lines of Python. It uses Claude's tool use API to coordinate three specialized agents: one for visual analysis, one for defect classification, and one for report generation.
By the end of this tutorial, you'll have a working system you can point at any product image and get back a JSON defect report with severity ratings, defect locations, and recommended actions. This is the same pattern we use at Naples AI when building computer vision quality control systems for manufacturing clients in Southwest Florida.
📦 Full Source Code
The complete, working code for this tutorial is broken into steps below. Each step builds on the last, so by Step 5 you'll have the entire system running. Copy each block in order, or grab the full file from the final step.
The complete, working code for this tutorial is broken into steps below. Each step builds on the last, so by Step 5 you'll have the entire system running. Copy each block in order, or grab the full file from the final step.
Prerequisites
- Python 3.10 or higher
- An Anthropic API key (get one at console.anthropic.com)
anthropicSDK installed:pip install anthropicPillowfor image processing:pip install Pillow- Basic familiarity with Python classes and dictionaries
- A sample product image (JPG or PNG) to test with
Step 1: Set Up Your Claude API Environment
First, let's make sure your environment is wired up correctly before we write a single line of agent logic. Getting the authentication right upfront saves a lot of headaches later.
Create a .env file in your project root and add your API key. Then install the required packages.
pip install anthropic Pillow python-dotenv
Now let's verify your connection works with a minimal test before building anything else.
test_connection.pyimport os
import anthropic
from dotenv import load_dotenv
load_dotenv()
client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
message = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=64,
messages=[{"role": "user", "content": "Reply with: Connection successful"}]
)
print(message.content[0].text)
# Expected output: Connection successful
If you see "Connection successful" printed to your terminal, you're good to go. If you see an AuthenticationError, double-check that your .env file is in the same directory you're running the script from.
Step 2: Create the Quality Inspection Agent Class
This is the core of the system. The QualityInspectionAgent class holds the Claude client, manages conversation state, and registers the tools that Claude will call during inspection. Think of it as the orchestrator that keeps everything organized.
The key design choice here is that each "agent" is really Claude operating with a specific set of tools and a focused system prompt. This multi-agent pattern lets Claude reason about which tool to call next based on what it's already learned about the product.
quality_inspector.pyimport os
import json
import base64
import anthropic
from pathlib import Path
from dotenv import load_dotenv
load_dotenv()
class QualityInspectionAgent:
"""
Multi-agent manufacturing quality inspector using Claude's tool use API.
Coordinates visual analysis, defect detection, and report generation.
"""
def __init__(self):
self.client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
self.model = "claude-sonnet-4-6"
self.inspection_results = {} # Stores intermediate agent outputs
self.tools = self._define_tools()
self.system_prompt = (
"You are a manufacturing quality control inspector AI. "
"When given a product image, you must: "
"1) Call inspect_product_image to perform visual analysis, "
"2) Call analyze_defects with those findings to classify any issues, "
"3) Call generate_report to produce the final structured report. "
"Always complete all three steps in order. Be precise and objective."
)
def _define_tools(self) -> list:
"""Returns the tool schema definitions Claude uses to understand available actions."""
return [
{
"name": "inspect_product_image",
"description": (
"Performs initial visual analysis of a product image. "
"Identifies surface anomalies, structural irregularities, "
"color deviations, and dimensional concerns. "
"Returns a dictionary of raw visual observations."
),
"input_schema": {
"type": "object",
"properties": {
"image_description": {
"type": "string",
"description": "Detailed visual description of what is observed in the image"
},
"anomalies_detected": {
"type": "array",
"items": {"type": "string"},
"description": "List of specific anomalies or irregularities observed"
},
"inspection_zones": {
"type": "object",
"description": "Observations broken down by product zone (top, bottom, left, right, center)",
"properties": {
"top": {"type": "string"},
"bottom": {"type": "string"},
"left": {"type": "string"},
"right": {"type": "string"},
"center": {"type": "string"}
}
}
},
"required": ["image_description", "anomalies_detected", "inspection_zones"]
}
},
{
"name": "analyze_defects",
"description": (
"Takes raw visual observations and classifies each defect by type, "
"severity, and location. Determines whether the product passes or fails inspection. "
"Returns structured defect classifications."
),
"input_schema": {
"type": "object",
"properties": {
"defects": {
"type": "array",
"description": "List of identified defects with classification details",
"items": {
"type": "object",
"properties": {
"defect_id": {"type": "string"},
"defect_type": {
"type": "string",
"enum": ["scratch", "crack", "discoloration", "deformation", "contamination", "missing_component", "dimensional_variance"]
},
"severity": {
"type": "string",
"enum": ["critical", "major", "minor"]
},
"location": {"type": "string"},
"description": {"type": "string"}
},
"required": ["defect_id", "defect_type", "severity", "location", "description"]
}
},
"overall_assessment": {
"type": "string",
"enum": ["pass", "fail", "conditional_pass"]
},
"confidence_score": {
"type": "number",
"description": "Confidence in the overall assessment, 0.0 to 1.0"
}
},
"required": ["defects", "overall_assessment", "confidence_score"]
}
},
{
"name": "generate_report",
"description": (
"Compiles all inspection findings into a final structured quality report. "
"Includes defect summary, pass/fail status, recommended actions, and inspector notes. "
"This is always the last tool called."
),
"input_schema": {
"type": "object",
"properties": {
"product_id": {
"type": "string",
"description": "Identifier for the inspected product"
},
"inspection_summary": {
"type": "string",
"description": "Plain-English summary of inspection findings"
},
"defect_count": {
"type": "object",
"properties": {
"critical": {"type": "integer"},
"major": {"type": "integer"},
"minor": {"type": "integer"}
}
},
"recommended_actions": {
"type": "array",
"items": {"type": "string"},
"description": "Ordered list of recommended next steps"
},
"disposition": {
"type": "string",
"enum": ["approved_for_shipment", "quarantine_for_review", "reject_and_scrap"]
}
},
"required": ["product_id", "inspection_summary", "defect_count", "recommended_actions", "disposition"]
}
}
]
Step 3: Implement the Tool Handler Functions
When Claude decides to call a tool, we need functions that actually execute that logic and return results back to Claude. These handlers store intermediate results so each subsequent agent has access to what the previous one found.
Add these methods to your QualityInspectionAgent class. Each one stores its output in self.inspection_results so the final report has everything it needs.
def _handle_inspect_product_image(self, tool_input: dict) -> str:
"""Processes and stores the initial visual inspection findings."""
self.inspection_results["visual_analysis"] = tool_input
zone_count = len([v for v in tool_input["inspection_zones"].values() if v])
anomaly_count = len(tool_input["anomalies_detected"])
return json.dumps({
"status": "visual_analysis_complete",
"zones_inspected": zone_count,
"anomalies_found": anomaly_count,
"message": f"Identified {anomaly_count} potential anomalies across {zone_count} inspection zones. Proceed to defect analysis."
})
def _handle_analyze_defects(self, tool_input: dict) -> str:
"""Processes defect classifications and stores for report generation."""
self.inspection_results["defect_analysis"] = tool_input
critical = sum(1 for d in tool_input["defects"] if d["severity"] == "critical")
major = sum(1 for d in tool_input["defects"] if d["severity"] == "major")
minor = sum(1 for d in tool_input["defects"] if d["severity"] == "minor")
return json.dumps({
"status": "defect_analysis_complete",
"total_defects": len(tool_input["defects"]),
"breakdown": {"critical": critical, "major": major, "minor": minor},
"overall_assessment": tool_input["overall_assessment"],
"confidence": tool_input["confidence_score"],
"message": "Defect analysis complete. Generate the final inspection report."
})
def _handle_generate_report(self, tool_input: dict) -> str:
"""Compiles the final report and merges all agent findings."""
visual = self.inspection_results.get("visual_analysis", {})
defects = self.inspection_results.get("defect_analysis", {})
# Build the complete report by merging all agent outputs
final_report = {
"report_metadata": {
"product_id": tool_input["product_id"],
"disposition": tool_input["disposition"],
"confidence_score": defects.get("confidence_score", 0.0)
},
"inspection_summary": tool_input["inspection_summary"],
"defects_found": defects.get("defects", []),
"defect_count": tool_input["defect_count"],
"inspection_zones": visual.get("inspection_zones", {}),
"recommended_actions": tool_input["recommended_actions"]
}
self.inspection_results["final_report"] = final_report
return json.dumps({"status": "report_generated", "report": final_report})
def _process_tool_call(self, tool_name: str, tool_input: dict) -> str:
"""Routes tool calls to the correct handler function."""
handlers = {
"inspect_product_image": self._handle_inspect_product_image,
"analyze_defects": self._handle_analyze_defects,
"generate_report": self._handle_generate_report
}
handler = handlers.get(tool_name)
if not handler:
return json.dumps({"error": f"Unknown tool: {tool_name}"})
return handler(tool_input)
Step 4: Implement the Agent Loop with Tool Use
This is where the multi-agent magic happens. The loop keeps running until Claude has called all three tools and returned a final response. Claude decides the order and what to pass between tools — we just keep feeding it results until it's done.
Add the run_inspection method to your class. It handles the full agentic loop, including the image encoding.
def run_inspection(self, image_path: str, product_id: str = "PROD-001") -> dict:
"""
Runs the full multi-agent inspection pipeline on a product image.
Returns the complete structured defect report.
"""
# Encode the image for the Claude API vision call
image_data = self._encode_image(image_path)
image_extension = Path(image_path).suffix.lower().replace(".", "")
media_type_map = {"jpg": "image/jpeg", "jpeg": "image/jpeg", "png": "image/png", "webp": "image/webp"}
media_type = media_type_map.get(image_extension, "image/jpeg")
# Initial message includes the image and a prompt to start inspection
messages = [
{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "base64",
"media_type": media_type,
"data": image_data
}
},
{
"type": "text",
"text": (
f"Please inspect this product image for quality control. "
f"Product ID: {product_id}. "
f"Use all three inspection tools in sequence to complete a full quality report."
)
}
]
}
]
print(f"\n🔍 Starting quality inspection for product: {product_id}")
print("━" * 50)
# Agentic loop — continues until Claude stops requesting tool calls
while True:
response = self.client.messages.create(
model=self.model,
max_tokens=4096,
system=self.system_prompt,
tools=self.tools,
messages=messages
)
print(f"Agent status: {response.stop_reason}")
# If Claude is done (no more tool calls), break the loop
if response.stop_reason == "end_turn":
print("✅ Inspection pipeline complete.")
break
# Process each tool call Claude requested
tool_results = []
for block in response.content:
if block.type == "tool_use":
print(f" → Calling tool: {block.name}")
result = self._process_tool_call(block.name, block.input)
tool_results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": result
})
# Append Claude's response and the tool results to maintain conversation context
messages.append({"role": "assistant", "content": response.content})
messages.append({"role": "user", "content": tool_results})
return self.inspection_results.get("final_report", {})
Step 5: Add Image Processing and the Main Runner
Last piece — we need the image encoding utility and the main() function that ties it all together. The encoding converts your image file to base64 so Claude's vision API can read it.
Add the encoder method to the class, then add the standalone main() block at the bottom of the file.
def _encode_image(self, image_path: str) -> str:
"""Reads an image file and returns base64-encoded string for the API."""
with open(image_path, "rb") as image_file:
return base64.standard_b64encode(image_file.read()).decode("utf-8")
# ── Entry point ──────────────────────────────────────────────────────────────
def main():
import sys
# Accept image path as a command-line argument or use a default test image
image_path = sys.argv[1] if len(sys.argv) > 1 else "test_product.jpg"
product_id = sys.argv[2] if len(sys.argv) > 2 else "PROD-2026-0710"
if not Path(image_path).exists():
print(f"Error: Image file not found at '{image_path}'")
print("Usage: python quality_inspector.py [product_id]")
sys.exit(1)
agent = QualityInspectionAgent()
report = agent.run_inspection(image_path, product_id)
print("\n📋 FINAL INSPECTION REPORT")
print("━" * 50)
print(json.dumps(report, indent=2))
# Save report to file
output_path = f"inspection_report_{product_id}.json"
with open(output_path, "w") as f:
json.dump(report, f, indent=2)
print(f"\n💾 Report saved to: {output_path}")
if __name__ == "__main__":
main()
Run it from your terminal like this:
terminalpython quality_inspector.py my_product.jpg PROD-2026-0710
Example Inspection Output
Here's what a real inspection run looks like in your terminal, followed by the JSON report that gets saved to disk. The agent works through all three tools sequentially before finalizing the report.
terminal output🔍 Starting quality inspection for product: PROD-2026-0710 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Agent status: tool_use → Calling tool: inspect_product_image Agent status: tool_use → Calling tool: analyze_defects Agent status: tool_use → Calling tool: generate_report Agent status: end_turn ✅ Inspection pipeline complete. 📋 FINAL INSPECTION REPORT ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
{
"report_metadata": {
"product_id": "PROD-2026-0710",
"disposition": "quarantine_for_review",
"confidence_score": 0.91
},
"inspection_summary": "Product shows two surface defects on the upper-right quadrant. A hairline crack approximately 3mm in length was detected near the mounting point, classified as major severity. A minor discoloration patch (approx. 8mm diameter) was found on the top surface, likely due to heat exposure during assembly. No missing components detected. Dimensional variance within acceptable tolerance.",
"defects_found": [
{
"defect_id": "DEF-001",
"defect_type": "crack",
"severity": "major",
"location": "upper-right quadrant, near mounting point",
"description": "Hairline crack approximately 3mm in length. Structural integrity may be compromised under load."
},
{
"defect_id": "DEF-002",
"defect_type": "discoloration",
"severity": "minor",
"location": "top surface center",
"description": "Discoloration patch approximately 8mm in diameter, consistent with heat exposure. No structural impact."
}
],
"defect_count": {
"critical": 0,
"major": 1,
"minor": 1
},
"inspection_zones": {
"top": "Minor discoloration detected, no deformation",
"bottom": "Clean, no anomalies",
"left": "Surface within normal parameters",
"right": "Hairline crack near mounting point",
"center": "Normal surface texture, no contamination"
},
"recommended_actions": [
"Quarantine unit and flag for