Summarize this article with:
- Label detection automatically identifies and categorizes objects, people, and scenes in videos or images.
- Developers can Detect Labels (Objects) in Videos Using Python using Eden AI's unified REST API with a single API key across all supported providers.
- Eden AI lets you experiment with various AI models before integration, making it easy to compare and choose the provider that best fits your needs.
- Label detection in videos now goes through a multimodal LLM: you send the video URL and get the labels back as JSON in the same response.
- Each label comes with the start and end times where it appears in the video.
Label Detection is a key task in video and image analysis that involves identifying and categorizing elements within each frame such as objects, people, animals, or landmarks.
This helps with content tagging, indexing, and search optimization. In this tutorial, we’ll show you how to implement label detection using Python and the Eden AI API.
With just a few lines of code, you can run label detection and extract meaningful insights from visual content.
What is Label/Object Detection?

Label detection automatically identifies and categorizes objects, people, and scenes in videos or images. It assigns labels like animals, buildings, or landmarks to visual elements, making content easier to search, organize, and analyze.
How to Detect Labels in Videos
Get Access to Eden AI API
1. Sign up: Visit Eden AI and create an account for free to get started. Once registered, navigate to the API section to find your personal API key which gives you access to a wide range of AI services, including Label Detection.

2. Navigate to Video Technologies – Log in and head to the Video Technologies section.
3. Select Label Detection– From there, select the Label Detection option.
Live Test Models on Eden AI
Eden AI lets you experiment with various AI models before integration, making it easy to compare and choose the provider that best fits your needs.
Implementing Label Detection in Python
Step 1: Install Python Requests Module
Before we start, make sure you have the requests module installed. This module allows your Python script to make HTTP requests to APIs like Eden AI.
pip install requests
This module is required to send HTTP requests to the Eden AI API.
Code Explanation
Label detection in videos now goes through a multimodal LLM that accepts video as input, called with Eden AI’s OpenAI-compatible chat completions endpoint. It works in two steps:
- Sending the request: you send the video URL, your instruction and the JSON format you expect for the answer.
- Reading the results: the labels come back in the same response, as JSON, so there is no job to poll.
A 10-second clip is processed in a few seconds. For long videos, set a generous timeout on your HTTP client.
Sending the Label Detection Request
This part of the code sends the video and the instruction to Eden AI.
import requests
# Replace with your actual API Key
headers = {"Authorization": "Bearer {API_KEY}"}
# The JSON format expected for the answer: each label with the time range where it appears
schema = {
"type": "object",
"properties": {
"labels": {
"type": "array",
"items": {
"type": "object",
"properties": {
"label": {"type": "string"},
"start_time": {"type": "number"},
"end_time": {"type": "number"},
},
"required": ["label", "start_time", "end_time"],
},
}
},
"required": ["labels"],
}
# Replace with the actual URL of your video
json_payload = {
"model": "google/gemini-2.5-flash", # Any model that accepts video input
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "List the objects, people and scenes in this video, with start and end times in seconds."},
{"type": "image_url", "image_url": {"url": "🔗 URL of your video"}},
],
}
],
"response_format": {"type": "json_schema", "json_schema": {"name": "labels", "schema": schema}},
}
# Eden AI's OpenAI-compatible chat endpoint
url = "https://api.edenai.run/v3/chat/completions"
# Send the POST request
response = requests.post(url, json=json_payload, headers=headers)
# Get the response and display it
result = response.json()
print(result)
Explanation:
- headers: Contains your API key to authenticate with Eden AI.
- url: Eden AI’s chat completions endpoint, https://api.edenai.run/v3/chat/completions.
- json_payload:
- model: The multimodal LLM to use, in the provider/model format. It must accept video: check for video in input_modalities at https://api.edenai.run/v3/models (for example google/gemini-2.5-flash or amazon/amazon.nova-2-lite-v1:0).
- messages: Your instruction in a text block, and the public URL of your video in an image_url block.
- response_format: The JSON schema the answer must follow, here a list of labels with start and end times.
- model: The multimodal LLM to use, in the provider/model format. It must accept video: check for video in input_modalities at https://api.edenai.run/v3/models (for example google/gemini-2.5-flash or amazon/amazon.nova-2-lite-v1:0).
- response: This POST request sends the video and your instruction to Eden AI.
- result: The full response. The answer is in choices[0].message.content, the cost of the request in cost and the provider that served it in provider.
Reading the Results
The labels are in the same response. This part of the code reads them from the result of the previous step.
import json
# The labels are in the model's answer, as a JSON string
labels = json.loads(result["choices"][0]["message"]["content"])["labels"]
for item in labels:
print(f'{item["label"]}: {item["start_time"]}s to {item["end_time"]}s')
print("Cost:", result["cost"], "USD")
Explanation:
- json.loads: The answer is a JSON string that follows your schema, so it is turned into a Python dictionary.
- The loop prints each label with the time range where it appears, then the cost of the request.
Interpreting the Results
Here’s an example of the model’s answer, read from choices[0].message.content:
{
"labels": [
{"label": "Snowy mountain landscape", "start_time": 0.0, "end_time": 9.0},
{"label": "Red-haired character", "start_time": 0.0, "end_time": 9.0},
{"label": "Large muscular character", "start_time": 1.0, "end_time": 9.0},
{"label": "Man with a knife", "start_time": 6.0, "end_time": 7.0}
]
}
Explanation of Output Fields:
- label: Name of the detected object, person or scene.
- start_time / end_time: When the label appears in the video, in seconds.
The timestamps are estimated by the model: check them on a few of your own videos before relying on them in production.
Managing Your Video Files
To analyze a video stored on your machine, upload it first with POST https://api.edenai.run/v3/upload, then send a file block with the returned file_id instead of the image_url block. Eden AI provides additional optional endpoints to manage these files:
Video Files - List Files
GET Request:
https://api.edenai.run/v3/upload
This lists the files you uploaded, with their file_id and expiration date.
Video Files - Delete Files
POST Request:
https://api.edenai.run/v3/upload/delete
Send the list of file_ids you no longer need in the file_ids field to delete them.
Why Eden AI is the Best Tool for Label Detection
Eden AI offers a robust Label Detection solution with several advantages.

Multiple Provider Options
Eden AI lets you compare multimodal models from providers like Google and Amazon by changing a single parameter, model.
Ease of Integration
Eden AI simplifies the integration of label detection into your applications with clear and straightforward Python code examples.
Security and Trust
The Eden AI platform prioritizes data security and privacy, ensuring that your image data is handled securely.
Conclusion
Label detection adds powerful visual intelligence to your applications, and Eden AI makes it easy.
With just a few lines of Python and a simple two-step process, you can analyze video content using top AI providers like Google and Amazon.
Eden AI’s unified API, OpenAI-compatible endpoint, and multi-provider flexibility make it an ideal solution for scalable video analysis, whether you're working on media apps, security systems, or content moderation tools.

.jpg)

