How to Detect Labels (Objects) in Videos Using Python

Label Detection is a key task in video and image analysis that involves identifying and categorizing elements within each frame such as objects, people, animals, or landmarks.

This helps with content tagging, indexing, and search optimization. In this tutorial, we’ll show you how to implement label detection using Python and the Eden AI API.

With just a few lines of code, you can run label detection and extract meaningful insights from visual content.

‍

What is Label/Object Detection?

Label detection automatically identifies and categorizes objects, people, and scenes in videos or images. It assigns labels like animals, buildings, or landmarks to visual elements, making content easier to search, organize, and analyze.

‍

How to Detect Labels in Videos

‍

Get Access to Eden AI API

1. Sign up: Visit Eden AI and create an account for free to get started. Once registered, navigate to the API section to find your personal API key which gives you access to a wide range of AI services, including Label Detection.

2. Navigate to Video Technologies – Log in and head to the Video Technologies section.

3. Select Label Detection– From there, select the Label Detection option.

‍

Live Test Models on Eden AI

Eden AI lets you experiment with various AI models before integration, making it easy to compare and choose the provider that best fits your needs.

‍

Implementing Label Detection in Python

‍

Step 1: Install Python Requests Module

Before we start, make sure you have the requests module installed. This module allows your Python script to make HTTP requests to APIs like Eden AI.

pip install requests

This module is required to send HTTP requests to the Eden AI API.

‍

Code Explanation

The label detection API in Eden AI is asynchronous, meaning it works in two stages:

1. POST request – Launching the Job: This initiates the label detection process by sending the video URL and provider(s) to Eden AI.
   
2. GET request – Retrieving the Results: Once the job is processed, you can fetch the detection results using a unique public_id received from the first request.
   
   

This two-phase system is essential for processing larger media files, which can take time to analyze. Instead of waiting for the process to finish in a single request, the async model allows you to start the job and check back later.

‍

Launching the Label Detection Job

This part of the code sends a request to Eden AI to start the label detection job.

import requests

# Replace with your actual API Key
headers = {"Authorization": "Bearer {API_KEY}"}

# Replace with the actual URL of your video
json_payload = {
    "providers": "google", 
    "file_url": "🔗 URL of your video"
}

# The URL for launching the shot change detection job
url = "https://api.edenai.run/v2/video/shot_change_detection_async"

# Send the POST request to initiate the job
response = requests.post(url, json=json_payload, headers=headers)

# Get the response and display it
result = response.json()
print(result)

Explanation:

• headers: Contains your API key to authenticate with Eden AI.
  
• url: Endpoint for launching the label detection job.
  
• json_payload:
  
  • providers: AI providers you want to use (e.g., Google, Amazon).
    
  • file_url: Public URL to your video.
    
• response: This POST request sends the job to Eden AI.
  
• result: Includes metadata like the public_id, which you will need to fetch the results later.

‍

Getting the Job Results

Once your job is submitted and processed, retrieve the results using the public_id.

import requests

headers = {"Authorization": "Bearer {API_TOKEN}"}

url = "https://api.edenai.run/v2/video/label_detection_async"
json_payload = {
    "providers": "google,amazon",
    "file_url": "🔗 URL of your video"
}

response = requests.post(url, json=json_payload, headers=headers)
result = response.json()
print(result)

Explanation:

• {public_id}: Replace this with the ID from the first response.
  
• This call returns the actual label detection results from the selected AI providers.

‍

Interpreting the Results

Here’s an example of what the response might look like:

{
  "google": {
    "status": "success",
    "label_annotations": [
      {"description": "Car", "confidence": 0.95, "start_time": "00:01", "end_time": "00:03"},
      {"description": "Tree", "confidence": 0.88, "start_time": "00:02", "end_time": "00:04"}
    ]
  },
  "amazon": {
    "status": "success",
    "labels": [
      {"label": "Vehicle", "confidence": 0.92, "timestamp": "00:01"},
      {"label": "Nature", "confidence": 0.90, "timestamp": "00:03"}
    ]
  }
}

Explanation of Output Fields:

• description / label: Name of the detected object.
  
• confidence: Accuracy level (0 to 1) of the prediction.
  
• start_time / end_time / timestamp: Indicates when the label appears in the video.

‍

Managing Label Detection Jobs

For better management of your label detection tasks, Eden AI provides additional optional steps:

Label Detection - List Jobs

GET Request:
https://api.edenai.run/v2/video/label_detection_async/
This step allows you to retrieve a list of all jobs launched for label detection. Using the unique ID of each job, you can check its status and results. You can check the full documentation here.

Label Detection - Delete Jobs

DELETE Request:
https://api.edenai.run/v2/video/label_detection_async/
This endpoint allows you to delete jobs, providing an easy way to manage and clean up completed or unnecessary tasks. You can check the full documentation here.

By leveraging these optional steps, you can more effectively manage and track your label detection jobs.

‍

Why Eden AI is the Best Tool for Label Detection

Eden AI offers a robust Label Detection solution with several advantages.

‍

Multiple Provider Options

Eden AI allows you to compare and use multiple providers like Google and Amazon at once.

Ease of Integration

Eden AI simplifies the integration of label detection into your applications with clear and straightforward Python code examples.

Security and Trust

‍The Eden AI platform prioritizes data security and privacy, ensuring that your image data is handled securely.

‍

Conclusion

Label detection adds powerful visual intelligence to your applications, and Eden AI makes it easy.

With just a few lines of Python and a simple two-step process, you can analyze video content using top AI providers like Google and Amazon.

Eden AI’s unified API, asynchronous processing, and multi-provider flexibility make it an ideal solution for scalable, high-performance video analysis—whether you're working on media apps, security systems, or content moderation tools.