.png)
Summarize this article with:
- Person Tracking API allows you to track people and their activity within a scene or segment.
- Person Tracking API can be used as a standalone tool, or it can be combined with face detection to generate more accurate results.
- The first step to start using person tracking API is to set Axios, a promise-based HTTP client for the browser and Node.js, that will allow you to call Eden AI API.
- Developers can Track and Localize a Person in a Video with JavaScript using a REST API that accepts standard inputs and returns structured JSON responses.
- The integration works with Python, JavaScript, PHP, and any HTTP-capable language with minimal setup.
In this tutorial, you will learn how to use person tracking API in 5 minutes using JavaScript and Eden AI person tracking API. Eden AI provides an easy and developer-friendly API that allows you to track persons in videos.
What is Video Person Tracking API ?
Person Tracking API allows you to track people and their activity within a scene or segment. The way that person tracking works is simple: person detection identifies individuals in a video, then tracks them over time by their face, clothing or posture.
Person Tracking API can be used as a standalone tool, or it can be combined with face detection to generate more accurate results. You can use person tracking to analyze situations with high levels of complexity, such as crowded scenes with multiple individuals, crowded transportation areas such as airports, trains and metro stations.
Getting Started with Video Person Tracking API
The first step to start using person tracking API is to set Axios, a promise-based HTTP client for the browser and Node.js, that will allow you to call Eden AI API.
.png)
Next, you'll need to initialize the File System module in order to access local files on your computer.
.png)
Finally, you'll need to create your multipart/formdata parameters form:
How to use Person Tracking API with JavaScript
You are now ready to process your video file into Eden AI's person tracking API.
1. Get your Person Tracking API Key on Eden AI
To perform person tracking, you'll need to create an account on Eden AI for free. Then, you will be able to get your API key direclty from the homepage with free credits offered by Eden AI.
.svg)
2. Let’s Track Persons in your Video File
Now that you have initialized the required JavaScript modules and got your API key, you will be able to track persons in videos. Person tracking in videos now goes through a multimodal LLM that accepts video as input, called with Eden AI’s OpenAI-compatible chat completions endpoint: the model lists the people with a short description and the time ranges where they appear, rather than frame-by-frame bounding boxes. You can use any model that has video in its input_modalities: see our documentation here.
Here is the JavaScript code to add your video to the form data and set your API key:
form.append("file", fs.createReadStream("path/to/your/video.mp4"));
const apiKey = "Your_API_Key";
Once the file is added, you can configure your requests: the video is first uploaded to Eden AI, which returns a file_id, then the request asks a multimodal LLM, here google/gemini-2.5-flash, to track the people and to answer in the JSON format set by the schema:
// Step 1: upload your video to Eden AI, which returns a file_id
const uploadOptions = {
method: "POST",
url: "https://api.edenai.run/v3/upload",
headers: {
authorization: `Bearer ${apiKey}`,
...form.getHeaders(),
},
data: form,
};
// Step 2: ask a multimodal LLM to track the people, with the JSON format of the answer
const schema = {
type: "object",
properties: {
persons: {
type: "array",
items: {
type: "object",
properties: {
person: { type: "string" },
start_time: { type: "number" },
end_time: { type: "number" },
},
required: ["person", "start_time", "end_time"],
},
},
},
required: ["persons"],
};
const options = (fileId) => ({
method: "POST",
url: "https://api.edenai.run/v3/chat/completions",
headers: {
authorization: `Bearer ${apiKey}`,
},
data: {
model: "google/gemini-2.5-flash", // any model with video in its input_modalities
messages: [
{
role: "user",
content: [
{ type: "text", text: "List the people who appear in this video, each with a short description and the time range in seconds where they appear." },
{ type: "file", file: { file_id: fileId } },
],
},
],
response_format: { type: "json_schema", json_schema: { name: "persons", schema: schema } },
},
});
Next, you'll need to create launchJob() function that will execute POST request:
.png)
Finally, create the run() function that uploads the video, launches the request and reads the answer. There is no job to poll, as the result comes back in the same response:
const run = async () => {
const upload = await axios.request(uploadOptions);
const response = await launchJob(options(upload.data.file_id));
console.log(response.data);
// The answer is a JSON string that follows the schema
const result = JSON.parse(response.data.choices[0].message.content);
console.log(result.persons);
};
run().catch((error) => {
console.error(error.response ? error.response.data : error);
});
You will get this response, where the answer is a JSON string in choices[0].message.content:
{
"status": "success",
"model": "gemini-2.5-flash",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "{\n \"persons\": [\n {\n \"person\": \"Sintel\",\n \"start_time\": 5,\n \"end_time\": 8\n },\n {\n \"person\": \"Elder\",\n \"start_time\": 12,\n \"end_time\": 13\n },\n {\n \"person\": \"Sintel\",\n \"start_time\": 14,\n \"end_time\": 15\n },\n {\n \"person\": \"Sintel\",\n \"start_time\": 17,\n \"end_time\": 17\n },\n {\n \"person\": \"Thug\",\n \"start_time\": 17,\n \"end_time\": 17\n },\n {\n \"person\": \"Sintel\",\n \"start_time\": 19,\n \"end_time\": 20\n },\n {\n \"person\": \"Sintel\",\n \"start_time\": 22,\n \"end_time\": 23\n },\n {\n \"person\": \"Sintel\",\n \"start_time\": 24,\n \"end_time\": 25\n },\n {\n \"person\": \"Sintel\",\n \"start_time\": 27,\n \"end_time\": 35\n },\n {\n \"person\": \"Dragon (Scales)\",\n \"start_time\": 25,\n \"end_time\": 35\n },\n {\n \"person\": \"Sintel\",\n \"start_time\": 37,\n \"end_time\": 41\n }\n ]\n}"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 15635,
"completion_tokens": 747,
"total_tokens": 16382
},
"provider": "google",
"cost": 0.0077277
}
Once parsed, this JSON string gives the result for Person Tracking task (the time ranges are estimated by the model, so check them on a few of your own videos before relying on them in production):
[
{
"person": "Sintel",
"start_time": 5,
"end_time": 8
},
{
"person": "Elder",
"start_time": 12,
"end_time": 13
},
{
"person": "Sintel",
"start_time": 14,
"end_time": 15
},
{
"person": "Sintel",
"start_time": 17,
"end_time": 17
},
...
]
Benefits of using Person Tracking API with Eden AI
Using person tracking with Eden AI API is quick and easy.
.gif)
Save time and cost
We offer a unified API for all providers: simple and standard to use, with a quick switch between providers and an access to the specificic features of each provider.
Easy to integrate
The JSON output format is the same for all suppliers thanks to Eden AI's standardisation work. The response elements are also standardised thanks to Eden AI's powerful matching algorithms.
Customization
With Eden AI you have the possibility to integrate a third party platform: we can quickly develop connectors. To go further and customize your person tracking request with specific parameters, check out our documentation.
.jpg)
