Computer vision & object detection: Which provider to choose?
Tutorial

Computer vision & object detection: Which provider to choose?

In this article, we test several pre-trained Object Detection APIs . We test these solutions on various relevant use cases.


What is Computer Vision and Object Detection? 

In recent years,within the world of Artificial Intelligence (AI), one of the most popular applications is computer vision. This popularity is due to the huge diversity of applications and needs: medical imaging, industry, transport, surveillance, etc. Nowadays, every fields use cameras and pictures in their activities. Computer vision includes various functionalities:

A table of different Compute Vision functionalities
Eden AI  - Computer Vision & Object Detection : Computer Vision functionalities

This table does not represent an exhaustive list of all computer vision functionalities. Many solutions are based on several functionalities combined.


It is very important to distinguish pre-trained APIs and AutoML APIs :

  • Pre-trained APIs are APIs built on models that have previously been trained using the providers' databases. Common things, people, animals, situations, cars, faces, and other use cases involving common objects and people are typically managed by these models. However, as the providers are constantly improving their APIs, it is always vital to test these APIs before using bespoke models.
  • It could be preferable to use AutoML APIs for some scenarios where extremely high precision is required. Google Cloud Platform, Amazon Web Services, Microsoft Azure, IBM Watson, Clarifai, Chooch AI, and a few other companies offer these APIs. Users using autoML APIs are able to create unique models that are trained just on user databases. It enables the creation of bespoke models made especially for a certain project.

This article briefly treats about pre-trained Object Detection APIs. The aim is to inform about which problematics can be solved with this kind of API. Who are the main providers on the market? What is the optimal process when using pre-trained APIs?


What are the Object Detection API Providers?

During our study on Object Detection pre-trained APIs, we decided to choose 6 providers APIs that provide high performance according to many blog articles, Gartner and Forrester rankings:


Logos of Google Cloud, AWS, Microsoft Azure, IBM Watson, Clarifai & Chooch AI
Eden AI  - Computer Vision & Object Detection : Google Cloud, AWS, Microsoft Azure, IBM Watson, Clarifai & Chooch AI

This is the list of providers APIs we are going to test. It is interesting to note that some other solutions and open source solutions exist.


What are the Object Detection API Use Cases?

As said previously, object detection APIs are used in hundreds of fields, for many various use cases. In this article, we are going to test different object detection APIs with various types of pictures representing common use cases.

We chose 3 use cases from different fields, represented by 3 pictures :

  • People counting : a park wants to count people at a precise spot analyzing pictures taken at regular intervals
  • animal detection : some searchers want to know which animals are living at a precise spot
  • home picture analysis : example : High Tech company wants to check on pictures sent by their technicians if TVs were well installed

For each use case, we tested the Object Detection API from the 6 providers, with one picture per use case. Of  course, for a real project you will need to test on a representative  part of your database (not only one picture) to have the right view  about different performance.


Better than comparing results from different APIs, Eden AI provides the Genius functionality. This functionality returns a smart  combination of all results. For our examples, we will see what we can  get with this combined result.



Object Detection Tests

The API response is only a text response. This response (often json format) will be used to develop applications. For our example, the way to proceed is:

1- Benchmark object detections APIs available on the market

  • Search for providers
  • Test solutions with some pictures according to the project
  • Analyze prices

2- Choose the API provider that best fits with your project OCR combine multiples providers APIs results

3- Integrate final API in your project / software

  • Look how to manage API in production
  • Add pre-processing and post-processing according to your project

Finally, depending on the project, the visual results with bounding boxes printed on the pictures can be useful or not. But for the benchmark, this is the best and fastest way to find and visualize performances.

Google, IBM, Clarifai, AWS, Azure and Chooch provide API for multiple computer vision functionality. They also provide a graphic interface only to test and compute few pictures.


Animals case :

Object detection by different providers on a photo of animals
Eden AI — Object Detection API benchmark (Google, Amazon, Microsoft & IBM)

Object detection by Genius on a photo of animals
Eden AI — Object Detection API benchmark (Google, Amazon, Microsoft & IBM)


For Clarifai, here is the result:

Object Detection API for Clarifai
Eden AI — Object Detection API benchmark (Clarifai)

Chooch AI result:

Object Detection API for Chooch AI
Eden AI — Object Detection API benchmark (Chooch)AI)

Eden AI API returns responses for AWS, GCP, IBM and Azure APIs:

AWS, GCP, IBM and Azure APIs coding by Eden AI
Eden AI — Eden AI Object Detection API

For Clarifai, here is the API response:

Clarifai's response coding
Eden AI — Clarifai API

For Chooch AI, here is the API response:

Chooch's response coding
Eden AI — Chooch API

With the visual results, we can see that Google, Chooch and Microsoft are  the most efficient APIs, but they can all be complementary. Here, the Genius functionality seems to give the best results.

You can also look at the API requests in Python and the json responses. This response data will be use to realize the projects.


People use case:

Object detection by different providers on a photo of a group of people
Eden AI — Object Detection API benchmark (Google, Amazon, Microsoft & IBM)

Object detection by Genius on a photo of a group of people
Eden AI — Object Detection API benchmark (Genius)

Clarifai:

People Detection by Clarifai API
Eden AI — Clarifai API

Chooch AI:

People Detection by Chooch API
Eden AI — Chooch API

For this use case, we can note that Google Cloud API is really performing. On this picture, any people is detected by a provider but not Google. The Genius  feature is interesting here insofar as to the person detection that Google does very well, it will add the context elements (park, trees, etc.) that are better detected by other providers (Chooch AI, AWS and  IBM in particular).


TV detection use case:

Object detection by different providers on a photo of a television
Eden AI — Object Detection API benchmark (Google, Amazon, Microsoft & IBM)

Clarifai:

TV Detection by Clarifai API
Eden AI — Clarifai API


Chooch AI:

TV Detection by Chooch API
Eden AI — Chooch API

For this use case, all the provider APIs seem to give good performance. In this case, the Genius feature will essentially validate that AI providers are predicting the same thing and therefore secure those predictions. Moreover, the API provider choice will be based on :

  • pricing
  • execution speed
  • facility for integration
  • familiarity with a provider

So we have chosen 3 random use cases, and for the 3, it seems that the way to manage the project will be different :

  • One API chosen
  • The Genius combination of all APIs chosen
  • Custom model (with providers AutoML solutions or with open source development)

Depending on the use case, the best way to obtain the highest performance is always different. For some use case, an API from the provider A will be the best, for another use case provider B’ API is better. For a more complex use case, maybe a custom model can be better, or develop a proprietary model based on open source solutions.

Why choose Eden AI?

For GCP, AWS, Azure and Watson, we don’t need to use their API directly. In fact, the Eden AI Object Detection API  allows to get the 4 providers APIs results with only one simple  request. With few lines of code, we can have access to the results from the 4 providers. Clarifai and Chooch AI are not implemented yet on Eden AI, so we use their API directly.

With Eden AI,  you can get a fast access to various results from various providers. So you can have a better idea about which is the solution that best fits for you.

The decision making is as following :

First you run your datas on Eden AI to benchmark solutions available on the market. Then you have 3 options :

a. You find a result that push you to choose one API that fits with your attempted performance

b. Different providers give good results so you use the Genius functionality to gather forces and get a combined result, better than any single result from a provider.

c. Pre-trained APIs cannot provide good results for your project :

  • you decide to try Custom Vision models
  • you decide to develop your own models using open source solutions

This process garanties you to make the right choice to success in your project. Eden AI is only a tool that allows you to realize a benchmark very easily and quickly. Finally, it is possible to use Eden AI API to realize the entire project avoiding accounts and billings from many providers, and keeping the flexibility to not just choose one provider.

You are a solution provider and want to integrate Eden AI, contact us at : contact@edenai.co

Related Posts

Try Eden AI for free.

You can directly start building now. If you have any questions, feel free to schedule a call with us!

Get startedContact sales