Step-by-Step Guide to Image Embeddings and Visual Similarity Search

Do you want to build a web application that can find images similar to a given photo?

In this beginner-friendly guide, you'll learn how to create an image similarity search API step-by-step using FastAPI and FAISS.

We'll explain every line of the provided Python code, so even if you're new to programming, you'll follow along easily.

You can watch the full tutorial on our YouTube channel by our developer advocate, Krishna Kompalli, for a full breakdown:

‍

Step 1: Setup Your Environment

First, install the required Python libraries:

pip install fastapi uvicorn faiss-cpu numpy requests python-dotenv

‍

• fastapi: A modern framework for building APIs quickly.
• uvicorn: A web server to run your FastAPI app.
• faiss-cpu: Library for fast similarity search.
• numpy: Numeric operations library.
• requests: To make HTTP requests.

python-dotenv: Load environment variables from .env file.

‍

Step 2: Understand the Main Components

FastAPI

FastAPI is a lightweight yet powerful web framework that enables you to rapidly build secure and performant APIs with minimal effort, providing automatic data validation and documentation.

FAISS

FAISS (Facebook AI Similarity Search) is an advanced library optimized for quickly searching through large datasets of vectors, ideal for finding similar items such as images or text embeddings.

‍

Step 3: API Initialization

Initialize your FastAPI application:

from fastapi import FastAPI

app = FastAPI(title="Image Embedding Search API")

‍

• FastAPI: Imports the framework.
• app: Creates an instance of FastAPI to define routes.

‍

Step 4: Setting Up the Vector Database (FAISS)

This class manages your FAISS vector database for storing embeddings:

import faiss
import numpy as np

class FAISSVectorDB:
    def __init__(self, dimension=1024):
        self.index = faiss.IndexFlatL2(dimension)
        self.dimension = dimension
        self.embeddings = []

    def add_embedding(self, embedding):
        self.index.add(np.array([embedding], dtype=np.float32))
        self.embeddings.append(embedding)

    def search(self, query_embedding, top_k=5):
        distances, indices = self.index.search(np.array([query_embedding], dtype=np.float32), top_k)
        return indices, distances

‍

• __init__: Initializes FAISS index and specifies dimensions of embeddings.
• add_embedding: Adds embeddings (numeric vectors) to the FAISS index.
• search: Finds and returns indices and distances of the most similar embeddings.

‍

Step 5: Upload Image and Generate Embeddings

Here's how to upload an image and generate embeddings using Eden AI:

@app.post("/embeddings/upload")
async def upload_image(file: UploadFile = File(...)):
    # Read image file, send it to Eden AI, and obtain an embedding vector
    # Then store this embedding using FAISSVectorDB

‍

• @app.post: Defines a POST endpoint for uploading images.
• UploadFile: Facilitates handling of file uploads within FastAPI, allowing you to read and process uploaded files efficiently

‍

Step 6: Search Similar Images

Implement similarity search for uploaded images:

@app.post("/search/upload")
async def search_similar_images(file: UploadFile = File(...)):
    # Generate embedding for the uploaded query image
    # Perform similarity search using FAISSVectorDB and return results

‍

• This endpoint is similar to the upload function but specifically designed to handle queries, generating embeddings for the query images and finding matches within the database.

‍

Step 7: Detailed Code Explanation

Loading Environment Variables

from dotenv import load_dotenv
import os

load_dotenv()
EDEN_AI_API_KEY = os.getenv("EDEN_AI_API_KEY")

‍

Securely loads sensitive configuration details, such as your Eden AI API key, from a hidden .env file, keeping it out of your main source code.

‍

Calling Eden AI to Generate Embeddings

import requests

response = requests.post(
    "https://api.edenai.run/v2/image/embeddings",
    headers={"authorization": f"Bearer {EDEN_AI_API_KEY}"},
    files={"file": (file.filename, await file.read())}
)

‍

• Sends your image to Eden AI's service to get embedding.
• Uses your API key to authenticate the request securely.

‍

Extracting Embeddings from Eden AI's Response

embedding = response.json()["provider_name"]["embedding"]

‍

• Parses the JSON response received from Eden AI and retrieves the embedding vector provided by the service.

‍

Adding Embedding to FAISS

vector_db.add_embedding(embedding)

‍

• Stores the extracted embedding vector into the FAISS database, making it available for similarity searches.

‍

Performing Similarity Search

indices, distances = vector_db.search(query_embedding)

‍

• Performs a similarity search in your FAISS database using the query embedding, quickly identifying the most similar stored embeddings.

‍

Step 8: Deploy Your API

Run your FastAPI application:

uvicorn main:app --host 0.0.0.0 --port 8000

‍

• uvicorn: Launches your FastAPI application on a local server, making it accessible through the specified port (8000).

‍

Conclusion

Congratulations! You've successfully built an image similarity search API using FastAPI, FAISS, and Eden AI.

You've learned how to handle image uploads, generate embeddings, and efficiently search similar images.

By understanding these foundational concepts, you can now expand this basic API into more complex systems, integrate it into real-world applications, and explore additional functionalities such as user interfaces, automated categorization, and enhanced search features. Keep experimenting and practicing to further strengthen your development skills!