Llama 3.1 8B Instruct FP8

llama-3.1-8b-instruct-fp8 by Meta Released July 23, 2024

ReasoningFunction callingStructured outputPrompt cachingWeb searchComputer useImage inputFile inputAudio inputVideo inputEU hostingFree endpointUp to % off

Input from

$0.02 / 1M

Output from

$0.05 / 1M

Cached input from

$0.003 / 1M

Context window

131K

Endpoints

from 2 providers

Regions

Global

About Llama 3.1 8B Instruct FP8

Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 8B instruct-tuned version is fast and efficient.

Llama 3.1 8B Instruct FP8 is a chat model by Meta, available on Eden AI through 2 endpoints from 2 providers, from $0.02 per million input tokens and $0.05 per million output tokens, with a context window of up to 131K tokens.

Providers and prices

2 endpoints from 2 providers, cheapest first, in USD per million tokens. Every one is called with the same Eden AI key: pin one with its endpoint ID, or let Eden AI route for you. .

Endpoint

Region

Context

Input / 1M

Output / 1M

Cached / 1M

Supports

FlexAIOfficial−%flexai/Meta-Llama-3.1-8B-Instruct-FP8

Global

131K

$0.02$0.02

$0.05

$0.003

ReasoningToolsJSONCacheVisionWeb

Cloudflare (CF/META/LLAMA-3.1-8B-INSTRUCT-FP8)Official−%cloudflare/@cf/meta/llama-3.1-8b-instruct-fp8

CF/META/LLAMA-3.1-8B-INSTRUCT-FP8

32K

$0.152$0.152

$0.287

ReasoningToolsJSONCacheVisionWeb

Performance

Measured on real Eden AI traffic over the last 30 days. Endpoints with too little traffic are not shown.

Fastest first token

Lowest latency

0.86 s

FlexAI

Highest throughput

5.73 tok/s

FlexAI

Best availability

100%

FlexAI

Endpoint

First token

Latency p50

Latency p95

Throughput

Availability

Cache hits

Tool errors

JSON errors

FlexAI

0.86 s

5.45 s

5.73 tok/s

100%

61.53%

Cloudflare (CF/META/LLAMA-3.1-8B-INSTRUCT-FP8)

tok/s

Estimate your monthly cost

Enter your expected volume to compare every endpoint at once.

Call Llama 3.1 8B Instruct FP8 with Eden AI

Use the model ID to let Eden AI pick an endpoint, or an endpoint ID from the table above to pin one.

import requests

response = requests.post(
    "https://api.edenai.run/v3/chat/completions",
    headers={"Authorization": "Bearer YOUR_EDENAI_API_KEY"},
    json={
        "model": "",
        "messages": [{"role": "user", "content": "Hello!"}],
    },
)
print(response.json())

curl https://api.edenai.run/v3/chat/completions \
  -H "Authorization: Bearer YOUR_EDENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "", "messages": [{"role": "user", "content": "Hello!"}]}'

const response = await fetch("https://api.edenai.run/v3/chat/completions", {
  method: "POST",
  headers: {
    Authorization: "Bearer YOUR_EDENAI_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "",
    messages: [{ role: "user", content: "Hello!" }],
  }),
});
console.log(await response.json());

Questions about Llama 3.1 8B Instruct FP8

How much does Llama 3.1 8B Instruct FP8 cost?

On Eden AI, prices for this model start at $0.02 per million input tokens and $0.05 per million output tokens. Prices differ by endpoint; the providers table lists each one. .

What is the context window of Llama 3.1 8B Instruct FP8?

Up to 131K tokens, depending on the endpoint.

Which providers serve Llama 3.1 8B Instruct FP8?

2 endpoints from 2 providers, available in: Global. All of them are called with the same Eden AI API key.

Can I run Llama 3.1 8B Instruct FP8 in the EU?

Yes. At least one endpoint is hosted in Europe; pin it with its endpoint ID from the providers table to keep requests in the EU.

How fast is Llama 3.1 8B Instruct FP8?

Over the last 30 days, the fastest first token was s () and the highest throughput was 5.73 tokens per second (FlexAI).

About Llama 3.1 8B Instruct FP8

Providers and prices

Performance

Estimate your monthly cost

Call Llama 3.1 8B Instruct FP8 with Eden AI

Questions about Llama 3.1 8B Instruct FP8

More from Meta