Text Moderation API, also called Content Moderation or Text Filtering, analyses text using NLP and machine learning to identify harmful content, including hate speech, offensive language, harassment, and spam.This API can filter and remove inappropriate material in user-generated content such as comments, reviews, messages, and social media posts. It creates a safer and more respectful online environment by detecting and filtering out such content.
Once processed, the API supplies feedback showing whether the content is acceptable or not, and may give additional details on the moderation decision, like a summary or reasons for moderation. Certain providers also give a filter for profanity and personal data detection within texts.
For users seeking a cost-effective engine, opting for an open-source model is the recommended choice. Here is the list of the best Text Filtering Open Source Models:
All-encompassing content moderation solution featuring detection of not-safe-for-work (NSFW) material, weapons, alcohol, drugs, gore, offensive symbols, and profanity. Additionally, the solution includes filters for spam, spam emails, and malicious URLs.
An open source text moderation model that helps you filter out unsafe content. It is trained on 60+ gigs of data.
Llama 2 comprises pre-trained and finely-tuned generative text models, with a parameter range spanning 7 billion to 70 billion. Llama-2-Chat models demonstrate superior performance compared to open-source chat models across several benchmark tests and remain on par with select closed-source models such as ChatGPT and PaLM. This can be used for text moderation also.
PaLM is an AI model that generates text by working with human language. It has learned how to read and create new, grammatically correct text from an extensive corpus of general training data. Google has introduced PaLM 2 (Pathways Language Model), its latest LLM, boasting exceptional prowess in advanced reasoning, coding, and mathematics. PaLM 2 is offered in diverse sizes, facilitating easy deployment across a wide range of usage scenarios. This can be used for text moderation also.
GPT-NeoX-20B is a language model with 20 billion parameters that was trained using the GPT-NeoX library on the Pile. Its architecture closely resembles that of GPT-3 and is nearly identical to GPT-J-6B. The model's training dataset includes a wide range of English-language texts, highlighting its general-purpose functionality. This can be used for text moderation also.
GPT-J or GPT-J-6B is a large language model created by EleutherAI in 2021. Its aim is to generate text that sounds like a human being speaking, and it does this by using pre-existing knowledge to continue from a prompt. This can be used for text moderation also.
BLOOM is a powerful language model that uses extensive text data and advanced computing resources to create coherent sentences. It can accurately replicate human writing in 46 languages and 13 programming languages. This can be used for text moderation also.
While open source models offer many advantages, they also come with some potential drawbacks and challenges. Here are some cons of using open source models:
Given the potential costs and challenges related to open-source models, one cost-effective solution is to use APIs. Eden AI smoothens the incorporation and implementation of AI technologies with its API, connecting to multiple AI engines.
Eden AI presents a broad range of AI APIs on its platform, customized to suit your specific needs and financial limitations. These technologies include data parsing, language identification, sentiment analysis, logo recognition, question answering, data anonymization, speech recognition, and numerous other capabilities.
To get started, we offer free $10 credits for you to explore our APIs.
Our standardized API enables you to integrate Text Filtering APIs into your system with ease by utilizing various providers on Eden AI. Here is the list (in alphabetical order):
Clarifai's Text Moderation API automatically filters and moderates text content to ensure it adheres to your content guidelines. It identifies explicit or inappropriate language, categorizes text as safe or unsafe, and enables you to filter or replace any inappropriate text.
Google Cloud's text moderation algorithm evaluates documents against a set of safety attributes that encompass topics that are deemed harmful or sensitive. You can assess the efficacy of Google's safety filters and set customized confidence thresholds that suit your business needs, enabling you to identify and promptly address any material infringing on Google's usage regulations or terms of service through systematic monitoring.
Microsoft Azure's technology can assess text in various contexts such as chat rooms, discussion boards, chatbots, e-commerce catalogs, and documents. It can locate and highlight offensive language in numerous languages employing a built-in directory of prohibited terms. Furthermore, Azure's software can organize text into three classifications via machine-assisted classification and can even discover personal information and correct the text automatically, where necessary.
The tool relies on OpenAI’s advanced natural language processing technology, ensuring the detection of unsuitable content with high precision and dependability. OpenAI's offering distinguishes itself by being able to comprehend the meaning of language and recognise subtle abusive language types, including hate speech, cyberbullying, and self-harming material as well as detecting potentially misleading or fraudulent content.
Eden AI offers a user-friendly platform for evaluating pricing information from diverse API providers and monitoring price changes over time. As a result, keeping up-to-date with the latest pricing is crucial. The pricing chart below outlines the rates for smaller quantities for November 2023, as well as you can get discounts for potentially large volumes.
Eden AI is the future of AI usage in companies: our app allows you to call multiple AI APIs.
You can see Eden AI documentation here.
The Eden AI team can help you with your Text Moderation integration project. This can be done by :