Addis AI

Multimodal

Chat with images and audio files using our unified Multimodal Model.

The Multimodal capability extends Addis-፩-አሌፍ beyond text. You can upload Images and Audio files alongside your prompts to perform complex reasoning tasks.

  • Vision: Visual Q&A, Scene Description, and Object Detection.
  • Audio Analysis: Summarize voice notes, analyze sentiment, or extract action items from meetings (distinct from simple transcription).
  • OCR: Extract text from scanned African documents.

Important: Data Format

Unlike standard text generation, this request must be sent as multipart/form-data.

The configuration (prompt, model, parameters) must be passed as a stringified JSON object in a field named request_data.

Usage Guide

Chat with Images (Vision)

Upload an image and ask questions about it. Ideal for explaining diagrams, identifying objects, or analyzing screenshots.

import AddisAI, { fileFromPath } from "addisai";

const addis = new AddisAI();
const response = await addis.chat.completions.create({
  messages: [{ role: "user", content: "Describe this image in Amharic." }],
  attachments: [{ file: await fileFromPath("market.jpg", "image/jpeg") }],
});

console.log(response.choices[0].message.content);
from addisai import AddisAI

addis = AddisAI()
with open("market.jpg", "rb") as image:
    response = addis.chat.completions.create(
        messages=[{"role": "user", "content": "Describe this image in Amharic."}],
        attachments=[image],
    )

print(response["choices"][0]["message"]["content"])
curl https://api.addisassistant.com/api/v1/chat_generate \\
  -H "x-api-key: $ADDIS_API_KEY" \\
  -F 'attachment_0=@market.jpg;type=image/jpeg' \\
  -F 'request_data={"prompt":"Describe this image in Amharic.","target_language":"am","attachment_field_names":["attachment_0"]}'

Chat with Audio

Upload an audio file (e.g., a voice note or meeting recording) and ask the model to perform reasoning on it.

Difference from STT: Speech-to-Text simply transcribes words. This endpoint listens to the audio and answers questions about it (e.g., "Summarize", "What was the tone?", "Extract action items").

import AddisAI, { fileFromPath } from "addisai";

const addis = new AddisAI();
const response = await addis.chat.completions.create({
  messages: [{ role: "user", content: "Follow the instruction in this recording." }],
  audio: await fileFromPath("command.wav", "audio/wav"),
});

console.log(response.transcription?.clean);
console.log(response.choices[0].message.content);
from addisai import AddisAI

addis = AddisAI()
with open("command.wav", "rb") as audio:
    response = addis.chat.completions.create(
        messages=[{"role": "user", "content": "Follow the instruction in this recording."}],
        audio=audio,
    )

print(response.get("transcription", {}).get("clean"))
print(response["choices"][0]["message"]["content"])
curl https://api.addisassistant.com/api/v1/chat_generate \\
  -H "x-api-key: $ADDIS_API_KEY" \\
  -F 'chat_audio_input=@command.wav;type=audio/wav' \\
  -F 'request_data={"prompt":"Follow the instruction in this recording.","target_language":"am"}'

Document OCR (Text Extraction)

Use the vision model to extract text from scanned African documents, ID cards, or handwritten notes.

Dedicated Endpoint

We are finalizing a dedicated OCR endpoint (/api/v1/ocr) for bulk processing. For now, you can use the chat endpoint with extraction prompts.

import AddisAI, { fileFromPath } from "addisai";

const addis = new AddisAI();
const response = await addis.chat.completions.create({
  messages: [{ role: "user", content: "Extract every visible line exactly as written." }],
  attachments: [{ file: await fileFromPath("document.png", "image/png") }],
});

console.log(response.choices[0].message.content);
from addisai import AddisAI

addis = AddisAI()
with open("document.png", "rb") as document:
    response = addis.chat.completions.create(
        messages=[{"role": "user", "content": "Extract every visible line exactly as written."}],
        attachments=[document],
    )

print(response["choices"][0]["message"]["content"])
curl https://api.addisassistant.com/api/v1/chat_generate \\
  -H "x-api-key: $ADDIS_API_KEY" \\
  -F 'attachment_0=@document.png;type=image/png' \\
  -F 'request_data={"prompt":"Extract every visible line exactly as written.","target_language":"am","attachment_field_names":["attachment_0"]}'

Conversation History with Attachments

The API is stateless, meaning it treats every request independently. To continue a conversation involving attachments, you must provide the relevant context in the conversation_history array.

Note

Do not re-upload files. Instead, switch from multipart/form-data to a standard JSON request and reference the previously uploaded file using its fileUri inside the parts array.

This approach maintains continuity while minimizing latency and bandwidth.

The SDK examples retrieve the reusable attachment metadata from the first upload. The cURL tab shows the subsequent JSON request with that returned URI.

import AddisAI, { fileFromPath } from "addisai";

const addis = new AddisAI();
const first = await addis.chat.completions.create({
  messages: [{ role: "user", content: "Describe this file." }],
  attachments: [{ file: await fileFromPath("report.pdf", "application/pdf") }],
});

console.log(first.uploaded_attachments);
from addisai import AddisAI

addis = AddisAI()
with open("report.pdf", "rb") as report:
    first = addis.chat.completions.create(
        messages=[{"role": "user", "content": "Describe this file."}],
        attachments=[report],
    )

print(first.get("uploaded_attachments"))
curl --location 'https://api.addisassistant.com/api/v1/chat_generate' \
  --header 'Content-Type: application/json' \
  --header 'X-API-Key: $ADDIS_API_KEY' \
  --data '{
    "prompt": "What was the document talking about in detail?",
    "target_language": "am",
    "conversation_history": [
      {
        "role": "user",
        "parts": [
          {
            "fileData": {
              "fileUri": "YOUR_FILE_URI_HERE",
              "mimeType": "application/pdf"
            }
          },
          { "text": "Describe this attachment" }
        ]
      },
      {
        "role": "assistant",
        "parts": [
          { "text": "ይህ ሰነድ በታህሳስ 18 ቀን 2025 በተካሄደው..." }
        ]
      }
    ]
  }'

Reusing returned file URIs

The SDK upload helper returns uploaded_attachments metadata after the first request. The current high-level SDK methods accept files for uploads; use the raw REST flow shown above when you need to reuse a returned fileUri without uploading the file again.

Conversation History Object Schema

The conversation_history array consists of message objects defined below.

Prop

Type

Part Object Structure: Each item in the parts array must be one of the following:

Prop

Type

FileData Object Structure:

Prop

Type

API Reference

Form Data Parameters

These parameters are sent in the multipart/form-data body.

Prop

Type

Request Data Object

These parameters go inside the request_data stringified JSON.

Prop

Type

Basic Multimodal Response Schema

When files are uploaded, the response includes an uploaded_attachments array confirming the upload.

{
    "status": "success",
    "data": {
        "response_text": "ይህ ሰነድ በ12ኛው ክፍለ ዘመን በንጉሥ ላሊበላ ስለተገነቡት የላሊበላ ውቅር አብያተ ክርስቲያናት ያብራራል። እነዚህ 11 አብያተ ክርስቲያናት ከወጥ ድንጋይ የተወቀሩ ሲሆኑ፣ በዓለም ቅርስነት የተመዘገቡ ድንቅ የኢትዮጵያ ስልጣኔ ማሳያዎች ናቸው። ሰነዱ በተለይ ስለ ቤተ ጊዮርጊስ የመስቀል ቅርጽ ግንባታ እና ስለ ውስብስብ የውሃ ማስወገጃ ስርዓታቸው ዝርዝር ትንታኔ ይሰጣል።",
        "finish_reason": "STOP",
        "usage_metadata": {
            "prompt_token_count": 2427,
            "candidates_token_count": 269,
            "total_token_count": 2696
        },
        "modelVersion": "Addis-፩-አሌፍ",
        "uploaded_attachments": [
            {
                "fileUri": "YOUR_FILE_URI_VARIABLE",
                "mimeType": "application/pdf"
            }
        ]
    }
}

Response Schema With Conversation History

{
    "status": "success",
    "data": {
        "response_text": "በቀረበው የውይይት ታሪክ መሠረት፣ ፋሲል ግቢ (የጎንደር ቤተመንግስት) በ17ኛው ክፍለ ዘመን በአፄ ፋሲለደስ የተገነባ ድንቅ የኪነ-ሕንጻ ውጤት ነው። ግቢው በውስጡ ስድስት ዋና ዋና ቤተ-መንግስቶችን፣ ቤተ-መጻሕፍትን እና አብያተ ክርስቲያናትን ይዟል። ሰነዱ በተለይ ስለ ግንባታው የህንድ እና የአረብ ኪነ-ሕንጻ ተፅእኖ እንዲሁም ስለ ህንፃው የመሬት መንቀጥቀጥ መቋቋም ችሎታ ዝርዝር ትንታኔ ይሰጣል።",
        "finish_reason": "STOP",
        "usage_metadata": {
            "prompt_token_count": 3500,
            "candidates_token_count": 145,
            "total_token_count": 3645
        },
        "modelVersion": "Addis-፩-አሌፍ"
    }
}

Best Practices

Optimize your multimodal integration with these architectural patterns.

Context Management

Don't Re-upload: Never re-upload the same file in a multi-turn chat. Upload it once, get the fileUri from the response, and reference that URI in future JSON requests.

Latency: Passing a URI is milliseconds; re-uploading a 10MB PDF takes seconds.

Document Prep

PDF Format: Native PDFs (text-selectable) process faster and more accurately than Scanned PDFs (images inside PDF).

Orientation: Ensure scanned documents are upright. Rotated text significantly degrades Amharic OCR performance.

Visual Prompting

Be Specific: Instead of "What's in this image?", ask "Extract the date and total amount from this receipt."

Language: For best results on African documents, prompt in Amharic (e.g., "በምስሉ ላይ ያለውን ጽሑፍ አውጣ").

Constraints

Max File Size10 MB

Token Cost: Images and PDFs consume significantly more tokens than text. Monitor your usage_metadata to avoid hitting rate limits.

Speech to Text (Transcription)

Transcribe audio into text for Amharic and Afan Oromo.

Realtime API

Low-latency, bidirectional voice conversations via WebSockets.

On this page

Usage Guide
Chat with Images (Vision)
Chat with Audio
Document OCR (Text Extraction)
Conversation History with Attachments
Conversation History Object Schema
API Reference
Form Data Parameters
Request Data Object
Basic Multimodal Response Schema
Response Schema With Conversation History
Best Practices