{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Extract Text from PDF Documents with IONOS AI Model Hub\n",
    "\n",
    "This notebook demonstrates how to extract text from PDF documents by rendering pages as images with [pypdfium2](https://pypi.org/project/pypdfium2/) and sending them to the IONOS Cloud OpenAI-compatible API.\n",
    "\n",
    "## Prerequisites\n",
    "- An IONOS API token (set as `IONOS_API_TOKEN` environment variable)\n",
    "- Python packages: `pypdfium2`, `pillow`, `openai`, `python-dotenv`\n",
    "- A PDF file to process"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "Load the API token, initialize the OpenAI client, and define the PDF rendering helper."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import base64\n",
    "import io\n",
    "import os\n",
    "\n",
    "import pypdfium2 as pdfium\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "\n",
    "load_dotenv()\n",
    "IONOS_API_TOKEN = os.getenv(\"IONOS_API_TOKEN\")\n",
    "\n",
    "client = OpenAI(\n",
    "    api_key=IONOS_API_TOKEN,\n",
    "    base_url=\"https://openai.inference.de-txl.ionos.com/v1\",\n",
    ")\n",
    "\n",
    "LIGHTON_MODEL = \"lightonai/LightOnOCR-2-1B\"\n",
    "MISTRAL_MODEL = \"mistralai/Mistral-Small-24B-Instruct\"\n",
    "\n",
    "# Replace with the path to your local PDF file\n",
    "pdf_path = \"document.pdf\"\n",
    "\n",
    "print(\"IONOS_API_TOKEN: \", IONOS_API_TOKEN[:10], \"...\")\n",
    "\n",
    "\n",
    "def pdf_page_to_base64(pdf_path: str, page_index: int, scale: float = 2.0) -> str:\n",
    "    doc = pdfium.PdfDocument(pdf_path)\n",
    "    bitmap = doc[page_index].render(scale=scale)\n",
    "    buf = io.BytesIO()\n",
    "    bitmap.to_pil().save(buf, format=\"PNG\")\n",
    "    return base64.b64encode(buf.getvalue()).decode()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Extract text from page 1 using LightOnOCR-2-1B\n",
    "\n",
    "LightOnOCR-2-1B is optimized for pure text extraction from scanned documents and complex layouts. It always returns Markdown-formatted text and does not require a prompt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "image_b64 = pdf_page_to_base64(pdf_path, page_index=0)\n",
    "\n",
    "start = time.time()\n",
    "response = client.chat.completions.create(\n",
    "    model=LIGHTON_MODEL,\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": [\n",
    "                {\n",
    "                    \"type\": \"image_url\",\n",
    "                    \"image_url\": {\"url\": f\"data:image/png;base64,{image_b64}\"},\n",
    "                }\n",
    "            ],\n",
    "        }\n",
    "    ],\n",
    "    max_tokens=4096,\n",
    "    temperature=0.0,\n",
    ")\n",
    "print(f\"Response time: {time.time() - start:.2f}s\")\n",
    "print(f\"\\nExtracted text:\\n{response.choices[0].message.content}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Extract text from page 1 using Mistral Small 24B\n",
    "\n",
    "Mistral Small 24B is a multimodal language model that accepts a text prompt alongside the image. Use it when you need to combine extraction with document understanding or Q&A."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "start = time.time()\n",
    "response = client.chat.completions.create(\n",
    "    model=MISTRAL_MODEL,\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": [\n",
    "                {\n",
    "                    \"type\": \"text\",\n",
    "                    \"text\": \"Extract all text from this document page and preserve the structure.\",\n",
    "                },\n",
    "                {\n",
    "                    \"type\": \"image_url\",\n",
    "                    \"image_url\": {\"url\": f\"data:image/png;base64,{image_b64}\"},\n",
    "                },\n",
    "            ],\n",
    "        }\n",
    "    ],\n",
    "    max_tokens=4096,\n",
    "    temperature=0.0,\n",
    ")\n",
    "print(f\"Response time: {time.time() - start:.2f}s\")\n",
    "print(f\"\\nExtracted text:\\n{response.choices[0].message.content}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Process all pages in a PDF\n",
    "\n",
    "Iterate over the page count and collect results for each page."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "doc = pdfium.PdfDocument(pdf_path)\n",
    "total_pages = len(doc)\n",
    "print(f\"Document has {total_pages} page(s). Processing with LightOnOCR-2-1B...\\n\")\n",
    "\n",
    "parts = []\n",
    "for i in range(total_pages):\n",
    "    img_b64 = pdf_page_to_base64(pdf_path, i)\n",
    "    resp = client.chat.completions.create(\n",
    "        model=LIGHTON_MODEL,\n",
    "        messages=[\n",
    "            {\n",
    "                \"role\": \"user\",\n",
    "                \"content\": [\n",
    "                    {\n",
    "                        \"type\": \"image_url\",\n",
    "                        \"image_url\": {\"url\": f\"data:image/png;base64,{img_b64}\"},\n",
    "                    }\n",
    "                ],\n",
    "            }\n",
    "        ],\n",
    "        max_tokens=4096,\n",
    "        temperature=0.0,\n",
    "    )\n",
    "    parts.append(f\"--- Page {i + 1} ---\\n\\n{resp.choices[0].message.content}\")\n",
    "    print(f\"Page {i + 1}/{total_pages} done.\")\n",
    "\n",
    "print(\"\\nFull document output:\")\n",
    "print(\"\\n\\n\".join(parts))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}