AI Chat Panel

The AI chat panel adds an LLM-powered chat interface to any Panel application or Panelini dashboard. It supports multiple providers, tool execution, streaming responses, and a live preview pane.

Overview

        graph TB
    subgraph ai [" AI Chat Panel "]
        direction TB
        frontend(["AiChat"])
        backend(["AiBackend"])
        iface(["AiInterface"])
        config(["Config Loader"])
        tools(["Tools"])
    end

    subgraph panelini [" Panelini "]
        sidebar(["Left Sidebar"])
        main(["Main Content"])
    end

    subgraph providers [" LLM Providers "]
        anthropic(["Anthropic"])
        azure(["Azure OpenAI"])
    end

    frontend -- "sidebar_objects" --> sidebar
    frontend -- "main_objects" --> main
    frontend --> backend
    backend --> iface
    backend --> config
    backend --> tools
    iface --> anthropic
    iface --> azure

    classDef aiNode fill:#0d7377,stroke:#095c5f,color:#ffffff
    classDef paneliniNode fill:#1e293b,stroke:#334155,color:#f8fafc
    classDef providerNode fill:#6366f1,stroke:#4f46e5,color:#ffffff

    class frontend,backend,iface,config,tools aiNode
    class sidebar,main paneliniNode
    class anthropic,azure providerNode
    

The panel is structured in layers:

• AiChat – UI widgets (chat interface, sidebar controls, preview pane)

• AiBackend – Business logic (provider management, tool execution, message routing)

• AiInterface – Provider-agnostic LLM wrapper (LangChain-based)

• Config – YAML configuration with environment variable resolution

• Tools – Extensible LangChain tool system

Installation

The AI chat panel is an optional extra:

uv add panelini[ai]

or with pip:

pip install panelini[ai]

This installs the required dependencies: langchain, langchain-anthropic, langchain-openai, pyyaml, and python-dotenv.

Quick Start

from panelini import Panelini

app = Panelini(title="My AI App", use_ai=True)
app.servable()

That’s it. The use_ai=True flag adds the chat interface to the main area and provider/model controls to the left sidebar.

Standalone Usage (without Panelini)

import panel as pn

from panelini.panels.ai import AiChat

chat = AiChat(
    system_message="You are a helpful assistant.",
)

# Use sidebar_objects and main_objects in any Panel layout
app = pn.Row(*chat.main_objects)
app.servable()

Configuration Parameters

app = Panelini(
    title="My AI App",
    use_ai=True,
    ai_system_message="You are a data analysis assistant.",
    ai_welcome_message="Hello! How can I help you today?",
    ai_config_path="/path/to/config.yml",
)

Parameter	Type	Description
use_ai	Boolean	Enable the AI chat panel (default: False).
ai_system_message	String	System message for the AI backend (default: "You are a helpful assistant.").
ai_welcome_message	String	Optional greeting posted into a new chat. None (the default) starts it empty.
ai_show_preview	Boolean	Show the markdown preview pane next to the chat (default: False).
ai_config_path	str | Path	Path to a custom config.yml. Auto-discovered if None.

Provider Configuration

Providers and models are defined in a YAML configuration file. The panel searches for configuration in this order:

1. PANELINI_AI_CONFIG_PATH environment variable

2. Walk upward from the working directory looking for config.yml or config.yaml

3. Fall back to the bundled default configuration

Configuration Format

providers:
  anthropic:
    display_name: "Anthropic"
    client_type: "anthropic"
    env_vars:
      api_key: "${ANTHROPIC_API_KEY}"
      endpoint: "${ANTHROPIC_ENDPOINT}"
    models:
      - name: "Claude Sonnet 4.5"
        value: "anthropic/claude-sonnet-4-5"
      - name: "Claude Haiku 4.5"
        value: "anthropic/claude-haiku-4-5"

  azure_openai:
    display_name: "Azure OpenAI"
    client_type: "azure_openai"
    env_vars:
      api_key: "${AZURE_OPENAI_API_KEY}"
      endpoint: "${AZURE_OPENAI_ENDPOINT}"
      api_version: "${AZURE_OPENAI_API_VERSION}"
    models:
      - name: "GPT-4o"
        value: "azure_openai/gpt-4o-2024-11-20"

Model values use the LiteLLM naming convention: provider_prefix/model-id. The prefix is stripped before passing the model name to LangChain. When client_type is omitted, it is derived directly from the first model’s prefix. Bare model names (without a prefix) still work when client_type is set explicitly.

Environment variables referenced with ${VAR_NAME} are resolved at load time. A ValueError is raised if a referenced variable is not set.

Supported Providers

Provider	client_type	Required Environment Variables
Anthropic	anthropic	ANTHROPIC_API_KEY, ANTHROPIC_ENDPOINT
Azure OpenAI	azure_openai	AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_VERSION

UI Layout

When use_ai=True, the panel injects two areas into the Panelini dashboard:

Sidebar Controls

The left sidebar receives two icon tabs. The conversations tab (💬, active by default) holds a single card:

• Conversations – New chat, new folder, import/export chat as JSON, a tree/list view toggle, search over titles and messages, and the user’s conversations in a drag-and-drop folder tree (inline rename, delete with undo); the list view groups them by date instead

The setup tab (⚙️) holds the model controls:

• Provider Settings – Select the LLM provider

• Model Settings – Select the model and adjust temperature

• Basic Tools – Toggle available tools on/off

Main Content

The main area receives the chat interface with streaming responses. With ai_show_preview=True (show_preview=True on AiChat) it turns into a two-column layout:

• Chat (left) – The chat interface with streaming responses

• Preview (right) – A markdown preview pane updated by the update_preview tool

Conversation History

Every chat keeps its conversations per user. Without configuration they live in memory for the lifetime of the process; set PANELINI_HISTORY_DB to a file path to persist them in SQLite, pass your own store as ai_history_store, or pass ai_history_store="browser" to keep each user’s history in their browser’s localStorage (per-browser persistence across reloads and restarts, no server-side database, ~5MB quota). ai_history_view picks the initial sidebar style: "tree" (a drag-and-drop folder tree, the default: organize chats like a filesystem) or "list" (date-grouped); an icon in the New Chat row switches between the two at runtime (per session; a reload starts from ai_history_view again, and a typed search filter carries over the switch). Conversations are owned by the resolved user id (see user_resolver); anonymous visitors get a cookie-backed id.

Folder tree

A chat can be filed in several folders at once, so it appears as one row under each of them. Dragging a row moves that filing; holding Ctrl or Alt over the drop files the chat in the target as well and leaves the row it came from. Taking a row to the top level takes that filing out of its folder and leaves the chat’s other rows alone, the top level being the absence of a folder rather than another folder to file into. A row is one filing, so moving it never changes a row left behind. Rename, delete, outdent and indent are in the row’s context menu, on F2, Delete and Alt+Arrow, and delete is a button on the row as well, at its trailing edge and dimmed until the row is hovered. New chat and new folder are the header buttons. Outdent takes the selected row out of its folder without touching the chat, which is the distinction deleting a row cannot make, and indent files it in the folder above; both are reported as the move a drag is. Deleting a folder that holds chats kept nowhere else asks for a second Delete before removing the folder, its subfolders and those chats; the folder turns red while it waits, and a Cancel appears beside the warning for anyone who changes their mind.

Document model

Each conversation is stored as one JSON document with embedded messages, defined by the bundled chat_history_schema.json. The schema is an OO-LD document: a plain JSON-Schema carrying a JSON-LD @context that maps properties to vocabulary terms (schema.org where a term exists). Its root is a oneOf over two document kinds sharing that context: a conversation and a folder. The same document is the import/export format of the sidebar icons, so a downloaded chat re-imports losslessly here or in any other store.

Three parts of the document the panel carries but does not yet author, so that a chat exported from elsewhere survives the round trip: folder_ids (the set of folders a chat is filed in), parent_id with forked_from_message_id (which chat a fork came from and where it diverged), and per-message attachments. An attachment carries its payload in url, a data: URI when inline and a reference otherwise; a store that dropped the bytes for size reasons says so with omitted rather than leaving the reader to guess.

All backends implement a shared document contract (DocumentHistoryStore): SQLite keeps one documents row per conversation or folder, the in-memory store keeps plain dicts, and the shape maps 1:1 onto a Postgres JSONB column or a browser object store.

Tools

The panel includes a tool system based on LangChain’s BaseTool. Tools are toggled via sidebar checkboxes.

Built-in Tools

Tool	Description
get_current_time	Returns the current date and time with optional timezone support. Enabled by default.
update_preview	Renders markdown content in the preview pane. Supports headers, tables, code blocks, and images.

Streaming vs Tool Mode

• No tools selected – Responses stream token-by-token inside a collapsible <details> block, then expand when complete.

• Tools selected – The model runs a tool execution loop (up to 10 iterations). The final text response is displayed after all tool calls complete.

Adding Custom Tools

Custom tools can be passed directly to AiChat via the tools parameter. They appear as additional checkboxes in the sidebar alongside the built-in tools:

from panelini.panels.ai import AiChat

chat = AiChat(tools=[MyTool()])

To create a custom tool, subclass LangChain’s BaseTool:

from langchain_core.tools import BaseTool
from pydantic import BaseModel, Field


class MyToolInput(BaseModel):
    query: str = Field(description="The search query")


class MyTool(BaseTool):
    name: str = "my_tool"
    description: str = "Describe what this tool does."
    args_schema: type[BaseModel] = MyToolInput

    def _run(self, query: str) -> str:
        return f"Result for: {query}"

    async def _arun(self, query: str) -> str:
        return self._run(query=query)

See examples/panels/ai/ai_chat_custom_tool.py for a complete working example with a LocalStorage tool.

Module Structure

panelini/panels/ai/
├── __init__.py
├── frontend.py          # UI layer (AiChat class)
├── backend.py           # Business logic (AiBackend class)
├── default_config.yml   # Bundled default provider config
├── history/
│   ├── __init__.py
│   ├── store.py                      # Records + ChatHistoryStore interface
│   ├── document.py                   # Document layer + in-memory backend
│   ├── sqlite_store.py               # SQLite backend
│   ├── local_storage_store.py        # Browser localStorage backend
│   ├── default.py                    # Shared default store
│   ├── panel.py                      # Date-grouped sidebar list
│   ├── tree.py                       # TanstackTable folder tree
│   ├── chat_history_schema.json      # Conversation + folder document schema (OO-LD)
│   └── chat_history_schema.sql       # SQLite document table DDL
├── tools/
│   ├── __init__.py
│   └── basic_tools.py   # Built-in tools
└── utils/
    ├── __init__.py
    ├── ai_interface.py   # Provider-agnostic LLM interface
    └── config.py         # YAML config loader

API Reference

AiChat

Standalone AI chat panel. Can be used independently in any Panel app or integrated into Panelini. Exposes widget lists for integration.

Property / Method	Description
AiChat(system_message, welcome_message, config_path, tools)	Constructor. All parameters are optional. Pass custom BaseTool instances via tools.
sidebar_objects	Property returning a list of Panel viewables for the sidebar.
main_objects	Property returning a list of Panel viewables for the main area.

AiBackend

Business logic layer managing providers, models, tools, and message processing.

Method	Description
AiBackend(system_message, config_path)	Constructor. Loads config and creates the initial AI interface.
get_available_providers()	Returns {display_name: ProviderConfig} dict.
get_available_models(provider)	Returns {display_name: ModelConfig} dict for a given provider.
update_provider(provider)	Switch provider, reset model, clear history.
update_model(model)	Switch model, preserve history.
update_temperature(temperature)	Update sampling temperature, preserve history.
update_tools(tools)	Update available tools, preserve history.
process_message(message, use_tools)	Process a user message. Returns {"response": str, "preview_updates": list}.
stream_message(message)	Async generator yielding response token chunks.
clear_history()	Clear conversation history.
export_chat_data(provider, model, temperature)	Export the active conversation as a v2 conversation document.
restore_chat_data(chat_data)	Restore the model context from a v2 document (or legacy export).

AiInterface

Low-level provider-agnostic LLM interface built on LangChain.

Method	Description
AiInterface(provider, model_name, temperature, max_tokens, tools, system_message)	Constructor. Initializes the LLM client and binds tools.
get_response(user_message, stream)	Get a response (streaming or non-streaming).
get_response_with_tools(user_message)	Get a response that may include tool calls.
add_tool(tool)	Dynamically add a tool to the interface.
clear_history()	Clear conversation history.

Configuration Classes

Class / Function	Description
AppConfig	Top-level config dataclass. Contains a providers dict and a default_provider property.
ProviderConfig	Frozen dataclass for a provider (key, display_name, client_type, env_vars, models).
ModelConfig	Frozen dataclass for a model (name, value).
load_config(path=None)	Load and validate a YAML config file. Auto-discovers if path is None.

Data Flow

        sequenceDiagram
    participant User as User
    participant Chat as ChatInterface
    participant FE as AiChat
    participant BE as AiBackend
    participant AI as AiInterface
    participant LLM as LLM Provider

    User->>Chat: Type message
    Chat->>FE: _handle_message()
    FE->>BE: process_message() or stream_message()
    BE->>AI: get_response() or get_response_with_tools()
    AI->>LLM: ainvoke() / astream()
    LLM-->>AI: Response / chunks
    AI-->>BE: Text + tool_calls
    BE->>BE: Execute tools (if any)
    BE-->>FE: {"response", "preview_updates"}
    FE-->>Chat: Yield response
    Chat-->>User: Display message