AI-Reporter

Automated transcription, analysis, and publishing of audio recordings using AI

Quick Start

# Complete pipeline: Audio → PDF
./voiceprocess pipeline event_2025_09_17c

# Individual pipeline stages
./voiceprocess transcribe event_2025_09_17c
./voiceprocess classify event_2025_09_17c
./voiceprocess revise event_2025_09_17c     # Includes sectioning
./voiceprocess publish event_2025_09_17c

Architecture

AI-Reporter uses a consolidated 4-step pipeline with human-in-the-loop review:

Audio → Transcribe → Classify → Revise+Section → Publish → PDF
                         ↓           ↑
                    ┌─────────────────┐
                    │  Human Review   │
                    │  (EventJournal) │
                    └─────────────────┘

Each stage receives an XML tree, transforms it, and passes it to the next stage. Classification happens early to provide context for all subsequent processing. The Review workflow allows users to approve or correct speaker mappings, event types, and sections before final processing.

🧠 Learning-Based Speaker Classification

The system includes a learning-based speaker classification system that:

• Loads historical patterns from 100+ previous events
• Analyzes speaker-role mappings from past classifications
• Provides confidence scores based on pattern frequency
• Automatically activates when AI classification fails
• Ensures reliable speaker assignment even during AI outages

Example: For projekt_planung_ai_reporter events: - speaker1 → Gerd Graßhoff (project_lead): 0.89 confidence, 28 occurrences - speaker2 → Josias Strelow (developer): 0.88 confidence, 28 occurrences

See docs/LEARNING_SYSTEM.md for complete technical details.

🔄 Human-in-the-Loop Review

The new Review Workflow integrates with the EventJournal frontend:

1. Speaker Mapping: Approve or correct AI-suggested speaker names
2. Event Type Selection: Confirm or change event classification
3. Section Structure: Adjust section boundaries and titles
4. Context Enhancement: Add additional context to Kontor

See EventJournal Documentation for details.

Core Components

• XMLContentManager: Central XML handling with lxml
• Pipeline Transforms: Functional transforms (classify, revise_with_sections, publish)
• PipelineRunner: Orchestrates the transformation sequence
• ReviewStateManager: Manages human review state and approvals
• GeminiClient: AI integration for transcription and analysis
• PublishingManager: PDF generation with Typst

Consolidated 4-Step Pipeline

┌─────────────┐     ┌─────────────┐     ┌───────────────────┐     ┌─────────────┐
│  TRANSCRIBE │ ──▶ │  CLASSIFY   │ ──▶ │  REVISE+SECTION   │ ──▶ │   PUBLISH   │
│  Audio→XML  │     │  Type/Role  │     │  Merge+Edit+Sect  │     │  Final XML  │
└─────────────┘     └─────────────┘     └───────────────────┘     └─────────────┘
                           │                      ▲
                           ▼                      │
                    ┌─────────────┐        ┌──────────────┐
                    │   REVIEW    │   ──▶  │   APPROVED   │
                    │  (EventUI)  │        │    CONFIG    │
                    └─────────────┘        └──────────────┘

Step 1: Transcribe

• Audio → XML with speaker diarization
• Output: {event}_transcript.xml

Step 2: Classify

• Identify event type, participants, roles
• Generate pipeline suggestions for review
• Output: {event}_classified.xml + kontor/pipeline_input.yaml

Step 3: Revise with Sections (Consolidated)

• Load approved configuration from review workflow
• Apply speaker name mappings
• Merge consecutive speaker segments
• AI-enhanced narrative revision
• Apply section structure
• Output: {event}_sectioned.xml

Step 4: Publish

• Final publishing format with namespace
• Add processing history
• Output: {event}_publish.xml

File Structure

EventCollections/event_journal/EVENT_NAME/
├── EVENT_NAME.m4a                 # Original audio
├── EVENT_NAME_transcript.xml      # Step 1: Transcription
├── EVENT_NAME_classified.xml      # Step 2: Classification
├── EVENT_NAME_sectioned.xml       # Step 3: Revision + Sectioning
├── EVENT_NAME_publish.xml         # Step 4: Publishing XML
├── EVENT_NAME.pdf                 # Final PDF output
├── kontor/                        # Event-specific configuration
│   ├── review_state.json          # Review status and approvals
│   ├── pipeline_input.yaml        # AI suggestions
│   ├── pipeline_approved.yaml     # User-approved configuration
│   ├── speaker_mapping.yaml       # Speaker ID → Name
│   └── classification.json        # Classification results

Pre-Configured Pipelines

Standard Pipeline (Production)

./voiceprocess pipeline EVENT_NAME

Stages: transcribe → classify → revise_with_sections → publish → PDF

From Transcript (Skip Transcription)

./voiceprocess fromtrans EVENT_NAME

Stages: classify → revise_with_sections → publish → PDF

With Review (Interactive)

from ai_reporter.pipeline_runner import PipelineRunner

runner = PipelineRunner(event_dir=event_dir)
runner.run_with_review()  # Stops for human review after classify

Documentation

• Pipeline Architecture: Complete technical reference
• EventJournal System: Human-in-the-loop review workflow
• Workflow Guide: Two-tier event and reportage architecture
• Kontor System: Knowledge base hierarchy

Installation

# Install dependencies
pip install -r requirements.txt

# Set environment variables
export GEMINI_API_KEY="your-api-key"

# Verify installation
./voiceprocess status

Development

Adding a New Transform

# In ai-reporter/ai_reporter/pipeline_transforms.py

def my_transform(
    xml: XMLContentManager,
    context: PipelineContext,
    **kwargs
) -> XMLContentManager:
    """My custom transformation"""
    logger.info("Transform: my_transform")

    # Transform the XML tree
    # ...

    return xml

Creating a Custom Pipeline

from ai_reporter.pipeline_runner import PipelineRunner
from ai_reporter.pipeline_transforms import transform_classify, transform_revise_with_sections

runner = PipelineRunner(event_dir=event_dir)
runner.add_step('classify', transform_classify)
runner.add_step('revise', transform_revise_with_sections)

final_xml = runner.execute()

Design Principles

1. Consolidated Pipeline: 4 steps instead of 5-7 for clarity
2. Human-in-the-Loop: Review workflow between classify and revise
3. Pure Functions: Each step transforms XML without side effects
4. Tree Preservation: Hierarchical structure maintained throughout
5. Clean Handshake: XML tree passed from step to step

License

OpenScienceTechnology GmbH

Author

Gerd Graßhoff, OpenScienceTechnology GmbH