From Manual to Automated: AI for Online Conferences

Organizing an online conference with 500+ participants means a stream of repetitive questions, constant program changes, and misdirected mailings. Attendees get lost, session lateness increases, and support struggles. Every minute of waiting for an answer — lost trust. We designed an AI system for online conferences with RAG, Q&A moderation, and email automation that takes over the routine: personalizes the program, moderates Q&A, and automates follow-up emails. The organizer focuses on content, not on answering "When will recordings be available?" Result: 95% reduction in response time and 18% increase in attendance. Our solution combines AI conference automation, RAG assistant for events, and AI question moderation for online conferences, leveraging LLM in events for natural language understanding. Estimated cost savings: $5,000 per event based on 40 hours saved at $125/hour.

"AI assistant cut question response time from 30 minutes to 90 seconds — 20x faster than manual support" — from a report on an IT conference with 800 participants.

Problems We Solve

Stream of repetitive questions. Attendees ask: "Where is the link?", "When does it start?", "Who is the speaker?" Answering manually wastes time. The AI assistant loads the event knowledge base and responds instantly with a source citation.

Low attendance. Forgetfulness is the main enemy of conferences. Personalized reminders 24 hours and 1 hour before boost attendance rate by 18% (our case with an IT conference).

Spam in Q&A. Without moderation, questions drown in duplicates and off-topic content. The AI moderator evaluates quality, groups duplicates, and delivers the top-5 questions to the speaker every 20 minutes.

AI System Components

How the System Architecture Works

The system consists of four modules: RAG-based knowledge base (technology RAG), attendee assistant, Q&A moderator, and email automation. Below is an implementation example in Python using LangChain and Anthropic.

from anthropic import Anthropic
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from datetime import datetime, timedelta
import json
import asyncio

client = Anthropic()
emveddings = OpenAIEmbeddings(model="text-embedding-3-small")

class EventKnowledgeBase:
    """Event knowledge base for the AI assistant"""

    def __init__(self, event_id: str):
        self.vectorstore = Chroma(
            collection_name=f"event_{event_id}",
            embedding_function=embeddings,
        )

    def index_event_data(self, event_data: dict):
        """Index event data"""
        texts = []
        metadatas = []

        # Program
        for session in event_data.get("sessions", []):
            text = f"""Section: {session['title']}
Speaker: {session['speaker']} ({session['speaker_bio']})
Time: {session['start_time']} — {session['end_time']}
Description: {session['description']}
Room/Track: {session.get('track', 'general')}"""
            texts.append(text)
            metadatas.append({"type": "session", "session_id": session["id"]})

        # FAQ
        for item in event_data.get("faq", []):
            texts.append(f"Question: {item['q']}\nAnswer: {item['a']}")
            metadatas.append({"type": "faq"})

        # Speakers
        for speaker in event_data.get("speakers", []):
            text = f"""Speaker: {speaker['name']}
Position: {speaker['title']} at {speaker['company']}
Bio: {speaker['bio']}
Topics: {', '.join(speaker.get('topics', []))}"""
            texts.append(text)
            metadatas.append({"type": "speaker", "speaker_name": speaker["name"]})

        self.vectorstore.add_texts(texts=texts, metadatas=metadatas)


class EventAssistant:
    """AI assistant for attendees"""

    def __init__(self, event_id: str, event_name: str):
        self.event_name = event_name
        self.kb = EventKnowledgeBase(event_id)
        self.sessions: dict[str, list] = {}  # dialog history

    def answer(self, participant_id: str, question: str, current_time: datetime = None) -> str:
        results = self.kb.vectorstore.similarity_search_with_score(question, k=5)
        context = "\n\n".join([doc.page_content for doc, score in results if (1 - score) > 0.5])

        time_context = f"Current time: {current_time.strftime('%H:%M')}" if current_time else ""

        history = self.sessions.get(participant_id, [])
        messages = history + [{
            "role": "user",
            "content": f"{time_context}\n\nAttendee question: {question}\n\nEvent information:\n{context}"
        }]

        response = client.messages.create(
            model="claude-haiku-4-5",
            max_tokens=512,
            system=f"""You are an assistant for attendees of the event "{self.event_name}".
Be concise and specific. If information is not available, say so honestly.""",
            messages=messages,
        )

        answer = response.content[0].text
        history.append({"role": "user", "content": question})
        history.append({"role": "assistant", "content": answer})
        self.sessions[participant_id] = history[-10:]

        return answer


class QAModerator:
    """AI moderation for Q&A sessions"""

    def __init__(self):
        self.question_queue: list[dict] = []
        self.answered: list[dict] = []

    def moderate_question(self, question: str, participant: dict) -> dict:
        """Moderate question: relevance, duplicates, quality"""
        # Check duplicate
        if self.question_queue or self.answered:
            existing = [q["question"] for q in self.question_queue + self.answered[-20:]]
            existing_text = "\n".join(f"- {q}" for q in existing[-15:])
        else:
            existing_text = "none"

        response = client.messages.create(
            model="claude-haiku-4-5",
            max_tokens=256,
            messages=[{
                "role": "user",
                "content": f"""Evaluate the question for Q&A session:

Question: "{question}"
Existing questions: {existing_text}

Return JSON:
{{
  "approve": true/false,
  "is_duplicate": true/false,
  "duplicate_of": "similar question or null",
  "quality_score": 1-5,
  "category": "technical|business|personal|off_topic",
  "cleaned_question": "edited version (remove rudeness, typos)",
  "reject_reason": "rejection reason or null"
}}

Approve on-topic questions, not spam or rudeness. Only JSON."""
            }],
        )

        text = response.content[0].text
        result = json.loads(text[text.find("{"):text.rfind("}") + 1])

        if result.get("approve"):
            self.question_queue.append({
                "question": result.get("cleaned_question", question),
                "original": question,
                "participant": participant,
                "quality": result.get("quality_score", 3),
                "category": result.get("category"),
                "submitted_at": datetime.now().isoformat(),
            })

        return result

    def get_top_questions(self, n: int = 5) -> list[dict]:
        """Return top questions by quality score"""
        sorted_queue = sorted(
            self.question_queue,
            key=lambda q: q.get("quality", 3),
            reverse=True
        )
        return sorted_queue[:n]

    def get_summary_for_speaker(self, session_topic: str) -> str:
        """Prepare a summary for the speaker before Q&A"""
        if not self.question_queue:
            return "No questions yet."

        questions_text = "\n".join([
            f"{i+1}. [{q['category']}] {q['question']}"
            for i, q in enumerate(self.get_top_questions(10))
        ])

        response = client.messages.create(
            model="claude-haiku-4-5",
            max_tokens=512,
            messages=[{
                "role": "user",
                "content": f"""Session topic: {session_topic}

Audience questions:
{questions_text}

Group them by theme, highlight the most common ones — help the speaker prepare for Q&A.
Be concise and to the point."""
            }],
        )

        return response.content[0].text

AI Moderator Assessment of Question Quality

The moderator checks each question for duplicates, relevance, quality, and category (technical, business, off-topic). Low-quality questions are rejected, duplicates grouped. The top 5 questions by quality are sent to the speaker every 20 minutes.

Real-Time Engagement Analytics

For collecting and analyzing attendee action logs, the EngagementAnalytics class is used. It records events (views, questions, reactions, drop-offs) and generates a report with metrics: average viewing time, drop-off points, number of questions. The AI processes the statistics and offers recommendations for future events.

Benefits and Implementation

Advantages of AI Moderation Over Manual Moderation

Manual moderation requires a dedicated person per track. The AI moderator processes questions in parallel, evaluates their quality on a 1–5 scale, and filters out duplicates. In our case with an IT conference of 800 participants, the AI moderated 340 questions over 2 days, delivering only the top 5 every 20 minutes to the speaker. Speakers noted that questions became more relevant and insightful.

Personalized Emails for Attendance Boost

Each attendee receives an email with a reminder of their track start, a link to materials, and a personal greeting. The mailing only includes sessions the attendee marked in the program. A result from our client's practice: +18% session attendance compared to a conference with generic emails.

Implementation Process

1. Event audit: collect data on program, speakers, attendees.
2. RAG knowledge base integration: upload documents, configure embeddings.
3. AI assistant setup: model configuration, response testing.
4. Q&A moderation configuration: filter rules, question categories.
5. Email automation: templates, segmentation, triggers.
6. Launch engagement analytics: dashboard, reports.
7. Testing and team training.

What's Included in the Work

• AI assistant for attendees: integration with conference platform, knowledge base upload, personalization setup.
• Q&A moderation: duplicate detection algorithms, quality assessment, automatic speaker summary.
• Email automation: reminder templates, session follow-ups, post-event emails with recordings.
• Engagement analytics: dashboard with metrics (average viewing time, drop-off points, number of questions), reports after each session.
• Team training: system administration instructions, log access, first-day support.

Comparison: Manual Management vs AI System

Implementation Timelines

Practical Case: IT Conference, 800 Participants

Scale: 2-day online conference, 22 sessions, 800 registered attendees, 3 parallel tracks.

What we automated:

• Q&A chatbot answered questions about the program, speakers, and technical connection issues (78% of questions)
• Q&A moderation: 340 questions over 2 days, speakers received top 5 quality questions every 20 minutes
• Personalized follow-up emails across 3 tracks (technical, business, management)
• Reminders 24h and 1h before: +18% attendance rate vs previous conference without reminders

Results:

• Organizing team (3 people): saved ~40 hours of operational work over 2 days
• Attendee question response time: 30 min → 90 sec
• Engagement report for each session: ready 5 minutes after session end

We guarantee SLA 99.9%, work under contract with quality assurance. Implementation experience — over 10 projects with audiences from 200 to 2000 attendees. Contact us to calculate the cost for your event. Request a consultation — we'll prepare a proposal within 24 hours. Get demo access to the system — write to us.

LLM Development: Fine-Tuning, RAG, Agents, and Production Deployment

Using GPT‑4 or Claude 3.5 Sonnet through a public API is not a solution — it's just a tool. When the requirement is to "make it like ChatGPT, but on our data," there is a real engineering challenge behind it: from prompt engineering to training a 70B model on your own infrastructure. End-to-end LLM solution development is a complex stack, and we have been doing it for over 5 years. During this time, we have completed over 20 projects in generative AI: from RAG systems for legal departments to custom support agents. Where exactly your task falls depends on data, latency requirements, budget, and how critical confidentiality is.

A typical situation: the client has already tried ChatGPT, but results are unstable — sometimes accurate, sometimes hallucinating. Or they need integration into a corporate portal while complying with security policies. Let's break down each layer of the stack in detail — from RAG to production deployment.

Why Do RAG Systems Break and How to Fix It?

RAG (Retrieval-Augmented Generation) looks simple: find relevant documents, put them in context, get an answer. In practice, it fails in several places.

Chunking without overlap. Classic mistake: chunk_size=512, overlap=0. If the answer lies across two chunks, retrieval won't find either with sufficient confidence. Solution: overlap 15–25% of chunk_size, or better yet, sentence-aware splitting with spaCy or NLTK instead of naive character splitting.

Poor embedder. text-embedding-ada-002 is good for general use, but on legal or medical texts, specialized models like E5-large-v2, BGE-M3, or fine-tuned sentence-transformers on domain data outperform it. Recall@5 differences can be 15–25%.

No re-ranking. Vector search optimizes for speed, not relevance. A cross-encoder re-ranker (ms-marco-MiniLM-L-6-v2, bge-reranker-large) after initial retrieval improves top-3 accuracy with acceptable latency (+50–150ms). This is often more impactful than improving the embedding model.

Hybrid search. Dense vectors alone work poorly on exact queries: names, SKUs, codes. BM25 (sparse) finds exact matches but misses semantics. Hybrid via RRF (Reciprocal Rank Fusion) is the optimal compromise. Qdrant, Weaviate, and pgvector 0.7+ support hybrid search natively.

When to Fine-Tune Instead of Prompt Engineering?

Prompt engineering solves ~70% of LLM adaptation tasks for a domain. The remaining 30% require fine-tuning. Three indicators: the model ignores a specific output format even with detailed prompting; the task requires deep knowledge of specialized vocabulary (medicine, law); you need to significantly reduce token costs by replacing a large model with a smaller specialized one.

LoRA and QLoRA are the standard for SFT. LoRA adds trainable low-rank matrices to attention layers. A typical configuration for Llama-3 8B: r=64, lora_alpha=128, target_modules=["q_proj","v_proj","k_proj","o_proj"] yields ~0.8% trainable parameters, training on one A100 40GB. QLoRA adds 4-bit quantization (NF4) and allows fine-tuning 70B models on two A100 40GB, though speed drops by half compared to bf16.

DPO instead of RLHF. Direct Preference Optimization requires only (chosen, rejected) pairs, not scalar reward signals. DPOTrainer from the trl library (Hugging Face) implements it in a few dozen lines.

Common mistake. A dataset of 500 examples, 5 epochs, validation loss 0.8 — seems fine. But on test, the model degrades on general instructions. Cause: catastrophic forgetting. Solution: add 10–20% general instruction-following examples (Alpaca, FLAN) to the training set to preserve original capabilities.

How to Choose a Base Model: 8B or 70B?

For most tasks, fine-tuning an 8B model is sufficient. 70B is needed when deep reasoning is required or the 8B baseline does not reach the required quality even after fine-tuning. Inference cost for Llama-3 8B via vLLM on A100 is efficient; the exact cost depends on volume.

What Does PagedAttention Bring to Production?

vLLM is the first choice for serving open-source models. PagedAttention is the key technical innovation: KV-cache is managed like virtual memory in an OS, without fragmentation. This yields 2–4x higher throughput compared to naive HuggingFace Transformers inference. The vLLM documentation confirms that continuous batching and PagedAttention are the standard for high-load LLM services.

Typical numbers on A100 80GB for Llama-3 8B (bf16): 400–600 req/s, P50 latency 200–400ms, P99 latency 600–900ms at concurrency 64. For 70B on two A100 with tensor parallelism: 80–120 req/s, P99 latency 1.5–2.5s. AWQ or GPTQ quantization reduces memory consumption by 2x with quality loss within 1–3%.

Multi-Agent Systems

Agents are LLMs with access to tools: search, code execution, API calls, database interaction. Common patterns:

• ReAct (Reason + Act): the model reasons → chooses a tool → observes the result → reasons again. LangChain and LlamaIndex implement it out of the box.
• Multi-agent orchestration: multiple specialized agents with a coordinator on top. Example: coordinator → researcher (search + summarization) → coder (code generation and execution) → critic (verification). Tools: AutoGen (Microsoft), CrewAI, custom implementation on LangGraph.

In production, agent systems are non-deterministic. Essential: guardrails, step limits, logging of each step, human-in-the-loop for critical actions.

How We Work: Stages, Timeline, Deliverables

What Is Included

We deliver:

• Technical documentation (model card, configs, deployment instructions)
• Access to infrastructure (code repository, trained weights)
• 1 month of post-deployment support (consultations, bug fixes)
• Customer team training (2–3 sessions on system operation)

Timeline: basic RAG prototype — 1–2 weeks. Fine-tuning with customer data — 3–6 weeks (including data preparation). Production system with monitoring and retraining — 2–4 months. Cost is calculated individually based on data volume, model complexity, and infrastructure requirements.

We guarantee the quality of the final model with performance benchmarks and ongoing monitoring. Our engineers have hands‑on experience with dozens of production LLM systems.

Want to evaluate your project? Leave a request — we will prepare a preliminary summary within 1–2 business days. Or get a consultation on choosing the approach: RAG, fine-tuning, or hybrid — we will tell you what works best for you. Contact us to discuss your LLM development needs. Schedule a free consultation today.