AI Solution for LMS: Grading, Tests, Analytics

Imagine: 300 students, each submitting an essay, lab work, and code review. An instructor spends 15 minutes per submission — that's 75 hours for just one check. Add tests, forum, struggling students. Our AI layer automates 70% of the routine, saving up to 90% of instructor time and cutting infrastructure costs by 40%. For a typical university with 1,200 students, this translates to over $50,000 saved per semester in grading costs alone.

We integrate ML models directly into your LMS (Moodle, Canvas, Teachable). Result: assignment grading in seconds, tests from lecture notes in minutes, and an early warning system that flags at-risk students two weeks before the deadline. Our AI LMS provides automated learning and intelligent assignment grading, test generation, and early warning systems.

One of our clients — a university with 1,200 students, 30 courses — deployed our system based on Claude 3.5 to grade essays in history and philosophy. Over one semester, 18,000 assignments were processed, achieving 88% rubric-based accuracy on automatic grading. Instructors now spend 3 minutes on selective verification instead of 15 per submission. Department budget savings reached 55% due to reduced assistant positions. According to industry data, automation of routine tasks in education reduces operational costs by 30-50%.

How AI Reduces Instructor Workload

Automatic assignment grading is the main driver of savings. LLMs (Claude 3.5, LLaMA 3) grade essays against a rubric, and code is evaluated via tests in Docker plus quality analysis. Typical result: 85% accuracy in full automation, the rest with manual verification. AI checks essays 450 times faster than a human.

from anthropic import Anthropic
import pandas as pd
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

class AssignmentGrader:
    """AI grading for open-ended assignments"""

    def __init__(self, rubric: dict):
        self.rubric = rubric
        self.llm = Anthropic()

    def grade_essay(self, submission: str, model_answer: str) -> dict:
        """Grade essay against rubric using LLM"""
        criteria_text = '\n'.join([
            f"- {criterion}: {max_points} points. {description}"
            for criterion, (max_points, description) in self.rubric.items()
        ])

        response = self.llm.messages.create(
            model="claude-3-5-sonnet-20241022",
            max_tokens=500,
            messages=[{
                "role": "user",
                "content": f"""Grade this student essay according to the rubric.

RUBRIC:
{criteria_text}

MODEL ANSWER (for reference):
{model_answer[:500]}

STUDENT SUBMISSION:
{submission[:800]}

Return JSON:
{{
  "scores": {{"criterion_name": score, ...}},
  "total": total_score,
  "max_total": max_possible,
  "feedback": "specific feedback in Russian",
  "strengths": ["..."],
  "improvements": ["..."]
}}"""
            }]
        )

        import json
        try:
            return json.loads(response.content[0].text)
        except Exception:
            return {'total': 0, 'feedback': 'Automatic grading error', 'error': True}

    def grade_code_assignment(self, code: str, test_cases: list[dict]) -> dict:
        """Grade code: run tests + quality analysis"""
        # Run test cases (in isolated environment)
        test_results = []
        passed = 0
        for tc in test_cases:
            try:
                # In production: Docker sandbox, timeout
                result = self._run_safely(code, tc['input'])
                correct = str(result).strip() == str(tc['expected']).strip()
                test_results.append({'input': tc['input'], 'passed': correct})
                if correct:
                    passed += 1
            except Exception as e:
                test_results.append({'input': tc['input'], 'passed': False, 'error': str(e)})

        functional_score = passed / len(test_cases) * 100

        # Code quality analysis via LLM
        quality_response = self.llm.messages.create(
            model="claude-3-5-sonnet-20241022",
            max_tokens=200,
            messages=[{
                "role": "user",
                "content": f"""Evaluate code quality (1-10) and give brief feedback in Russian.
Consider: readability, efficiency, edge cases, style.

Code:

{code[:600]}

Return JSON: {{"quality_score": 7, "feedback": "..."}}"""
            }]
        )

        import json
        try:
            quality = json.loads(quality_response.content[0].text)
        except Exception:
            quality = {'quality_score': 5, 'feedback': ''}

        return {
            'functional_score': functional_score,
            'quality_score': quality.get('quality_score', 5),
            'total_score': functional_score * 0.7 + quality.get('quality_score', 5) * 3,
            'tests_passed': f"{passed}/{len(test_cases)}",
            'feedback': quality.get('feedback', ''),
            'test_details': test_results
        }

    def _run_safely(self, code: str, input_data) -> str:
        """Placeholder — in production: subprocess + Docker + timeout"""
        return "placeholder"


class QuizGenerator:
    """Generate tests from learning materials"""

    def __init__(self):
        self.llm = Anthropic()

    def generate_quiz(self, content: str, n_questions: int = 5,
                       difficulty: str = 'medium',
                       question_types: list = None) -> list[dict]:
        """Generate quiz from learning material"""
        if question_types is None:
            question_types = ['multiple_choice', 'true_false', 'fill_blank']

        response = self.llm.messages.create(
            model="claude-3-5-sonnet-20241022",
            max_tokens=1000,
            messages=[{
                "role": "user",
                "content": f"""Generate {n_questions} quiz questions in Russian.

Content:
{content[:1500]}

Requirements:
- Difficulty: {difficulty}
- Mix of types: {', '.join(question_types)}
- Test understanding, not memorization
- Include distractors for multiple choice

Return JSON array:
[{{
  "type": "multiple_choice",
  "question": "...",
  "options": ["A) ...", "B) ...", "C) ...", "D) ..."],
  "correct_answer": "A",
  "explanation": "Why this answer is correct"
}}]"""
            }]
        )

        import json
        try:
            return json.loads(response.content[0].text)
        except Exception:
            return []


class EarlyWarningSystem:
    """Early identification of at-risk students"""

    def compute_risk_scores(self, engagement_data: pd.DataFrame) -> pd.DataFrame:
        """
        Risk indicators for dropout/course abandonment:
        - Drop in activity over the last 2 weeks
        - Low grades + slow response time
        - Missed deadlines
        """
        risk_df = engagement_data.copy()

        # Activity trend
        risk_df['activity_trend'] = (
            risk_df['logins_last_week'] - risk_df['logins_week_before']
        ) / (risk_df['logins_week_before'] + 1)

        # Normalized risk factors
        risk_factors = pd.DataFrame({
            'low_grades': (risk_df['avg_score_last_3'] < 0.6).astype(float),
            'declining_activity': (risk_df['activity_trend'] < -0.3).astype(float),
            'missed_deadlines': (risk_df['missed_deadlines_count'] > 1).astype(float),
            'no_login_7d': (risk_df['days_since_last_login'] > 7).astype(float),
            'low_forum_activity': (risk_df['forum_posts_total'] == 0).astype(float),
        })

        # Weighted risk score
        weights = {
            'low_grades': 0.25,
            'declining_activity': 0.25,
            'missed_deadlines': 0.30,
            'no_login_7d': 0.15,
            'low_forum_activity': 0.05
        }

        risk_df['risk_score'] = sum(
            risk_factors[factor] * weight
            for factor, weight in weights.items()
        )

        risk_df['risk_level'] = pd.cut(
            risk_df['risk_score'],
            bins=[0, 0.3, 0.6, 1.0],
            labels=['low', 'medium', 'high']
        )

        return risk_df.sort_values('risk_score', ascending=False)

    def generate_intervention(self, student: dict) -> dict:
        """Recommended intervention by risk level"""
        risk_level = student.get('risk_level', 'low')

        interventions = {
            'low': {
                'action': 'automated_reminder',
                'message': 'Automatic reminder about active assignments',
                'urgency': 'low'
            },
            'medium': {
                'action': 'personalized_email',
                'message': 'Personalized support email generated by LLM',
                'urgency': 'medium',
                'assigned_to': 'system'
            },
            'high': {
                'action': 'mentor_outreach',
                'message': 'Personal contact from mentor/counselor',
                'urgency': 'high',
                'assigned_to': 'human_mentor'
            }
        }

        return interventions.get(risk_level, interventions['low'])

Why the Early Warning System Works

The algorithm analyzes 5 factors: declining logins, low grades, missed deadlines, forum absence. A weighted risk score automatically assigns intervention — from a reminder to a mentor call. Our projects show that implementing such a system reduces dropout rate by 15-25%, directly impacting the institution's budget.

Types of Assignments for Automation

What's Included in the Work

• LMS audit: analyze current architecture, API, constraints.
• ML layer design: select model (Claude, LLaMA, Mistral), vector DB (pgvector, ChromaDB), integration scheme.
• Development: assignment grading, test generation, early warning (as in code above), analytics dashboards.
• Testing: A/B comparison with manual grading, p99 latency measurement, accuracy on your data.
• Deployment: on your server or cloud (SageMaker, Vertex AI), set up CI/CD for model updates.
• Documentation and training: instructions for instructors, API documentation for developers.
• Support: warranty service, model fine-tuning when new courses appear.

Process of Work

1. Analytics: we examine your LMS, collect historical data (grades, logins).
2. Design: choose architecture (RAG, fine-tuning, rule-based), agree on metrics.
3. Implementation: write code, integrate with LMS, deploy infrastructure.
4. Testing: load testing (100+ concurrent requests), quality verification.
5. Deployment: phased rollout — first on 10% of students, then full rollout.

Timeline and How to Start

Project estimation takes 3 to 8 weeks depending on LMS complexity and module set. Cost is calculated individually for your scenario. We offer a turnkey solution — Retrieval-Augmented Generation (RAG) (Wikipedia) is a key pattern used in our architecture. With 5 years in EdTech and over 30 successful projects, we guarantee results. Contact us for a free project assessment: we will analyze your LMS and propose the optimal solution.

Order a pilot project on one course — assess the effect before full implementation. Get a consultation from our engineer: we will assess your project at no cost.

Recommender System Development: From Collaborative Filtering to Real-Time Serving

On one e-commerce project with a catalog of 300k SKUs, we boosted CTR from 1.8% to 4.4% — a 2.4x increase. The first leap came from switching from 'popular in the last 7 days' to collaborative filtering; the second from adding content features and re-ranking. The difference between showing popular items and showing personalized recommendations is measurable and significant. Below is the engineering experience that made this possible, along with architectures that actually work in production.

Collaborative Filtering: Matrix Factorization and Neural Approaches

Matrix Factorization is the classic approach for implicit feedback (clicks, views, purchases without explicit ratings). ALS (Alternating Least Squares) from the Implicit library handles user×item matrices with hundreds of millions of non-zero values in minutes on GPU. Latent factors 64–256, regularization λ=0.01–0.1 are starting parameters. Cold start problem: no history for new users or items — pure CF fails; content features or hybrid approach needed.

Neural Collaborative Filtering (NCF) replaces the dot product with a neural network. In practice, the gain over a well-tuned ALS is modest, but NCF is easier to extend with additional features (age, category, time of day). Sequence-aware models (SASRec, BERT4Rec) account for the order of interactions — state-of-the-art for session-based recommendations.

How to Choose Recommender System Architecture?

The answer depends on data, load, and cold start requirements. Below are three main approaches with selection criteria.

Hybrid architecture outperforms pure CF by 20–40% in long-tail coverage — validated on catalogs from 100k SKU.

Content-Based Filtering: When Interaction History is Scarce

Content-based recommends based on item characteristics rather than other users' behavior — solves cold start for new items. Text embeddings via sentence-transformers (multilingual-e5-base, BGE-M3) → similarity search using FAISS IndexFlatIP — query in <5 ms for 100k items. Item2Vec (Word2Vec on view sequences) yields interpretable 'similar items' in a couple hours of training.

Structured features (category, brand, price) are fed through embedding layers or gradient boosting — CatBoost handles categories without manual encoding.

Why Hybrid Models Work Better?

Production systems are almost always two-level. Stage 1 (Retrieval) — fast selection of 100–500 candidates from 300k items using ALS or Two-Tower model with vector search (FAISS, Qdrant). Stage 2 (Ranking) — heavy ranker on LightGBM or neural network with cross-features, time, device, and session context. LightFM is a good starting point for medium scale without heavy infrastructure. Our practice shows: moving from single-stage to two-stage yields a 15–25% accuracy improvement with only 20–30 ms additional latency.

Real-Time Serving: Architecture Under Load

Latency SLA — 50–100 ms at thousands of requests per second. Base recommendations precomputed (batch job hourly) → Redis by user_id → <5 ms. Real-time re-ranking via Kafka for events (clicks, cart adds) → update of context features. Feature serving — Redis with TTL (views in 24 hours, last clicked item). At 10k req/s, we deploy Redis Cluster with replication.

A/B testing is the only reliable way to measure improvements. Offline metrics do not always correlate with online. Kohavi et al., 'Online Controlled Experiments at Large Scale' (KDD 2013) — a must-read for the team. Test on 5–10% of traffic, monitor CTR, conversion, revenue per session. One of our client systems after hybridization increased revenue by 18% over a month of A/B.

Recommender System Development Timeline

The stages and typical time frames are in the table below. Costs are calculated individually based on catalog scale and latency requirements.

What's Included in Turnkey Development

1. Data audit — user×item matrix density (typically <0.1%), activity distribution, temporal patterns, cold start statistics.
2. Baseline — 'popular' as a simple threshold that is often hard to beat.
3. Iterative improvement — ALS → content features → two-stage → sequence-aware. Each step with A/B.
4. Serving infrastructure — batch precomputation, Redis, real-time re-ranking, Grafana monitoring.
5. Documentation — model card with metrics, deployment instructions, feature descriptions.
6. Team training — session on interpreting results and model fine-tuning.
7. Support — 1 month post-launch (incident fixes, pipeline tuning).

We are a team with 7+ years of experience in recommender systems, having delivered over 30 projects for e-commerce and media. We guarantee transparent A/B testing and documented metric improvements.

Want to assess the growth potential of your catalog? Contact us for a free data audit. Order recommender system development — first prototype within two weeks.

from implicit.als import AlternatingLeastSquares

model = AlternatingLeastSquares(
    factors=64,
    regularization=0.05,
    iterations=15,
    use_gpu=True
)
model.fit(user_item_matrix)

More about the mathematics of recommender systems — in specialized literature.