AI Candidate Ranking System Development

AI Candidate Ranking System

We develop AI candidate ranking systems that solve the problem of multi-criteria selection: technical skills, relevant experience, cultural fit, career trajectory. Traditional methods like keyword matching achieve ~50% precision on real data. We use multi-vector scoring with separate components for each criterion and weighted aggregation, customizable per role. This not only improves accuracy but also provides an explanation for each score.

A typical recruiting scenario: 400 resumes for a position, an HR manager has time to read 30. The rest are filtered out by surface-level signals—job titles, university names, presence of keywords. This isn't selecting the best candidates—it's selecting the first ones that come along. Our AI ranking system gives a weighted score to each candidate relative to a specific vacancy, explains the score in terms understandable to HR, and doesn't miss strong profiles with non-traditional backgrounds. That's what true AI recruiting and recruitment automation is about.

Where the Naive Approach Breaks Down

The simplest approach—keyword matching or TF-IDF between job description and resume—yields precision of 0.45–0.55 on real data. A candidate with 10 years of experience in "software development" loses to someone who wrote the exact phrase from the JD. Semantics are lost.

The next step—embedding similarity via sentence-transformers—has a different problem: general-purpose models (all-MiniLM-L6-v2, text-embedding-3-small) perform poorly on HR-specific semantics. "Python" in a JD and "Python" in a data analyst's resume versus a backend developer's are different contexts, but the embedding doesn't distinguish. Semantic matching of resumes without context is ineffective.

Even trickier are implicit requirements. A JD requires "team management experience", a candidate writes "led a team of 5 analysts"—semantic match might work, but a fuzzy threshold on cosine similarity (<0.72) would cut it off. Our approach with cross-encoders overcomes this limitation.

How Multi-Dimensional Scoring Works

Not a single score, but a vector of evaluations across key dimensions:

• Hard skills match—semantic matching of technical skills. We use a bi-encoder for initial retrieval (FAISS, top-100), then a cross-encoder for precise scoring (ms-marco-MiniLM-L-12-v2 or fine-tuned on HR data from MS MARCO).
• Experience level—extract years of relevant experience from the resume via NER + regex. Compare with JD requirements.
• Career trajectory—analyze progression: growth in responsibility, relevance of previous positions. LLM call with structured output.
• Education & certifications—rule-based extraction + normalization of university names and certifications.

The final rank is a weighted sum with weights specific to the role. For junior developers, education carries more weight; for seniors, experience trajectory.

Fine-Tuning Models on Historical Data

If historical hiring data is available (who was hired, who passed interviews, who failed), it's gold for training. We build a ranking model: input (vacancy, resume), output—probability of successful hire. LambdaRank or ListNet on top of embedding features.

Caution with bias: if historical data contains bias (e.g., 90% of hires are male for technical positions), the model will reproduce it. A fairness audit before deployment is mandatory. We conduct fairness audits using the AIF360 library.

Explainable AI for HR

A score without explanation is a black box that HR ignores. For each candidate, we generate:

• Top-3 matches with job requirements (with quotes from resume)
• Top-2 gaps (what is required, what is missing)
• One recommendation for additional verification during interviews

This makes ranking a tool, not a replacement for judgment. Explainable AI is a key advantage of our system.

Practical Case: How We Cut Screening Time by 3x

Our client—an outsourcing IT company with 200+ open positions simultaneously. Existing process: manual review in an ATS (Huntflow). Funnel processing time: 8–12 days to first phone call.

We built the system: resume parsing (PDF/DOCX → structured JSON via LLM extraction + regex) → bi-encoder indexing in Qdrant → cross-encoder reranking of top-50 per vacancy → LLM-generated explanations → integration via Huntflow API, scores displayed directly in the candidate card.

Results after 3 months: average time to first call reduced from 9.4 to 3.1 days (3x faster compared to manual screening). HR managers reviewed top-15 instead of top-30 on average (fewer irrelevant ones). Offer rate for hires through the system: +22% compared to historical baseline. Average savings for the client: 4.2 million rubles per year per 100 vacancies. Contact us to discuss a similar project.

Technical Stack

Comparison of ranking methods (precision@10 on our data):

Our approach is 34% more accurate than embedding similarity and provides full explainability. This is an HR Tech and ML for HR solution.

What's Included

• Audit of current recruiting process and data quality
• Prototype development in 2 weeks: bi-encoder + scalar ranking
• Weight calibration with HR department
• Fairness audit and model debiasing
• Integration with ATS (Huntflow, Lever, Greenhouse, Workday)
• Documentation and team training
• 3-month post-launch support

Work Stages

1. Data Audit—quality of resumes in the database, availability of historical hiring data, structure of JDs. This determines the approach.
2. Prototype in 2 weeks—bi-encoder + simple scalar ranking, demo on 50 vacancies.
3. Iterations with HR—weight calibration, validation on cases of "well-known" candidates. Without recruiter input, quality cannot be tuned.
4. Fairness audit—mandatory before deployment, especially if historical data is used.
5. ATS integration—via API for Huntflow, Lever, Greenhouse, or Workday. Or a standalone interface.

Timeline: MVP in 4–6 weeks, full system with fine-tuning and ATS integration in 3–4 months.

Get a consultation on AI ranking implementation by contacting our team. We have 10+ years of experience in production and 40+ successful projects in NLP and recruiting. Request a demo to see the system in action.

Explainable ML: SHAP, LIME, Integrated Gradients, and EU AI Act Requirements

Imagine: a credit scoring model rejects an application. The client demands an explanation, the compliance officer wants detailed documentation. Without built-in explainability (XAI) methods, compliance with modern regulatory requirements is impossible. Our experience includes over 50 projects implementing SHAP, LIME, and Integrated Gradients in production. We guarantee your AI solution becomes transparent, interpretable, and passes audits on the first attempt. Average time to implement basic explanations is 2-4 weeks; full compliance solution takes 6-14 weeks. Contact us for a preliminary assessment of your project.

Why is AI explainability critical for business and compliance?

Explainability is not one task but three distinct requirements. Global explainability shows how the model works overall: which features matter and how they affect predictions on average. Tools: SHAP summary plots, partial dependence plots (PDP), permutation importance. Local explainability explains a specific prediction: why this credit was rejected, which pixels led to a 'cat' classification. Tools: SHAP waterfall, LIME, Integrated Gradients. Contrastive/counterfactual explains what needs to change for a different outcome: 'If income were $10k higher, would the credit be approved?' Tools: DiCE (Diverse Counterfactual Explanations), alibi.

How does SHAP help explain tabular models?

SHAP (SHapley Additive exPlanations) is the standard for tabular data. Based on cooperative game theory: each feature gets a contribution to the deviation of the prediction from the dataset mean. Mathematically correct — satisfies efficiency, symmetry, dummy, and additivity properties.

import shap

explainer = shap.TreeExplainer(lgbm_model)
shap_values = explainer.shap_values(X_test)

# Waterfall plot for a single prediction
shap.plots.waterfall(explainer(X_test)[0])

# Summary for the entire sample
shap.summary_plot(shap_values, X_test, feature_names=feature_names)

TreeExplainer is a fast, accurate algorithm for tree-based models (LightGBM, XGBoost, Random Forest, CatBoost). It computes exact SHAP values in O(TLD²), where T is trees, L is leaves, D is depth. On a model with 1000 trees of depth 6 — milliseconds per explanation. LinearExplainer — for linear models (logistic regression, Ridge) — instant analytical solution. KernelExplainer is model-agnostic, works with any model, but slower: O(2^M) samples for M features. In practice, we use nsamples=1000–5000 as an approximation. For neural networks — DeepExplainer or GradientExplainer.

A common mistake: SHAP values for correlated features are distributed evenly between them — mathematically correct but visually confusing. Features income and income_log have similar SHAP, even though only one is used. Solution: remove duplicate features before training.

When is LIME indispensable?

LIME (Local Interpretable Model-Agnostic Explanations) builds a local linear approximation around the explained example. Faster than SHAP for complex neural networks, but unstable: two runs on the same example may yield different explanations. LIME's strength is text explanations. LimeTextExplainer shows which words influenced the classification. For quick debugging of a text classifier — a convenient tool.

from lime.lime_text import LimeTextExplainer
explainer = LimeTextExplainer(class_names=['neg', 'pos'])
exp = explainer.explain_instance(text, classifier.predict_proba, num_features=10)
exp.show_in_notebook()

What does Integrated Gradients offer for neural networks?

For deep learning models (CNN, Transformer), neither SHAP KernelExplainer nor LIME provides satisfactory explanations: both are too slow or inaccurate. Integrated Gradients (IG) is a gradient-based method, theoretically grounded (axioms completeness, sensitivity, implementation invariance). IG computes the integral of gradients along a straight line from a baseline input (zero or average values) to the actual input. The result is an attribution map showing the contribution of each pixel/token.

from captum.attr import IntegratedGradients

ig = IntegratedGradients(model)
attributions = ig.attribute(
    inputs=input_tensor,
    baselines=baseline_tensor,
    target=predicted_class,
    n_steps=300,
)

The captum library from Meta is the standard for PyTorch. It includes IG, GradCAM, SHAP DeepLift, LayerConductance. GradCAM is simpler, faster, but theoretically weaker. It visualizes which areas of an image the CNN looks at. Sufficient for debugging CV models, but not for compliance documentation.

Comparison of Explainability Methods

EU AI Act: What You Need in Practice

The recently enacted EU AI Act (phased implementation) requires for high-risk systems (credit scoring, medical AI, recruitment systems, law enforcement): technical model documentation, logging of all decisions with audit capability, explanation of each individual decision upon user request, risk assessment and mitigation measures, human oversight. Technically, this means: each prediction must be stored with input features, output, timestamp, model version, and pre-computed explanation. SHAP values are computed at inference and saved alongside the prediction.

For LLM systems, requirements are more complex: there is no standard explanation method, attention weights are not reliable attributions. Current practice: logging full context, retrieved chunks in RAG, chain-of-thought reasoning as proxy explanations. We help determine if the system falls under the high-risk category per Annex III of the EU AI Act, develop a technical model passport (architecture, training data, quality metrics, limitations), configure a decision logging system with retention period (minimum 10 years for some categories), integrate explanation mechanisms into the production pipeline, and implement a user decision appeal procedure.

How We Implement Explainability: Step-by-Step Process

1. Audit and Regulatory Assessment — we determine if the system falls under high-risk category (EU AI Act, GDPR Art. 22, industry requirements Basel IV, MDR). 2-5 days.
2. Integration of Explanations into Inference Pipeline — we connect SHAP, LIME, or IG to the existing service. Configure asynchronous computation with caching. 1-2 weeks.
3. UI Development for Explanations — if a client interface is needed (web dashboard, PDF export). 2-4 weeks.
4. Logging and Audit Setup — we store all inputs, outputs, pre-computed explanations, model version, timestamp. 1-2 weeks.
5. Model Card Documentation — per Google's Model Card Toolkit with breakdown by demographics/subgroups. 1 week.
6. Team Training and Support — documentation handover, engineer training, 3-month SLA support.

What the Result Includes

• Technical model documentation (model card) with intended use, evaluation results by subgroups, limitations, ethical considerations.
• Integrated explanation mechanism (SHAP/LIME/IG) in the production pipeline with automatic saving at inference.
• UI for viewing explanations (web interface or API) with export capability.
• Logging system with retention field configured for EU AI Act requirements.
• Instructions for user decision appeals (for client portal).
• Customer team training (2-3 workshops) and support documentation.

Typical Mistakes in XAI Implementation (and How to Avoid Them)

Feedback and Next Steps

If you need to implement explainability under the EU AI Act, obtain a certified solution, or simply assess the current transparency of your model — request a consultation. We are ready to offer an individualized implementation plan considering your stack (PyTorch, TensorFlow, XGBoost, LLM) and regulatory requirements. Contact us for a detailed cost and timeline estimate for your project.