AI Scoring Limitations and Best Practices

Understand what AI scoring can and cannot evaluate to write better criteria and get more accurate results.

What AI Scoring Can Assess

AI analyzes these data sources:

• Resume content
• Profile fields
• Questionnaire responses
• Evaluation form answers
• Email message content
• Discussion notes
• Job description

What AI Scoring Cannot Assess

The AI can only evaluate information that exists in the data sources. It cannot assess:

• Speaking accent or verbal communication (unless from interview notes)
• Personality traits (unless documented in feedback)
• Physical presence or demeanor
• Skills not mentioned in resume or profile
• Cultural fit (unless specific behaviors are documented)

Key principle: If it's not written down, AI cannot score it.

Premium Model vs Standard

Standard Model

• Uses available profile data only
• Lower credit cost per candidate
• Good for high-volume initial screening

Premium Model

• 2x smarter analysis
• Searches external information about candidates
• Finds company details: products, headcount, industry
• Costs ~10x more AI credits per run
• Best for senior or critical positions

Writing Effective Criteria

Be Specific

Good:

"Evaluate B2B SaaS sales experience. Look for: ARR/MRR metrics, enterprise deal sizes ($50K+), multi-stakeholder sales cycles. Score 0-30 if SMB only, 40-70 if some enterprise experience, 80-100 if proven enterprise track record."

Too vague:

"Good sales experience"

Match Criteria to Available Data

Before writing criteria, consider what data you'll have:

Use Scoring Guidelines

Include explicit scoring instructions:

"Score 0-25 if no relevant experience, 26-50 if 1-2 years, 51-75 if 3-5 years with progression, 76-100 if 5+ years with leadership"

This helps AI apply consistent standards.

Best Practices

Start Simple

• Begin with 3-5 criteria
• Add more only if needed
• Focus on must-have requirements

Weight Appropriately

• Use weights 1-10 for each criterion
• Higher weights = more impact on total score
• Don't make everything weight 10

Pre-Screen First

To save AI credits:

1. Use knockout questions to eliminate unqualified candidates
2. Only AI score candidates who pass basic requirements
3. This reduces credit usage significantly

Review and Adjust

• Check AI scores against your own assessments
• If scores don't match expectations, refine criteria
• More specific prompts = better accuracy

Common Mistakes

1. Criteria too broad: "Good communication" - instead specify what evidence to look for
2. Assessing unavailable info: "Speaking ability" from resume only
3. Missing scoring guidelines: No clear scale for what constitutes 50 vs 80
4. Too many criteria: 10+ criteria dilutes each one's impact
5. All high weights: If everything is weight 10, nothing is prioritized

When to Use Each Model

Use Standard Model For:

• High-volume roles
• Initial screening rounds
• Junior positions
• When profile data is sufficient

Use Premium Model For:

• Executive searches
• Specialized technical roles
• When company background matters
• Final candidate comparisons

Updated on: 22/12/2025