← All insights

AI interviews

What an AI interviewer actually measures

Inside the rubric: how role-specific questions, calibrated scoring, and bias checks turn a conversation into evidence.

6 min read

The reasonable fear about AI interviews is that a black box is grading candidates on something nobody can see. That fear is worth taking seriously, because it describes a real failure mode. It is also, in a well-built system, the opposite of what happens. An AI interviewer measures what it has been told to measure, and nothing else. The work is in deciding what that should be.

Here is how ours decides, in the order the decisions get made.

The rubric comes first

Before a single question is written, the role has a brief, and the brief becomes a rubric: the specific capabilities this role needs, weighted by how much they matter. A regulatory affairs specialist and a bioinformatician both sit in life sciences, but their rubrics share almost nothing.

That rubric is written and reviewed by people who understand the field. It is the contract for the whole interview. If a capability is not on it, the interview does not score it.

Questions are built for the role, not the industry

Each rubric item produces questions designed to surface evidence for that item. They are not trivia. They ask candidates to reason through situations from the actual work: a design trade-off, a failed experiment, a contract clause with a hidden risk.

Because the questions come from the rubric, two candidates for the same role answer the same core set. Because they come from the role, candidates for different roles do not.

Follow-ups adapt, scoring does not

A good human interviewer probes. If an answer is vague, they ask what happened next; if it is strong, they push on the hardest part. The interviewer does the same, choosing follow-ups based on the answer it just heard.

What does not adapt is the scoring. Every answer is graded against the same written criteria, anchored with examples of what a weak, adequate and strong response looks like. Calibration means a seven for one candidate means the same thing as a seven for another.

What it deliberately ignores

Accent. Name. Where someone studied. Gaps in a timeline. Whether they used the word 'leverage'. The system is instructed not to weigh any of it, and the scorecard makes it possible to check that it did not, because every score points to the evidence behind it.

This is the part a phone screen could never offer. A human's first impression is formed in seconds and is almost impossible to audit afterwards. A scored transcript can be read by anyone.

What comes out the other end

Not a number. A scorecard: a score per rubric item, the quoted evidence that justified it, and a short summary of where the candidate was strongest and weakest. Hiring managers read it in minutes, and can disagree with it, because the reasoning is on the page.

What it does not measure

Whether you will enjoy working with this person. Whether they will thrive under this particular manager. Whether they are the right bet for where the team is going. Those are judgements, and they belong to the people making the hire. The interviewer's job is to make sure those people are choosing between candidates who can actually do the work.

Hiring for depth?

Submit a brief and receive a ranked shortlist within 48 hours.