Short answer
An automated HR interview is useful when it gathers specific employee evidence consistently and leaves interpretation and action with an accountable person. Before buying one, define the decision it will inform, the participant's choices, the follow-up logic, the source record a reviewer can inspect, and the conditions that would stop the pilot. Do not treat scheduling, listening and employment assessment as the same workflow.
First decide what you are automating
The phrase “automated interview” hides several different products. A buyer should name the intended workflow before comparing features.
| Workflow | What the technology does | Appropriate evidence | Main buyer question |
|---|---|---|---|
| Human-led interview | Schedules, prompts or records a live conversation | Interviewer notes and approved source records | Does the tool reduce administration without changing the interviewer's judgment? |
| Fixed asynchronous interview | Presents the same questions to each participant | Written, audio or video responses | Are the questions job-related, accessible and reviewed consistently? |
| Adaptive listening conversation | Asks a follow-up when an answer needs an example or clarification | Participant responses, follow-up trail and cited excerpts | Can a reviewer see how the conclusion relates to the source? |
| Automated assessment | Scores, ranks or recommends a people decision | Validation evidence, subgroup testing and decision records | Is the procedure lawful, job-related and demonstrably fit for the decision? |
The last row belongs in a separate procurement track. The US Equal Employment Opportunity Commission explains that algorithmic tools can disadvantage people with disabilities and that employers need an accommodation process. The UK government's responsible AI in recruitment guidance similarly calls for attention to data protection, accessibility and tool performance across groups. A listening pilot should not drift into candidate or employee scoring simply because the vendor can generate a number.
For the broader distinction between employee-service chatbots and listening conversations, see Conversational AI for HR. This guide starts one step later: commissioning a specific interview workflow and deciding whether its evidence is usable.
Write a one-page scope before seeing a demo
A polished conversation can distract from an unclear purpose. Complete this brief first:
- Decision: What decision or review will the interview inform?
- Participants: Who is invited, who may decline, and who needs another way to take part?
- Moment: Is this onboarding, a check-in, a stay conversation, an exit process or a research exercise?
- Question boundary: Which topics are relevant, and which should route to a person or another process?
- Output: Will the reviewer receive a transcript, attributed excerpts, grouped themes, a manager brief or another defined artefact?
- Human owner: Who checks the source, challenges an inference and decides what happens next?
- Access: Which roles can see individual responses, grouped results and the decision record?
- Retention: What is kept, for how long, and how can a participant ask about the record?
- Evaluation: What observations would justify continuing, changing or stopping the workflow?
“Improve engagement” is too broad. “Understand which parts of the first-week handover confuse new warehouse starters, then let the onboarding owner revise one instruction” is testable. It also makes the interview easier to design because every question can be checked against a concrete use.
Design the evidence trail, not only the questions
A useful adaptive interview does more than ask a friendly question. It preserves the path from an employee's words to a human decision.
Start with a small question guide. Ask for an observable example, the point in a process where it occurred and what the participant tried. A follow-up should resolve a gap in that account. It should not steer the person toward a preferred answer.
Then define the review record. A practical version has four columns:
| Field | Example entry | Reviewer task |
|---|---|---|
| Participant statement | “The handover sheet lists a code I had not seen in training.” | Check the wording and surrounding exchange |
| Supporting context | Week-one starter, late shift, one named document | Confirm whether the context is relevant and sufficiently specific |
| System interpretation | Possible mismatch between training and handover language | Mark as supported, revise it or leave it unresolved |
| Human decision | Compare the two documents and ask two supervisors where the code is introduced | Assign an owner and record the next check |
This structure prevents a plausible summary from becoming evidence by repetition. The NIST AI Risk Management Framework Core recommends documenting scope, knowledge limits, human oversight, testing and feedback processes. Those practices translate directly into an HR interview review record.
Test accessibility as part of the interview
Participation rates alone can conceal who could not use the format. During a pilot, offer a clear route to an alternative and record access problems separately from non-response.
Test the actual devices, browsers, work settings and assistive technologies participants use. Include people who type, dictate, use a keyboard without a mouse or need more time. The EEOC's AI and ADA resources explain why employers should tell people how a tool evaluates them and provide a way to request an accommodation when AI informs employment decisions.
For an internal listening exercise, the safest practical rule is simpler: do not infer lack of interest or competence from failure to complete one interface. Offer another channel and investigate the access failure.
Run a pilot with observable review tests
Avoid a single headline success metric. Review a compact set of observations:
- Reach: invited, started, completed, declined and unable to access, with denominators defined.
- Follow-up quality: whether each follow-up sought relevant clarification rather than repeating or leading.
- Source traceability: whether a reviewer could find the passage behind each material theme.
- Unsupported interpretation: how often reviewers removed or rewrote a conclusion that the source did not support.
- Access failures: where device, channel, language configuration or assistive technology blocked participation.
- Escalation handling: whether a request for human help reached the named owner.
- Action closure: whether the owner recorded a decision and told participants what was reviewed.
Set no universal threshold without baseline data. A pilot is a comparison against your current process under the intended conditions. Keep reviewer corrections visible so the team can see which failure modes persist.
Fictional example: an onboarding handover review
Consider a fictional UK distribution business testing an adaptive interview with one group of new starters. The purpose is not to rate employees or supervisors. It is to find where the written handover and week-one training disagree.
The invitation explains the purpose, who will review the responses and how to request another format. Participants can type or dictate. The interview asks for the first confusing instruction and a concrete example. When someone names a code on the handover sheet, the follow-up asks where they expected that code to have been introduced.
The onboarding owner reads the cited exchange, compares the current training pack with the handover sheet and logs one of three decisions: revise the material, gather more evidence or leave it unchanged with a reason. A later message tells the group what was reviewed. No individual score is created, and the system does not decide whether anyone performed well.
Questions to put to a vendor
Ask for demonstrations against your scope rather than a standard tour:
- Show how a vague answer becomes a specific follow-up.
- Show the exact source behind a generated theme or brief.
- Show what happens when the source does not support a conclusion.
- Explain which channels and accessibility tests were run in conditions like yours.
- Identify every human role in configuration, review, escalation and action.
- Explain access, retention and deletion with the actual product settings.
- State whether any output is used to score or recommend a people decision.
- Show how reviewer corrections are retained and used in later evaluation.
The ICO's AI and data protection guidance provides a useful UK framework for fairness, transparency and risk assessment. Legal review still needs to match the jurisdiction, data and employment context of the proposed workflow.
Where a Lontra trial can fit
Lontra is relevant to an employee-listening workflow in which invited employees type or use dictation, the conversation asks for follow-up detail, and a human reviews what the exchange supports. Managers receive a prepared brief rather than raw employee responses; HR views grouped results only when the group has at least five respondents. These controls do not turn a listening conversation into an employment assessment.
The Lontra product overview shows the current conversation and review model. A trial can test one campaign with up to 30 invitations for 60 days, without a payment card. Use the commissioning brief above to choose one bounded question, record reviewer corrections and make a deliberate continue, change or stop decision.
