Short answer
Evaluate an AI HR tool against one work decision, not a generic promise of insight or efficiency. Bring a realistic fictional case, define the expected output and human review, test sources and access roles, and record what the tool cannot establish. A good demonstration should end with a bounded pilot decision or a clear reason not to proceed.
Define the job before comparing products
“AI for HR” includes very different work: drafting a policy summary, preparing a manager conversation, identifying a missing handover step or helping an HR team search approved guidance. Do not compare those jobs in the same scorecard.
Write a one-sentence test brief:
A manager needs to prepare for a discussion about unclear first-week handovers. Can an authorised reviewer use the output to identify examples, missing context and one question to ask?
This describes a work decision, the population, the permitted material and the person who will check the result. It also prevents a vendor demonstration from becoming a tour of unrelated features.
The CIPD's AI and technology factsheet frames technology selection and use as a people-profession responsibility. Use that as a prompt to examine the work, not only the interface.
Use an evaluation card
| Field | What to test |
|---|---|
| Work question | The decision or conversation the output must support |
| Test material | Fictional, approved examples including missing or conflicting context |
| Expected behaviour | What a useful answer should preserve, ask or decline to infer |
| Evidence | Source, date and context behind each material claim |
| Roles | What participant, manager and reviewer can each see or correct |
| Human action | Who checks the output and what they may decide next |
Ask the presenter to show an unanswerable question and a correction. A system that says it lacks enough information can be more useful than a fluent answer that invents an explanation.
Fictional demonstration test
Fictional example: a UK service team supplies three test accounts. One describes a late handover, one says “everything is fine” but gives no example, and one corrects an earlier statement. The buyer asks the tool to organise the accounts for an HR reviewer.
The buyer checks whether the output separates the employee statement from an interpretation, keeps the correction visible and asks for missing context rather than assigning a cause. They then switch from reviewer to manager access and inspect the actual output, not just a product claim about permissions. The record notes what was shown, what needs retesting and who owns the next decision.
Questions that make a demonstration useful
- What source or user-provided material supports this statement?
- What does the tool do when the source is missing, old or contradictory?
- Can the accountable reviewer inspect, amend or reject the output?
- Which role sees the same material, a summary or nothing at all?
- Is the workflow shown available in the proposed configuration, or simulated?
- What work remains for people after the output is generated?
NIST's AI Risk Management Framework is a voluntary framework for considering trustworthiness through design, use and evaluation. In a buyer evaluation, the practical point is to test the system in conditions close to the intended workflow and make limits visible before relying on it.
Where Lontra fits
Lontra can be evaluated where a focused employee conversation and manager brief would help prepare a human discussion. Managers receive briefs rather than raw employee conversations. Test that narrow workflow with a fictional work question, then decide whether the context is useful to the responsible people. Lontra is not a substitute for an HRIS or an entire people process. Explore the product.
Frequently asked questions
How should an HR team evaluate an AI tool?
Choose one workflow, test realistic fictional cases, inspect sources and roles, record failures and decide whether a bounded pilot is justified.
Does a good AI demonstration prove an HR tool will work?
No. A demonstration shows behaviour in the conditions presented. A pilot must test the intended users, workflow, review effort and operating controls.