Organizational Memory Platform: A Practical Evaluation Pack

Build a small test collection

Use synthetic records or material approved for the evaluation. Include cases where the correct result is uncertainty or a referral to the process owner.

• A current approved instruction with its owner and effective date.
• An older version that has been superseded.
• A recent account that conflicts with the instruction.
• A restricted record available to only one test role.
• A question that the collection cannot answer.

Copy the knowledge-record fields

Complete one record manually so you can judge what the platform preserves. A source link is useful only when it supports the specific statement being made.

• Practice or decision: what someone may need to understand or do.
• Source and date: permitted reference, observation date and recording date.
• Scope: task, location, tools and conditions in which it applies.
• Status: draft, checked for a defined use, disputed, superseded or withdrawn.
• Owner: person responsible for clarifying and maintaining the record.
• Review trigger: date, process change or evidence that should reopen it.
• Use check: what a new reader must confirm before applying it.

Run the same questions through each option

Use the same collection, test roles and expected behaviour when comparing tools. Record the actual result rather than only a yes/no claim from a demonstration.

• Find the current instruction and show what supports it.
• Explain what remains unresolved when two sources disagree.
• Ask from an unauthorised role; inspect the answer, preview and citation for restricted details.
• Correct or withdraw a record and repeat the query, checking copies and cached answers where relevant.
• Ask an unanswerable question and check that the system does not invent a procedure.

Record the result and the operating effort

For each test, keep the expected result, actual output, source checked, failure found and owner of the correction. Add the time needed to prepare, review and maintain the record. A successful search alone does not establish a worthwhile return on the whole system.

Choose a bounded next step

Proceed with the use case that passed the relevant checks, repeat a failed case after correction, or keep a simpler repository. Before extending the collection, assign who handles conflicting instructions and stale records. Treat content imported without an owner as work still to be reviewed.

Why this guide is useful

Use this original evaluation pack during a demonstration or a small internal pilot. It tests the maintenance and use of knowledge, beyond whether a search returns a fluent answer. An existing shared document may be enough for a small collection if someone owns the records and their review.

Test a record through its useful life

NASA’s public Lessons Learned system provides an example of reviewed knowledge organised around the event and recommendations. The following pack is an original business evaluation method; it does not imply that a workplace platform is equivalent to NASA’s system or that a reviewed lesson applies in every setting.

Concrete example

Fictional test: a service team has a February handover guide and an approved August revision. A recent discussion still refers to the old guide. Ask which handover rule applies today. A useful answer identifies the approved revision and the conflicting account, rather than treating the newest conversation as the strongest authority. Repeat the test after restricting access to one source and after the process owner withdraws the revision.

Sources

• NASA: reviewed lessons and their originating events