A museum description can tell us who created an object, where it travelled and how its interpretation has changed. Turning that story into connected data could help people discover relationships across collections. But how much of the original meaning survives when AI performs the conversion?
Testing AI on museum descriptions
A REEVALUATE-supported study compares a single-prompt approach with a staged AI workflow for building knowledge graphs. It also examines whether validation during processing or repairs afterwards improve the results.
The researchers introduce HeritageKGBench: thirty museum texts annotated using CIDOC-CRM and CACAO, frameworks for representing cultural heritage knowledge.
Correct structure does not guarantee correct meaning
All three approaches tested on the heritage benchmark achieved low extraction accuracy. Repairing the output made it machine-readable for 27 of the 30 texts, but did not resolve the accuracy problem.
The researchers traced important failures to the extraction itself: simplifying events, losing connections and omitting temporal information, figurative language or subjective interpretations.
The distinction matters: software may successfully read a graph even when that graph fails to preserve the source’s meaning.
Keeping uncertainty and context visible
Consider an illustrative catalogue description stating that a vessel was “probably produced in a particular workshop” and “later attributed to another maker.” Converting this into a single, definite creator relationship would remove both uncertainty and the history of interpretation.
For someone searching a collection, that difference could change which objects appear related and what conclusions they draw. A useful digital record should help users understand the evidence behind a connection, including where knowledge remains incomplete.
Why this matters for REEVALUATE
For REEVALUATE, the practical implication is to evaluate AI-generated collection data against the original descriptions, alongside checking its technical structure.
Curators and technical teams could review whether proposed connections preserve dates, qualifications and different interpretations. Keeping links to the source text would make those checks easier and help users investigate a claim.
The openly available HeritageKGBench resource provides a starting point for further testing. Its value lies in helping researchers identify weaknesses and assess improvements using cultural heritage material.
Read the publication
LLM-Based Knowledge Graph Construction for Cultural Heritage: An Extraction-Failure Diagnostic and CIDOC-CRM Benchmark
Ruben Peeters; Xuemin Duan; Anastasia Dimou.
Knowledge Engineering and Knowledge Management, EKAW 2026, Springer, pp. 115–133. First published online on 16 October 2026; proceedings citation year: 2027.
Explore the open HeritageKGBench resource
REEVALUATE is funded by the European Union under Grant Agreement No. 101132389. Views and opinions expressed are those of the authors and do not necessarily reflect those of the European Union or the European Research Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.
