Auditing Archive-Size Effects in Transformer-Based Semantic Monitoring of AI Research Literature

Abstract

Field-level semantic distinctiveness can be estimated by measuring the semantic distance between a focal paper and previous papers. The methodological background comes from previous research on transformer-based document representation, semantic novelty, and topic dynamics, and the challenges of longitudinal nearest-neighbor monitoring arise from the growth of archives, the choice of representation, and the definition of the neighborhood. This study audits these sources of sensitivity in historical semantic distinctiveness. We establish that, for nested prior-paper sets, adding candidates cannot increase the mean distance to the k nearest prior papers. The audit is based on the title-abstract text of a text-audited, relevance-ranked OpenAlex sample of 5,429 artificial intelligence (AI)-related records from 2018–2025. MiniLM, SciBERT, SPECTER and past-only term frequency–inverse document frequency (TF-IDF) representations are evaluated using exact cosine k-nearest-neighbor search (k=5; k=10 sensitivity) under cumulative memory, and MiniLM is also tested under three-year rolling and fixed-capacity memory, while BERTopic-based topic drift and retrieval provenance are explored as a form of exploratory context. The primary comparison is for 2020–2025. Under cumulative memory, endpoint distinctiveness changed by −12.85% for MiniLM, −8.11% for SciBERT, −22.09% for SPECTER, and −2.27% for TF-IDF. A fixed capacity MiniLM memory, by contrast, resulted in a +9.84% endpoint change. This reversal is considered a sensitivity finding, not evidence of a trend in creativity, research quality, or the whole AI field. The study demonstrates that archive dependence needs to be audited prior to the interpretation of semantic distinctiveness as novelty.