ROMEM: Reshaping Temporal Knowledge Graphs with Geometric Rotation to Crack the Core Problem of Agent Memory
As agent technology driven by Large Language Models (LLMs) matures, building stable and scalable long-term memory systems has become crucial. Knowledge graphs, as an efficient structured memory carrier, face a long-standing challenge: how to effectively handle facts that change over time. Recently, a research team from institutions including the University of Edinburgh and the University of St Andrews released an innovative module called ROMEM, which fundamentally solves this problem by modeling time as continuous geometric rotations in a complex vector space.
The “Static-Dynamic Dilemma” in Agent Memory
Current knowledge graph-based memory systems typically treat time as discrete timestamps attached to facts (such as triples). When faced with the dynamism of the real world, this approach falls into a “static-dynamic dilemma,” manifesting in three flawed strategies:
- Destructive Overwriting: New facts directly overwrite old ones, leading to a permanent loss of historical information and preventing the system from performing historical look-up queries.
- LLM Arbitration: An LLM is invoked for judgment during each data write. In long-term memory scenarios with millions of facts, this introduces unacceptable cost and latency.
- Temporal Recency Sorting: This is the most common solution, but it erroneously prioritizes newer, yet potentially trivial, entries over older, permanently valid knowledge like “Obama was born in Hawaii,” which interferes with the reasoning of downstream LLMs.
ROMEM’s core idea is to move beyond the traditional framework of time as metadata and resolve temporal conflicts internally within the model through geometric methods.
Core Mechanisms: Continuous Geometric Operator and Semantic Velocity Gate
ROMEM is not an entirely new memory system but a plug-and-play temporal engine. It internalizes the processing of temporal information into geometric operations within the embedding space through two core designs.
Continuous Geometric Masking
The key insight of ROMEM is to model time as a continuous phase rotation in a complex vector space. The validity of a fact is no longer determined by a discrete label but by the phase position of its vector at a specific query time. For example, for a dynamic relation like “President of the United States,” when the query time is 2025, the entity vector for “Trump” will rotate to a phase that aligns with the relation vector, thus receiving a high score. When the query time is moved back to 2010, its vector will go out of phase due to continuous rotation, lowering its score, while the vector for “Obama” will rotate into the aligned position. This process is called “geometric masking.” It resolves temporal conflicts in an append-only manner, eliminating the need to delete any historical data and naturally supporting historical look-ups.
Semantic Velocity Gate
To distinguish the temporal properties of different relations (e.g., “born in” is static, while “employed at” is dynamic), ROMEM introduces the “Semantic Velocity Gate.” This is a lightweight multi-layer perceptron (MLP) that takes the semantic embedding of the relation text and outputs a volatility scalar α_r between 0 and 1, which controls the rotation speed of facts under that relation.
- When α_r approaches 0, the relation is considered static, and its vectors barely rotate, thus remaining permanently valid.
- When α_r approaches 1, the relation is considered highly dynamic, and its vectors rotate quickly, ensuring that outdated facts are rapidly “masked.”
This mechanism has powerful zero-shot generalization capabilities, allowing it to assign an appropriate volatility to unseen relations (e.g., “CEO of”) based on semantic understanding, without relying on fixed relation IDs.
Comprehensive Experimental Validation: SOTA on Multiple Benchmarks
The research team rigorously evaluated ROMEM on several tasks, including temporal knowledge graph completion, agent reasoning, and zero-shot generalization, with results showing that its performance comprehensively surpasses existing baselines.
- Temporal Knowledge Graph Completion: On the standard ICEWS05-15 dataset, the ChronoR model integrated with ROMEM (ROMEM-ChronoR) achieved an MRR (Mean Reciprocal Rank) of 72.6, significantly higher than the baseline ChronoR’s 68.4 and the previous SOTA model 3DG-TE’s 69.4.
- Agent Memory Tasks: On the strong temporal reasoning benchmark MultiTQ, ROMEM boosted the baseline system’s MRR from 0.203 to 0.337 and LLM answer accuracy from 0.112 to 0.366. On the static memory benchmark DMR-MSC, ROMEM achieved an MRR of 0.856, demonstrating that its temporal modeling does not harm static knowledge retrieval capabilities.
- Zero-Shot Generalization: On the unseen financial domain benchmark FinTMMBench, ROMEM achieved an MRR of 0.728, showcasing its cross-domain adaptability.
Engineering Value and Industry Impact
ROMEM’s most significant engineering value lies in its “plug-and-play” nature. As an independent temporal module, it can seamlessly replace the knowledge graph embedding component in existing graph memory systems without re-architecting the system. Its append-only design avoids costly database updates and LLM arbitration calls, substantially reducing the operational costs of long-term memory systems. Furthermore, its query mechanism is compatible with mainstream vector retrieval libraries like FAISS, further lowering the engineering barrier for implementation.
In summary, ROMEM offers an efficient, scalable, and easily integrable solution for temporal knowledge graphs and agent long-term memory from a novel geometric perspective, paving the way for building general-purpose agents capable of long-term autonomous operation.
