Non-Stationary Transformer Architecture: A Versatile Framework for Recommendation Systems

Yuchen Liu, Gangmin Li*, Terry R. Payne, Yong Yue, Ka Lok Man

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Recommendation systems are crucial in navigating the vast digital market. However, user data’s dynamic and non-stationary nature often hinders their efficacy. Traditional models struggle to adapt to the evolving preferences and behaviours inherent in user interaction data, posing a significant challenge for accurate prediction and personalisation. Addressing this, we propose a novel theoretical framework, the non-stationary transformer, designed to effectively capture and leverage the temporal dynamics within data. This approach enhances the traditional transformer architecture by introducing mechanisms accounting for non-stationary elements, offering a robust and adaptable solution for multi-tasking recommendation systems. Our experimental analysis, encompassing deep learning (DL) and reinforcement learning (RL) paradigms, demonstrates the framework’s superiority over benchmark models. The empirical results confirm our proposed framework’s efficacy, which provides significant performance enhancements, approximately 8% in LogLoss reduction and up to 2% increase in F1 score with other attention-related models. It also underscores its potential applicability across accumulative reward scenarios with pure reinforcement learning models. These findings advocate adopting non-stationary transformer models to tackle the complexities of today’s recommendation tasks.

Original languageEnglish
JournalElectronics (Switzerland)
Volume13
Issue number11
DOIs
Publication statusPublished - Jun 2024

Keywords

  • deep learning
  • non-stationary transformer
  • recommendation systems
  • reinforcement learning
  • user-centric systems

Fingerprint

Dive into the research topics of 'Non-Stationary Transformer Architecture: A Versatile Framework for Recommendation Systems'. Together they form a unique fingerprint.

Cite this