Welcome to the GitHub repository dedicated to exploring and advancing large recommendation models. This repository will be continuously updated with the latest works and insights in this rapidly evolving field.
Project Page [This Page] | Paper
- Scalability Analysis: This pioneering paper delves into the scalability of large recommendation model architectures, leveraging popular Transformers such as HSTU, Llama, GPT, and SASRec. 🌟
- Comprehensive Study: We conduct an extensive ablation study and parameter analysis on HSTU, uncovering the origins of scaling laws. Our work also enhances the scalability of the traditional Transformer-based sequential recommendation model, SASRec, by integrating effective modules from scalable large recommendation models. 🌟
- Complex User Behavior: This is the first study to assess the performance of large recommendation models on complex user behavior sequence data, pinpointing areas for improvement in modeling intricate user behaviors, including auxiliary information, multi-behaviors, and cross-domain joint modeling. 🌟
- Ranking Tasks Evaluation: To our knowledge, this is the first comprehensive evaluation of large recommendation models on ranking tasks, demonstrating their scalability. Our findings offer valuable insights into designing efficient large ranking recommendation models, with a focus on datasets and hyperparameters. 🌟
🔥🔥🔥 Predictive Models in Sequential Recommendations: Bridging Performance Laws with Data Quality Insights
- Scalability Analysis: This paper introduces a Performance Law to address the scalability of Sequential Recommendation (SR) models by analyzing model performance rather than loss, aiming to optimize computational resource management. 🌟
- Data Quality Extension: The study emphasizes understanding users' interest patterns through their historical interactions and introduces Approximate Entropy (ApEn) as a significant measure of data quality, improving the interaction data analysis critical for scaling law. 🌟
- Comprehensive Study: We propose a novel correlation between model size and performance by fitting metrics such as hit rate (HR) and normalized discounted cumulative gain (NDCG), validated theoretically and experimentally across different models and datasets. 🌟
- Optimizing Model Parameters: This approach facilitates the determination of optimal parameters for embedding dimensions and model layers, as calculated using the Performance Law, and observes potential performance gains when scaling the model across different frameworks. 🌟
- Comprehensive Review: We present the first systematic review and analysis of the application of generative large models in recommendation systems, offering a foundational understanding of this innovative field. 🌟
- Categorical Framework: Our research classifies current studies on large language models in recommendation systems into three distinct paradigms. This categorization provides a clear and structured overview, facilitating a deeper understanding of the diverse approaches within this emerging discipline. 🌟
- Analysis of Strengths and Challenges: We evaluate the strengths and weaknesses of existing methods, identify key challenges faced by LLM-based recommendation systems, and offer insights to inspire future research in this promising area. 🌟
- Large Recommendation Models
| Title | Link | Date | Code |
|---|---|---|---|
| IFA: Interaction Fidelity Attention for Entire Lifelong Behaviour Sequence Modeling | 2024-6-14 | - | |
| Learning to retrieve user behaviors for click-through rate estimation | 2023-4-8 | GitHub | |
| PinnerFormer: Sequence Modeling for User Representation at Pinterest | 2022-8-14 | - | |
| Linear-time self attention with codeword histogram for efficient recommendation | 2021-6-3 | GitHub | |
| Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction | 2020-10-19 | - | |
| User behavior retrieval for click-through rate prediction | 2020-7-25 | GitHub | |
| Practice on long sequential user behavior modeling for click-through rate prediction | 2019-7-25 | GitHub | |
| Lifelong sequential modeling with personalized memorization for user response prediction | 2019-7-18 | GitHub | |
| Sequential recommendation with user memory networks | 2018-1-2 | - |
| Title | Link | Date | Code |
|---|---|---|---|
| MEANTIME: Mixture of Attention Mechanisms with Multi-temporal Embeddings for Sequential Recommendation | 2020-9-22 | GitHub | |
| Time Matters: Sequential Recommendation with Complex Temporal Information | 2020-7-25 | - | |
| Time Interval Aware Self-Attention for Sequential Recommendation | 2020-1-22 | - | |
| BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer | 2019-11-3 | GitHub | |
| Self-Attentive Sequential Recommendation | 2018-12-30 | GitHub |
| Title | Link | Date | Code |
|---|---|---|---|
| Multi-Behavior Generative Recommendation | 2024-10-21 | GitHub | |
| Denoising Pre-Training and Customized Prompt Learning for Efficient Multi-Behavior Sequential Recommendation | 2024-8-21 | - | |
| Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations | 2024-5-6 | GitHub | |
| Efficient Noise-Decoupling for Multi-Behavior Sequential Recommendation | 2024-3-13 | - | |
| Personalized Behavior-Aware Transformer for Multi-Behavior Sequential Recommendation | 2023-10-27 | GitHub | |
| Coarse-to-fine knowledge-enhanced multi-interest learning framework for multi-behavior recommendation | 2023-8-18 | - | |
| Hierarchical projection enhanced multi-behavior recommendation | 2023-8-4 | GitHub | |
| Compressed interaction graph based framework for multi-behavior recommendation | 2023-4-30 | - | |
| Multi-behavior sequential transformer recommender | 2022-7-7 | GitHub | |
| Multi-view multi-behavior contrastive learning in recommendation | 2022-4-8 | GitHub | |
| Deep multifaceted transformers for multi-objective ranking in large-scale e-commerce recommender systems | 2020-10-19 | GitHub | |
| Multiplex behavioral relation learning for recommendation via memory augmented transformer network | 2020-7-25 | GitHub | |
| Buying or browsing?: Predicting real-time purchasing intent using attention-based deep network with multiple behavior | 2019-7-25 | - | |
| Neural Multi-task Recommendation from Multi-behavior Data | 2019-6-6 | - |
| Title | Link | Date | Code |
|---|---|---|---|
| MF-GSLAE: A Multi-Factor User Representation Pre-training Framework for Dual-Target Cross-Domain Recommendation | 2024-10-24 | GitHub | |
| MDAP: A Multi-view Disentangled and Adaptive Preference Learning Framework for Cross-Domain Recommendation | 2024-10-8 | GitHub | |
| Learning Partially Aligned Item Representation for Cross-Domain Sequential Recommendation | 2024-8-21 | - | |
| Exploring User Retrieval Integration towards Large Language Models for Cross-Domain Sequential Recommendation | 2024-6-5 | GitHub | |
| A Unified Framework for Adaptive Representation Enhancement and Inversed Learning in Cross-Domain Recommendation | 2024-5-30 | - | |
| Triple Sequence Learning for Cross-Domain Recommendation | 2024-2-9 | - | |
| Learning vector-quantized item representation for transferable sequential recommenders | 2023-4-30 | GitHub | |
| Contrastive Cross-Domain Sequential Recommendation | 2022-10-17 | GitHub | |
| Towards universal sequence representation learning for recommender systems | 2022-8-14 | GitHub | |
| RecGURU: Adversarial Learning of Generalized User Representations for Cross-domain Recommendation | 2022-2-15 | GitHub |
| Title | Link | Date | Code |
|---|---|---|---|
| Dataset Regeneration for Sequential Recommendation | 2024-8-24 | GitHub | |
| Entropy Law: The Story Behind Data Compression and LLM Performance | 2024-7-11 | GitHub | |
| A Survey on Data-Centric Recommender Systems | 2024-3-28 | - | |
| Data Management For Training Large Language Models: A Survey | 2023-12-1 | - | |
| Robust preference-guided denoising for graph based social recommendation | 2023-4-30 | GitHub | |
| Autodenoise: Automatic data instance denoising for recommendations | 2023-4-30 | - | |
| An empirical analysis of compute-optimal large language model training | 2022-11-1 | - | |
| Hierarchical item inconsistency signal learning for sequence denoising in sequential recommendation | 2022-10-17 | GitHub | |
| The Datasets Dilemma: How Much Do We Really Know About Recommendation Datasets? | 2022-2-15 | GitHub | |
| Mixgcf: An improved training method for graph neural network-based recommender systems | 2021-8-14 | GitHub | |
| Joint item recommendation and attribute inference: An adaptive graph convolutional network approach | 2020-7-25 | - | |
| Scaling laws for neural language models | 2020-1-23 | - | |
| Enhancing collaborative filtering with generative augmentation | 2019-7-25 | - |
| Title | Link | Date | Code |
|---|---|---|---|
| Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model | 2024-11-1 | - | |
| Toward a Theory of Tokenization in LLMs | 2024-4-12 | - | |
| Text is all you need: Learning language representations for sequential recommendation | 2023-8-4 | - | |
| Recommender systems with generative retrieval | 2023-5-18 | - | |
| Sinkhorn Collaborative Filtering | 2021-6-3 | GitHub | |
| Automated hate speech detection on Twitter | 2019-9-21 | - | |
| ANR: Aspect-based Neural Recommender | 2018-10-17 | - | |
| Neural attentional rating regression with review-level explanations | 2018-4-10 | - | |
| Transnets: Learning to transform for recommendation | 2017-8-27 | - | |
| Neural Collaborative Filtering | 2017-4-3 | GitHub | |
| Joint deep modeling of users and items using reviews for recommendation | 2017-2-2 | - | |
| Variational graph auto-encoders | 2016-11-21 | - | |
| Convolutional matrix factorization for document context-aware recommendation | 2016-9-7 | - | |
| node2vec: Scalable feature learning for networks | 2016-8-13 | - |
If you find our work useful, please cite it using the following references:
@article{scalingnewfrontiers,
title={Scaling New Frontiers: Insights into Large Recommendation Models},
author={Guo, Wei and Wang, Hao and Zhang, Luankang and Chin, Jin Yao and Liu, Zhongzhou and Cheng, Kai and Pan, Qiushi and Lee, Yi Quan and Xue, Wanqi and Shen, Tingjia and Song, Kenan and Wang, Kefan and Xie, Wenjia and Ye, Yuyang and Guo, Huifeng and Liu, Yong and Lian, Defu and Tang, Ruiming and Chen, Enhong},
journal={arXiv preprint arXiv:2412.00714},
year={2024}
}@article{PerformanceLaws,
title={Predictive Models in Sequential Recommendations: Bridging Performance Laws with Data Quality Insights},
author={Shen, Tingjia and Wang, Hao and Wu, Chuhan and Chin, Jin Yao and Guo, Wei and Liu, Yong and Guo, Huifeng and Lian, Defu and Tang, Ruiming and Chen, Enhong},
journal={arXiv preprint arXiv:2412.00430},
year={2024}
}@article{wu2024survey,
title={A survey on large language models for recommendation},
author={Wu, Likang and Zheng, Zhi and Qiu, Zhaopeng and Wang, Hao and Gu, Hongchao and Shen, Tingjia and Qin, Chuan and Zhu, Chen and Zhu, Hengshu and Liu, Qi and others},
journal={World Wide Web},
volume={27},
number={5},
pages={60},
year={2024},
publisher={Springer}
}