Mohammadreza Soltaniyeh
Santosh Nagarakatte
ABSTRACT
Deep learning-based recommendation systems are extensively deployed in numerous internet services, including social media, entertainment services, and search engines, to provide users with the most relevant and personalized content. Production scale deep learning models consist of large embedding tables with billions of parameters. DRAM-based recommendation systems incur a high infrastructure cost and limit the size of the deployed models. Recommendation systems based on solid-state drives (SSDs) are a promising alternative for DRAM-based systems. Systems based on SSDs can offer ample storage required for deep learning models with large embedding tables. This paper proposes SmartRec, an inference engine for deep learning-based recommendation systems that utilizes Samsung SmartSSD, an SSD with an on-board FPGA that can process data in-situ. We evaluate SmartRec with state-of-the-art recommendation models from Facebook and compare its performance and energy efficiency to a DRAM-based system on a CPU. We show SmartRec improves the energy efficiency of the recommendation inference task up to 10x in comparison to the baseline CPU implementation. In addition, we propose a novel application-specific caching system for SmartSSDs that allows the kernel on the FPGA to use its DRAM as a cache to minimize high latency SSD accesses. Finally, we demonstrate the scalability of our design by offloading the computation to multiple SmartSSDs to further improve performance.
- Assaf Eisenman, Maxim Naumov, Darryl Gardner, Misha Smelyanskiy, Sergey Pupyrev, Kim Hazelwood, Asaf Cidon, and Sachin Katti. 2018. Bandana: Using Non-volatile Memory for Storing Deep Learning Models. arxiv: 1811.05922 [cs.LG]Google Scholar
- Phil Francisco. 2011. The Netezza Data Appliance Architecture: A Platform for High Performance Data Warehousing and Analytics. IBM Redbook (2011).Google Scholar
- Udit Gupta, Samuel Hsia, Vikram Saraph, Xiaodong Wang, Brandon Reagen, Gu-Yeon Wei, Hsien-Hsin S. Lee, David Brooks, and Carole-Jean Wu. 2020 a. DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). 982--995. https://doi.org/10.1109/ISCA45697.2020.00084Google ScholarDigital Library
- Udit Gupta, Carole-Jean Wu, Xiaodong Wang, Maxim Naumov, Brandon Reagen, David Brooks, Bradford Cottel, Kim Hazelwood, Mark Hempstead, Bill Jia, Hsien-Hsin S. Lee, Andrey Malevich, Dheevatsa Mudigere, Mikhail Smelyanskiy, Liang Xiong, and Xuan Zhang. 2020 b. The Architectural Implications of Facebook's DNN-Based Personalized Recommendation. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). 488--501. https://doi.org/10.1109/HPCA47549.2020.00047Google ScholarCross Ref
- Malay Haldar, Mustafa Abdool, Prashant Ramanathan, Tao Xu, Shulin Yang, Huizhong Duan, Qing Zhang, Nick Barrow-Williams, Bradley C. Turnbull, Brendan M. Collins, and et al. 2019. Applying Deep Learning to Airbnb Search. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Jul 2019). https://doi.org/10.1145/3292500.3330658Google ScholarDigital Library
- Kim Hazelwood, Sarah Bird, David Brooks, Soumith Chintala, Utku Diril, Dmytro Dzhulgakov, Mohamed Fawzy, Bill Jia, Yangqing Jia, Aditya Kalro, James Law, Kevin Lee, Jason Lu, Pieter Noordhuis, Misha Smelyanskiy, Liang Xiong, and Xiaodong Wang. 2018. Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective. In 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA) . 620--629. https://doi.org/10.1109/HPCA.2018.00059Google ScholarCross Ref
- Ranggi Hwang, Taehun Kim, Youngeun Kwon, and Minsoo Rhu. 2020. Centaur: A Chiplet-Based, Hybrid Sparse-Dense Accelerator for Personalized Recommendations .IEEE Press, 968--981. https://doi.org/10.1109/ISCA45697.2020.00083Google ScholarDigital Library
- Wenqi Jiang, Zhenhao He, Shuai Zhang, Thomas B. Preuß er, Kai Zeng, Liang Feng, Jiansong Zhang, Tongxuan Liu, Yong Li, Jingren Zhou, Ce Zhang, and Gustavo Alonso. 2021 a. MicroRec: Efficient Recommendation Inference by Hardware and Data Structure Solutions. In Proceedings of Machine Learning and Systems , , A. Smola, A. Dimakis, and I. Stoica (Eds.), Vol. 3. 845--859.Google Scholar
- Wenqi Jiang, Zhenhao He, Shuai Zhang, Kai Zeng, Liang Feng, Jiansong Zhang, Tongxuan Liu, Yong Li, Jingren Zhou, Ce Zhang, and Gustavo Alonso. 2021 b. FleetRec: Large-Scale Recommendation Inference on Hybrid GPU-FPGA Clusters .Association for Computing Machinery, New York, NY, USA, 3097--3105. https://doi.org/10.1145/3447548.3467139Google ScholarDigital Library
- Liu Ke, Udit Gupta, Carole-Jean Wu, Benjamin Youngjae Cho, Mark Hempstead, Brandon Reagen, Xuan Zhang, David Brooks, Vikas Chandra, Utku Diril, Amin Firoozshahian, Kim Hazelwood, Bill Jia, Hsien-Hsin S. Lee, Meng Li, Bert Maher, Dheevatsa Mudigere, Maxim Naumov, Martin Schatz, Mikhail Smelyanskiy, and Xiaodong Wang. 2019. RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing. arxiv: 1912.12953 [cs.DC]Google Scholar
- H.T. Kung, Bradley McDanel, and Sai Qian Zhang. 2019. Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (Providence, RI, USA) (ASPLOS '19). Association for Computing Machinery, New York, NY, USA, 821--834. https://doi.org/10.1145/3297858.3304028Google ScholarDigital Library
- Youngeun Kwon, Yunjae Lee, and Minsoo Rhu. 2019. TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learning. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (Columbus, OH, USA) (MICRO '52). Association for Computing Machinery, New York, NY, USA, 740--753. https://doi.org/10.1145/3352460.3358284Google ScholarDigital Library
- Veronica Lagrange Moutinho dos Reis, Harry (Huan) Li, and Anahita Shayesteh. 2020. Modeling Analytics for Computational Storage. In Proceedings of the ACM/SPEC International Conference on Performance Engineering (Edmonton AB, Canada) (ICPE '20). Association for Computing Machinery, New York, NY, USA, 88--99. https://doi.org/10.1145/3358960.3375794Google ScholarDigital Library
- Yang Li and Zhitao Dai. 2019. Design and Implementation of Hardware Accelerator for Recommendation System based on Heterogeneous Computing Platform. https://doi.org/10.2991/icmeit-19.2019.150Google Scholar
- Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherniavskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, Volodymyr Kondratenko, Stephanie Pereira, Xianjie Chen, Wenlin Chen, Vijay Rao, Bill Jia, Liang Xiong, and Misha Smelyanskiy. 2019. Deep Learning Recommendation Model for Personalization and Recommendation Systems. CoRR , Vol. abs/1906.00091 (2019). https://arxiv.org/abs/1906.00091Google Scholar
- Jongsoo Park, Maxim Naumov, Protonu Basu, Summer Deng, Aravind Kalaiah, Daya Khudia, James Law, Parth Malani, Andrey Malevich, Satish Nadathur, Juan Pino, Martin Schatz, Alexander Sidorov, Viswanath Sivakumar, Andrew Tulloch, Xiaodong Wang, Yiming Wu, Hector Yuen, Utku Diril, Dmytro Dzhulgakov, Kim Hazelwood, Bill Jia, Yangqing Jia, Lin Qiao, Vijay Rao, Nadav Rotem, Sungjoo Yoo, and Mikhail Smelyanskiy. 2018. Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications. arxiv: 1811.09886 [cs.LG]Google Scholar
- Sahand Salamat, Armin Haj Aboutalebi, Behnam Khaleghi, Joo Hwan Lee, Yang Seok Ki, and Tajana Rosing. 2021. NASCENT: Near-Storage Acceleration of Database Sort on SmartSSD. In The 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (Virtual Event, USA) (FPGA '21). Association for Computing Machinery, New York, NY, USA, 262--272. https://doi.org/10.1145/3431920.3439298Google ScholarDigital Library
- Mohammadreza Soltaniyeh, Richard P. Martin, and Santosh Nagarakatte. 2021 a. SPOTS: An Accelerator for Sparse CNNs Leveraging General Matrix-Matrix Multiplication. arxiv: 2107.13386 [cs.AR]Google Scholar
- Mohammadreza Soltaniyeh, Veronica Lagrange Moutinho Dos Reis, Matthew Bryson, Richard Martin, and Santosh Nagarakatte. 2021 b. Near-Storage Acceleration of Database Query Processing with SmartSSDs. In 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). 265--265. https://doi.org/10.1109/FCCM51124.2021.00052Google Scholar
- Moshe Tennenholtz and Oren Kurland. 2019. Rethinking Search Engines and Recommendation Systems: A Game Theoretic Perspective. Commun. ACM , Vol. 62, 12 (Nov. 2019), 66--75. https://doi.org/10.1145/3340922Google ScholarDigital Library
- Jizhe Wang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-Scale Commodity Embedding for E-Commerce Recommendation in Alibaba. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ; Data Mining (London, United Kingdom) (KDD '18). Association for Computing Machinery, New York, NY, USA, 839--848. https://doi.org/10.1145/3219819.3219869Google ScholarDigital Library
- Mark Wilkening, Udit Gupta, Samuel Hsia, Caroline Trippel, Carole-Jean Wu, David Brooks, and Gu-Yeon Wei. 2021. RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (Virtual, USA) (ASPLOS 2021). Association for Computing Machinery, New York, NY, USA, 717--729. https://doi.org/10.1145/3445814.3446763Google ScholarDigital Library
- Xilinx-Samsung. 2021. SmartSSD. https://www.xilinx.com/applications/data-center/computational-storage/smartssd.htmlGoogle Scholar
- Weijie Zhao, Jingyuan Zhang, Deping Xie, Yulei Qian, Ronglai Jia, and Ping Li. 2019 b. AIBox: CTR Prediction Model Training on a Single Node. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (Beijing, China) (CIKM '19). Association for Computing Machinery, New York, NY, USA, 319--328. https://doi.org/10.1145/3357384.3358045Google ScholarDigital Library
- Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019 a. Recommending What Video to Watch next: A Multitask Ranking System. In Proceedings of the 13th ACM Conference on Recommender Systems (Copenhagen, Denmark) (RecSys '19). Association for Computing Machinery, New York, NY, USA, 43--51. https://doi.org/10.1145/3298689.3346997Google ScholarDigital Library
- Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2018a. Deep Interest Evolution Network for Click-Through Rate Prediction. arxiv: 1809.03672 [stat.ML]Google Scholar
- Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018b. Deep Interest Network for Click-Through Rate Prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery; Data Mining (London, United Kingdom) (KDD '18). Association for Computing Machinery, New York, NY, USA, 1059--1068. https://doi.org/10.1145/3219819.3219823Google ScholarDigital Library
Index Terms
- Near-Storage Processing for Solid State Drive Based Recommendation Inference with SmartSSDs®
Recommendations
RecSSD: near data processing for solid state drive based recommendation inference
ASPLOS '21: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating SystemsNeural personalized recommendation models are used across a wide variety of datacenter applications including search, social media, and entertainment. State-of-the-art models comprise large embedding tables that have billions of parameters requiring ...
Towards a deep learning model for hybrid recommendation
WI '17: Proceedings of the International Conference on Web IntelligenceThe deep learning wave is propagating through many research areas and communities. In the last years it quickly propagated to Recommendation Systems, a research area which aims to recommend items to users. Indeed, many deep learning models and ...
A General Rating Recommended Weight-Aware Model for Recommendation System
HCC 2016: Revised Selected Papers of the Second International Conference on Human Centered Computing - Volume 9567In recommendation system, the ratings represent the users' preference and play an important role in recommending items to users. However, the ratings of items may be influenced by many factors, such as time the latest ratings are more able to reflect ...
Comments