Pattern recognition of user behavior plays an important role in extracting portrait, and critical event sequence extraction is also valuable. Addressing these two issues, an approach of Transformer-based multi-task user behavior prediction is investigated in this paper, named LogSeqTrans model, to enhance the accuracy of predicting user actions and extract critical event sequences. By serializing user behavior data and employing information entropy to identify key events, the proposed LogSeqTrans model processes data through an embedding layer, an encoding layer, and an output layer. The embedding layer converts events and their temporal information into high-dimensional vectors. The encoding layer leverages a multi-head self-attention mechanism to capture sequence dependencies, while the output layer simultaneously predicts behavior types, event occurrence times, and remaining durations. Experimental results demonstrate that the proposed model surpasses other models across three open datasets. Specifically, the average accuracy of LogSeqTrans model for the next activity prediction task significantly outperforming alternative models; Similarly, in the tasks of predicting the next activity occurrence time and the remaining time, the mean absolute errors of LogSeqTrans model are all outperforming comparative models. These results indicate that LogSeqTrans is highly effective in multi-task prediction and capturing complex sequence patterns.

1. [1] L. Guo, L. Hua, R. Jia et al., “Buying or browsing?: Predicting real-time purchasing intent using attention-based deep network with multiple behavior,” Proc. of the 25th ACM SIGKDD Int. Conf. on Knowledge Discovery & Data Mining, pp. 1984-1992, 2019. https://doi.org/10.1145/3292500.3330670
2. [2] D. Koehn, S. Lessmann, and M. Schaal, “Predicting online shop behavior from clickstream data using deep learning,” Expert Systems with Applications, Vol.150, Article No.113342, 2020. https://doi.org/10.1016/j.eswa.2020.113342
3. [3] Y. Seo, “Analyzing factors influencing customer satisfaction with organic coffee in e-commerce,” Int. J. Automation Technol., Vol.18, No.6, pp. 794-801, 2024. https://doi.org/10.20965/ijat.2024.p0794
4. [4] J. Chen and A. Abdul, “A session-based customer preference learning method by using the gated recurrent units with attention function,” IEEE Access, Vol.7, pp. 17750-17759, 2019. https://doi.org/10.1109/ACCESS.2019.2895647
5. [5] A. Adadi, “A survey on data‐efficient algorithms in big data era,” J. of Big Data, Vol.8, No.1, Article No.24, 2021. https://doi.org/10.1186/s40537-021-00419-9
6. [6] L. Tao, “Improving the Quality of Vocational Education in Higher Vocational Colleges Based on Deep Learning Technology: Student Learning Prediction and Personalized Recommendation,” J. Adv. Comput. Intell. Intell. Inform., Vol.29 No.2, pp. 407-416, 2025. https://doi.org/10.20965/jaciii.2025.p0407
7. [7] Z. Cui, X. Xu, X. Fei et al., “Personalized recommendation system based on collaborative filtering for IoT scenarios,” IEEE Trans. on Services Computing, Vol.13, No.4, pp. 685-695, 2020. https://doi.org/10.1109/TSC.2020.2964552
8. [8] X. Ran, Y. Wang, L. Y. Zhang et al., “A differentially private nonnegative matrix factorization for recommender system,” Information Sciences, Vol.592, pp. 21-35, 2022. https://doi.org/10.1016/j.ins.2022.01.050
9. [9] H. Yuan and A. A. Hernandez, “User Cold Start Problem in Recommendation Systems: A Systematic Review,” IEEE Access, Vol.11, pp. 136958-136977, 2023. https://doi.org/10.1109/ACCESS.2023.3338705
10. [10] S. Zhang, L. Yao, A. Sun et al., “Deep learning based recommender system: A survey and new perspectives,” ACM Computing Surveys (CSUR), Vol.52, No.1, pp. 1-38, 2019. https://doi.org/10.1145/3285029
11. [11] X. Yang and J. A. Esquivel, “Time-aware LSTM neural networks for dynamic personalized recommendation on business intelligence,” Tsinghua Science and Technology, Vol.29, No.1, pp. 185-196, 2023. https://doi.org/10.26599/TST.2023.9010025
12. [12] Y. Xu, Y. Zhu, and J. Yu, “Modeling multiple coexisting category-level intentions for next item recommendation,” ACM Trans. on Information Systems (TOIS), Vol.39, No.3, pp. 1-24, 2021. https://doi.org/10.1145/3441642
13. [13] S. Wu, F. Sun, W. Zhang, X. Ye, and B. Cui, “Graph neural networks in recommender systems: a survey,” ACM Computing Surveys, Vol.55, No.5, pp. 1-37, 2022. https://doi.org/10.1145/3535101
14. [14] C. Wu, F. Wu, Y. Huang, X. Ye, and B. Cui, “User-as-graph: User modeling with heterogeneous graph pooling for news recommendation,” Int. Joint Conf. on Artificial Intelligence, pp. 1624-1630, 2021. https://doi.org/10.24963/ijcai.2021/224
15. [15] L. Cui, Y. Wang, W. Chen, W. Wen, and M. S. Han, “Predicting determinants of consumers’ purchase motivation for electric vehicles: An application of Maslow’s hierarchy of needs model,” Energy Policy, Vol.151, Article No.112167, 2021. https://doi.org/10.1016/j.enpol.2021.112167
16. [16] N. Ding, C. Hu, K. Sun, S. Mensah, and R. Zhang, “Explicit role interaction network for event argument extraction,” Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 3475-3485, 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.254
17. [17] X. He, L. Liao, H. Zhang et al., “Neural collaborative filtering,” Proceedings of the 26th Int. Conf. on World Wide Web, pp. 173-182, 2017. https://doi.org/10.1145/3038912.3052569
18. [18] J. Bu, X. Shen, B. Xu et al., “Improving collaborative recommendation via user-item subgroups,” IEEE Trans. on Knowledge and Data Engineering, Vol.28, No.9, pp. 2363-2375, 2016. https://doi.org/10.1109/TKDE.2016.2566622
19. [19] Q. Yang, S. Zhou, H. Li, and J. Zhang, “Embedding Implicit User Importance for Group Recommendation,” Computers, Materials & Continua, Vol.64, No.3, pp. 1691-1704, 2020. https://doi.org/10.32604/cmc.2020.010256
20. [20] R. Shen, “Prediction method of user behavior label based on the BP neural network,” Scientific Programming, Vol.2022, No.1, Article No.7241956, 2022. https://doi.org/10.1155/2022/7241956
21. [21] R. Chen, J. Zhang, Z. Zhang et al., “A comprehensive social matrix factorization for recommendations with prediction and feedback mechanisms by fusing trust relationships and social tags,” Soft Computing, Vol.26, No.21, pp. 11479-11496, 2022. https://doi.org/10.1007/s00500-022-07440-x
22. [22] X. Cen, Z. Chen, H. Chen et al., “User repurchase behavior prediction for integrated energy supply stations based on the user profiling method,” Energy, Vol.286, Article No.129625, 2024. https://doi.org/10.1016/j.energy.2023.129625
23. [23] R. Jia, R. Li, M. Yu et al., “E-commerce purchase prediction approach by user behavior data,” 2017 Int. Conf. on Computer, Information and Telecommunication Systems (CITS), 2017. https://doi.org/10.1109/CITS.2017.8035294
24. [24] B. Requena, G. Cassani, J. Tagliabue et al., “Shopper intent prediction from clickstream e-commerce data with minimal browsing information,” Scientific Reports, Vol.10, No.1, Article No.16983, 2020. https://doi.org/10.1038/s41598-020-73622-y
25. [25] L. García-Montoya and J. Mahoney, “Critical event analysis in case study research,” Sociological Methods & Research, Vol.52, No.1, pp. 480-524, 2023. https://doi.org/10.1177/0049124120926201
26. [26] J. C. Flanagan, “The critical incident technique,” Psychological Bulletin, Vol.51, No.4, Article No.327, 1954. https://doi.org/10.1037/h0061470
27. [27] Z. Liu, Y. Zhang, Y. Li et al., “Key news event detection and event context using graphic convolution, clustering, and summarizing methods,” Applied Sciences, Vol.13, No.9, Article No.5510, 2023. https://doi.org/10.3390/app13095510
28. [28] A. Javed, A. Irtaza, Y. Khaliq et al., “Replay and key-events detection for sports video summarization using confined elliptical local ternary patterns and extreme learning machine,” Applied Intelligence, Vol.49, pp. 2899-2917, 2019. https://doi.org/10.1007/s10489-019-01410-x
29. [29] K. Toyoda, R. G. K. Ying, A. N. S. Zhang, and P. S. Tan, “Identifying the Key Attributes in an Unlabeled Event Log for Automated Process Discovery,” IEEE Trans. on Services Computing, Vol.17, No.1, pp. 74-81, 2024. https://doi.org/10.1109/TSC.2023.3330175
30. [30] J. Yin, R. Wang, S. Ju et al., “An asymptotic statistical learning algorithm for prediction of key trading events,” IEEE Intelligent Systems, Vol.35, No.2, pp. 25-35, 2020. https://doi.org/10.1109/MIS.2020.2977590
31. [31] Y. Leng, W. Zhao, C. Lin et al., “LDA-based data augmentation algorithm for acoustic scene classification,” Knowledge-Based Systems, Vol.195, Article No.105600, 2020. https://doi.org/10.1016/j.knosys.2020.105600
32. [32] J. Wang, J. Huang, X. Ma, Z. Li, Y. Wang, and D. Yu, “MTLFormer: Multi-Task Learning Guided Transformer Network for Business Process Prediction,” IEEE Access, Vol.11, pp. 76722-76738, 2023. https://doi.org/10.1109/ACCESS.2023.3298305
33. [33] A. P. Yunus, K. Morita, N. C. Shirai, and T. Wakabayashi, “Time series self-attention approach for human motion forecasting: A baseline 2d pose forecasting,” J. Adv. Comput. Intell. Intell. Inform., Vol.27, No.3, pp. 445-457, 2023. https://doi.org/10.20965/jaciii.2023.p0445
34. [34] L. Chen and H. Zhu, “Behavior prediction based on a Commodity Utility-Behavior Sequence model,” Machine Learning with Applications, Vol.9, Article No.100314, 2022. https://doi.org/10.1016/j.mlwa.2022.100314
35. [35] Z. T. Liu, X. Xu, J. She, Z. Yang, and D. Chen, “Electroencephalography emotion recognition based on rhythm information entropy extraction,” J. Adv. Comput. Intell. Intell. Inform., Vol.28, No.5, pp. 1095-1106, 2024. https://doi.org/10.20965/jaciii.2024.p1095
36. [36] M. Camargo, M. Dumas, and O. González-Rojas, “Learning accurate LSTM models of business processes,” 17th Int. Conf. on Business Process Management (BPM 2019), pp. 286-302, 2019. https://doi.org/10.1007/978-3-030-26619-6_19
37. [37] N. DiMauro, A. Appice, and T. M. A. Basile, “Activity prediction of business process instances with inception CNN models,” AI*IA 2019–Advances in Artificial Intelligence: 18th Int. Conf. of the Italian Association for Artificial Intelligence, pp. 348-361, 2019. https://doi.org/10.1007/978-3-030-35166-3_25
38. [38] N. Navarin, B. Vincenzi, M. Polato et al., “LSTM networks for data-aware remaining time prediction of business process instances,” 2017 IEEE Symp. Series on Computational Intelligence (SSCI), 2017. https://doi.org/10.1109/SSCI.2017.8285184