single-jc.php

JACIII Vol.30 No.6 pp. 2084-2101
(2026)

Research Paper:

CxLSTM: Deploying xLSTM-Based Custom Keyword Spotting to Edge Devices

Haitao Cao ORCID Icon, Tianyou Xiao, Chao Chang ORCID Icon, Luyao Teng ORCID Icon, Xi Liu† ORCID Icon, and Dan Deng ORCID Icon

School of Information Engineering, Guangzhou Polytechnic University
No.1342 Shiliang Road, Shawan Street, Panyu District, Guangzhou, Guangdong 511483, China

†Corresponding author

Received:
December 18, 2025
Accepted:
September 1, 2026
Published:
November 20, 2026
Keywords:
custom keyword spotting, extended long short-term memory, edge devices, dynamic quantization
Abstract

Keyword spotting (KWS) serves as the critical entry point for voice interaction in intelligent systems. Traditional KWS systems rely on predefined keywords with limited customization, whereas existing custom keyword spotting (CKWS) approaches incur heavy computational overhead, limiting their deployment on resource-limited edge devices. To address these challenges, we present CxLSTM, an optimized extended LSTM module with only 57.6K parameters, 318.6 KB static memory, 593.1 KB peak memory, and 3.54 MACs per inference. Based on CxLSTM, we propose an on-device CKWS method and perform extensive experiments. In offline evaluation, the proposed method obtains an equal error rate (EER) of 0.27 and an area under the ROC curve (AUC) of 0.78 on the public Google Speech Commands V2 dataset. Superior performance is further validated on a private Mandarin dataset, yielding an EER of 0.06 and an AUC of 0.98. Furthermore, in real-world tests for Mandarin CKWS across three scenarios, the proposed method achieves a recognition accuracy above 95% on Android devices, with a false alarm rate below 0.75/24 h and an average latency of 2.1 ms. When deployed on STM32 microcontrollers, it maintains an accuracy of over 92.3%, with moderately increased false alarms (15.8/24 h) and latency (264.7 ms). Despite performance degradation on low-power microcontrollers due to limited hardware resources, the proposed CKWS method still enables efficient voice interaction on edge devices.

Custom keyword spotting with CxLSTM

Custom keyword spotting with CxLSTM

Cite this article as:
H. Cao, T. Xiao, C. Chang, L. Teng, X. Liu, and D. Deng, “CxLSTM: Deploying xLSTM-Based Custom Keyword Spotting to Edge Devices,” J. Adv. Comput. Intell. Intell. Inform., Vol.30 No.6, pp. 2084-2101, 2026.
Data files:
References
  1. [1] O. Friha, M. A. Ferrag, B. Kantarci, B. Cakmak, A. Ozgun, and N. Ghoualmi-Zine, “Llm-based edge intelligence: A comprehensive survey on architectures, applications, security and trustworthiness,” IEEE Open J. of the Communications Society, Vol.5, pp. 5799-5856, 2024. https://doi.org/10.1109/OJCOMS.2024.3456549
  2. [2] S. Anusha, M. Ezhilvandan, S. Gayathri, V. Sivaraman et al., “Emerging trends in edge AI for Industry 4.0 and 5.0: Technologies, applications, and challenges,” AI Innovations in Service and Tourism Marketing, pp. 211-228, 2024. https://doi.org/10.4018/979-8-3693-7909-7.ch010
  3. [3] Y. Sone and J. Woo, “Design of a Human-Centric Robotic System for User Support Based on Gaze Information,” J. Adv. Comput. Intell. Intell. Inform., Vol.29, No.4, pp. 796-802, 2025. https://doi.org/10.20965/jaciii.2025.p0796
  4. [4] Z. Zhang, R. Li, and K. Chen, “A Parallel CNN-Transformer Framework for Speech Age Recognition,” J. Adv. Comput. Intell. Intell. Inform., Vol.29, No.5, pp. 1137-1144, 2025. https://doi.org/10.20965/jaciii.2025.p1137
  5. [5] C. Mao and S. Liu, “A Study on Speech Recognition by a Neural Network Based on English Speech Feature Parameters,” J. Adv. Comput. Intell. Intell. Inform., Vol.28, No.3, pp. 679-684, 2024. https://doi.org/10.20965/jaciii.2024.p0679
  6. [6] M. Li, “A Lightweight Architecture for Query-by-Example Keyword Spotting on Low-Power IoT Devices,” IEEE Trans. on Consumer Electronics, Vol.69, No.1, pp. 65-75, 2023. https://doi.org/10.1109/TCE.2022.3213075
  7. [7] Y. Wei, Z. Gong, S. Yang, K. Ye, and Y. Wen, “EdgeCRNN: an edge-computing oriented model of acoustic feature enhancement for keyword spotting,” J. of Ambient Intelligence and Humanized Computing, Vol.13, No.3, pp. 1525-1535, 2022. https://doi.org/10.1007/s12652-021-03022-1
  8. [8] L. Lei, G. Yuan, H. Yu, D. Kong, and Y. He, “Multilingual Customized Keyword Spotting Using Similar-Pair Contrastive Learning,” IEEE/ACM Trans. on Audio, Speech, and Language Processing, Vol.31, pp. 2437-2447, 2023. https://doi.org/10.1109/TASLP.2023.3284523
  9. [9] M. Rusci and T. Tuytelaars, “Few-Shot Open-Set Learning for On-Device Customization of KeyWord Spotting Systems,” Interspeech 2023, pp. 2768-2772, 2023. https://doi.org/10.21437/Interspeech.2023-1904
  10. [10] Q. Shao, J. Hou, Y. Hu, Q. Wang, L. Xie, and X. Lei, “Target speaker extraction for customizable query-by-example keyword spotting,” 2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conf. (APSIPA ASC), pp. 672-678, 2021.
  11. [11] Z. Yang, S. Sun, J. Li, X. Zhang, X. Wang, L. Ma, and L. Xie, “CaTT-KWS: A Multi-stage Customized Keyword Spotting Framework based on Cascaded Transducer-Transformer,” Interspeech 2022, pp. 1681-1685, 2022. https://doi.org/10.21437/Interspeech.2022-10258
  12. [12] H. Yan, Q. He, and W. Xie, “CRNN-CTC based mandarin keywords spotting,” ICASSP 2020-2020 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 7489-7493, 2020. https://doi.org/10.1109/ICASSP40776.2020.9054618
  13. [13] S. Li and H. Zhang, “Keyword spotting based on CTC and Similarity Matching for Chinese Speech,” 2023 IEEE/ACIS 23rd Int. Conf. on Computer and Information Science (ICIS), pp. 79-84, 2023. https://doi.org/10.1109/ICIS57766.2023.10210227
  14. [14] T. N. Sainath and C. Parada, “Convolutional neural networks for small-footprint keyword spotting,” Interspeech 2015, pp. 1478-1482, 2015. https://doi.org/10.21437/Interspeech.2015-352
  15. [15] G. Chen, C. Parada, and T. N. Sainath, “Query-by-example keyword spotting using long short-term memory networks,” 2015 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 5236-5240, 2015. https://doi.org/10.1109/ICASSP.2015.7178970
  16. [16] S. Ö. Arik, M. Kliegl, R. Child, J. Hestness, A. Gibiansky, C. Fougner, R. Prenger, and A. Coates, “Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting,” Interspeech 2017, pp. 1606-1610, 2017. https://doi.org/10.21437/Interspeech.2017-1737
  17. [17] S. Majumdar and B. Ginsburg, “MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition,” Interspeech 2020, pp. 3356-3360, 2020. https://doi.org/10.21437/Interspeech.2020-1058
  18. [18] B. Kim, S. Chang, J. Lee, and D. Sung, “Broadcasted Residual Learning for Efficient Keyword Spotting,” Interspeech 2021, pp. 4538-4542, 2021. https://doi.org/10.21437/Interspeech.2021-383
  19. [19] S. Choi, S. Seo, B. Shin, H. Byun, M. Kersner, B. Kim, D. Kim, and S. Ha, “Temporal Convolution for Real-Time Keyword Spotting on Mobile Devices,” Interspeech 2019, pp. 3372-3376, 2019. https://doi.org/10.21437/Interspeech.2019-1363
  20. [20] J. Huang, W. Gharbieh, Q. Wan, H. S. Shim, and H. C. Lee, “QbyE-MLPMixer: Query-by-Example Open-Vocabulary Keyword Spotting using MLPMixer,” Interspeech 2022, pp. 5200-5204, 2022. https://doi.org/10.21437/Interspeech.2022-11080
  21. [21] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems, pp. 6000-6010, 2017.
  22. [22] A. Berg, M. O’Connor, and M. T. Cruz, “Keyword Transformer: A Self-Attention Model for Keyword Spotting,” Interspeech 2021, pp. 4249-4253, 2021. https://doi.org/10.21437/Interspeech.2021-1286
  23. [23] K. Ding, M. Zong, J. Li, and B. Li, “LETR: A Lightweight and Efficient Transformer for Keyword Spotting,” 2022 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 7987-7991, 2022. https://doi.org/10.1109/ICASSP43922.2022.9747295
  24. [24] N. Kitaev, Ł. Kaiser, and A. Levskaya, “Reformer: The efficient transformer,” arXiv preprint, arXiv:2001.04451, 2020. https://doi.org/10.48550/arXiv.2001.04451
  25. [25] Y. Li, G. Yuan, Y. Wen, J. Hu, G. Evangelidis, S. Tulyakov, Y. Wang, and J. Ren, “EfficientFormer: Vision Transformers at MobileNet Speed,” Advances in Neural Information Processing Systems, Vol.35, pp. 12934-12949, 2022. https://doi.org/10.52202/068431-0940
  26. [26] M. Beck, K. Pöppel, M. Spanring, A. Auer, O. Prudnikova, M. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter, “xLSTM: Extended Long Short-Term Memory,” Advances in Neural Information Processing Systems, Vol.37, pp. 107547-107603, 2024. https://doi.org/10.52202/079017-3417
  27. [27] S. Yadav, S. Theodoridis, and Z.-H. Tan, “AxLSTMs: learning self-supervised audio representations with xLSTMs,” Interspeech 2025, pp. 3524-3528, 2025. https://doi.org/10.21437/Interspeech.2025-426
  28. [28] B. Alkin, M. Beck, K. Pöppel, S. Hochreiter, and J. Brandstetter, “Vision-LSTM: xLSTM as Generic Vision Backbone,” The Thirteenth Int. Conf. on Learning Representations, 2025.
  29. [29] Z. Liu, T. Li, and P. Zhang, “RNN-T based open-vocabulary keyword spotting in mandarin with multi-level detection,” 2021 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 5649-5653, 2021. https://doi.org/10.1109/ICASSP39728.2021.9413588
  30. [30] S.-G. Leem, I.-C. Yoo, and D. Yook, “Multitask Learning of Deep Neural Network-Based Keyword Spotting for IoT Devices,” IEEE Trans. on Consumer Electronics, Vol.65, No.2, pp. 188-194, 2019. https://doi.org/10.1109/TCE.2019.2899067
  31. [31] E. van der Westhuizen, H. Kamper, R. Menon, J. Quinn, and T. Niesler, “Feature learning for efficient ASR-free keyword spotting in low-resource languages,” Computer Speech & Language, Vol.71, Article No.101275, 2022. https://doi.org/10.1016/j.csl.2021.101275
  32. [32] K. R, V. K. Kurmi, V. Namboodiri, and C. V. Jawahar, “Generalized Keyword Spotting using ASR embeddings,” Interspeech 2022, pp. 126-130, 2022. https://doi.org/10.21437/Interspeech.2022-10450
  33. [33] Y. Yang, B. Du, Y. Zhang, W. Wang, and Y. Li, “BBS-KWS: The Mandarin Keyword Spotting System Won the Video Keyword Wakeup Challenge,” arXiv preprint, arXiv:2112.01757, 2021. https://doi.org/10.48550/arXiv.2112.01757
  34. [34] E. T. Mekonnen, A. Brutti, and D. Falavigna, “End-to-end low resource keyword spotting through character recognition and beam-search re-scoring,” 2022 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 8182-8186, 2022. https://doi.org/10.1109/ICASSP43922.2022.9746012
  35. [35] B. Labrador, G. Zhao, I. L. Moreno, A. S. Scarpati, L. Fowl, and Q. Wang, “Exploring Sequence-to-Sequence Transformer-Transducer Models for Keyword Spotting,” 2023 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2023. https://doi.org/10.17023/r23c-4m17
  36. [36] A. Parnami and M. Lee, “Few-Shot Keyword Spotting With Prototypical Networks,” Proc. of the 2022 7th Int. Conf. on Machine Learning Technologies (ICMLT ’22), pp. 277-283, 2022. https://doi.org/10.1145/3529399.3529443
  37. [37] J. Lin, K. Kilgour, D. Roblek, and M. Sharifi, “Training keyword spotters with limited and synthesized speech data,” 2020 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 7474-7478, 2020. https://doi.org/10.1109/ICASSP40776.2020.9053193
  38. [38] M. Xu and X.-L. Zhang, “Depthwise Separable Convolutional ResNet with Squeeze-and-Excitation Blocks for Small-Footprint Keyword Spotting,” Interspeech 2020, pp. 2547-2551, 2020. https://doi.org/10.21437/Interspeech.2020-1045
  39. [39] M. Sun, A. Raju, G. Tucker, S. Panchapagesan, G. Fu, A. Mandal, S. Matsoukas, N. Strom, and S. Vitaladevuni, “Max-pooling loss training of long short-term memory networks for small-footprint keyword spotting,” 2016 IEEE Spoken Language Technology Workshop (SLT), pp. 474-480, 2016. https://doi.org/10.1109/SLT.2016.7846306
  40. [40] M. Mazumder, C. Banbury, J. Meyer, P. Warden, and V. J. Reddi, “Few-Shot Keyword Spotting in Any Language,” Interspeech 2021, pp. 4214-4218, 2021. https://doi.org/10.21437/Interspeech.2021-1966
  41. [41] Y. Xi, T. Tan, W. Zhang, B. Yang, and K. Yu, “Text adaptive detection for customizable keyword spotting,” 2022 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 6652-6656, 2022. https://doi.org/10.1109/ICASSP43922.2022.9746647
  42. [42] B. Wei, M. Yang, T. Zhang, X. Tang, X. Huang, K. Kim, J. Lee, K. Cho, and S.-U. Park, “End-to-End Transformer-Based Open-Vocabulary Keyword Spotting with Location-Guided Local Attention,” Interspeech 2021, pp. 361-365, 2021. https://doi.org/10.21437/Interspeech.2021-1335
  43. [43] H. Cao, X. Liu, Z. Tan, Z. Yang, and X. Qin, “Custom Mandarin Keyword Spotting with Extended Long Short-Term Memory,” IAENG Int. J. of Computer Science, Vol.51, No.12, Article No.1933, 2024.
  44. [44] D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V. Le, “SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition,” Interspeech 2019, pp. 2613-2617, 2019. https://doi.org/10.21437/Interspeech.2019-2680
  45. [45] T. Devries and G. W. Taylor, “Improved Regularization of Convolutional Neural Networks with Cutout,” arXiv preprint, arXiv:1708.04552, 2017. https://doi.org/10.48550/arXiv.1708.04552
  46. [46] P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition,” arXiv preprint, arXiv:1804.03209, 2018. https://doi.org/10.48550/arXiv.1804.03209
  47. [47] A. Mesaros, T. Heittola, A. Diment, B. Elizalde, A. Shah, E. Vincent, B. Raj, and T. Virtanen, “DCASE 2017 Challenge Setup: Tasks, Datasets and Baseline System,” Proc. of the Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE2017), pp. 85-92, 2017.
  48. [48] N. Turpault, R. Serizel, A. P. Shah, and J. Salamon, “Sound event detection in domestic environments with weakly labeled data and soundscape synthesis,” Workshop on Detection and Classification of Acoustic Scenes and Events, 2019. https://doi.org/10.33682/006b-jx26

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Nov. 20, 2026