Control Technology for Power Resources Based on Improved Q Learning Algorithm for Automated Intelligent Control

Run Ma

doi:10.20965/jaciii.2018.p1065

single-jc.php

« previous

JACIII Vol.22 No.7 pp. 1065-1070

doi: 10.20965/jaciii.2018.p1065

(2018)

Paper:

Views over last 60 days: 678

Control Technology for Power Resources Based on Improved Q Learning Algorithm for Automated Intelligent Control

Run Ma

School of Automation, Chongqing Industry Polytechnic College
No.1000 Taoyuan Road, Yubei District, Chongqing 401120, China

Received:

August 23, 2018

Accepted:

June 11, 2018

Published:

November 20, 2018

Keywords:

femtocell, power resource control, Q learning algorithm, intellectualization, message capacity

Abstract

With the advancement in internet technologies, requirements for quality of indoor wireless communication have increased. Femtocell, which is an effective approach to improve indoor communication quality, can provide highly-efficient indoor network services for users. This study puts forward a power resource control method based on Q learning algorithm for improved solutions to the problems of frequency spectrum and power resource allocation of a two-tier femtocell network. The algorithm was further improved, and was compared with the traditional algorithm via a simulation experiment. It was found that the improved Q learning algorithm could enhance the message capacity and control power resource; this provides a reference for the application of Q learning algorithm in femtocell communication.

Tow-tier femtocell network structure

Cite this article as:

R. Ma, “Control Technology for Power Resources Based on Improved Q Learning Algorithm for Automated Intelligent Control,” J. Adv. Comput. Intell. Intell. Inform., Vol.22 No.7, pp. 1065-1070, 2018.

Data files:

References

[1] W. Skulski, A. Ruben, and S. BenZvi, “FemtoDAQ: A Low-Cost Digitizer for SiPM-Based Detector Studies and its Application to the HAWC Detector Upgrade,” IEEE Trans. on Nuclear Science, Vol.64, No.7, pp. 1677-1682, 2017.
[2] W. Zheng, T. Su, W. Li, Z. Lu, and X. Wen, “Distributed energy-efficient power optimization in two-tier femtocell networks,” IEEE Int. Conf. on Communications, pp. 5767-5771, 2012.
[3] M. H. Tai, N. H. Tran, S. M. A. Kazmi, D. H. Kim, and C. S. Hong, “Distributed resource allocation for interference management and QoS guarantee in underlay cognitive femtocell networks,” 2016 18th Asia-Pacific Network Operations and Management Symp., pp. 1-4, 2016.
[4] H. L. Jing, R. B. Ahmad, M. Jusoh, and T. Sabapathy, “Power Management in LTE Femtocell Networks,” Lecture Notes in Electrical Engineering, Vol.344, pp. 265-273, 2015.
[5] D. Kim, T. Park, S. Kim, H. Kim, and S. Choi, “Load Balancing in Two-Tier Cellular Networks With Open and Hybrid Access Femtocells,” IEEE/ACM Trans. on Networking, Vol.24, No.6, pp. 3397-3411, 2016.
[6] B. Banitalebi and J. Abouei, “An efficient multiple access interference suppression scheme in asynchronous femtocells,” IET Communications, Vol.7, No.14, pp. 1439-1448, 2013.
[7] U. Mudenagudi and D. G. Narayan, “A Cross-Layer Interference and Delay-aware Routing Metric for Infrastructure Wireless Mesh Networks,” Int. J. of Ad Hoc and Ubiquitous Computing, Vol.1, No.1, pp. 1, 2016.
[8] Q. Wei, R. Song, and Q. Sun, “Nonlinear Neuro-Optimal Tracking Control via Stable Iterative Q-Learning Algorithm,” Neurocomputing, Vol.168, No.C, pp. 520-528, 2015.
[9] H. Cheng, X. Zhang, J. Yu, and F. Li, “Markov Process Based Retrieval for Encrypted JPEG Images,” Int. Conf. on Availability, Reliability and Security, pp. 417-421, 2015.
[10] R. Zajdel, “Prioritized epoch-incremental Q-learning algorithm,” Theoretical and Applied Informatics, Vol.24, No.2, pp. 159-171, 2012.
[11] A. Berry, A. Gutierrez, M. Huchard, A. Napoli, and A. Sigayret, “Hermes: a simple and efficient algorithm for building the AOC-poset of a binary relation,” Annals of Mathematics and Artificial Intelligence, Vol.72, No.1-2, pp. 45-71, 2014.
[12] K. Lakshmanan and S. Bhatnagar, “A novel Q-learning algorithm with function approximation for constrained Markov decision processes,” 2012 50th Annual Allerton Conf. on Communication, Control, and Computing, pp. 400-405, 2013.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] W. Skulski, A. Ruben, and S. BenZvi, “FemtoDAQ: A Low-Cost Digitizer for SiPM-Based Detector Studies and its Application to the HAWC Detector Upgrade,” IEEE Trans. on Nuclear Science, Vol.64, No.7, pp. 1677-1682, 2017.

[2] [2] W. Zheng, T. Su, W. Li, Z. Lu, and X. Wen, “Distributed energy-efficient power optimization in two-tier femtocell networks,” IEEE Int. Conf. on Communications, pp. 5767-5771, 2012.

[3] [3] M. H. Tai, N. H. Tran, S. M. A. Kazmi, D. H. Kim, and C. S. Hong, “Distributed resource allocation for interference management and QoS guarantee in underlay cognitive femtocell networks,” 2016 18th Asia-Pacific Network Operations and Management Symp., pp. 1-4, 2016.

[4] [4] H. L. Jing, R. B. Ahmad, M. Jusoh, and T. Sabapathy, “Power Management in LTE Femtocell Networks,” Lecture Notes in Electrical Engineering, Vol.344, pp. 265-273, 2015.

[5] [5] D. Kim, T. Park, S. Kim, H. Kim, and S. Choi, “Load Balancing in Two-Tier Cellular Networks With Open and Hybrid Access Femtocells,” IEEE/ACM Trans. on Networking, Vol.24, No.6, pp. 3397-3411, 2016.

[6] [6] B. Banitalebi and J. Abouei, “An efficient multiple access interference suppression scheme in asynchronous femtocells,” IET Communications, Vol.7, No.14, pp. 1439-1448, 2013.

[7] [7] U. Mudenagudi and D. G. Narayan, “A Cross-Layer Interference and Delay-aware Routing Metric for Infrastructure Wireless Mesh Networks,” Int. J. of Ad Hoc and Ubiquitous Computing, Vol.1, No.1, pp. 1, 2016.

[8] [8] Q. Wei, R. Song, and Q. Sun, “Nonlinear Neuro-Optimal Tracking Control via Stable Iterative Q-Learning Algorithm,” Neurocomputing, Vol.168, No.C, pp. 520-528, 2015.

[9] [9] H. Cheng, X. Zhang, J. Yu, and F. Li, “Markov Process Based Retrieval for Encrypted JPEG Images,” Int. Conf. on Availability, Reliability and Security, pp. 417-421, 2015.

[10] [10] R. Zajdel, “Prioritized epoch-incremental Q-learning algorithm,” Theoretical and Applied Informatics, Vol.24, No.2, pp. 159-171, 2012.

[11] [11] A. Berry, A. Gutierrez, M. Huchard, A. Napoli, and A. Sigayret, “Hermes: a simple and efficient algorithm for building the AOC-poset of a binary relation,” Annals of Mathematics and Artificial Intelligence, Vol.72, No.1-2, pp. 45-71, 2014.

[12] [12] K. Lakshmanan and S. Bhatnagar, “A novel Q-learning algorithm with function approximation for constrained Markov decision processes,” 2012 50th Annual Allerton Conf. on Communication, Control, and Computing, pp. 400-405, 2013.