Deep Reinforcement Learning-Based Algorithm for Dynamic Resource Allocation in Edge Computing
DOI:
https://doi.org/10.63846/fb7zns45Keywords:
Edge Computing, Dynamic Resource Allocation, Deep Reinforcement Learning, Transfer Learning, ScalabilityAbstract
Edge computing has emerged as a pivotal technology to address the demands of low-latency and high-bandwidth applications by processing data closer to the source. However, the dynamic nature of edge environments, characterized by fluctuating workloads and constrained resources, poses significant challenges for efficient resource allocation. Traditional heuristic-based approaches often fail to adapt to real-time variations, while existing reinforcement learning (RL) models struggle with the high-dimensional state and action spaces inherent in edge scenarios. This study proposes a novel deep reinforcement learning (DRL)-based algorithm tailored for dynamic resource allocation in edge computing. Key innovations include the development of a hierarchical or multi-agent DRL model to enhance coordination among decentralized edge nodes, the integration of transfer learning techniques for rapid adaptation to new environments, and the design of lightweight architectures optimized for resource-constrained edge devices. Experimental results demonstrate that the proposed algorithm outperforms traditional methods and state-of-the-art RL models in terms of efficiency, adaptability, and scalability, thereby contributing to the advancement of intelligent edge computing.
References
A. K. Shukla, “Distributed Algorithms for Resource Allocation and Load Balancing,” 2023, doi: 10.52783/tojqi.v11i4.10021.
J. Datta and I. Pan, “Greedy Load Balancing for Cloud Computing Framework,” Indian Journal of Computer Science and Engineering, 2021, doi: 10.21817/indjcse/2021/v12i3/211203255.
J. Kumar, A. Singh, and A. Mohan, “Resource‐efficient Load‐balancing Framework for Cloud Data Center Networks,” Etri Journal, 2020, doi: 10.4218/etrij.2019-0294.
L. Cristancho-Fajardo, P. Ezanno, and E. Vergu, “Dynamic Resource Allocation for Controlling Pathogen Spread on a Large Metapopulation Network,” J R Soc Interface, 2022, doi: 10.1098/rsif.2021.0744.
O. H. Türkakın, D. Arditi, and E. Manisalı, “Comparison of Heuristic Priority Rules in the Solution of the Resource-Constrained Project Scheduling Problem,” Sustainability, 2021, doi: 10.3390/su13179956.
Y. Luo, Q. Hu, Y. Wang, J. Wang, O. Alfarraj, and A. Tolba, “Revenue Optimization of a UAV-Fog Collaborative Framework for Remote Data Collection Services,” Ieee Access, 2020, doi: 10.1109/access.2020.3016779.
K. A. Guimarães Araújo, J. Guedes, and B. de Athayde Prata, “Hybrid Matheuristics for the Multi-Capacitated Clustering Problem,” Rairo - Operations Research, 2022, doi: 10.1051/ro/2022048.
X. Gao, R. Liu, and A. Kaushik, “Hierarchical Multi-Agent Optimization for Resource Allocation in Cloud Computing,” Ieee Transactions on Parallel and Distributed Systems, 2021, doi: 10.1109/tpds.2020.3030920.
A. Ilankumaran, “An Energy-Aware QoS Load Balance Scheduling Using Hybrid GAACO Algorithm for Cloud,” Cybernetics and Information Technologies, 2023, doi: 10.2478/cait-2023-0009.
G. Yıldız and F. E. Sevilgen, “Hyper‐heuristic Method for Processor Allocation in Parallel Tasks Scheduling,” Concurr Comput, 2023, doi: 10.1002/cpe.7757.
M. I. Arasu, S. Rani, and G. R. Geoffery, “Efficient Heuristic for Optimal MILP-LoRa Adaptive Resource Allocation for Aquaculture,” Intelligent Automation & Soft Computing, 2022, doi: 10.32604/iasc.2022.021973.
K. S. Shalini, “Decentralized Machine Learning for Dynamic Resource Optimization in Wireless Networks Using Reinforcement Learning,” Jes, 2024, doi: 10.52783/jes.2539.
J. Liu and L. Zhu, “Joint Resource Allocation Optimization of Wireless Sensor Network Based on Edge Computing,” Complexity, 2021, doi: 10.1155/2021/5556651.
M. Yan, B. Chen, G. Feng, and S. Qin, “Federated Cooperation and Augmentation for Power Allocation in Decentralized Wireless Networks,” Ieee Access, 2020, doi: 10.1109/access.2020.2979323.
X. Liu, L. Zhou, X. Zhang, X. Tan, and J. Wei, “Joint Radio Map Construction and Dissemination in MEC Networks: A Deep Reinforcement Learning Approach,” Wirel Commun Mob Comput, 2022, doi: 10.1155/2022/4621440.
Z. Qin, Z. Cheng, C. Lin, L. Zhou, and L. Wang, “Optimal Workload Allocation for Edge Computing Network Using Application Prediction,” Wirel Commun Mob Comput, 2021, doi: 10.1155/2021/5520455.
A. H. Salem and G. H. Algaphari, “Resource Allocation in Fog Computing: A Systematic Review,” Journal of Science and Technology, 2023, doi: 10.20428/jst.v27i2.2052.
S. Narayanasamy, P. Chellammal, G. Keerthana, and M. A. Amarnath, “Decentralized Data Storage and Processing for Iot Devices: Unlocking the Potential,” Nq, 2023, doi: 10.48047/nq.2020.18.8.nq20233.
S. Kaliraj, “Snake Swarm Optimization‐based Deep Reinforcement Learning for Resource Allocation in Edge Computing Environment,” Concurr Comput, 2024, doi: 10.1002/cpe.8130.
H. Ning, “Design of the Physical Education Teaching System by Using Edge Calculation and the Fuzzy Clustering Algorithm,” Mobile Information Systems, 2022, doi: 10.1155/2022/7473614.
Y. Cai, “Multi-Agent DRL for Task Offloading With Delay Reliability Assurances in Ris-Aided MEC in UDN,” J Phys Conf Ser, 2024, doi: 10.1088/1742-6596/2807/1/012034.
S. A. Syed et al., “Design of Resources Allocation in 6G Cybertwin Technology Using the Fuzzy Neuro Model in Healthcare Systems,” J Healthc Eng, 2022, doi: 10.1155/2022/5691203.
R. Zhang and W. Shi, “Research on Resource Allocation and Management of Mobile Edge Computing Network,” Informatica, 2020, doi: 10.31449/inf.v44i2.3166.
S. Kuang, “Reliable Information Delivery and Dynamic Link Utilization in MANET Cloud Using Deep Reinforcement Learning,” Transactions on Emerging Telecommunications Technologies, 2024, doi: 10.1002/ett.5028.
S. K. Balachandar, “Dynamic Adaptive Resource Allocation for Edge Computing in Big Data Analytics Using GBDT, DQN, and GA Algorithms,” Jes, 2024, doi: 10.52783/jes.2561.
A. Zendebudi and S. Choudhury, “Designing a Deep Q-Learning Model With Edge-Level Training for Multi-Level Task Offloading in Edge Computing Networks,” Applied Sciences, 2022, doi: 10.3390/app122010664.
Y. Zhang, M. Zhang, C. Fan, F. Li, and B. Li, “Computing Resource Allocation Scheme of IOV Using Deep Reinforcement Learning in Edge Computing Environment,” EURASIP J Adv Signal Process, 2021, doi: 10.1186/s13634-021-00750-6.
Q. Wu, J. Wu, B. Yong, and Q. Zhou, “An Edge Based Multi-Agent Auto Communication Method for Traffic Light Control,” Sensors, 2020, doi: 10.3390/s20154291.
Z. Gao, W. Hao, Z. Han, and S. Yang, “Q-Learning-Based Task Offloading and Resources Optimization for a Collaborative Computing System,” Ieee Access, 2020, doi: 10.1109/access.2020.3015993.
Y. Yamagishi, T. Kaneko, M. Akai-Kasaya, and T. Asai, “Hardware-Oriented Deep Reinforcement Learning for Edge Computing,” Nonlinear Theory and Its Applications Ieice, 2021, doi: 10.1587/nolta.12.526.
A. Ndikumana, K. K. Nguyen, and M. Cheriet, “Federated Learning Assisted Deep Q-Learning for Joint Task Offloading and Fronthaul Segment Routing in Open RAN,” Ieee Transactions on Network and Service Management, 2023, doi: 10.1109/tnsm.2023.3245544.
Y. Wang, “A Federated Network Intrusion Detection System With Multi-Branch Network and Vertical Blocking Aggregation,” Electronics (Basel), 2023, doi: 10.3390/electronics12194049.
Y. Li, “Task-Offloading Strategy of Mobile Edge Computing for WBANs,” Electronics (Basel), 2024, doi: 10.3390/electronics13081422.
J. Shuai, “Q‐learning‐based Task Offloading Strategy for Satellite Edge Computing,” International Journal of Communication Systems, 2023, doi: 10.1002/dac.5691.
B. Ali, “Trust‐aware Task Load Balancing in Multi‐access Edge Computing Based on Blockchain and a Zero Trust Security Capability Framework,” Transactions on Emerging Telecommunications Technologies, 2023, doi: 10.1002/ett.4845.
Y. Li and Y. Hong, “Prediction of Football Match Results Based on Edge Computing and Machine Learning Technology,” International Journal of Mobile Computing and Multimedia Communications, 2022, doi: 10.4018/ijmcmc.293749.
S. Zhang, F. Li, and Y. Zhang, “Hierarchical Federated Learning With Mobile Edge Computing in the Internet of Vehicles,” Frontiers in Computing and Intelligent Systems, 2023, doi: 10.54097/fcis.v3i2.6998.
V. Patsias, “Task Allocation Methods and Optimization Techniques in Edge Computing: A Systematic Review of the Literature,” Future Internet, 2023, doi: 10.3390/fi15080254.
K. Κaur, A. Singh, and A. Sharma, “A Systematic Review on Resource Provisioning in Fog Computing,” Transactions on Emerging Telecommunications Technologies, 2023, doi: 10.1002/ett.4731.
X. Li, “5G Converged Network Resource Allocation Strategy Based on Reinforcement Learning in Edge Cloud Computing Environment,” Comput Intell Neurosci, 2022, doi: 10.1155/2022/6174708.
W. Y. Bryan Lim et al., “Decentralized Edge Intelligence: A Dynamic Resource Allocation Framework for Hierarchical Federated Learning,” Ieee Transactions on Parallel and Distributed Systems, 2022, doi: 10.1109/tpds.2021.3096076.
C. Avasalcai, C. Tsigkanos, and S. Dustdar, “Resource Management for Latency-Sensitive IoT Applications With Satisfiability,” IEEE Trans Serv Comput, 2022, doi: 10.1109/tsc.2021.3074188.
X. Song, Y. Wang, Z. Xie, and L. Xia, “A Cloud-Edge Collaborative Computing Task Scheduling and Resource Allocation Algorithm for Energy Internet Environment,” Ksii Transactions on Internet and Information Systems, 2021, doi: 10.3837/tiis.2021.06.019.
N. Li, “Cache Allocation Policy Based on User Preference Using Reinforcement Learning in Mobile Edge Computing,” Concurr Comput, 2023, doi: 10.1002/cpe.7991.
J. Lee, “Personalized Treatment Policies With the Novel Buckley-James Q-Learning Algorithm,” Axioms, 2024, doi: 10.3390/axioms13040212.
P. S. Hong, “Development and Data Storage of Sports Teaching Live Broadcast Platform Based on Mobile Device Edge Computing,” International Journal of Science and Engineering Applications, 2023, doi: 10.7753/ijsea1204.1045.
M. K. Kasi, S. A. Ghazalah, R. N. Akram, and D. Sauveron, “Secure Mobile Edge Server Placement Using Multi-Agent Reinforcement Learning,” Electronics (Basel), 2021, doi: 10.3390/electronics10172098.
S. Goudarzi, M. H. Anisi, H. Ahmadi, and L. Musavian, “Dynamic Resource Allocation Model for Distribution Operations Using SDN,” IEEE Internet Things J, 2021, doi: 10.1109/jiot.2020.3010700.
J.-H. Park, J.-W. Lee, T. Kim, I. Ahn, and J. Park, “Co-Evolution of Predator-Prey Ecosystems by Reinforcement Learning Agents,” Entropy, 2021, doi: 10.3390/e23040461.
G. Zhang, “MARL-Based Multi-Satellite Intelligent Task Planning Method,” Ieee Access, 2023, doi: 10.1109/access.2023.3337358.
Z. Xu, Y. Bai, B. Zhang, D. Li, and G. Fan, “HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning With Dual Coordination Mechanism,” Proceedings of the Aaai Conference on Artificial Intelligence, 2023, doi: 10.1609/aaai.v37i10.26386.
Q. Yin, “Distributed Deep Reinforcement Learning: A Survey and a Multi-Player Multi-Agent Learning Toolbox,” Machine Intelligence Research, 2024, doi: 10.1007/s11633-023-1454-4.
B. Hazarika, “Multi-Agent DRL-Based Task Offloading in Multiple RIS-Aided IoV Networks,” IEEE Trans Veh Technol, 2024, doi: 10.1109/tvt.2023.3302010.
S. Chen, Z. Yao, X. Jiang, J. Yang, and L. Hanzo, “Multi-Agent Deep Reinforcement Learning-Based Cooperative Edge Caching for Ultra-Dense Next-Generation Networks,” Ieee Transactions on Communications, 2021, doi: 10.1109/tcomm.2020.3044298.
Y. Yang, “A Policy Gradient Algorithm to Alleviate the Multi-Agent Value Overestimation Problem in Complex Environments,” Sensors, 2023, doi: 10.3390/s23239520.
A. Lupu and D. Precup, “Gifting in Multi-Agent Reinforcement Learning (Student Abstract),” Proceedings of the Aaai Conference on Artificial Intelligence, 2020, doi: 10.1609/aaai.v34i10.7208.
I. Jang, D. Lee, Y.-S. Son, and S. Kim, “Knowledge Transfer for on-Device Deep Reinforcement Learning in Resource Constrained Edge Computing Systems,” Ieee Access, 2020, doi: 10.1109/access.2020.3014922.
Y.-P. Zhao, I. Niemegeers, and S. M. H. de Groot, “Dynamic Power Allocation for Cell-Free Massive MIMO: Deep Reinforcement Learning Methods,” Ieee Access, 2021, doi: 10.1109/access.2021.3097243.
F. Li, “Collaborative Computation Offloading and Resource Management in Space–Air–Ground Integrated Networking: A Deep Reinforcement Learning Approach,” Electronics (Basel), 2024, doi: 10.3390/electronics13101804.
D. Wei, “Intelligent Hierarchical Admission Control for Low-Earth Orbit Satellites Based on Deep Reinforcement Learning,” Sensors, 2023, doi: 10.3390/s23208470.
Y. Abiko, T. Saito, D. Ikeda, K. Ohta, T. Mizuno, and H. Mineno, “Flexible Resource Block Allocation to Multiple Slices for Radio Access Network Slicing Using Deep Reinforcement Learning,” Ieee Access, 2020, doi: 10.1109/access.2020.2986050.
S. Kalantari, R. S. Ramhormozi, Y. Wang, S. Sun, and W. Xin, “Trailer Allocation and Truck Routing Using Bipartite Graph Assignment and Deep Reinforcement Learning,” Transactions in Gis, 2023, doi: 10.1111/tgis.13057.
J. Zhao, M. Kong, Q. Li, and X. Sun, “Contract-Based Computing Resource Management via Deep Reinforcement Learning in Vehicular Fog Computing,” Ieee Access, 2020, doi: 10.1109/access.2019.2963051.
D. Tian, “An Intelligent Optimization Method for Wireless Communication Network Resources Based on Reinforcement Learning,” J Phys Conf Ser, 2023, doi: 10.1088/1742-6596/2560/1/012036.
S. Wen, R. Han, C. H. Liu, and L. Y. Chen, “Fast DRL-based Scheduler Configuration Tuning for Reducing Tail Latency in Edge-Cloud Jobs,” Journal of Cloud Computing Advances Systems and Applications, 2023, doi: 10.1186/s13677-023-00465-z.
F. Li, C. Fang, M. Liu, N. Li, and T. Sun, “Intelligent Computation Offloading Mechanism with Content Cache in Mobile Edge Computing,” Electronics (Basel), vol. 12, no. 5, p. 1254, Mar. 2023, doi: 10.3390/electronics12051254.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Muhammad Adi Sulistyo, Doni Setiawan
This work is licensed under a Creative Commons Attribution 4.0 International License.
