Spark-Based Parallel Genetic Algorithm for Simulating a Solution of Optimal Deployment of an Underwater Sensor Network
<p>The process framework of location updates of an UWSN.</p> "> Figure 2
<p>(<b>a</b>) Shubert function simulation view 1 (along X-axis and Y-axis); and (<b>b</b>) Shubert function simulation view 2 (along Z-axis).</p> "> Figure 3
<p>Single-node GA for solving ESMPF.</p> "> Figure 4
<p>Hadoop-based parallel GAESMPF.</p> "> Figure 5
<p>Spark operating mechanism.</p> "> Figure 6
<p>Spark-based parallel GAESMPF.</p> "> Figure 7
<p>Illustration of crossover.</p> "> Figure 8
<p>Illustration of mutation.</p> "> Figure 9
<p>Searching optimal extremums for SMPF based on RDD.</p> "> Figure 10
<p>Running efficiency comparison of GAESMPF.</p> "> Figure 11
<p>Solution accuracy comparison of GAESMPF.</p> "> Figure 12
<p>Comparison of the run time of a single iteration of GAESMPF.</p> "> Figure 13
<p>Comparison of the speedup ratio with different evolution times.</p> "> Figure 14
<p>Comparison of speedup ratio with different node numbers.</p> ">
Abstract
:1. Introduction
- (1)
- The Shubert multi-peak function (SMPF) is used to simulate deployment of underwater sensor network. By calculating the extremums of the Shubert multi-peak function (ESMPF), the simulating optimal deployment sites can be obtained.
- (2)
- Based on RDD computation model of the Spark framework, a parallel GA for optimizing the deployment of a UWSN (DUWSN) is designed and implemented.
- (3)
- By the comparison with the GAs based on single-node and Hadoop, it is verified that the proposed GA runs more efficiently while showing a higher accuracy.
2. System Application Model
3. Single-Node GA for Solving ESMPF
3.1. Shubert Multi-Peak Function
3.2. The GA Dataset Encoding for SMPF
3.3. Single-Node Implementation
- Population Initialization: To input 262,144 pairs of binary coding string into the sample database of Shubert function and set them as the initial population.
- Fitness Evaluation: Given that the individual fitness should be non-negative, a linear non-negative conversion should be done to the Shubert function and the fitness corresponding to each individual should be calculated.
- Selection Operator: According to the individual fitness and selection probability, the next generation can be selected out by means of roulette.
- Crossover Operator: All the individuals of the population should be coupled in a random and pairwise way, and single-point crossover should be conducted according to the crossover probability.
- Mutation Operator: To traverse through each gene locus of every individual and reverse the loci which satisfy the mutation probability.
- Decoding and Calculation: When the termination conditions are met, the individuals of the last generation are decoded with functions to obtain the corresponding function-value set and then the values are sorted to locate the extremums.
4. Hadoop-Based Parallel GAESMPF
4.1. Process of Parallel Genetic Iteration
Algorithm 1. Parallel genetic iteration. | |
Input: N: Shubert sample code set T: maximum times of iteration | |
Output: the individual encoding key-value pair of the last generation | |
|
|
4.2. Process of Searching Extremums
Algorithm 2. Searching extremums. | |
Input: N: Shubert sample code set of the last generation | |
Output: the extremums of Shubert function | |
|
|
5. Spark-Based Parallel GAESMPF
5.1. Phase of Parallel Genetic Operation
5.2. Phase of Searching Extremums
6. Experiment and Analysis
- High efficiency: Through the comparison of running time, speedup ratio, and average time consumed for iterations on single-node, Hadoop, and Spark, respectively, the high efficiency of the Spark-based GAESMPF is verified.
- Accuracy and stability: Through comparison of the Shubert function extremums obtained with different times of iterations, the accuracy and stability of parallel GAESMPF based on Spark is verified.
6.1. Experiment Configuration
6.2. General Running Efficiency
6.3. Solution Accuracy with Evolution Times
6.4. Run Time of a Single Iteration
6.5. Speedup Ratio Analysis
6.5.1. Speedup Ratio with Different Evolution Times
6.5.2. Speedup Ratio with Different Node Numbers
7. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Peng, J.; Yang, F.; Feng, W. Fruit fly inspired underwater sensor network deployment algorithm. J. Chin. J. Electron. 2017, 45, 1403–1407. [Google Scholar]
- Alam, S.M.N.; Haas, Z.J. Coverage and Connectivity in Three-Dimensional Networks with Random Node Deployment; Elsevier Science Publishers B. V.: Amsterdam, The Netherlands, 2015; pp. 285–286. [Google Scholar]
- Ismail, N.S.N.; Hussein, L.A.; Ariffin, S.H.S. Analyzing the Performance of Acoustic Channel in Underwater Wireless Sensor Network (UWSN). In Proceedings of the Fourth Asia International Conference on Mathematical/Analytical Modelling & Computer Simulation, Kota Kinabalu, Malaysia, 26–28 May 2010. [Google Scholar]
- Anguita, D.; Brizzolara, D.; Parodi, G. Building an Underwater Wireless Sensor Network Based on Optical Communication: Research Challenges and Current Results. In Proceedings of the International Conference on Sensor Technologies & Applications, Venice, Italy, 15–18 July 2010. [Google Scholar]
- Yang, G.; Wei, Z.; Cong, Y. Analysis of Security and Threat of Underwater Wireless Sensor Network Topology. In Proceedings of the International Conference on Digital Image Processing, Kuala Lumpur, Malaysia, 7–8 April 2012. [Google Scholar]
- Gupta, S.K.; Kuila, P.; Jana, P.K. Genetic algorithm approach for k -coverage and m -connected node placement in target based wireless sensor networks. J. Comput. Electr. Eng. 2015, 56, 544–556. [Google Scholar] [CrossRef]
- Cao, B.; Zhao, J.W.; Lu, Z.H. Deployment optimization for 3D industrial wireless sensor networks based on particle swarm optimizers with distributed parallelism. J. Netw. Comput. Appl. 2018, 103, 225–238. [Google Scholar] [CrossRef]
- Han, G.; Jiang, J.; Shu, L. Localization algorithms of Underwater Wireless Sensor Networks: A survey. Sensors 2012, 12, 2026–2061. [Google Scholar] [CrossRef] [PubMed]
- Luque, G.; Alba, E. Parallel Genetic Algorithms: Theory and Real World Applications. In Parallel Genetic Algorithms: Theory and Real World Applications; Springer: Berlin, Germany, 2011. [Google Scholar]
- Arthur, L.C.; Roger, L.W. A Parallel Island Model Genetic Algorithm for the Multi-processor Scheduling Problem. In Proceedings of the ACM/SIGAPP Symposium on Applied Computing, Phoenix, AZ, USA, 6–8 March 1994; pp. 483–487. [Google Scholar]
- Alba, E.; Troya, J.M. A survey of parallel distributed genetic algorithms. J. Complex. 1999, 4, 31–52. [Google Scholar] [CrossRef]
- Wang, Z.Q.; Lu, Y.Y. Research on Big Data Processing Technology Based on Hadoop. In Proceedings of the International Conference on Electronics, Electrical Engineering and Information Science, Guangzhou, China, 7–9 August 2015. [Google Scholar]
- Jin, C.; Vecchiola, C.; Buyya, R. MRPGA: An Extension of MapReduce for Parallelizing Genetic Algorithms. In Proceedings of the IEEE 5th International Conference on Escience, Oxford, UK, 9–11 December 2009; pp. 214–221. [Google Scholar]
- Han, Z.; Zhang, Y. Spark: A Big Data Processing Platform Based on Memory Computing. In Proceedings of the Seventh International Symposium on Parallel Architectures, Nanjin, China, 12–14 December 2016. [Google Scholar]
- Nair, L.R.; Shetty, S.D.; Shetty, S.D. Applying spark based machine learning model on streaming big data for health status prediction. J. Comput. Electr. Eng. 2017, 65, 393–399. [Google Scholar] [CrossRef]
- Jia, G.; Han, G.; Du, J. A Maximum Cache Value Policy in Hybrid Memory based Edge Computing for 5G Mobile Devices. IEEE Internet Things J. 2018. [Google Scholar] [CrossRef]
- Jia, G.; Han, G.; Li, A. SSL: Smart Street Lamp based on Fog Computing for Smarter Cities. J. IEEE Trans. Ind. Inf. 2018, 14, 4995–5004. [Google Scholar] [CrossRef]
- Jia, G.; Han, G.; Xie, H. Hybrid-LRU Caching for Optimizing Data Storage and Retrieval in Edge Computing-Based Wearable Sensors. IEEE Internet Things J. 2018, 6, 1342–1351. [Google Scholar] [CrossRef]
- Jia, G.; Han, G.; Rao, H.; Shu, L. Edge Computing-based Intelligent Manhole Cover Management System for Smart Cities. IEEE Internet Things J. 2018, 5, 1648–1656. [Google Scholar] [CrossRef]
- Jia, G.; Han, G.; Rodrigues, J.J. Coordinate Memory Deduplication and Partition for Improving Performance in Cloud Computing. J. IEEE Trans. Cloud Comput. 2015, 7, 357–368. [Google Scholar] [CrossRef]
- Zongwei, Z.; Fan, W.; Jing, C.; Xi, L.; Gangyong, J. A Thread-Oriented Memory Resource Management Framework for Mobile Edge Computing. IEEE Access 2019, 7, 45881–45890. [Google Scholar]
- Zongwei, Z.; Jing, C.; Xi, L.; Junneng, Z.; Youqing, X.; Gangyong, J. Impacts of Memory Address Mapping Scheme on Reducing DRAM Self-Refresh Power for Mobile Computing Devices. IEEE Access 2018, 6, 78513–78520. [Google Scholar]
- Jiafen, G.; Xiufen, Z.; Hui, J. A Dynamic Two-Level Evolutionary Algorithm for Solving Multi-peak Function Optimization Problem. J. Prog. Intell. Comput. Appl. 2005, 50–55. [Google Scholar]
- Tomassini, M.; Vanneschi, L. Special issue on parallel and distributed evolutionary algorithms. J. Genet. Program. Evolvable Mach. 2009, 10, 339–341. [Google Scholar] [CrossRef]
- Min, X.; Shi, W.R.; Jiang, C.J. Energy efficient clustering algorithm for maximizing lifetime of wireless sensor networks. J. AEUE Int. J. Electron. Commun. 2010, 64, 289–298. [Google Scholar] [CrossRef]
- Dean, J.; Ghemawat, S. MapReduce: Simplified Data Processing on Large Clusters. In Proceedings of the 6th Symposium on Operating System Design and Implementation (OSDI), San Francisco, CA, USA, 6–8 Decmber 2004. [Google Scholar]
- Verma, A.; Xavier, L.; Goldberg, D.E. Scaling Genetic Algorithms using MapReduce. In Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications, Pisa, Italy, 30 November–2 December 2009. [Google Scholar]
- Flores, P.; Yoonsik, C.P. WiseGen: Generating Test Cases for Pairwise Testing Using Genetic Algorithms. In Proceedings of the IEEE International Conference on Computer Science and Automation Engineering (CSAE), Zhangjiajie, China, 10–11 June 2011. [Google Scholar]
- Geronimo, D.L.; Ferrucci, F.; Murolo, A.; Sarro, F. A Parallel Genetic Algorithm Based on Hadoop MapReduce for the Automatic Generation of JUnit Test Suites. In Proceedings of the 5th IEEE International Conference on Software Testing, Verification and Validation, Montreal, QC, Canada, 17–21 April 2012. [Google Scholar]
- Dean, J.; Ghemawat, S. MapReduce: Simplified data processing on large clusters. J. Commun. ACM 2008, 51, 107–113. [Google Scholar] [CrossRef]
- Zaharia, M.; Chowdhury, M.; Das, T.; Dave, A.; Ma, J.; McCauly, M.; Franklin, M.J.; Shenker, S.; Stoica, I. Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. In Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, San Jose, CA, USA, 25–27 April 2012. [Google Scholar]
- Zhu, Z.; Li, X.; Wang, C. Memory Power Optimization on Different Memory Address Mapping Schemas. In Proceedings of the IEEE International Conference on Embedded & Real-time Computing Systems & Applications, Daegu, Korea, 17–18 August 2016. [Google Scholar]
Node | Maximum Population | Crossover Rate | Mutation Rate | CPU (Core) | Memory | Network | JDK Version | Hadoop Version | Spark Version |
---|---|---|---|---|---|---|---|---|---|
Master | 262,144 | 0.9 | 0.05 | 4 | 6 GB | 1 GB/s | 1.7.0 | 1.2.1 | 1.6.0 |
Slave (1~19) | 262,144 | 0.9 | 0.05 | 4 | 6 GB | 1 GB/s | 1.7.0 | 1.2.1 | 1.6.0 |
Iterations | Single-Node | Hadoop | Spark | |||
---|---|---|---|---|---|---|
Time/s | Extremum | Time/s | Extremum | Time/s | Extremum | |
30 | 1152.645 | 175.27 | 94.810 | 205.32 | 9.169 | 206.67 |
50 | 2231.642 | 182.45 | 156.104 | 206.89 | 11.008 | 207.73 |
70 | 3081.259 | 196.65 | 217.609 | 206.12 | 14.507 | 208.83 |
90 | 3953.328 | 206.15 | 278.967 | 207.73 | 18.997 | 209.52 |
110 | 4881.409 | 208.14 | 340.735 | 209.12 | 23.050 | 209.32 |
130 | 5560.156 | 208.16 | 401.761 | 210.22 | 27.633 | 210.32 |
150 | 6515.616 | 209.67 | 463.883 | 210.42 | 32.771 | 210.43 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, P.; Ye, S.; Wang, C.; Zhu, Z. Spark-Based Parallel Genetic Algorithm for Simulating a Solution of Optimal Deployment of an Underwater Sensor Network. Sensors 2019, 19, 2717. https://doi.org/10.3390/s19122717
Liu P, Ye S, Wang C, Zhu Z. Spark-Based Parallel Genetic Algorithm for Simulating a Solution of Optimal Deployment of an Underwater Sensor Network. Sensors. 2019; 19(12):2717. https://doi.org/10.3390/s19122717
Chicago/Turabian StyleLiu, Peng, Shuai Ye, Can Wang, and Zongwei Zhu. 2019. "Spark-Based Parallel Genetic Algorithm for Simulating a Solution of Optimal Deployment of an Underwater Sensor Network" Sensors 19, no. 12: 2717. https://doi.org/10.3390/s19122717