All Issue

2021 Vol.63, Issue 1 Preview Page
Comparison of Artificial Neural Network Model Capability for Runoff Estimation about Activation Functions

활성화 함수에 따른 유출량 산정 인공신경망 모형의 성능 비교

Maga Kima
,
Jin-Yong Choib†
,
Jehong Banga
,
Pureun Yoona
,
Kwihoon Kima
김 마가a
,
최 진용b†
,
방 재홍a
,
윤 푸른a
,
김 귀훈a
aPh.D. Student, Department of Rural Systems Engineering, Seoul National University
bProfessor, Department of Rural Systems Engineering, Research Institute of Agriculture and Life Sciences, Global Smart Farm Convergence Major, Seoul National University
a서울대학교 농촌시스템공학부 박사과정 학생
b글로벌 스마트팜 융합전공, 서울대 농업생명과학연구소 교수
† Corresponding author Tel.: +82-2-880-4583 E-mail: iamchoi@snu.ac.kr
2021. pp. 103-116
PDF XML Citation
Abstract

Analysis of runoff is substantial for effective water management in the watershed. Runoff occurs by reaction of a watershed to the rainfall and has non-linearity and uncertainty due to the complex relation of weather and watershed factors. ANN (Artificial Neural Network), which learns from the data, is one of the machine learning technique known as a proper model to interpret non-linear data. The performance of ANN is affected by the ANN’s structure, the number of hidden layer nodes, learning rate, and activation function. Especially, the activation function has a role to deliver the information entered and decides the way of making output. Therefore, It is important to apply appropriate activation functions according to the problem to solve. In this paper, ANN models were constructed to estimate runoff with different activation functions and each model was compared and evaluated. Sigmoid, Hyperbolic tangent, ReLU (Rectified Linear Unit), ELU (Exponential Linear Unit) functions were applied to the hidden layer, and Identity, ReLU, Softplus functions applied to the output layer. The statistical parameters including coefficient of determination, NSE (Nash and Sutcliffe Efficiency), NSEln (modified NSE), and PBIAS (Percent BIAS) were utilized to evaluate the ANN models. From the result, applications of Hyperbolic tangent function and ELU function to the hidden layer and Identity function to the output layer show competent performance rather than other functions which demonstrated the function selection in the ANN structure can affect the performance of ANN.

Keywords
Activation function
artificial neural network
multi-layer perceptron
rainfall-runoff
stream flow
References

REFERENCES

1
S. J. Ahn, K. W. Jun and K. I. Kim, Journal of Korean Water Resources Association, Forecasting of runoff hydrograph using neural network algorithms, 33(4); 505-515, (in Korean) (2000)
2
H. Asadi, H. Shahedi, B. Jarihani and R. C. Sidle, Water, Rainfall-runoff modelling using hydrological connectivity index and artificial neural network approach, 11(2); 212 (2019)10.3390/w11020212
3
M. Campolo, P. Andreussi and A. Soldati, Water Resources Research, River flood forecasting with a neural network model, 35(4); 1191-1197 (1999)10.1029/1998wr900086
4
M. A. Ghorbani, H. A. Zadeh and M. Isazadeh, Environmental Earth Sciences, A comparative study of artificial neural network (MLP, RBF) and support vector machine models for river flow prediction, 75; 476 (2016)10.1007/s12665-015-5096-x
5
X. Glorot, A. Bordes and Y. Bengio, Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, Deep sparse rectifier neural networks, 15; 315-323 (2011)
6
G. S. da S. Gomes, T. B. Ludermir and L. M. M. R. Lima, Neural Computing and Applications, Comparison of new activation functions in neural network for forecasting financial time series, 20; 417-439 (2011)10.1007/s00521-010-0407-3
7
Frauke Günther and S. Fritsch, Journal of Hydrologic Engineering, Neuralnet: training of neural networks, 2(1); 30-38 (2010)
8
H. V. Gupta, S. Sorooshian and P. O. Yapo, Journal of Hydrologic Engineering, Status of automatic calibration for hydrologic models: comparison with multilevel expert calibration, 4(2); 135-143 (1999)10.1061/(asce)1084-0699(1999)4:2(135)
9
S. H. Jung, D. E. Lee and K. S. Lee, Journal of the Korean Society of Hazard Mitigation, Prediction of river water level using deep-learning open library, 18(1); 1-11 (2018)10.9798/kosham.2018.18.1.1
10
N. Karunanithi, W. J. Grenney, D. Whitley and K. Bovee, Journal of Computing in Civil Engineering, Neural networks for river flow prediction, 8(2); 201-220 (1994)10.1061/(asce)0887-3801(1994)8:2(201)
11
S. W. Kim, Journal of Korean Water Resources Association, A study on the forecasting of daily streamflow using the multilayer neural networks model, 33(5); 537-550, (in Korean) (2000)
12
D. P. Kingma and J. L. Ba, 3rd International Conference for Learning Representations, Adam: a method for stochastic optimization (2015)
13
Ö. Kişi, Journal of Hydrologic Engineering, Streamflow forecasting using different artificial neural network algorithms, 15(5); 532-539 (2007)10.1061/(asce)1084-0699(2007)12:5(532)
14
A. R. S. Kumar, K. P. Sudheer, S. K. Jain and P. K. Agarwal, Hydrological Processes, Rainfall-runoff modelling using artificial neural networks: comparison of network types, 19(6); 127-1291 (2005)10.1002/hyp.5581
15
S. Kurma, T. Roshini and D. Himayoun, Civil Engineering Journal, A comparison of emotional neural network (ENN) and artificial neural network (ANN) approach for rainfall- runoff modelling, 5(10); 2120-2130 (2019)10.28991/cej-2019-03091398
16
K. S. Lee, S. C. Park, H. M. Lee and Y. H. Jin, Journal of The Korean Society of Civil Engineers, The study on the forecasting of runoff applied the B.P. Algorithm of the artificial neural network in the Young-San river, 20(5B); 679-688, (in Korean) (2000)
17
Y. Lin, H. Wen and S. Liu, Journal of Water and Climate Change, Surface runoff response to climate change based on artificial neural network (ANN) models: a case study with Zagunao catchment in upper Minjiang river, southwest China, 10(1); 158-166 (2019)10.2166/wcc.2018.130
18
P. K. Mishra and S. Karmakar, International Journal of Environmental Science and Technology, Performance of optimum neural network in rainfall-runoff modeling over a river basin, 16; 1289-1302 (2019)10.1007/s13762-018-1726-7
19
D. N. Moriasi, M. W. Gitau, N. Pai and P. Daggupati, Transactions of the ASABE, Hydrologic and water quality models: performance measures and evaluation criteria, 50(6); 1763-1785 (2015)10.13031/trans.58.10715
20
J. E. Nash and V. Sutcliffe, Journal of Hydrology, River flow forecasting through conceptual models partⅠ - a discussion of principles, 10(3); 282-290 (1970)10.1016/0022-1694(70)90255-6
21
A. B. Patel and G. S. Joshi, Civil Engineering Journal, Modeling of rainfall- runoff correlations using artificial neural network-a case study of Dharoi watershed of a Sabarmati river basin, India, 3(2); 78-87 (2017)10.28991/cej-2017-00000074
22
V. Poonia and H. L. Tiwari, Arabian Journal of Geosciences, Rainfall-runoff modeling for the Hoshangabad basin of Narmada river using artificial neural network, 13; 944 (2020)10.1007/s12517-020-05930-6
23
R. Pushpalatha, C. Perrin, N. L. Moine and V. Andréassian, Journal of Hydrology, A review of efficiency criteria suitable for evaluating low-flow simulations, 420; 171-182 (2012)10.1016/j.jhydrol.2011.11.055
24
P. Ramachandran, B. Zoph and Q. V. Le, 6th International Conference on Learning Representations, Searching for activation functions (2017)
25
A. Sarkar and R. Kumar, Journal of Water Resource and Protection, Artificial networks for event based rainfall-runoff modeling, 4(10); 891-897 (2012)10.4236/jwarp.2012.410105
26
A. Y. Shamseldin, Journal of Hydroinformatics, Artificial neural network model for river flow forecasting in a developing country, 12(1); 22-35 (2010)10.2166/hydro.2010.027
27
A. Y. Shamseldin, A. E. Nasr and K. M. O’Connor, Hydrology and Earth System Sciences, Comparison of differeny forms of the multi-layer feed-forward neural network method used for river flow forecasting, 6(4); 671-684 (2002)10.5194/hess-6-671-2002
28
S. Sharma, S. Sharma and A. Athaiya, International Journal of Engineering Applied Sciences and Technology, Activation functions in neural networks, 4(12); 310-316 (2020)10.33564/ijeast.2020.v04i12.054
29
P. Sibi, S. A. Jones and P. Siddarth, Journal of Theoretical and Applied Information Technology, Analysis of different activation functions using back propagation neural networks, 47(3); 1344-1348 (2013)
30
C. M. Song and K. H. Lee, Ecology and Resilient Infrastructure, Applicability evaluation for discharge model using curve number and convolution neural network, 7(2); 114-125 (2020)10.17820/eri.2020.7.2.114.
31
K. P. Sudheer, P. C. Nayak and K. S. Ramasastri, Hydrological Processes, Improving peak flow estimation in artificial neural network river flow models, 17(3); 677-686 (2003)10.1002/hyp.5103
32
A. S. Tokar and M. Markus, Journal of Hydrologic Engineering, Precipitation-runoff modeling using artificial neural networks and conceptual models, 5(2); 156-161 (2000)10.1061/(ASCE)1084-0699(1999)4:3(232)
33
A. K. Yadav, V. K. Chandola, A. Singh and B. P. Singh, International Journal of Current Microbiology and Applied Sciences, Rainfall-runoff modelling using artificial neural networks (ANNs) model, 9(3); 127-135 (2020)10.20546/ijcmas.2020.903.016
34
W. K. Yeo, Y. M. Seo, S. Y. Lee and H. K. Ji, Journal of Korean Water Resources Association, Study on water stage prediction using hybrid model of artificial neural network and genetic algorithm, 43(8); 721-731, (in Korean) (2010)10.3741/jkwra.2010.43.8.721
35
K. H. Yoon, B. C. Seo and H. S. Shin, Journal of Korean Water Resources Association, Dam inflow forecasting for short term flood based on neural networks in Nakdong river basin, 37(1); 67-75, (in Korean) (2004)10.3741/jkwra.2004.37.1.067
36
M. R. Zadeh, S. Amin, D. Khalili and V. P. Singh, Water Resources Management, Daily outflow prediction by multi layer perceptron with logistic sigmoid and tangent sigmoid activation functions, 24; 2673-2688 (2010)10.1007/s11269-009-9573-4
Information
  • Publisher :The Korean Society of Agricultural Engineers
  • Publisher(Ko) :한국농공학회
  • Journal Title :Journal of Korean Society of Agricultural Engineers
  • Journal Title(Ko) :한국농공학회논문집
  • Volume : 63
  • No :1
  • Pages :103-116
  • Received Date : 2020-12-22
  • Revised Date : 2021-01-18
  • Accepted Date : 2021-01-18
  • DOI :https://doi.org/10.5389/KSAE.2021.63.1.103
Journal Informaiton Journal of Korean Society of Agricultural Engineers Journal of Korean Society of Agricultural Engineers
  • NRF
  • KOFST
  • crossref crossmark
  • crossref cited-by
  • crossref funder-registry
  • crosscheck
  • orcid
  • open access
Journal Informaiton Journal Informaiton - close