All Issue

2021 Vol.63, Issue 1 Preview Page
Impact of Activation Functions on Flood Forecasting Model Based on Artificial Neural Networks

홍수량 예측 인공신경망 모형의 활성화 함수에 따른 영향 분석

Jihye Kima
,
Sang-Min Junb
,
Soonho Hwangc
,
Hak-Kwan Kimd
,
Jaemin Heoe
,
Moon-Seong Kangf†
김 지혜a
,
전 상민b
,
황 순호c
,
김 학관d
,
허 재민e
,
강 문성f†
aPh.D. Student, Department of Rural Systems Engineering, Seoul National University
bPh.D. Student, Department of Rural Systems Engineering, Seoul National University
cResearch Professor, Research Institute of Agriculture and Life Sciences, Seoul National University
dAssistant Professor, Graduate School of International Agricultural Technology, Institutes of Green Bio Science and Technology, Seoul National University
eUndergraduate Student, Department of Rural Systems Engineering, Seoul National University
fProfessor, Department of Rural Systems Engineering, Research Institute of Agriculture and Life Sciences, Institutes of Green Bio Science and Technology, Seoul National University
a서울대학교 농촌시스템공학부 박사과정 학생
b서울대학교 농촌시스템공학부 박사과정 학생
c서울대 농업생명과학연구소 연구교수
d서울대 녹색바이오과학기술원 국제농업기술대학원 조교수
e서울대학교 농촌시스템공학부 학부생
f서울대 녹색바이오과학기술원 농촌시스템공학과 교수, 농업생명과학연구소
† Corresponding author Tel.: +82-2-880-4582, Fax: +82-2-873-2087 E-mail: mskang@snu.ac.kr
2021. pp. 11-25
PDF XML Citation
Abstract

The objective of this study was to analyze the impact of activation functions on flood forecasting model based on Artificial neural networks (ANNs). The traditional activation functions, the sigmoid and tanh functions, were compared with the functions which have been recently recommended for deep neural networks; the ReLU, leaky ReLU, and ELU functions. The flood forecasting model based on ANNs was designed to predict real-time runoff for 1 to 6-h lead time using the rainfall and runoff data of the past nine hours. The statistical measures such as R2, Nash-Sutcliffe Efficiency (NSE), Root Mean Squared Error (RMSE), the error of peak time (ETp), and the error of peak discharge (EQp) were used to evaluate the model accuracy. The tanh and ELU functions were most accurate with R2=0.97 and RMSE=30.1 (m3/s) for 1-h lead time and R2=0.56 and RMSE=124.6~124.8 (m3/s) for 6-h lead time. We also evaluated the learning speed by using the number of epochs that minimizes errors. The sigmoid function had the slowest learning speed due to the ‘vanishing gradient problem’ and the limited direction of weight update. The learning speed of the ELU function was 1.2 times faster than the tanh function. As a result, the ELU function most effectively improved the accuracy and speed of the ANNs model, so it was determined to be the best activation function for ANNs-based flood forecasting.

Keywords
Artificial neural networks
activation function
flood forecasting
ReLU
sigmoid
References

REFERENCES

1
J. Adamowski and S. O. Prasher, Journal of Water and Land Development, Comparison of machine learning methods for runoff forecasting in mountainous watersheds with limited data, 17(1); 89-97 (2012)10.2478/v10025-012-0038-4
2
C. C. Aggarwal, Neural networks and deep learning, Springer. (2018)
3
F. Chollet, Keras, https://keras.io. Accessed 20Jul. 2020. (2015)
4
F. Chollet, Deep Learning with Python, Manning. (2017)
5
D. A. Clevert, T. Unterthiner and S. Hochreiter, arXiv preprint, Fast and accurate deep network learning by exponential linear units (ELUs), arXiv:1511.07289 (2016)
6
C. W. Dawson and R. Wilby, Hydrological Sciences, An artificial neural network approach to rainfall-runoff modeling., 43(1); 47-66 (1998)10.1080/02626669809492102
7
J. P. Donate, C. Paulo, G. S. German and S. M. Araceli, Neurocompution, Time series forecasting using a weighted cross- validation evolutionary artificial neural network ensemble, 109; 27-32 (2013)10.1016/j.neucom.2012.02.053
8
X. Glorot and Y. Bengio, Thirteenth International Conference on Artificial Intelligence and Statistics, Understanding the difficulty of training deep feedforward neural networks. In Proc; 249-256 (2010)
9
I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT press. (2016)
10
K. N. Hsu, H. V. Gupta and S. sorooshian, Water Resources Research, Artificial neural network modeling of the rainfall-runoff process, 31(10); 2517-2530 (1995)10.1029/95wr01955
11
C. Hu, Q. Wu, H. Li, S. Jian, N. Li and Z. Lou, Water, Deep learning with a long short-term memory networks approach for rainfall-runoff simulation, 10(11); 1543 (2018)10.3390/w10111543
12
M. Jeung, S. Baek, J. Beom, K. H. Cho, Y. Her and K. Yoon, Journal of Hydrology, Evaluation of random forest and regression tree methods for estimation of mass first flush ratio in urban catchments, 575; 1099-1110 (2019)10.1016/j.jhydrol.2019.05.079
13
M. S. Kang and S. W. Park, Journal of The Korean Society of Agricultural Engineers, Short-term flood forecasting using artificial neural networks, 45(2); 45-57, (in Korean) (2003)
14
M. S. Kang, M. G. Kang, S. W. Park, J. J. Lee and R. H. Yoo, Journal of the American Water Resources Association, Application of grey model and artificial neural networks to flood forecasting, 42(2); 473-486 (2006)10.1111/j.1752-1688.2006.tb03851.x
15
J. H. Kim, A study on hydrologic forecasting of stream flows based on artificial neural network. Ph.D. diss, (in Korean), Incheon, Republic of Korea. Inha University. (1993)
16
D. P. Kingma and J. Ba, arXiv preprint, Adam: A method for stochastic optimization, arXiv:1412.6980 (2014)
17
A. Krizhevsky, I. Sutskever and G. E. Hinton, Communications of the ACM, Imagenet classification with deep convolutional neural networks, 60(6); 84-90 (2017)10.1145/3065386
18
M. M. Lau and K. H. Lim, 2017 2nd International Conference on Control and Robotics Engineering (ICCRE), Investigation of activation functions in deep belief network; 201-206 (2017)10.1109/iccre.2017.7935070
19
G. S. Lee, S. C. Park, H. M. Lee and Y. H. Jin, Journal of Korea Water Resources Association, The prediction of runoff using artificial neural network in the Young-san River, 33(1); 251-256, (in Korean) (2000)
20
C. Marina, P. Andreussi and A. Soldati, Water Resources Research, River flood forecasting with a neural network model, 35(4); 1191-1197 (1999)10.1029/1998wr900086
21
Ministry of the Interior and Safety, Statistical yearbook of natural disaster, (in Korean) (2018)
22
D. N. Moriasi, M. W. Gitau, N. Pai and P. Daggupati, Transactions of the ASABE, Hydrologic and water quality models: performance measures and evaluation criteria, 58(6); 1763-1785 (2015)10.13031/trans.58.10715
23
A. Sarkar and R. Kumar, Journal of Water Resource and Protection, Artificial neural networks for event based rainfall-runoff modeling, 4(10); 891 (2012)10.4236/jwarp.2012.410105
24
D. Solomatine, L. M. See and R. J. Abrahart, In Practical Hydroinformatics, Data-driven modelling: concepts, approaches and experiences.; 17-30 (2009)
25
J. H. Song, Y. Her, K. Suh, M. S. Kang and H. Kim, Water, Regionalization of a Rainfall-Runoff Model: Limitations and Potentials, 11(11); 2257 (2019)10.3390/w11112257
26
B. Xu, N. Wang, T. Chen and M. Li, arXiv preprint, Empirical evaluation of rectified activations in convolutional network, arXiv:1505.00853 (2015)
27
C. C. Young, W. C. Liu and M. C. Wu, Applied Soft Computing, A physically based and machine learning hybrid approach for accurate rainfall-runoff modeling during extreme typhoon events, 53; 205-216 (2017)10.1016/j.asoc.2016.12.052
28
C. M. Zealnad, D. H. Burn and S. P. Simonovic, Journal of Hydrology, Short term stream flow forecasting using artificial neural networks, 214; 32-48 (1999)10.1016/s0022-1694(98)00242-x
Information
  • Publisher :The Korean Society of Agricultural Engineers
  • Publisher(Ko) :한국농공학회
  • Journal Title :Journal of Korean Society of Agricultural Engineers
  • Journal Title(Ko) :한국농공학회논문집
  • Volume : 63
  • No :1
  • Pages :11-25
  • Received Date : 2020-07-30
  • Revised Date : 2020-10-12
  • Accepted Date : 2020-10-14
  • DOI :https://doi.org/10.5389/KSAE.2021.63.1.011
Journal Informaiton Journal of Korean Society of Agricultural Engineers Journal of Korean Society of Agricultural Engineers
  • NRF
  • KOFST
  • crossref crossmark
  • crossref cited-by
  • crossref funder-registry
  • crosscheck
  • orcid
  • open access
Journal Informaiton Journal Informaiton - close