Vol. 20, No. 2, pp. 169-178 (2024)
SUPER-RESOLUTION RECONSTRUCTION AND HIGH-PRECISION
TEMPERATURE MEASUREMENT OF THERMAL IMAGES UNDER HIGH-
TEMPERATURE SCENES BASED ON NEURAL NETWORK
Yi-Chuan Dong, Jian Jiang *, Qing-Lin Wang, Wei Chen and Ji-Hong Ye
Jiangsu Key Laboratory of Environmental Impact and Structural Safety in Engineering, Xuzhou, China
*(Corresponding author: E-mail:This email address is being protected from spambots. You need JavaScript enabled to view it.)
Received: 4 June 2024; Revised: 4 June 2024; Accepted: 11 June 2024
DOI:10.18057/IJASC.2024.20.2.9
 View Article |
Export Citation: Plain Text | RIS | Endnote |
ABSTRACT
Accurate temperature readings are vital in fire resistance tests, but conventional thermal imagers often lack sufficient resolution, and applying super-resolution algorithms can disrupt the temperature and color correspondence, leading to limited efficiency. To address these issues, a convolutional network tailored for high-temperature scenes is designed for image super-resolution with the internal joint attention sub-residual blocks (JASRB) efficiently integrating channel, spatial attention mechanisms, and convolutional modules. Furthermore, a segmented method is developed for predicting thermal image temperature using color temperature measurements and an interpretable artificial neural network. This approach predicts temperatures in super-resolution thermal images ranging from 400 to 1200°C. Through comparative validation, it is found that the three-neuron neural network approach demonstrates superior prediction accuracy compared to other machine learning methods. The seamlessly combined proposed super-resolution architecture with the temperature measurement method has a predicted RMSE of 20°C for the whole temperature range with over 85% of samples falling within errors of 30°C.
KEYWORDS
High-temperature scene, Infrared thermal imaging, Neural networks, Image super-resolution, Color temperature prediction
REFERENCES
[1] Lou, G. B., Hou, J., Qi, H. H., Jiang, Y., Zhong, B., Pang, Y., and Li, G. Q. Experimental, numerical and analytical analysis of a single-span steel portal frame exposed to fire. Engineering Structures, 302, 117417, 2024.
[2] Chudasama, V., Patel, H., Prajapati, K., Upla, K. P., Ramachandra, R., Raja, K., and Busch, C. Therisurnet - a computationally efficient thermal image super-resolution network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 86-87), 2024.
[3] Keys, R. Cubic convolution interpolation for digital image processing. IEEE transactions on acoustics, speech, and signal processing, 29(6), 1153-1160, 1981.
[4] Kim, K. I., and Kwon, Y. Single-image super-resolution using sparse regression and natural image prior. IEEE transactions on pattern analysis and machine intelligence, 32(6), 1127-1133, 2010.
[5] Irani, M., and Peleg, S. Improving resolution by image registration. CVGIP: Graphical models and image processing, 53(3), 231-239, 1991.
[6] Li, J., Brown, C., Dzikowicz, D. J., Carey, M. G., Tam, W. C., and Huang, M. X. Towards real-time heart health monitoring in firefighting using convolutional neural networks. Fire Safety Journal, 140, 103852, 2023.
[7] Jiang, Y., Liu, Y., Zhan, W., and Zhu, D. Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion. Entropy, 25(6), 914, 2023.
[8] Hodges, J. L., Lattimer, B. Y., and Luxbacher, K. D. Compartment fire predictions using transpose convolutional neural networks. Fire Safety Journal, 108, 102854, 2019.
[9] Wang, Z., Chen, J., and Hoi, S. C. Deep learning for image super-resolution: A survey. IEEE transactions on pattern analysis and machine intelligence, 43(10), 3365-3387, 2020.
[10] Kim, J., Lee, J. K., and Lee, K. M. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1646-1654), 2016.
[11] Kim, J., Lee, J. K., and Lee, K. M. Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1637-1645), 2016.
[12] Tai, Y., Yang, J., and Liu, X. Image super-resolution via deep recursive residual network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3147-3155), 2017.
[13] Lim, B., Son, S., Kim, H., Nah, S., and Mu Lee, K. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 136-144), 2017.
[14] Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., ... and Wang, Z. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1874-1883), 2016.
[15] Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., and Fu, Y. Image super-resolution using very deep residual channel attention networks. In Proceedings of the European conference on computer vision (ECCV) (pp. 286-301), 2018.
[16] Woo, S., Park, J., Lee, J. Y., and Kweon, I. S. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV) (pp. 3-19), 2018.
[17] Hu, J., Shen, L., and Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141), 2018.
[18] Choi, Y., Kim, N., Hwang, S., and Kweon, I. S. Thermal image enhancement using convolutional neural network. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 223-230). IEEE, 2016.
[19] Lee, K., Lee, J., Lee, J., Hwang, S., and Lee, S. Brightness-based convolutional neural network for thermal image enhancement. IEEE Access, 5, 26867-26879, 2017.
[20] Liu, S., Yang, Y., Li, Q., Feng, H., Xu, Z., Chen, Y., and Liu, L. Infrared image super resolution using gan with infrared image prior. In 2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP) (pp. 1004-1009). IEEE, 2019.
[21] Rivadeneira, R. E., Sappa, A. D., and Vintimilla, B. X. Thermal Image Super-resolution: A Novel Architecture and Dataset. In VISIGRAPP (4: VISAPP) (pp. 111-119), 2020.
[22] Hwang, S., Park, J., Kim, N., Choi, Y., and So Kweon, I. Multispectral pedestrian detection: Benchmark dataset and baseline. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1037-1045), 2015.
[23] He, Z., Tang, S., Yang, J., Cao, Y., Yang, M. Y., and Cao, Y. Cascaded deep networks with multiple receptive fields for infrared image super-resolution. IEEE transactions on circuits and systems for video technology, 29(8), 2310-2322, 2018.
[24] Miklavec, A., Pušnik, I., Batagelj, V., and Drnovšek, J. A large aperture blackbody bath for calibration of thermal imagers. Measurement Science and Technology, 24(2), 025001, 2012.
[25] Pelzmann, T., Dupont, F., Sauté, B., and Robert, É. Two-color pyrometry for backface temperature and emissivity measurement of burning materials. International Journal of Thermal Sciences, 184, 107894, 2023.
[26] Zamarripa-Ramírez, J. C. I., Moreno-Hernández, D., and Martinez-Gonzalez, A. Simultaneous measurement of temperature and color spectrum of axisymmetric premixed flames using digital laser speckle photography and an image processing approach. Measurement Science and Technology, 32(10), 105903, 2021.
[27] Shan, L., Huang, H., Hong, B., Zhao, J., Wang, D., and Kong, M. Temperature measurement method of flame image fusion with different exposures. Energies, 13(6), 1487, 2020.
[28] Zhong, Y., Li, X., Yang, Z., Liu, X., and Yao, E. A model for minimum ignition energy prediction of sugar dust clouds based on interactive orthogonal experiments and machine learning. Fire Safety Journal, 104111, 2024.
[29] Qu, N., Li, Z., Li, X., Zhang, S., and Zheng, T. Multi-parameter fire detection method based on feature depth extraction and stacking ensemble learning model. Fire Safety Journal, 128, 103541, 2022.
[30] Özyurt, O. The effect of time correlation function, Sauter mean diameter and asymmetry ratio on machine learning classifiers for particle discrimination by using light scattering. Fire Safety Journal, 141, 104002, 2023.
[31] Qu, X. S., Deng, Y. X., and Sun, G. J. INVESTIGATION ON BEHAVIOR OF STEEL CABLES SUBJECT TO LOCALIZED FIRE IN LARGE-SPACE BUILDINGS. ADVANCED STEEL CONSTRUCTION, 20(1), 1-11, 2024.
[32] Nan, Z., Orabi, M. A., Huang, X., Jiang, Y., and Usmani, A. Structural-fire responses forecasting via modular AI. Fire Safety Journal, 140, 103863, 2023.
[33] Couto, C., Tong, Q., and Gernay, T. Predicting the capacity of thin-walled beams at elevated temperature with machine learning. Fire Safety Journal, 130, 103596, 2022.
[34] Sun, B., Liu, X., Xu, Z. D., and Xu, D. BP neural network-based adaptive spatial-temporal data generation technology for predicting ceiling temperature in tunnel fire and full-scale experimental verification. Fire Safety Journal, 130, 103577, 2022.
[35] Chen, L., Zhang, H. Y., Liu, S. W., and Chan, S. L. SECOND-ORDER ANALYSIS OF BEAM-COLUMNS BY MACHINE LEARNING-BASED STRUCTURAL ANALYSIS THROUGH PHYSICS-INFORMED NEURAL NETWORKS. Advanced Steel Construction, 19(4), 411-420, 2023.
[36] Wang, Z. Q., Zheng, Z. Mengxiang, J. W., and Jiang, W. Q. INVERSION METHOD OF UNCERTAIN PARAMETERS FOR TRUSS STRUCTURES BASED ON GRAPH NEURAL NETWORKS. ADVANCED STEEL CONSTRUCTION, 19(4): p. 366-374, 2023.
[37] Sharifi, Y., and Tohidi, S. Ultimate capacity assessment of web plate beams with pitting corrosion subjected to patch loading by artificial neural networks. Advanced Steel Construction, 10(3), 325-350, 2014.
[38] Nair, V., and Hinton, G. E. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10) (pp. 807-814), 2010.
[39] Kingma, D. P., and Ba, J. Adam: A method for stochastic optimization. arxiv preprint arxiv:1412.6980, 2014.
[40] Mittal, A., Soundararajan, R., and Bovik, A. C. Making a “completely blind” image quality analyzer. IEEE Signal processing letters, 20(3), 209-212, 2012.
[41] Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 31, No. 1), 2017.
[42] Zhao, H., Kong, X., He, J., Qiao, Y., and Dong, C. Efficient image super-resolution using pixel attention. In Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16 (pp. 56-72). Springer International Publishing, 2020.
[43] Ismail, I., Yefriadi, Y., Yuhefizar, Y., and Hendri, Z. The Generating Super Resolution of Thermal Image based on Deep Learning. Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), 6(2), 289-294, 2022.
[44] Wang, Y., Zhao, L., Liu, L., Hu, H., and Tao, W. URNet: a U-shaped residual network for lightweight image super-resolution. Remote Sensing, 13(19), 3848, 2021.
[45] Jiang, J., Wang, C., Lou, G., and Li, G. Q. Quantitative evaluation of progressive collapse process of steel portal frames in fire. Journal of Constructional Steel Research, 150, 277-287, 2018.
[46] Marquardt, D. W. An algorithm for least-squares estimation of nonlinear parameters. Journal of the society for Industrial and Applied Mathematics, 11(2), 431-441, 1963.
[47] Liu, X., and Aldrich, C. Assessing the influence of operational variables on process performance in metallurgical plants by use of Shapley value regression. Metals, 12(11), 1777, 2022.
[48] Lundberg, S. M., and Lee, S. I. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
[49] Andrew, A. M. An introduction to support vector machines and other kernel‐based learning methods. Kybernetes, 30(1), 103-115, 2001.
[50] Huang, P. S., Avron, H., Sainath, T. N., Sindhwani, V., and Ramabhadran, B. Kernel methods match deep neural networks on timit. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 205-209). IEEE, 2014.
[51] Loh, W. Y. Regression tress with unbiased variable selection and interaction detection. Statistica sinica, 361-386, 2002.
[52] Street, J. O., Carroll, R. J., & Ruppert, D. (1988). A note on computing robust regression estimates via iteratively reweighted least squares. The American Statistician, 42(2), 152-154.