Image Mining Based on Deep Belief Neural Network and Feature Matching Approach Using Manhattan Distance
Abstract
Over the past few decades multimedia content, particularly digital images, has increased at a rapid pace, with several complex images being uploaded to various social websites such as Instagram, Facebook and Twitter. Therefore, it is difficult to search and retrieve the relevant image in seconds. Search engines retrieve images based on traditional textbased methods that depend on metadata and captions. In the last few years, a wide range of research has focused on content-based image retrieval (CBIR) based on image mining approaches. This is a challenging research area due to the ever-increasing multimedia database and other image libraries. In order to offer an effective search and retrieval, a novel CBIR system is proposed using the image mining-based deep belief neural network (IMDBN) technique. The proposed method is designed to enhance retrieval accuracy while diminishing the semantic gap between human visual understanding and image feature representation. To achieve this objective, the proposed system carries several steps like preprocessing, feature extraction, classification, and feature matching. Initially, the input database images are fed into the proposed image mining-based CBIR system, whereas colour-shape-texture (CST) feature extraction technique is applied to extract relevant feature set. The extracted features are fused and stored in the feature vector and are
subjected to the proposed IMDBN classification step to retrieve similar images in one label. Whenever a new query content is created, the most relevant images are retrieved. This, in turn, achieves 94% accuracy, which is higher than in existing approaches.
Keywords:
mining, deep belief neural network, Manhattan distance, median filter, energy, correlation, contrast, dissimilarity, homogeneityReferences
1. Q. Zheng, M. Yang., J. Yang, Q. Zhang, X. Zhang, Improvement of generalization ability of deep CNN via implicit regularization in two-stage training process, IEEE Access, 6: 15844–15869, 2018, https://doi.org/10.1109/ACCESS.2018.2810849
2. Q. Zheng, X. Tian, N. Jiang, M Yang, Layer-wise learning based stochastic gradient descent method for the optimization of deep convolutional neural network, Journal of Intelligent & Fuzzy Systems, 37(4): 5641–5654, 2019, https://doi.org/10.3233/JIFS-190861
3. T.A. Ahmed, S. Ummesafi, A. Iqbal, Content based image retrieval using image features information fusion, Information Fusion, 51: 76–99, https://doi.org/10.1016/j.inffus.2018.11.004
4. M.S. Haji, M.H. Alkawaz, A. Rehman, T. Saba, Content-based image retrieval: A deep look at features prospectus, International Journal of Computational Vision and Robotics, 9(1): 14–38, 2019, https://doi.org/10.1504/IJCVR.2019.098004
5. Q. Zheng, X. Tian, M. Yang, H. Wang,Differential learning: a powerful tool for interactive content-based image retrieval, Engineering Letters, 27(1): 202–215, 2019.
6. S.P. Rana, M. Dey, P. Siarry, Boosting content based image retrieval performance through integration of parametric & nonparametric approaches, Journal of Visual Communication and Image Representation, 58: 205–219, 2019, https://doi.org/10.1016/j.jvcir.2018.11.015
7. U. Chaudhuri, B. Banerjee, A. Bhattacharya, Siamese graph convolutional network for content based remote sensing image retrieval, Computer Vision and Image Understanding, 184: 22–30, 2019, https://doi.org/10.1016/j.cviu.2019.04.004
8. R.R. Saritha, V. Paul, P.G. Kumar, Content based image retrieval using deep learning process, Cluster Computing, 22(2): 4187–4200, 2019, https://doi.org/10.1007/s10586-018-1731-0
9. U. Sharif, Z. Mehmood, T. Mahmood, M.A. Javid, A. Rehman, T. Saba, Scene analysis and search using local features and support vector machine for effective content-based image retrieval, Artificial Intelligence Review, 52(2): 901–925, 2019, https://doi.org/10.1007/s10462-018-9636-0
10. A. Rashno, E. Rashno, Content-based image retrieval system with most relevant features among wavelet and color features, arXiv preprint, arXiv:1902.02059, [eess.IV], 2019.
11. P. Tschandl, G. Argenziano, M. Razmara, J. Yap, Diagnostic accuracy of content-based dermatoscopic image retrieval with deep classification features, British Journal of Dermatology, 181(1): 155–165, 2019.
12. A. Sarwar, Z. Mehmood, T. Saba, K.A. Qazi, A. Adnan, H. Jamal, A novel method for content-based image retrieval to improve the effectiveness of the bag-of-words model using a support vector machine, Journal of Information Science, 45(1): 117–135, 2019, https://doi.org/10.1177/0165551518782825
13. Y.D. Mistry, Textural and color descriptor fusion for efficient content-based image retrieval algorithm, Iran Journal of Computer Science, 3(3): 169–183, 2020, https://doi.org/10.1007/s42044-020-00056-0
14. R. Ashraf, M. Ahmed, U. Ahmad, M.A. Habib, S. Jabbar, K. Naseer, MDCBIR-MF: multimedia data for content-based image retrieval by using multiple features, Multimedia Tools and Applications, 79(13): 8553–8579, 2020, https://doi.org/10.1007/s11042-018-5961-1
15. N.F. Haq, M. Moradi, Z.J. Wang, A deep community based approach for large scale content based X-ray image retrieval, Medical Image Analysis, 68: 101847, https://doi.org/10.1016/j.media.2020.101847
16. A.R. Zubair, O.A. Alo, Content-based image retrieval system using second-order statistics, International Journal of Computer Applications, 176(36), 9 pages, 2020, https://doi.org/10.5120/ijca2020920475
17. H.-H. Bu, N.-C. Kim, B.-J. Yun, S.-H. Kim, Content-based image retrieval using multiresolution multi-direction filtering-based CLBP texture features and color autocorrelogram features, Journal of Information Processing Systems, 16(4), 991–1000, 2020, https://doi.org/10.3745/JIPS.02.0138
18. G. Zhao, M. Zhang, J. Liu, Y. Li, J.-R. Wen, AP-GAN: Adversarial patch attack on content-based image retrieval systems, Geoinformatica, 1–31, 2020, https://doi.org/10.1007/s10707-020-00418-7
19. S. Singh, S. Batra, An efficient bi-layer content based image retrieval system, Multimedia Tools & Applications, 79(25/26): 17731–17759, 2020.
20. G. Xie, B. Guo, Z. Huang, Y. Zheng, Y. Yan, Combination of dominant color descriptor and Hu moments in consistent zone for content based image retrieval, IEEE Access, 8: 146284–146299, 2020, https://doi.org/10.1109/ACCESS.2020.3015285
21. P. Kavitha, P.V. Saraswathi, Segmentation for content based satellite image retrieval using fuzzy clustering, International Journal of Advanced Science and Technology, 29(9s): 3042–3049, 2020.
22. G. Rudrappa, N. Vijapur, Cloud classification using K-means clustering and content based image retrieval technique, [in]: 2020 International Conference on Communication and Signal Processing (ICCSP), pp. 0700–0704, 2020, https://doi.org/10.1109/ICCSP48568.2020.9182211
23. X.-Y. Wang, B.-B. Zhang, H.-Y. Yang, Content-based image retrieval by integrating color and texture features, Multimedia Tools and Applications, 68(3): 545–569, 2014.
24. T. Kato, Database architecture for content-based image retrieval, [in:] Image storage and retrieval systems, International Society for Optics and Photonics, Vol. 1662, pp. 112–123, 1992.
25. H.-H. Bu, N.-C. Kim, C.-J. Moon, J.-H. Kim, Content-based image retrieval using combined color and texture features extracted by multi-resolution multi-direction filtering, Journal of Information Processing Systems, 13(3): 464–475, 2017, https://doi.org/10.3745/JIPS.02.0060
26. J.J. Thiagarajan, K.N. Ramamurthy, P. Sattigeri, A. Spanias, Supervised local sparse coding of sub-image features for image retrieval, [in:] 2012 19th IEEE International Conference on Image Processing, pp. 3117–3120, 2012, https://doi.org/10.1109/ICIP.2012.6467560
27. A. Latif, A. Rasheed, U. Sajid, J. Ahmed, N. Ali, N.I. Ratyal, Z. Bushra, S.H. Dar, M. Sajid, T. Khalil, Content-based image retrieval and feature extraction: a comprehensive review, Mathematical Problems in Engineering, 2019, Article ID 9658350, 21 pages, 2019, https://doi.org/10.1155/2019/9658350
28. Y. Hou, Q. Wang, Research and improvement of content-based image retrieval framework, International Journal of Pattern Recognition and Artificial Intelligence, 32(12), 1850043, 2018.
29. S. Jabeen, Z. Mehmood, T. Mahmood, T. Saba, A. Rehman, M.T. Mahmood, An effective content-based image retrieval technique for image visuals representation based on the bag-of-visual-words model, PloS ONE, 13(4): e0194526, 2018, https://doi.org/10.1371/journal.pone.0194526
30. J.M. Medina, S. Jaime-Castillo, C.D. Barranco, J.R. Campana, On the use of a fuzzy object-relational database for flexible retrieval of medical images, IEEE Transactions on Fuzzy Systems, 20(4): 786–803. 2012, https://doi.org/10.1109/TFUZZ.2012.2201726
31. E. Yildizer, A.M. Balci, T.N. Jarada, R. Alhajj, Integrating wavelets with clustering and indexing for effective content-based image retrieval, Knowledge-Based Systems, 31: 55–66, 2012, https://doi.org/10.1016/j.knosys.2012.01.013
32. N. Jyothi, D. Madhavi, M.R. Patnaik, Optimization of log Gabor filters using genetic algorithm for query by image content systems, [in:] S. Choudhury, R. Mishra, R. Mishra, A. Kumar [Eds], Intelligent Communication, Control and Devices, pp. 799–806, Springer, Singapore, 2020.
33. M. Kokare, P.K. Biswas, B.N. Chatterji, Rotated complex wavelet based texture features for content based image retrieval, 17th International Conference on Pattern Recognition, ICPR 2004, Cambridge, UK, August 23–26, pp. 652–655, 2004, https://doi.org/10.1109/ICPR.2004.1334250
34. Q. Zheng, X. Tian, M. Yang, Y. Wu, H. Su, PAC-Bayesian framework based drop-path method for 2D discriminative convolutional network pruning, Multidimensional Systems and Signal Processing, 31: 793–827, 2020, https://doi.org/10.1007/s11045-019-00686-z
35. Q. Zheng, M. Yang, X. Tian, N. Jiang, D. Wang, A full stage data augmentation method in deep convolutional neural network for natural image classification, Discrete Dynamics in Nature and Society, 2020: Article ID 4706576, 11 pages, 2020, https://doi.org/10.1155/2020/4706576
36. Q. Zheng, P. Zhao, Y. Li, H. Wang, Y. Yang, Spectrum interference-based two-level data augmentation method in deep learning for automatic modulation classification, Neural Computing and Applications, 33: 7723–7745, 2021, https://doi.org/10.1007/s00521-020-05514-1
37. I. Loboda, Y. Feldshteyn, V. Ponomaryov, Neural networks for gas turbine fault identification: Multilayer perceptron or radial basis network?, International Journal of Turbo and Jet Engines, 29(1): 37–48, 2012, https://doi.org/10.1515/tjj-2012-0005
38. Q. Liang, Y. Wang, W. Nie, Q. Li, MVCLN: multi-view convolutional LSTM network for cross-media 3D shape recognition, IEEE Access, 8: 139792–139802, 2020, https://doi.org/10.1109/ACCESS.2020.3012692
39. S. Antani, L.R. Long, G.R. Thoma, R.J. Stanley, Vertebra shape classification using MLP for content-based image retrieval, [in:] Proceedings of the International Joint Conference on Neural Networks, Vol. 1, pp. 160–165, 2003, https://doi.org/10.1109/IJCNN.2003.1223324
40. M. Kaur, D. Singh, Fusion of medical images using deep belief networks, Cluster Computing, 23(2): 1439–1453, 2020, https://doi.org/10.1007/s10586-019-02999-x
41. S. Hussain, M. Hashmani, M. Moinuddin, M. Yoshida, H. Kanjo, Image retrieval based on color and texture feature using artificial neural network, [in:] B.S. Chowdhry, F.K. Shaikh, D.M.A. Hussain, M.A. Uqaili [Eds], Emerging Trends and Applications in Information Communication Technologies, IMTIC 2012, Communications in Computer and Information Science, vol. 281, pp. 501–511, Springer, Berlin, Heidelberg, 2012, https://doi.org/10.1007/978-3-642-28962-0_47
2. Q. Zheng, X. Tian, N. Jiang, M Yang, Layer-wise learning based stochastic gradient descent method for the optimization of deep convolutional neural network, Journal of Intelligent & Fuzzy Systems, 37(4): 5641–5654, 2019, https://doi.org/10.3233/JIFS-190861
3. T.A. Ahmed, S. Ummesafi, A. Iqbal, Content based image retrieval using image features information fusion, Information Fusion, 51: 76–99, https://doi.org/10.1016/j.inffus.2018.11.004
4. M.S. Haji, M.H. Alkawaz, A. Rehman, T. Saba, Content-based image retrieval: A deep look at features prospectus, International Journal of Computational Vision and Robotics, 9(1): 14–38, 2019, https://doi.org/10.1504/IJCVR.2019.098004
5. Q. Zheng, X. Tian, M. Yang, H. Wang,Differential learning: a powerful tool for interactive content-based image retrieval, Engineering Letters, 27(1): 202–215, 2019.
6. S.P. Rana, M. Dey, P. Siarry, Boosting content based image retrieval performance through integration of parametric & nonparametric approaches, Journal of Visual Communication and Image Representation, 58: 205–219, 2019, https://doi.org/10.1016/j.jvcir.2018.11.015
7. U. Chaudhuri, B. Banerjee, A. Bhattacharya, Siamese graph convolutional network for content based remote sensing image retrieval, Computer Vision and Image Understanding, 184: 22–30, 2019, https://doi.org/10.1016/j.cviu.2019.04.004
8. R.R. Saritha, V. Paul, P.G. Kumar, Content based image retrieval using deep learning process, Cluster Computing, 22(2): 4187–4200, 2019, https://doi.org/10.1007/s10586-018-1731-0
9. U. Sharif, Z. Mehmood, T. Mahmood, M.A. Javid, A. Rehman, T. Saba, Scene analysis and search using local features and support vector machine for effective content-based image retrieval, Artificial Intelligence Review, 52(2): 901–925, 2019, https://doi.org/10.1007/s10462-018-9636-0
10. A. Rashno, E. Rashno, Content-based image retrieval system with most relevant features among wavelet and color features, arXiv preprint, arXiv:1902.02059, [eess.IV], 2019.
11. P. Tschandl, G. Argenziano, M. Razmara, J. Yap, Diagnostic accuracy of content-based dermatoscopic image retrieval with deep classification features, British Journal of Dermatology, 181(1): 155–165, 2019.
12. A. Sarwar, Z. Mehmood, T. Saba, K.A. Qazi, A. Adnan, H. Jamal, A novel method for content-based image retrieval to improve the effectiveness of the bag-of-words model using a support vector machine, Journal of Information Science, 45(1): 117–135, 2019, https://doi.org/10.1177/0165551518782825
13. Y.D. Mistry, Textural and color descriptor fusion for efficient content-based image retrieval algorithm, Iran Journal of Computer Science, 3(3): 169–183, 2020, https://doi.org/10.1007/s42044-020-00056-0
14. R. Ashraf, M. Ahmed, U. Ahmad, M.A. Habib, S. Jabbar, K. Naseer, MDCBIR-MF: multimedia data for content-based image retrieval by using multiple features, Multimedia Tools and Applications, 79(13): 8553–8579, 2020, https://doi.org/10.1007/s11042-018-5961-1
15. N.F. Haq, M. Moradi, Z.J. Wang, A deep community based approach for large scale content based X-ray image retrieval, Medical Image Analysis, 68: 101847, https://doi.org/10.1016/j.media.2020.101847
16. A.R. Zubair, O.A. Alo, Content-based image retrieval system using second-order statistics, International Journal of Computer Applications, 176(36), 9 pages, 2020, https://doi.org/10.5120/ijca2020920475
17. H.-H. Bu, N.-C. Kim, B.-J. Yun, S.-H. Kim, Content-based image retrieval using multiresolution multi-direction filtering-based CLBP texture features and color autocorrelogram features, Journal of Information Processing Systems, 16(4), 991–1000, 2020, https://doi.org/10.3745/JIPS.02.0138
18. G. Zhao, M. Zhang, J. Liu, Y. Li, J.-R. Wen, AP-GAN: Adversarial patch attack on content-based image retrieval systems, Geoinformatica, 1–31, 2020, https://doi.org/10.1007/s10707-020-00418-7
19. S. Singh, S. Batra, An efficient bi-layer content based image retrieval system, Multimedia Tools & Applications, 79(25/26): 17731–17759, 2020.
20. G. Xie, B. Guo, Z. Huang, Y. Zheng, Y. Yan, Combination of dominant color descriptor and Hu moments in consistent zone for content based image retrieval, IEEE Access, 8: 146284–146299, 2020, https://doi.org/10.1109/ACCESS.2020.3015285
21. P. Kavitha, P.V. Saraswathi, Segmentation for content based satellite image retrieval using fuzzy clustering, International Journal of Advanced Science and Technology, 29(9s): 3042–3049, 2020.
22. G. Rudrappa, N. Vijapur, Cloud classification using K-means clustering and content based image retrieval technique, [in]: 2020 International Conference on Communication and Signal Processing (ICCSP), pp. 0700–0704, 2020, https://doi.org/10.1109/ICCSP48568.2020.9182211
23. X.-Y. Wang, B.-B. Zhang, H.-Y. Yang, Content-based image retrieval by integrating color and texture features, Multimedia Tools and Applications, 68(3): 545–569, 2014.
24. T. Kato, Database architecture for content-based image retrieval, [in:] Image storage and retrieval systems, International Society for Optics and Photonics, Vol. 1662, pp. 112–123, 1992.
25. H.-H. Bu, N.-C. Kim, C.-J. Moon, J.-H. Kim, Content-based image retrieval using combined color and texture features extracted by multi-resolution multi-direction filtering, Journal of Information Processing Systems, 13(3): 464–475, 2017, https://doi.org/10.3745/JIPS.02.0060
26. J.J. Thiagarajan, K.N. Ramamurthy, P. Sattigeri, A. Spanias, Supervised local sparse coding of sub-image features for image retrieval, [in:] 2012 19th IEEE International Conference on Image Processing, pp. 3117–3120, 2012, https://doi.org/10.1109/ICIP.2012.6467560
27. A. Latif, A. Rasheed, U. Sajid, J. Ahmed, N. Ali, N.I. Ratyal, Z. Bushra, S.H. Dar, M. Sajid, T. Khalil, Content-based image retrieval and feature extraction: a comprehensive review, Mathematical Problems in Engineering, 2019, Article ID 9658350, 21 pages, 2019, https://doi.org/10.1155/2019/9658350
28. Y. Hou, Q. Wang, Research and improvement of content-based image retrieval framework, International Journal of Pattern Recognition and Artificial Intelligence, 32(12), 1850043, 2018.
29. S. Jabeen, Z. Mehmood, T. Mahmood, T. Saba, A. Rehman, M.T. Mahmood, An effective content-based image retrieval technique for image visuals representation based on the bag-of-visual-words model, PloS ONE, 13(4): e0194526, 2018, https://doi.org/10.1371/journal.pone.0194526
30. J.M. Medina, S. Jaime-Castillo, C.D. Barranco, J.R. Campana, On the use of a fuzzy object-relational database for flexible retrieval of medical images, IEEE Transactions on Fuzzy Systems, 20(4): 786–803. 2012, https://doi.org/10.1109/TFUZZ.2012.2201726
31. E. Yildizer, A.M. Balci, T.N. Jarada, R. Alhajj, Integrating wavelets with clustering and indexing for effective content-based image retrieval, Knowledge-Based Systems, 31: 55–66, 2012, https://doi.org/10.1016/j.knosys.2012.01.013
32. N. Jyothi, D. Madhavi, M.R. Patnaik, Optimization of log Gabor filters using genetic algorithm for query by image content systems, [in:] S. Choudhury, R. Mishra, R. Mishra, A. Kumar [Eds], Intelligent Communication, Control and Devices, pp. 799–806, Springer, Singapore, 2020.
33. M. Kokare, P.K. Biswas, B.N. Chatterji, Rotated complex wavelet based texture features for content based image retrieval, 17th International Conference on Pattern Recognition, ICPR 2004, Cambridge, UK, August 23–26, pp. 652–655, 2004, https://doi.org/10.1109/ICPR.2004.1334250
34. Q. Zheng, X. Tian, M. Yang, Y. Wu, H. Su, PAC-Bayesian framework based drop-path method for 2D discriminative convolutional network pruning, Multidimensional Systems and Signal Processing, 31: 793–827, 2020, https://doi.org/10.1007/s11045-019-00686-z
35. Q. Zheng, M. Yang, X. Tian, N. Jiang, D. Wang, A full stage data augmentation method in deep convolutional neural network for natural image classification, Discrete Dynamics in Nature and Society, 2020: Article ID 4706576, 11 pages, 2020, https://doi.org/10.1155/2020/4706576
36. Q. Zheng, P. Zhao, Y. Li, H. Wang, Y. Yang, Spectrum interference-based two-level data augmentation method in deep learning for automatic modulation classification, Neural Computing and Applications, 33: 7723–7745, 2021, https://doi.org/10.1007/s00521-020-05514-1
37. I. Loboda, Y. Feldshteyn, V. Ponomaryov, Neural networks for gas turbine fault identification: Multilayer perceptron or radial basis network?, International Journal of Turbo and Jet Engines, 29(1): 37–48, 2012, https://doi.org/10.1515/tjj-2012-0005
38. Q. Liang, Y. Wang, W. Nie, Q. Li, MVCLN: multi-view convolutional LSTM network for cross-media 3D shape recognition, IEEE Access, 8: 139792–139802, 2020, https://doi.org/10.1109/ACCESS.2020.3012692
39. S. Antani, L.R. Long, G.R. Thoma, R.J. Stanley, Vertebra shape classification using MLP for content-based image retrieval, [in:] Proceedings of the International Joint Conference on Neural Networks, Vol. 1, pp. 160–165, 2003, https://doi.org/10.1109/IJCNN.2003.1223324
40. M. Kaur, D. Singh, Fusion of medical images using deep belief networks, Cluster Computing, 23(2): 1439–1453, 2020, https://doi.org/10.1007/s10586-019-02999-x
41. S. Hussain, M. Hashmani, M. Moinuddin, M. Yoshida, H. Kanjo, Image retrieval based on color and texture feature using artificial neural network, [in:] B.S. Chowdhry, F.K. Shaikh, D.M.A. Hussain, M.A. Uqaili [Eds], Emerging Trends and Applications in Information Communication Technologies, IMTIC 2012, Communications in Computer and Information Science, vol. 281, pp. 501–511, Springer, Berlin, Heidelberg, 2012, https://doi.org/10.1007/978-3-642-28962-0_47
