Machine Learning Meets the Semantic Web

Source:
By:Konstantinos Ilias Kotis, Konstantina Zachila, Evaggelos Paparidis

DOI: https://doi.org/10.30564/aia.v3i1.3178

Abstract: Remarkable progress in research has shown the efficiency of Knowledge Graphs (KGs) in extracting valuable external knowledge in various domains. A Knowledge Graph (KG) can illustrate high-order relations that connect two objects with one or multiple related attributes. The emerging Graph Neural Networks (GNN) can extract both object characteristics and relations from KGs. This paper presents how Machine Learning (ML) meets the Semantic Web and how KGs are related to Neural Networks and Deep Learning. The paper also highlights important aspects of this area of research, discussing open issues such as the bias hidden in KGs at different levels of graph representation. References:

[1] P. A. Bonatti, S. Decker, A. Polleres, and V. Presutti, “Knowledge graphs: New directions for knowledge representation on the semantic web (dagstuhl seminar 18371),” in Dagstuhl Reports, vol. 8, no. 9. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019. [2] C. d’Amato, “Machine learning for the semantic web: Lessons learnt and next research directions,” Semantic Web, no. Preprint, pp. 1–9, 2020. [3] M. Nickel, K. Murphy, V. Tresp, and E. Gabrilovich, “A review of relational machine learning for knowledge graphs,” Proceedings of the IEEE, vol. 104, no. 1, pp. 11–33, 2015. [4] E. Marchi and O. Miguel, “On the structure of the teaching-learning interactive process,” International Journal of Game Theory, vol. 3, no. 2, pp. 83–99, 1974. [5] H. van den Berg, “First-order logic in knowledge graphs,” Current Issues in Mathematical Linguistics, vol. 56, pp. 319–328, 1993. [6] R. R. Bakker, Knowledge Graphs: representation and structuring of scientific knowledge, 1987. [7] L. Ehrlinger and W. Woß, “Towards a definition of knowledge graphs.”¨ SEMANTiCS (Posters, Demos, SuCCESS), vol. 48, pp. 1–4, 2016. [8] M. Farber, F. Bartscherer, C. Menne, and A. Rettinger, “Linked data¨ quality of dbpedia, freebase, opencyc, wikidata, and yago,” Semantic Web, vol. 9, no. 1, pp. 77–129, 2018. [9] Z. Huang, J. Yang, F. van Harmelen, and Q. Hu, “Constructing diseasecentric knowledge graphs: a case study for depression (short version),” in Conference on Artificial Intelligence in Medicine in Europe. Springer, 2017, pp. 48–52. [10] H. Paulheim, “Knowledge graph refinement: A survey of approaches and evaluation methods,” Semantic web, vol. 8, no. 3, pp. 489–508, 2017. [11] C. Bizer, T. Heath, and T. Berners-Lee, “Linked data: The story so far,” in Semantic services, interoperability and web applications: emerging concepts. IGI global, 2011, pp. 205–227. [12] Z. Zhao, S.-K. Han, and I.-M. So, “Architecture of knowledge graph construction techniques,” International Journal of Pure and Applied Mathematics, vol. 118, no. 19, pp. 1869–1883, 2018. [13] J. P. McCusker, J. Erickson, K. Chastain, S. Rashid, R. Weerawarana, and D. McGuinness, “What is a knowledge graph,” Semantic Web Journal, 2018. [14] C. d’Amato, N. Fanizzi, and F. Esposito, “Inductive learning for the semantic web: what does it buy?” Semantic Web, vol. 1, no. 1, 2, pp. 53–59, 2010. [15] L. Deng and D. Yu, “Deep learning: methods and applications,” Foundations and trends in signal processing, vol. 7, no. 3–4, pp. 197–387, 2014. [16] C. d’Amato, N. Fanizzi, and F. Esposito, “Query answering and ontology population: An inductive approach,” in European Semantic Web Conference. Springer, 2008, pp. 288–302. [17] A. Rettinger, U. Losch, V. Tresp, C. d’Amato, and N. Fanizzi, “Mining¨ the semantic web,” Data Mining and Knowledge Discovery, vol. 24, no. 3, pp. 613–662, 2012. [18] S. Bloehdorn and Y. Sure, “Kernel methods for mining instance data in ontologies,” in The Semantic Web. Springer, 2007, pp. 58–71. [19] N. Fanizzi, C. d’Amato, and F. Esposito, “Induction of concepts in web ontologies through terminological decision trees,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 2010, pp. 442–457. [20] G. Rizzo, N. Fanizzi, C. d’Amato, and F. Esposito, “Approximate classification with web ontologies through evidential terminological trees and forests,” International Journal of Approximate Reasoning, vol. 92, pp. 340–362, 2018. [21] N. Fanizzi, C. d’Amato, and F. Esposito, “Dl-foil concept learning in description logics,” in International Conference on Inductive Logic Programming. Springer, 2008, pp. 107–121. [22] A. C. Tran, J. Dietrich, H. W. Guesgen, and S. Marsland, “An approach to parallel class expression learning,” in International Workshop on Rules and Rule Markup Languages for the Semantic Web. Springer, 2012, pp. 302–316. [23] J. Lehmann, S. Auer, L. Buhmann, and S. Tramp, “Class expression¨ learning for ontology engineering,” Journal of Web Semantics, vol. 9, no. 1, pp. 71–81, 2011. [24] G. Rizzo, N. Fanizzi, C. d’Amato, and F. Esposito, “A framework for tackling myopia in concept learning on the web of data,” in European Knowledge Acquisition Workshop. Springer, 2018, pp. 338–354. [25] A. C. Tran, J. Dietrich, H. W. Guesgen, and S. Marsland, “Parallel symmetric class expression learning,” The Journal of Machine Learning Research, vol. 18, no. 1, pp. 2145–2178, 2017. [26] F. Baader, D. Calvanese, D. McGuinness, P. Patel-Schneider, D. Nardi et al., The description logic handbook: Theory, implementation and applications. Cambridge university press, 2003. [27] C. d’Amato, A. G. Tettamanzi, and T. D. Minh, “Evolutionary discovery of multi-relational association rules from ontological knowledge bases,” in European knowledge acquisition workshop. Springer, 2016, pp. 113– 128. [28] J. Volker, D. Fleischhacker, and H. Stuckenschmidt, “Automatic acqui-¨ sition of class disjointness,” Journal of Web Semantics, vol. 35, pp. 124– 139, 2015. [29] J. Volker and M. Niepert, “Statistical schema induction,” in¨ Extended Semantic Web Conference. Springer, 2011, pp. 124–138. [30] A. Rettinger, M. Nickles, and V. Tresp, “Statistical relational learning with formal ontologies,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 2009, pp. 286–301. [31] M. Nickel, V. Tresp, and H.-P. Kriegel, “A three-way model for collective learning on multi-relational data,” in Icml, 2011. [32] Y. LeCun, Y. Bengio et al., “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks, vol. 3361, no. 10, p. 1995, 1995. [33] J. Schmidhuber and S. Hochreiter, “Long short-term memory,” Neural Comput, vol. 9, no. 8, pp. 1735–1780, 1997. [34] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997. [35] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, and L. Bottou, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion.” Journal of machine learning research, vol. 11, no. 12, 2010. [36] Y. Gao, Y.-F. Li, Y. Lin, H. Gao, and L. Khan, “Deep learning on knowledge graph for recommender system: A survey,” arXiv preprint arXiv:2004.00387, 2020. [37] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” IEEE transactions on neural networks and learning systems, 2020. [38] M. Gori, G. Monfardini, and F. Scarselli, “A new model for learning in graph domains,” in Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., vol. 2. IEEE, 2005, pp. 729– 734. [39] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE transactions on neural networks, vol. 20, no. 1, pp. 61–80, 2008. [40] C. Gallicchio and A. Micheli, “Graph echo state networks,” in The 2010 International Joint Conference on Neural Networks (IJCNN). IEEE, 2010, pp. 1–8. [41] Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel, “Gated graph sequence neural networks,” arXiv preprint arXiv:1511.05493, 2015. [42] H. Dai, Z. Kozareva, B. Dai, A. Smola, and L. Song, “Learning steadystates of iterative algorithms over graphs,” in International conference on machine learning. PMLR, 2018, pp. 1106–1114. [43] J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun, “Spectral networks and locally connected networks on graphs,” arXiv preprint arXiv:1312.6203, 2013. [44] M. Henaff, J. Bruna, and Y. LeCun, “Deep convolutional networks on graph-structured data,” arXiv preprint arXiv:1506.05163, 2015. [45] M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” arXiv preprint arXiv:1606.09375, 2016. [46] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907, 2016. [47] R. Levie, F. Monti, X. Bresson, and M. M. Bronstein, “Cayleynets: Graph convolutional neural networks with complex rational spectral filters,” IEEE Transactions on Signal Processing, vol. 67, no. 1, pp. 97–109, 2018. [48] A. F. Agarap, “Deep learning using rectified linear units (relu),” arXiv preprint arXiv:1803.08375, 2018. [49] A. Micheli, “Neural network for graphs: A contextual constructive approach,” IEEE Transactions on Neural Networks, vol. 20, no. 3, pp. 498–511, 2009. [50] J. Atwood and D. Towsley, “Diffusion-convolutional neural networks,” arXiv preprint arXiv:1511.02136, 2015. [51] M. Niepert, M. Ahmed, and K. Kutzkov, “Learning convolutional neural networks for graphs,” in International conference on machine learning. PMLR, 2016, pp. 2014–2023. [52] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” in International Conference on Machine Learning. PMLR, 2017, pp. 1263–1272. [53] Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,” arXiv preprint arXiv:1707.01926, 2017. [54] A. Jain, A. R. Zamir, S. Savarese, and A. Saxena, “Structural-rnn: Deep learning on spatio-temporal graphs,” in Proceedings of the ieee conference on computer vision and pattern recognition, 2016, pp. 5308– 5317. [55] S. Yan, Y. Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based action recognition,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018