Source Credibility Assessment in the Realm of Information Disorder: A Literature Review

Alessia Cosentino; Carmen De Maio; Domenico Furno; Mariacristina Gallo; Vincenzo Loia

doi:10.9781/ijimai.2025.01.002

Authors

Alessia Cosentino University of Salerno
Carmen De Maio University of Salerno https://orcid.org/0000-0003-0641-4792
Domenico Furno University of Salerno https://orcid.org/0009-0007-6937-2864
Mariacristina Gallo University of Salerno https://orcid.org/0000-0002-5474-2697
Vincenzo Loia University of Salerno https://orcid.org/0000-0003-4807-8942

DOI:

https://doi.org/10.9781/ijimai.2025.01.002

Keywords:

Credibility Assessment, Information Disorder, Reliability, Source

Supporting Agencies

This work was partially supported by project SERICS (PE00000014) under the MUR National Recovery and Resilience Plan funded by the European Union -NextGenerationEU.

Abstract

The proliferation of information disorder in the digital age has sparked a growing concern regarding the credibility of sources disseminating information. This review examines the evolving landscape of source credibility within information disorder. The review synthesizes key findings and trends related to the factors influencing source credibility, including available tools, shared indicators, and existing methods experimented with in calculating source credibility. The analysis highlights that from a more commercial point of view, several tools are aimed at analyzing the content’s credibility and studying the sources’ credibility. However, from a methodological point of view, there is still something more to do. Indicators that can be used to carry out a source credibility assessment focus on the structure and design of the source, excluding others indicating how the page traffic could be. As for the techniques to be used to assess the credibility of a source, it emerged that more innovative techniques, such as deep-learning, are being developed alongside slightly more classical statistical methods. The review analyzes 23 papers from Conferences and 22 from Journals published in recent years. It also identifies avenues for future inquiry and the development of effective strategies to combat the challenges posed by misinformation in the digital era.

Downloads

Download data is not yet available.

Author Biographies

Alessia Cosentino, University of Salerno

She received a bachelor’s degree in diplomatic, international, and global security studies from the University of Salerno, Italy, in 2022. She is currently a data science and innovation management student at the same university.

Carmen De Maio, University of Salerno

She graduated and received a Ph.D. degree in Computer Sciences, both from the University of Salerno, Italy, in 2008 and 2011, respectively. The research activity has focused mainly on the definition and experimentation of Knowledge Extraction methodologies adopting Conceptual Data Analysis techniques and processes relying on Fuzzy Logic and Computational Intelligence theories. She has over 50 publications in Fuzzy Decision Making, Knowledge Extraction and Management, Situation and Context Awareness, Semantic Information Retrieval, and Ontology Learning. More recently, she has been working on the definition of Time Aware Knowledge Extraction, Process Mining, and Social Media Analytics methodologies. She is currently an Associate Professor in Computer Science at the University of Salerno.

Domenico Furno, University of Salerno

He received master’s degree cum laude and PhD with evaluation excellent from University of Salerno, respectively, in 2007 and 2013. He has publications in Situation/Context Awareness, Soft Computing, Intelligent agents, Data Mining, Semantic Web and Knowledge Representation. He started his research career by defining and testing hybrid approaches based on Computational Intelligence and Semantic Web methodologies and techniques for distributed Situation and Context Awareness scenarios. He is currently a researcher at University of Salerno, and his research interests include Information Disorder Awareness.

Mariacristina Gallo, University of Salerno

She earned a master’s degree in computer science at the University of Salerno, Italy, in 2009. In 2021, she obtained a Ph.D. degree in Big Data Management at the same University. Research interests mainly focus on Computational Intelligence methods to support semanticenabled solutions and decision making. Research activities regard Knowledge Extraction and Management, Context Awareness, Semantic Information Retrieval, Ontology Learning. She is currently a research fellow at the University of Salerno.

Vincenzo Loia, University of Salerno

Graduated in Computer Science at the University of Salerno, Italy, in 1985 and received his Ph.D. in Computer Science in 1989 at the Universite’ Pierre and Marie Curie Paris VI, France. He is currently a Computer Science Full Professor at the University of Salerno, where he served as a researcher from 1989 to 2000 and as an associate professor from 2000 to 2004. Dr. Loia is the Co-Editorin- Chief of Soft Computing and the Editor-in-Chief of Ambient Intelligence and Humanized Computing. He serves as an Editor for 14 other international journals.

References

[1] D. Zhang, D. Wang, N. Vance, Y. Zhang, S. Mike, “On scalable and robust truth discovery in big data social media sensing applications,” IEEE Transactions on Big Data, vol. 5, no. 2, pp. 195–208, 2018.

[2] H. Elmimouni, A. Forte, J. Morgan, “Why people trust wikipedia articles: Credibility assessment strategies used by readers,” in Proceedings of the 18th International Symposium on Open Collaboration, 2022, pp. 1–10.

[3] S. Lee, J. Kim, D. Kim, K. J. Kim, E. Park, “Computational approaches to developing the implicit media bias dataset: Assessing political orientations of nonpolitical news articles,” Applied Mathematics and Computation, vol. 458, p. 128219, 2023.

[4] A. A. Shah, S. D. Ravana, S. Hamid, M. A. Ismail, “Web pages credibility scores for improving accuracy of answers in web-based question answering systems,” IEEE Access, vol. 8, pp. 141456–141471, 2020.

[5] X. Zhou, R. Zafarani, “A survey of fake news: Fundamental theories, detection methods, and opportunities,” ACM Computing Surveys (CSUR), vol. 53, no. 5, pp. 1–40, 2020.

[6] B. Umeogu, “Source credibility: A philosophical analysis,” Open Journal of Philosophy, vol. 2, no. 2, pp. 112–115, 2012.

[7] P. Meel, D. K. Vishwakarma, “Fake news, rumor, information pollution in social media and web: A contemporary survey of state-of-the-arts, challenges and opportunities,” Expert Systems with Applications, vol. 153, p. 112986, 2020.

[8] E. Saquete, D. Tomás, P. Moreda, P. Martínez- Barco, M. Palomar, “Fighting post-truth using natural language processing: A review and open challenges,” Expert Systems with Applications, vol. 141, p. 112943, 2020.

[9] S. M. Shariff, “A review on credibility perception of online information,” in 2020 14th International Conference on Ubiquitous Information Management and Communication (IMCOM), 2020, pp. 1–7, IEEE.

[10] M. Faraon, A. Jaff, L. P. Nepomuceno, V. Villavicencio, “Fake news and aggregated credibility: Conceptualizing a co-creative medium for evaluation of sources online,” International Journal of Ambient Computing and Intelligence (IJACI), vol. 11, no. 4, pp. 93–117, 2020.

[11] M. Mensio, H. Alani, “Misinfome: Who’s interacting with misinformation?,” in Proceedings of the 18th International Semantic Web Conference (ISWC 2019): Posters & Demonstrations, Industry and Outrageous Ideas Tracks, CEUR WS, October 2019, CEUR Workshop Proceedings.

[12] M. Fernandez, H. Alani, “Online misinformation: Challenges and future directions,” in Companion Proceedings of the Web Conference 2018, 2018, pp. 595–602.

[13] J. Park, R. D. Ellezhuthil, J. Isaac, C. Mergerson, L. Feldman, V. Singh, “Misinformation detection algorithms and fairness across political ideologies: The impact of article level labeling,” in Proceedings of the 15th ACM Web Science Conference 2023, 2023, pp. 107–116.

[14] Y. Jo, M. Kim, K. Han, “How do humans assess the credibility on web blogs: Qualifying and verifying human factors with machine learning,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 2019, pp. 1–12.

[15] C. Soohoo, D. R. Danielson, L. Marable, J. Stanford, E. R. Tauber, “How do users evaluate the credibility of web sites? a study with over 2,500 participants,” in Proceedings of the 2003 Conference on Designing for User Experiences, 2003, pp. 1–15.

[16] S. Wineburg, S. McGrew, “Lateral reading: Reading less and learning more when evaluating digital information,” 2017.

[17] S. S. Birunda, R. K. Devi, “A novel score-based multi- source fake news detection using gradient boosting algorithm,” in 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), 2021, pp. 406– 414, IEEE.

[18] A. Wang, L. M. De Jesus Sanchez, A. Wintner, Y. Zhu, E. Mustafaraj, “Assessing google search’s new features in supporting credibility judgments of unknown websites,” in Proceedings of the 2023 Conference on Human Information Interaction and Retrieval, 2023, pp. 303–307.

[19] H. Keshavarz, M. E. Givi, “Website evaluation frameworks: Is oriented vs. business oriented models,” in 2020 6th International Conference on Web Research (ICWR), 2020, pp. 223–228, IEEE.

[20] Z. E. Serman, J. Sims, “Source credibility theory: Sme hospitality sector blog posting during the covid 19 pandemic,” Information Systems Frontiers, vol. 25, no. 6, pp. 2317–2334, 2023.

[21] S. Padhye, M. Hastak, “A framework to evaluate information and source credibility: International construction decision-making,” Journal of Management in Engineering, vol. 40, no. 1, p. 04023046, 2024.

[22] A. V. Gorodishchev, A. N. Gorodishcheva, D. O. Baigozhina, G. P. Kovalev, “Web 3.0 credibility: Principles for ranking media sources,” in 2023 Communication Strategies in Digital Society Seminar (ComSDS), 2023, pp. 184–188, IEEE.

[23] H. Liu, L. Wang, X. Han, W. Zhang, X. He, “Detecting fake news on social media: A multi-source scoring framework,” in 2020 IEEE 5th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), 2020, pp. 524–531, IEEE.

[24] M. Fayaz, A. Khan, M. Bilal, S. U. Khan, “Machine learning for fake news classification with optimal feature selection,” Soft Computing, vol. 26, no. 16, pp. 7763–7771, 2022.

[25] I. Ahmad, M. Yousaf, S. Yousaf, M. O. Ahmad, “Fake news detection using machine learning ensemble methods,” Complexity, vol. 2020, pp. 1–11, 2020.

[26] M. Kakol, R. Nielek, A. Wierzbicki, “Understanding and predicting web content credibility using the content credibility corpus,” Information Processing & Management, vol. 53, no. 5, pp. 1043–1061, 2017.

[27] N. Sitaula, C. K. Mohan, J. Grygiel, X. Zhou, R. Zafarani, “Credibilitybased fake news detection,” Disinformation, Misinformation, and Fake News in Social Media: Emerging Research Challenges and Opportunities, pp. 163–182, 2020.

[28] Y. Zhao, L. Zhang, C. Zeng, Y. Chen, W. Lu, N. Song, “Factors influencing online health information credibility: a meta-analysis,” Aslib Journal of Information Management, 2024.

[29] M. M. Bhuiyan, M. Horning, S. W. Lee, T. Mitra, “Nudgecred: Supporting news credibility assessment on social media through nudges,” Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW2, pp. 1–30, 2021.

[30] M. Bink, S. Zimmerman, D. Elsweiler, “Featured snippets and their influence on users’ credibility judgements,” in Proceedings of the 2022 Conference on Human Information Interaction and Retrieval, 2022, pp. 113–122.

[31] M. Viviani, G. Pasi, “Credibility in social media: opinions, news, and health information—a survey,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 7, no. 5, p. e1209, 2017.

[32] D. Pehlivanoglu, T. Lin, F. Deceus, A. Heemskerk, N. C. Ebner, B. S. Cahill, “The role of analytical reasoning and source credibility on the evaluation of real and fake full-length news articles,” Cognitive Research: Principles and Implications, vol. 6, no. 1, pp. 1–12, 2021.

[33] V. Wagle, K. Kaur, P. Kamat, S. Patil, K. Kotecha, “Explainable ai for multimodal credibility analysis: Case study of online beauty health (mis)-information,” IEEE Access, vol. 9, pp. 127985–128022, 2021.

[34] A. Aker, K. Vincentius, K. Bontcheva, “Credibility and transparency of news sources: Data collection and feature analysis.,” in NewsIR@ SIGIR, 2019, pp. 15–20.

[35] K.-Y. Shin, W. Song, J. Kim, J.-H. Lee, “News credibility scroing: Suggestion of research methodology to determine the reliability of news distributed in sns,” in Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2019, pp. 737–740.

[36] Y. Lin, S. C. Han, B. H. Kang, “Machine learning for the peer assessment credibility,” in Companion Proceedings of the The Web Conference 2018, 2018, pp. 117–118.

[37] G. Hu, H. Liu, J. Wang, J. Ye, W. Gong, S. Song, “Research on information source detection based on machine learning algorithm,” in 2022 7th International Conference on Intelligent Computing and Signal Processing (ICSP), 2022, pp. 1507–1510, IEEE.

[38] T. H. Chiang, C.-S. Liao, W.-C. Wang, “Investigating the difference of fake news source credibility recognition between ann and bert algorithms in artificial intelligence,” Applied Sciences, vol. 12, no. 15, p. 7725, 2022.

[39] M.-A. Kaufhold, M. Bayer, D. Hartung, C. Reuter, “Design and evaluation of deep learning models for real-time credibility assessment in twitter,” in Artificial Neural Networks and Machine Learning–ICANN 2021: 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part V 30, 2021, pp. 396–408, Springer.

[40] J. A. Nasir, O. S. Khan, I. Varlamis, “Fake news detection: A hybrid cnnrnn based deep learning approach,” International Journal of Information Management Data Insights, vol. 1, no. 1, p. 100007, 2021.

[41] A. Amini, Y. E. Bayiz, A. Ram, R. Marculescu, U. Topcu, “News source credibility assessment: A reddit case study,” arXiv preprint arXiv:2402.10938, 2024.

[42] W. Lifferth, “Fake News. Kaggle,” 2018. [Online]. Available: https:// kaggle.com/competitions/fake-news.

[43] K. Shu, D. Mahudeswaran, S. Wang, D. Lee, H. Liu, “Fakenewsnet: A data repository with news content, social context and dynamic information for studying fake news on social media,” arXiv preprint arXiv:1809.01286, 2018.

[44] M. Szpakowski, “FakeNewsCorpus,” 2020. [Online]. Available: http://github.com/several27/FakeNewsCorpus.

[45] M. Zainab, “News credibility dataset,” 2021. [Online]. Available: https://dx.doi.org/10.21227/xjkz-c814.

Source Credibility Assessment in the Realm of Information Disorder: A Literature Review

Authors

DOI:

Keywords: