[Objective] This paper systematically summarizes the research on interpretable machine learning methods and their applications for information resource management. It identifies possible areas of improvements, and provides insights for future research. [Coverage] We searched interpretable machine learning papers from CNKI and Web of Science. A total of 44 related articles were retrieved for review. [Methods] First, from the machine learning process, we constructed a general interpretable machine learning framework. Then, we thoroughly reviewed the classification of interpretable machine learning methods. Finally, we discussed the interpretable machine learning applications for information resource management. [Results] The general interpretable machine learning framework consists of three different modules: pre-explanation, explainable models, and post-explanation. Post-explanation methods have been widely applied in health informatics, online public opinion, scientometrics, and social network user behavior, with the help of commonly used methods including SHAP and feature importance analysis. Many existing research are lack of diversity and integration in applied methods, insufficient exploration of causal relationships, inadequate explanations for multi-source heterogeneous data, and the need for broadening domain applications. [Limitations] This review focuses on the applications and shortcomings of interpretable machine learning. It does not delve into the algorithm principles. [Conclusions] In future research, efforts should be made to strengthen the integration of interpretable machine learning methods, explore interpretable machine learning based on causal machine learning, introduce interpretable machine learning methods for multi-source heterogeneous data. We should also broaden applications in various domains such as information recommendation, information retrieval, and informetrics.
刘智锋, 王继民. 可解释机器学习在信息资源管理领域的应用研究综述*[J]. 数据分析与知识发现, 2024, 8(1): 16-29.
Liu Zhifeng, Wang Jimin. Review of Interpretable Machine Learning for Information Resource Management. Data Analysis and Knowledge Discovery, 2024, 8(1): 16-29.
Gunning D, Stefik M, Choi J, et al. XAI—Explainable Artificial Intelligence[J]. Science Robotics, 2019, 4(37). DOI: 10.1126/scirobotics.aay7120.
[2]
Cambria E, Malandri L, Mercorio F, et al. A Survey on XAI and Natural Language Explanations[J]. Information Processing & Management, 2023, 60(1):Article No.103111.
(Developing Responsible Artificial Intelligence: New Generation of AI Governance Principles Released[EB/OL].[2023-08-18]. http://www.gov.cn/xinwen/2019-06/17/content_5401006.htm.)
(Luo Pengcheng, Wang Yibo, Wang Jimin. Automatic Discipline Classification for Scientific Papers Based on a Deep Pre-Training Language Model[J]. Journal of the China Society for Scientific and Technical Information, 2020, 39(10): 1046-1059.)
(Liu Jiangfeng, Lin Litao, Liu Chang, et al. Study on the Discipline Classification of Massive Humanities and Social Science Academic Literature Driven by Deep Learning[J]. Information Studies: Theory & Application, 2023, 46(2): 71-81.)
(Jin Qingwen, Chao Lemen, Meng Gang. Research on Algorithm Interpretation and Implementation Method in AI Governance[J]. Information and Documentation Services, 2022, 43(5): 16-23.)
(Zhang Chenghong, Chen Gang, Lu Tian, et al. Explainable Artificial Intelligence and Its Impact on Management: Research Status and Prospects[J]. Journal of Management Science, 2021, 34(3): 63-79.)
(Li Yao, Zuo Xingquan, Wang Chunlu, et al. Research Progress of Artificial Intelligence Interpretability Evaluation[J]. Navigation Positioning and Timing, 2022, 9(6): 13-24.)
(Liu Tong, Gu Xiaoqing. Opening the “Black Box”: Exploring the Interpretability of Artificial Intelligence in Education[J]. China Educational Technology, 2022(5): 82-90.)
[14]
van Lent M, Fisher W, Mancuso M. An Explainable Artificial Intelligence System for Small-Unit Tactical Behavior[C]// Proceedings of the 16th Conference on Innovative Applications of Artifical Intelligence. 2004: 900-907.
[15]
Adadi A, Berrada M. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)[J]. IEEE Access, 2018, 6: 52138-52160.
doi: 10.1109/ACCESS.2018.2870052
(Fan Hao, Li Shanshan, Reziya Aihaiti. Adoption and Influence of Machine Learning Algorithms in Information Science Research in China: From the Perspective of CSSCI Journal Papers[J]. Documentation, Information & Knowledge, 2022, 39(5): 96-108.)
(Ji Shouling, Li Jinfeng, Du Tianyu, et al. Survey on Techniques, Applications and Security of Machine Learning Interpretability[J]. Journal of Computer Research and Development, 2019, 56(10): 2071-2096.)
[18]
Loh H W, Ooi C P, Seoni S, et al. Application of Explainable Artificial Intelligence for Healthcare: A Systematic Review of the Last Decade (2011-2022)[J]. Computer Methods and Programs in Biomedicine, 2022, 226: Article No.107161.
[19]
林志萍. 可解释的机器学习及应用研究[D]. 广州: 华南理工大学, 2021.
[19]
(Lin Zhiping. Research on Interpretable Machine Learning and Its Application[D]. Guangzhou: South China University of Technology, 2021.)
[20]
Goldstein A, Kapelner A, Bleich J, et al. Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation[J]. Journal of Computational and Graphical Statistics, 2015, 24(1): 44-65.
doi: 10.1080/10618600.2014.907095
[21]
Kim J, Lee H J, Lee H. Mining the Determinants of Review Helpfulness: A Novel Approach Using Intelligent Feature Engineering and Explainable AI[J]. Data Technologies and Applications, 2022, 57(1): 108-130.
doi: 10.1108/DTA-12-2021-0359
[22]
Fisher A, Rudin C, Dominici F. Model Class Reliance: Variable Importance Measures for Any Machine Learning Model Class, from the “Rashomon” Perspective[OL]. arXiv Preprint, arXiv: 1801.01489.
(Yu Ying. Application of Machine Learning Method and Its Interpretable Technology in Automobile Insurance Pricing— Based on XGBoost Model[D]. Dalian: Dongbei University of Finance and Economics, 2021.)
[24]
Ribeiro M T, Singh S, Guestrin C. “Why Should I Trust You?”: Explaining the Predictions of Any Classifier[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016: 1135-1144.
[25]
Shapley L S, Shubik M. A Method for Evaluating the Distribution of Power in a Committee System[J]. American Political Science Review, 1954, 48(3): 787-792.
doi: 10.2307/1951053
[26]
Lundberg S M, Lee S I. A Unified Approach to Interpreting Model Predictions[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. ACM, 2017: 4768-4777.
(Yang Pengbo, Sang Jitao, Zhang Biao, et al. Survey on Interpretability of Deep Models for Image Classification[J]. Journal of Software, 2023, 34(1): 230-254.)
[28]
Abbasi-Asl R, Yu B. Interpreting Convolutional Neural Networks Through Compression[OL]. arXiv Preprint, arXiv: 1711.02329.
[29]
Szegedy C, Zaremba W, Sutskever I, et al. Intriguing Properties of Neural Networks[OL]. arXiv Preprint, arXiv: 1312.6199.
[30]
Koh P W, Liang P. Understanding Black-Box Predictions via Influence Functions[C]// Proceedings of the 34th International Conference on Machine Learning. ACM, 2017: 1885-1894.
[31]
Mahendran A, Vedaldi A. Understanding Deep Image Representations by Inverting Them[C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015: 5188-5196.
[32]
Kim B, Wattenberg M, Gilmer J, et al. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors(TCAV)[OL]. arXiv Preprint, arXiv: 1711.11279.
[33]
Simonyan K, Vedaldi A, Zisserman A. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps[OL]. arXiv Preprint, arXiv: 1312.6034.
[34]
Smilkov D, Thorat N, Kim B, et al. SmoothGrad: Removing Noise by Adding Noise[OL]. arXiv Preprint, arXiv: 1706.03825.
[35]
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. ACM, 2017: 6000-6010.
(Luo Wangcheng, Yang Xiang, Chen Yanhong. Interpretable Arrhythmia Classification Model Based on Attention Mechanism[J]. Computer Technology and Development, 2022, 32(9): 114-120.)
Hao Yaru, Dong Li, Xu Ke, et al. Interpretability of Pre-Trained Language Models: A Survey[J]. Journal of Guangxi Normal University (Natural Science Edition), 2022, 40(5): 59-71.)
[38]
Clark K, Khandelwal U, Levy O, et al. What does BERT Look at? An Analysis of BERT's Attention[OL]. arXiv Preprint, arXiv: 1906.04341.
(Han Lei, Xu Zhongyang, Qiu Junping. Review on the Application of Datametric Methods in Health Informatics[J]. Journal of Modern Information, 2022, 42(8): 157-167.)
doi: 10.3969/j.issn.1008-0821.2022.08.016
(Xu Liangchen, Guo Chonghui. Predicting Survival Rates for Gastric Cancer Based on Ensemble Learning[J]. Data Analysis and Knowledge Discovery, 2021, 5(8): 86-99.)
[41]
Rodriguez V A, Bhave S, Chen R J, et al. Development and Validation of Prediction Models for Mechanical Ventilation, Renal Replacement Therapy, and Readmission in COVID-19 Patients[J]. Journal of the American Medical Informatics Association, 2021, 28(7): 1480-1488.
doi: 10.1093/jamia/ocab029
pmid: 33706377
(Che Hongxin, Wang Tong, Wang Wei. Comparing Prediction Models for Prostate Cancer[J]. Data Analysis and Knowledge Discovery, 2021, 5(9): 107-114.)
[43]
Chen H C, Damarjati C, Putra K T, et al. Pulse-Line Intersection Method with Unboxed Artificial Intelligence for Hesitant Pulse Wave Classification[J]. Information Processing & Management, 2022, 59(2): Article No.102855.
[44]
Yu S C, Hofford M R, Lai A M, et al. Respiratory Support Status from EHR Data for Adult Population: Classification, Heuristics, and Usage in Predictive Modeling[J]. Journal of the American Medical Informatics Association, 2022, 29(5): 813-821.
doi: 10.1093/jamia/ocac005
[45]
Amrollahi F, Shashikumar S P, Meier A, et al. Inclusion of Social Determinants of Health Improves Sepsis Readmission Prediction Models[J]. Journal of the American Medical Informatics Association, 2022, 29(7): 1263-1270.
doi: 10.1093/jamia/ocac060
pmid: 35511233
[46]
Liu S, Schlesinger J J, McCoy A B, et al. New Onset Delirium Prediction Using Machine Learning and Long Short-Term Memory(LSTM) in Electronic Health Record[J]. Journal of the American Medical Informatics Association, 2022, 30(1): 120-131.
doi: 10.1093/jamia/ocac210
[47]
Kwak H, Chang J, Choe B, et al. Interpretable Disease Prediction Using Heterogeneous Patient Records with Self-Attentive Fusion Encoder[J]. Journal of the American Medical Informatics Association, 2021, 28(10): 2155-2164.
doi: 10.1093/jamia/ocab109
pmid: 34198329
[48]
Zhang X, Yan C, Malin B A, et al. Predicting Next-Day Discharge via Electronic Health Record Access Logs[J]. Journal of the American Medical Informatics Association, 2021, 28(12): 2670-2680.
doi: 10.1093/jamia/ocab211
pmid: 34592753
[49]
Jiao Y, Sharma A, Ben Abdallah A, et al. Probabilistic Forecasting of Surgical Case Duration Using Machine Learning: Model Development and Validation[J]. Journal of the American Medical Informatics Association, 2020, 27(12): 1885-1893.
doi: 10.1093/jamia/ocaa140
pmid: 33031543
[50]
Catling F J R, Wolff A H. Temporal Convolutional Networks Allow Early Prediction of Events in Critical Care[J]. Journal of the American Medical Informatics Association, 2020, 27(3): 355-365.
doi: 10.1093/jamia/ocz205
pmid: 31858114
(Shi Wei, Xue Guangcong, He Shaoyi. Literature Review of Network Public Opinion Research from the Perspective of Sentiment[J]. Documentation, Information & Knowledge, 2022, 39(1): 105-118.)
(An Lu, Hui Qiuyue. Prediction of Microblogging Public Opinion Reversal in the Context of Hot Events[J]. Journal of Information Resources Management, 2022, 12(3): 21-34.)
(Zeng Ziming, Zhang Yu, Li Tingting. Detection of Potential Rumor Spreaders in Public Health Emergencies Based on Multi-Feature Fusion[J]. Library and Information Service, 2022, 66(13): 80-90.)
doi: 10.13266/j.issn.0252-3116.2022.13.008
[54]
Sun R, An L, Li G, et al. Predicting Social Media Rumours in the Context of Public Health Emergencies[J]. Journal of Information Science, DOI:10.1177/01655515221137879.
[55]
Li Z M, Du X Y, Zhao Y, et al. Lifecycle Research of Social Media Rumor Refutation Effectiveness Based on Machine Learning and Visualization Technology[J]. Information Processing & Management, 2022, 59(6): Article No.103077.
[56]
Li Z M, Zhang Q, Du X Y, et al. Social Media Rumor Refutation Effectiveness: Evaluation, Modelling and Enhancement[J]. Information Processing & Management, 2021, 58(1): Article No.102420.
(Wei Zhiguang, Song Xiaokang, Zhu Qinghua, et al. Research on Health Rumor Sharing Intention Based on Random Forest Classification Algorithm[J]. Journal of Modern Information, 2020, 40(5): 78-87.)
doi: 10.3969/j.issn.1008-0821.2020.05.010
[58]
Tao J E, Zhou L N, Hickey K. Making Sense of the Black-Boxes: Toward Interpretable Text Classification Using Deep Learning Models[J]. Journal of the Association for Information Science and Technology, 2023, 74(6): 685-700.
doi: 10.1002/asi.v74.6
[59]
Ayoub J, Yang X J, Zhou F. Combat COVID-19 Infodemic Using Explainable Natural Language Processing Models[J]. Information Processing & Management, 2021, 58(4): Article No.102569.
[60]
Uyheng J, Moffitt J D, Carley K M. The Language and Targets of Online Trolling: A Psycholinguistic Approach for Social Cybersecurity[J]. Information Processing & Management, 2022, 59(5): Article No.103012.
(An Lu, Chen Miaomiao. Evaluating the Effectiveness of Government Microblog Information Release in Emergency Situations[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(7): 692-706.)
(Yi Ming, Yao Yujia, Hu Min. An Interpretable Model for New Government Media Public Value Consensus Integrating XGBoost and SHAP: Taking the Top 10 Municipal Government Accounts of the Jinri Toutiao as an Example[J]. Library and Information Service, 2022, 66(16): 36-47.)
doi: 10.13266/j.issn.0252-3116.2022.16.004
(Yang Donghong, Wu Bang’an, Sun Xiaochun. Research on the Helpfulness Prediction Model of Online Review Information Based on Machine Learning[J]. Information Science, 2019, 37(12): 34-39.)
[64]
Wang T, Han D M, Dai Y H. Popularity Prediction for Consumers’ Product Recommendation Articles[J]. Journal of Global Information Management, 2022, 30(8): 1-16.
(Yin Lichun, Wang Yue. Influencing Factors and Mechanism of Book Consumer Satisfaction Based on Online Comments[J]. Library and Information Service, 2019, 63(22): 106-117.)
doi: 10.13266/j.issn.0252-3116.2019.22.012
(Zhang Yunzhong, Qin Yiyuan. Research on Influencing Factors of Tag Quality in Social Tagging System: Based on Random Forest[J]. Library and Information Service, 2019, 63(24): 119-126.)
doi: 10.13266/j.issn.0252-3116.2019.24.013
(Men Xiuping, Wei Ruibin, Wu Xiaolan. Analysis and Detection of Language and Behavior Characteristics of Depression in Social Network[J]. Journal of Modern Information, 2020, 40(6): 76-87.)
doi: 10.3969/j.issn.1008-0821.2020.06.008
(Shen Hongzhou, Yuan Qinjian. A Classification Study on the Strength of Social Relationships Based on Social Networks[J]. Journal of the China Society for Scientific and Technical Information, 2014, 33(8): 846-859.)
[69]
Zhang Y, Zhang C Z, Mayr P, et al. An Editorial of “AI+Informetrics”: Multi-Disciplinary Interactions in the Era of Big Data[J]. Scientometrics, 2022, 127(11): 6503-6507.
doi: 10.1007/s11192-022-04561-w
[70]
Beranová L, Joachimiak M P, Kliegr T, et al. Why was This Cited? Explainable Machine Learning Applied to COVID-19 Research Literature[J]. Scientometrics, 2022, 127(5): 2313-2349.
doi: 10.1007/s11192-022-04314-9
(Ding Heng, Ruan Jinglong. Exploring the Factors Influencing LIS Scholars Citing Other’s Works: An Empirical Research Based on Algorithmic Attribution[J]. Documentation, Information & Knowledge, 2022, 39(2): 83-97.)
(Sun Ran, An Lu, Li Gang. Patent Value Prediction Based on Multi-Feature Fusion—Taking the 5G Technology as a Case[J]. Journal of Modern Information, 2022, 42(11): 87-96.)
doi: 10.3969/j.issn.1008-0821.2022.11.008
(Fu Zhenkang, Liu Bingxiang, Zhou Ziyu, et al. Research on Patent Quality Analysis and Classification Prediction Based on Ensemble Learning[J]. Journal of Intelligence, 2022, 41(10): 89-96.)
[74]
Zeng T, Acuna D E. Modeling Citation Worthiness by Using Attention-Based Bidirectional Long Short-Term Memory Networks and Interpretable Models[J]. Scientometrics, 2020, 124(1): 399-428.
doi: 10.1007/s11192-020-03421-9
[75]
Ma Y X, Li T T, Mao J, et al. Identifying Widely Disseminated Scientific Papers on Social Media[J]. Information Processing & Management, 2022, 59(3): Article No.102945.
[76]
Zhang Z J, Wu C, Qu S Y, et al. An Explainable Artificial Intelligence Approach for Financial Distress Prediction[J]. Information Processing & Management, 2022, 59(4): Article No.102988.
[77]
Wang T, He C, Jin F J, et al. Evaluating the Effectiveness of Marketing Campaigns for Malls Using a Novel Interpretable Machine Learning Model[J]. Information Systems Research, 2022, 33(2): 659-677.
doi: 10.1287/isre.2021.1078
[78]
Wang Z, Jiang C, Zhao H. Know Where to Invest: Platform Risk Evaluation in Online Lending[J]. Information Systems Research, 2022, 33(3): 765-783.
doi: 10.1287/isre.2021.1083
[79]
Liu R, Mai F, Shan Z, et al. Predicting Shareholder Litigation on Insider Trading from Financial Text: An Interpretable Deep Learning Approach[J]. Information & Management, 2020, 57(8): Article No.103387.
(Chen Chen, Li Yongnan, Wang Mingjian. Research on the Construction of Terrorist Attack Early Warning Model Based on LightGBM Algorithm[J]. Journal of Intelligence, 2022, 41(6): 21-28.)
(Luo Weiping, Zhou Bo. Research on Terrorist Organization Forecast Method Based on Terrorist Attack Characteristics Analysis[J]. Journal of Intelligence, 2020, 39(11): 18-24.)
[82]
Kalampokis E, Karamanou A, Tarabanis K. Applying Explainable Artificial Intelligence Techniques on Linked Open Government Data[C]// Proceedings of the 20th IFIP WG 8.5 International Conference on Electronic Government. Springer, 2021: 247-258.
(Cao Rui, Liao Bin, Li Min, et al. Predicting Prices and Analyzing Features of Online Short-Term Rentals Based on XGBoost[J]. Data Analysis and Knowledge Discovery, 2021, 5(6): 51-65.)
(Zhang Yunzhong, Zhang Rong, Guo Dong. Empirical Research on the Influencing Factors of Public Opinion Guidance of Media—Based Think Tanks[J]. Information Science, 2022, 40(7): 102-110.)
[85]
Rajabi E, Etminani K. Knowledge-Graph-Based Explainable AI: A Systematic Review[J]. Journal of Information Science, DOI:10.1177/01655515221112844.
[86]
郭若城, 程璐, 刘昊, 等. 因果推断与机器学习[M]. 北京: 电子工业出版社, 2023.
[86]
(Guo Ruocheng, Cheng Lu, Liu Hao, et al. Causal Inference and Machine Learning[M]. Beijing: Publishing House of Electronics Industry, 2023.)