|
|
Automatic Recognition of Exploratory and Lookup Intents Based on Berry Picking Model |
Liu Jie1,Gui Sisi2,Zhang Xiaojuan3( ) |
1College of Computer and Information Science, Southwest University, Chongqing 400715, China 2College of Information Management, Nanjing Agricultural University, Nanjing 210095, China 3School of Public Administration, Sichuan University, Chengdu 610065, China |
|
|
Abstract [Objective] This paper selects several new classification features to improve the accuracy of automatic recognition of exploratory and lookup intents. [Methods] Firstly, we collected 1805 queries from the AOL search log and manually labelled them. Then, we proposed classification features from three aspects: query nature, search process, and information source inspired by the Berry Picking model. Third, we evaluated the performance of the proposed features in Naive Bayes, SVM, Decision Tree, Random Forest, and Neural Network. Finally, we explored the classification performance of individual features and feature sets. [Results] The three types of classification features can effectively distinguish exploratory and lookup intentions, with query nature-based features achieving the best performance. Among the five classification models, the neural network algorithm-based model performed the best (Accuracy=0.817 2,Precision=0.849 4,Recall=0.774 7,F1 Score=0.810 3). [Limitations] We did not examine the performances of newly proposed classification features with multiple datasets. User searching behaviors need to be fully explored to form more effective classification features. Moreover, the dataset applied to exploratory/lookup intent recognition was limited due to the high time consumption and labor cost of manual labelling. [Conclusions] The proposed features based on the Berry Picking model can effectively distinguish between exploratory and lookup intents.
|
Received: 29 April 2023
Published: 13 September 2023
|
|
Fund:National Social Science Fund of China(19CTQ023) |
Corresponding Authors:
Zhang Xiaojuan,ORCID:0000-0002-5889-5922, E-mail: zxj0614@scu.edu.cn。
|
[1] |
Broder A. A Taxonomy of Web Search[J]. ACM SIGIR Forum, 2002, 36(2): 3-10.
|
[2] |
White R W, Roth R A. Exploratory Search: Beyond the Query-Response Paradigm[J]. Synthesis Lectures on Information Concepts, Retrieval, and Services, 2009, 1(1): 1-98.
|
[3] |
Marchionini G. Exploratory Search: From Finding to Understanding[J]. Communications of the ACM, 2006, 49(4): 41-46.
|
[4] |
袁红. 探索式搜寻意图对搜寻策略选择的影响机制研究[J]. 图书情报工作, 2022, 66(3): 93-105.
doi: 10.13266/j.issn.0252-3116.2022.03.011
|
[4] |
(Yuan Hong. Research on the Influence Mechanism of Exploratory Seeking Intention on Seeking Strategy Selection[J]. Library and Information Service, 2022, 66(3): 93-105.)
doi: 10.13266/j.issn.0252-3116.2022.03.011
|
[5] |
Crescenzi A, Ward A R, Li Y, et al. Supporting Metacognition during Exploratory Search with the OrgBox[C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2021: 1197-1207.
|
[6] |
Bates M J. The Design of Browsing and Berrypicking Techniques for the Online Search Interface[J]. Online Review, 1989, 13(5): 407-424.
|
[7] |
Savolainen R. Berrypicking and Information Foraging: Comparison of Two Theoretical Frameworks for Studying Exploratory Search[J]. Journal of Information Science, 2018, 44(5): 580-593.
|
[8] |
付煜文, 马志柔, 刘杰, 等. 面向问题意图识别的深度主动学习方法[J]. 中文信息学报, 2021, 35(4): 92-99, 109.
|
[8] |
(Fu Yuwen, Ma Zhirou, Liu Jie, et al. Deep Active Learning Method for Question Intention Recognition[J]. Journal of Chinese Information Processing, 2021, 35(4): 92-99, 109.)
|
[9] |
贺国秀, 张晓娟. 查询意图自动分类的方法改进探讨[J]. 数字图书馆论坛, 2018(1): 53-60.
|
[9] |
(He Guoxiu, Zhang Xiaojuan. Discussion on the Improvement of Methods for Automatic Classification of Query Intent[J]. Digital Library Forum, 2018(1): 53-60.)
|
[10] |
桂思思, 陆伟, 张晓娟. 基于查询表达式特征的时态意图识别研究[J]. 数据分析与知识发现, 2019, 3(3): 66-75.
|
[10] |
(Gui Sisi, Lu Wei, Zhang Xiaojuan. Temporal Intent Classification with Query Expression Feature[J]. Data Analysis and Knowledge Discovery, 2019, 3(3): 66-75.)
|
[11] |
郑新. 电子商务平台中用户查询意图识别与分析方法研究[J]. 西安文理学院学报(自然科学版), 2021, 24(1): 29-33.
|
[11] |
(Zheng Xin. Research on the Method of Identifying and Analyzing User's Query Intention in E-commerce Platform[J]. Journal of Xi'an University (Natural Science Edition), 2021, 24(1): 29-33.)
|
[12] |
赵一鸣, 潘沛, 毛进. 基于任务知识融合与文本数据增强的医学信息查询意图强度识别研究[J]. 数据分析与知识发现, 2023, 7(2): 38-47.
|
[12] |
(Zhao Yiming, Pan Pei, Mao Jin. Recognizing Intensity of Medical Query Intentions Based on Task Knowledge Fusion and Text Data Enhancement[J]. Data Analysis and Knowledge Discovery, 2023, 7(2): 38-47.)
|
[13] |
宋筱璇, 刘畅. 学习型搜索中用户信息源选择和使用策略研究[J]. 情报学报, 2019, 38(6): 655-666.
|
[13] |
(Song Xiaoxuan, Liu Chang. Exploring Users Information Source Selection and Use Strategies in Learning Related Search[J]. Journal of the China Society for Scientific and Technical Information, 2019, 38(6): 655-666.)
|
[14] |
Soufan A, Ruthven I, Azzopardi L. Searching the Literature: An Analysis of an Exploratory Search Task[C]// Proceedings of the 2022 Conference on Human Information Interaction and Retrieval. New York: ACM, 2022: 146-157.
|
[15] |
Rha E Y, Mitsui M, Belkin N J, et al. Exploring the Relationships Between Search Intentions and Query Reformulations[C]// Proceedings of the 79th ASIS&T Annual Meeting on Creating Knowledge, Enhancing Lives through Information & Technology. New York: ACM, 2016: 1-9.
|
[16] |
杨杰, 徐越, 余建桥, 等. 基于搜索引擎日志的用户查询意图分类[J]. 指挥信息系统与技术, 2019, 10(2): 74-79.
|
[16] |
(Yang Jie, Xu Yue, Yu Jianqiao, et al. User Query Intention Classification Based on Search Engine Log[J]. Command Information System and Technology, 2019, 10(2): 74-79.)
|
[17] |
王瑞雪, 方婧, 桂思思, 等. 基于深度学习算法的学术查询意图分类器构建[J]. 图书情报工作, 2021, 65(3): 93-99.
doi: 10.13266/j.issn.0252-3116.2021.03.012
|
[17] |
(Wang Ruixue, Fang Jing, Gui Sisi, et al. Deep Learning-Based Algorithm for Academic Query Intent Classification[J]. Library and Information Service, 2021, 65(3): 93-99.)
doi: 10.13266/j.issn.0252-3116.2021.03.012
|
[18] |
Tsai M F, Wu Y H. User Intent Prediction Search Engine System Based on Query Analysis and Image Recognition Technologies[J]. The Journal of Supercomputing, 2023, 79(5): 5327-5359.
|
[19] |
Kim Y M, Lee T H, Na S O. Constructing Novel Datasets for Intent Detection and NER in a Korean Healthcare Advice System: Guidelines and Empirical Results[J]. Applied Intelligence, 2023, 53(1): 941-961.
|
[20] |
Yu R, Limock, Dietze S. Still Haven't Found What You're Looking for-Detecting the Intent of Web Search Missions from User Interaction Features[OL]. arXiv Preprint, arXiv:2207.01256.
|
[21] |
Zamora J, Mendoza M, Allende H. Query Intent Detection Based on Query Log Mining[J]. Journal of Web Engineering, 2014, 13(1-2): 24-52.
|
[22] |
Hashemi H, Zamani H, Croft W B. Stochastic Optimization of Text Set Generation for Learning Multiple Query Intent Representations[C]// Proceedings of the 31st ACM International Conference on Information and Knowledge Management. New York: ACM, 2022: 4003-4008.
|
[23] |
Natarajan V, Shanmuganathan S. Understanding Query Intention in Search Queries of Learners in Blended Learning Environments[C]// Proceedings of the 4th International Conference on Cybernetics, Cognition and Machine Learning Applications (ICCCMLA). Piscataway: IEEE, 2022: 308-315.
|
[24] |
Wang Y Q, Wang S, Li Y Y, et al. Recognizing Medical Search Query Intent by Few-Shot Learning[C]// Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2022: 502-512.
|
[25] |
Shah C, Hendahewa C, González-Ibáñez R. Rain or Shine? Forecasting Search Process Performance in Exploratory Search Tasks[J]. Journal of the Association for Information Science and Technology, 2016, 67(7): 1607-1623.
|
[26] |
Etcheverry M, Moulin-Frier C, Oudeyer P Y. Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems[C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. New York: ACM, 2020: 4846-4859.
|
[27] |
Ward A R, Capra R. OrgBox: Supporting Cognitive and Metacognitive Activities During Exploratory Search[C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2021: 2570-2574.
|
[28] |
邢玉艳, 刘萍. 探索式搜索前后用户认知结构变化研究[J]. 图书情报工作, 2021, 65(22): 74-84.
doi: 10.13266/j.issn.0252-3116.2021.22.008
|
[28] |
(Xing Yuyan, Liu Ping. Research on Users' Cognitive Structure Changes Before and After Exploratory Search[J]. Library and Information Service, 2021, 65(22): 74-84.)
doi: 10.13266/j.issn.0252-3116.2021.22.008
|
[29] |
Athukorala K, Oulasvirta A, Głowacka D, et al. Narrow or Broad?: Estimating Subjective Specificity in Exploratory Search[C]// Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. New York: ACM, 2014: 819-828.
|
[30] |
Athukorala K, Głowacka D, Jacucci G, et al. Is Exploratory Search Different? A Comparison of Information Search Behavior for Exploratory and Lookup Tasks[J]. Journal of the Association for Information Science and Technology, 2016, 67(11): 2635-2651.
|
[31] |
袁红, 黄燕. 查询式搜索与探索式搜索行为比较研究——以健康信息为例[J]. 现代情报, 2019, 39(5): 48-56.
doi: 10.3969/j.issn.1008-0821.2019.05.007
|
[31] |
(Yuan Hong, Huang Yan. On the Comparation of Lookup and Exploratory Search Behavior—A Case Study of Health Information[J]. Journal of Modern Information, 2019, 39(5): 48-56.)
doi: 10.3969/j.issn.1008-0821.2019.05.007
|
[32] |
Agarwal M K, Sahu T. Lookup or Exploratory: What Is Your Search Intent?[OL]. arXiv Preprint, arXiv:2110.04640.
|
[33] |
Khattak A, Habib A, Asghar M Z, et al. Applying Deep Neural Networks for User Intention Identification[J]. Soft Computing, 2021, 25(3): 2191-2220.
|
[34] |
Hassan A, White R W, Dumais S T, et al. Struggling or Exploring?: Disambiguating Long Search Sessions[C]// Proceedings of the 7th ACM International Conference on Web Search and Data Mining. New York: ACM, 2014: 53-62.
|
[35] |
魏丽, 张云秋. 探索式搜索行为的日志挖掘研究[J]. 图书情报工作, 2014, 58(11): 21-28.
doi: 10.13266/j.issn.0252-3116.2014.11.003
|
[35] |
(Wei Li, Zhang Yunqiu. A Study on Exploratory Search Behaviors Based on Log Mining[J]. Library and Information Service, 2014, 58(11): 21-28.)
doi: 10.13266/j.issn.0252-3116.2014.11.003
|
[36] |
Mikolov T, Sutskever I, Chen K, et al. Distributed Representations of Words and Phrases and Their Compositionality[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. New York: ACM, 2013: 3111-3119.
|
[37] |
Kinley K, Tjondronegoro D, Partridge H, et al. Human-Computer Interaction: The Impact of Users’ Cognitive Styles on Query Reformulation Behaviour During Web Searching[C]// Proceedings of the 24th Australian Computer-Human Interaction Conference. New York: ACM, 2012: 299-307.
|
[38] |
Kinley K, Tjondronegoro D, Partridge H, et al. Relationship between the Nature of the Search Task Types and Query Reformulation Behaviour[C]// Proceedings of the 17th Australasian Document Computing Symposium. New York: ACM, 2012: 39-46.
|
[39] |
White R W, Huang J. Assessing the Scenic Route: Measuring the Value of Search Trails in Web Logs[C]// Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2010: 587-594.
|
[40] |
马超. 面向探索式搜索过程的查询推荐方法研究[D]. 沈阳: 东北大学, 2018.
|
[40] |
(Ma Chao. Research on Query Recommendation Methods Supporting Exploratory Search[D]. Shenyang: Northeastern University, 2018.)
|
[41] |
White R W, Drucker S M. Investigating Behavioral Variability in Web Search[C]// Proceedings of the 16th International Conference on World Wide Web. New York: ACM, 2007: 21-30.
|
[42] |
Kang I H, Kim G C. Query Type Classification for Web Document Retrieval[C]// Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2003: 64-71.
|
[43] |
Ma C, Zhang B. A New Query Recommendation Method Supporting Exploratory Search Based on Search Goal Shift Graphs[J]. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(11): 2024-2036.
|
[44] |
Craswell N, Campos D, Mitra B, et al. ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search[C]// Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York: ACM, 2020: 2983-2989.
|
[45] |
Yin D, Tan J W, Zhang Z, et al. Learning to Generate Personalized Query Auto-Completions via a Multi-View Multi-Task Attentive Approach[C]// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2020: 2998-3007.
|
[46] |
Li P F, Zhang Y, Zhang B. Understanding Query Combination Behavior in Exploratory Searches[J]. Applied Sciences, 2022, 12(2): 706-725.
|
[47] |
Fleiss J L, Cohen J. The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability[J]. Educational & Psychological Measurement, 1973, 33(3):613-619.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|