Automatic Recognition of Exploratory and Lookup Intents Based on Berry Picking Model

doi:10.11925/infotech.2096-3467.2023.0389

Data Analysis and Knowledge Discovery

2024, Vol. 8

Issue (4): 152-166 DOI: 10.11925/infotech.2096-3467.2023.0389

Current Issue | Archive | Adv Search

Automatic Recognition of Exploratory and Lookup Intents Based on Berry Picking Model

Liu Jie¹,Gui Sisi²,Zhang Xiaojuan³(

)

¹College of Computer and Information Science, Southwest University, Chongqing 400715, China
²College of Information Management, Nanjing Agricultural University, Nanjing 210095, China
³School of Public Administration, Sichuan University, Chengdu 610065, China

Download: PDF (1473 KB) HTML ( 4 )
Export: BibTeX | EndNote (RIS)

Abstract

[Objective] This paper selects several new classification features to improve the accuracy of automatic recognition of exploratory and lookup intents. [Methods] Firstly, we collected 1805 queries from the AOL search log and manually labelled them. Then, we proposed classification features from three aspects: query nature, search process, and information source inspired by the Berry Picking model. Third, we evaluated the performance of the proposed features in Naive Bayes, SVM, Decision Tree, Random Forest, and Neural Network. Finally, we explored the classification performance of individual features and feature sets. [Results] The three types of classification features can effectively distinguish exploratory and lookup intentions, with query nature-based features achieving the best performance. Among the five classification models, the neural network algorithm-based model performed the best (Accuracy=0.817 2,Precision=0.849 4,Recall=0.774 7,F1 Score=0.810 3). [Limitations] We did not examine the performances of newly proposed classification features with multiple datasets. User searching behaviors need to be fully explored to form more effective classification features. Moreover, the dataset applied to exploratory/lookup intent recognition was limited due to the high time consumption and labor cost of manual labelling. [Conclusions] The proposed features based on the Berry Picking model can effectively distinguish between exploratory and lookup intents.

Key words： Query Intent Recognition Exploratory Intent Lookup Intent BerryPicking Model

Received: 29 April 2023 Published: 13 September 2023

ZTFLH:	TP393
	G354

Fund:National Social Science Fund of China(19CTQ023)

Corresponding Authors: Zhang Xiaojuan，ORCID：0000-0002-5889-5922， E-mail： zxj0614@scu.edu.cn。

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Jie Liu
	Sisi Gui
	Xiaojuan Zhang

Cite this article:

Liu Jie, Gui Sisi, Zhang Xiaojuan. Automatic Recognition of Exploratory and Lookup Intents Based on Berry Picking Model. Data Analysis and Knowledge Discovery, 2024, 8(4): 152-166.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2023.0389 OR https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2024/V8/I4/152

Category System of Search Activities Defined by Marchionini

BerryPicking Model

查询重构类型	特征描述	查询示例
新建	$Q i$ 和 $Q i + 1$ 不包含任何共同术语	$Q i$ ：“back to the future” $Q i + 1$ ：“holiday mansion houseboat”
添加	$Q i$ 是 $Q i + 1$ 的子集，即 $Q i + 1$ 中术语大于等于 $Q i$ 中术语	$Q i$ ：“select business servic。es” $Q i + 1$ ：“select business services title”
替换	$Q i$ 和 $Q i + 1$ 包含至少一个共同术语和至少一个不同术语	$Q i$ ：“national real estate settlement services” $Q i + 1$ ：“Pennsylvania real estate settlement services”
删除	$Q i$ 是 $Q i + 1$ 的超集，即 $Q i$ 中的术语个数大于等于 $Q i + 1$ 中的术语个数	$Q i$ ：“auto locator Pennsylvania” $Q i + 1$ ：“auto locator”
重复	$Q i$ 和 $Q i + 1$ 包含完全相同的术语，这些术语的顺序可能不同	$Q i$ ：“coats tire equipment” $Q i + 1$ ：“coats tire equipment”

Definition and Examples of Query Reformulating Types

采莓模型特征		本文分类特征
查询性质类	查询术语多样性（DQT）	查询 $q$ 所在的所有session中相邻两查询之间非共现术语所占比值的平均值
	查询语义多样性（DQS）	查询 $q$ 所在的所有session中相邻两查询向量之间语义多样性的平均值
	查询重构相关特征	（1）所有查询重构类型数的平均值（AART）：查询 $q$ 所在session中，查询重构类型总数与session数的比值。（2）每种查询重构类型的平均频率（AFRT）：查询 $q$ 所在session中，某一重构类型数占同一session所有重构类型数的比值的总和与session数的比值
搜索过程类	查询重构路径平均长度（QRPL）	查询 $q$ 所在的所有session中全部查询重构路径的平均长度
	查询重构路径平均时间间隔（AVTI）	查询 $q$ 所在的所有session中完成一个session中全部查询重构路径的平均时间间隔
	重构路径类型数的平均值（ANRPT）	查询 $q$ 所在session中，查询重构路径类型总数与session数的比值
信息来源类	URL深度（UDP）	查询 $q$ 所在的所有session中共存于同一session的相邻两个URL路径中“/”的平均数量
信息来源类	URL多样性（UDV）	查询 $q$ 所在的所有session中共存于同一session的相邻两个URL之间的不同URL片段所占比值的平均值

Features of Multiple Categories Selected in This Paper

AOL Dataset Format

Comparison and Analysis of Intention Recognition Results

Classification Performance of Different Features on Different Classifiers

Classification Performance of Query Property Features

Classification Performance of Search Process Features

Classification Performance of Information Source Features

Classification Performance of Query Property and Search Process Features

Classification Performance of Query Property and Information Source Features

Classification Performance of Search Process and Information Source Features

Classification Performance of Three Types Features

A Matrix of Correlations Between Individual Features

Classification Performance of Removing Individual Features

[1]	Broder A. A Taxonomy of Web Search[J]. ACM SIGIR Forum, 2002, 36(2): 3-10.
[2]	White R W, Roth R A. Exploratory Search: Beyond the Query-Response Paradigm[J]. Synthesis Lectures on Information Concepts, Retrieval, and Services, 2009, 1(1): 1-98.
[3]	Marchionini G. Exploratory Search: From Finding to Understanding[J]. Communications of the ACM, 2006, 49(4): 41-46.
[4]	袁红. 探索式搜寻意图对搜寻策略选择的影响机制研究[J]. 图书情报工作, 2022, 66(3): 93-105. doi: 10.13266/j.issn.0252-3116.2022.03.011
[4]	(Yuan Hong. Research on the Influence Mechanism of Exploratory Seeking Intention on Seeking Strategy Selection[J]. Library and Information Service, 2022, 66(3): 93-105.) doi: 10.13266/j.issn.0252-3116.2022.03.011
[5]	Crescenzi A, Ward A R, Li Y, et al. Supporting Metacognition during Exploratory Search with the OrgBox[C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2021: 1197-1207.
[6]	Bates M J. The Design of Browsing and Berrypicking Techniques for the Online Search Interface[J]. Online Review, 1989, 13(5): 407-424.
[7]	Savolainen R. Berrypicking and Information Foraging: Comparison of Two Theoretical Frameworks for Studying Exploratory Search[J]. Journal of Information Science, 2018, 44(5): 580-593.
[8]	付煜文, 马志柔, 刘杰, 等. 面向问题意图识别的深度主动学习方法[J]. 中文信息学报, 2021, 35(4): 92-99, 109.
[8]	(Fu Yuwen, Ma Zhirou, Liu Jie, et al. Deep Active Learning Method for Question Intention Recognition[J]. Journal of Chinese Information Processing, 2021, 35(4): 92-99, 109.)
[9]	贺国秀, 张晓娟. 查询意图自动分类的方法改进探讨[J]. 数字图书馆论坛, 2018(1): 53-60.
[9]	(He Guoxiu, Zhang Xiaojuan. Discussion on the Improvement of Methods for Automatic Classification of Query Intent[J]. Digital Library Forum, 2018(1): 53-60.)
[10]	桂思思, 陆伟, 张晓娟. 基于查询表达式特征的时态意图识别研究[J]. 数据分析与知识发现, 2019, 3(3): 66-75.
[10]	(Gui Sisi, Lu Wei, Zhang Xiaojuan. Temporal Intent Classification with Query Expression Feature[J]. Data Analysis and Knowledge Discovery, 2019, 3(3): 66-75.)
[11]	郑新. 电子商务平台中用户查询意图识别与分析方法研究[J]. 西安文理学院学报(自然科学版), 2021, 24(1): 29-33.
[11]	(Zheng Xin. Research on the Method of Identifying and Analyzing User's Query Intention in E-commerce Platform[J]. Journal of Xi'an University (Natural Science Edition), 2021, 24(1): 29-33.)
[12]	赵一鸣, 潘沛, 毛进. 基于任务知识融合与文本数据增强的医学信息查询意图强度识别研究[J]. 数据分析与知识发现, 2023, 7(2): 38-47.
[12]	(Zhao Yiming, Pan Pei, Mao Jin. Recognizing Intensity of Medical Query Intentions Based on Task Knowledge Fusion and Text Data Enhancement[J]. Data Analysis and Knowledge Discovery, 2023, 7(2): 38-47.)
[13]	宋筱璇, 刘畅. 学习型搜索中用户信息源选择和使用策略研究[J]. 情报学报, 2019, 38(6): 655-666.
[13]	(Song Xiaoxuan, Liu Chang. Exploring Users Information Source Selection and Use Strategies in Learning Related Search[J]. Journal of the China Society for Scientific and Technical Information, 2019, 38(6): 655-666.)
[14]	Soufan A, Ruthven I, Azzopardi L. Searching the Literature: An Analysis of an Exploratory Search Task[C]// Proceedings of the 2022 Conference on Human Information Interaction and Retrieval. New York: ACM, 2022: 146-157.
[15]	Rha E Y, Mitsui M, Belkin N J, et al. Exploring the Relationships Between Search Intentions and Query Reformulations[C]// Proceedings of the 79th ASIS&T Annual Meeting on Creating Knowledge, Enhancing Lives through Information & Technology. New York: ACM, 2016: 1-9.
[16]	杨杰, 徐越, 余建桥, 等. 基于搜索引擎日志的用户查询意图分类[J]. 指挥信息系统与技术, 2019, 10(2): 74-79.
[16]	(Yang Jie, Xu Yue, Yu Jianqiao, et al. User Query Intention Classification Based on Search Engine Log[J]. Command Information System and Technology, 2019, 10(2): 74-79.)
[17]	王瑞雪, 方婧, 桂思思, 等. 基于深度学习算法的学术查询意图分类器构建[J]. 图书情报工作, 2021, 65(3): 93-99. doi: 10.13266/j.issn.0252-3116.2021.03.012
[17]	(Wang Ruixue, Fang Jing, Gui Sisi, et al. Deep Learning-Based Algorithm for Academic Query Intent Classification[J]. Library and Information Service, 2021, 65(3): 93-99.) doi: 10.13266/j.issn.0252-3116.2021.03.012
[18]	Tsai M F, Wu Y H. User Intent Prediction Search Engine System Based on Query Analysis and Image Recognition Technologies[J]. The Journal of Supercomputing, 2023, 79(5): 5327-5359.
[19]	Kim Y M, Lee T H, Na S O. Constructing Novel Datasets for Intent Detection and NER in a Korean Healthcare Advice System: Guidelines and Empirical Results[J]. Applied Intelligence, 2023, 53(1): 941-961.
[20]	Yu R, Limock, Dietze S. Still Haven't Found What You're Looking for-Detecting the Intent of Web Search Missions from User Interaction Features[OL]. arXiv Preprint, arXiv:2207.01256.
[21]	Zamora J, Mendoza M, Allende H. Query Intent Detection Based on Query Log Mining[J]. Journal of Web Engineering, 2014, 13(1-2): 24-52.
[22]	Hashemi H, Zamani H, Croft W B. Stochastic Optimization of Text Set Generation for Learning Multiple Query Intent Representations[C]// Proceedings of the 31st ACM International Conference on Information and Knowledge Management. New York: ACM, 2022: 4003-4008.
[23]	Natarajan V, Shanmuganathan S. Understanding Query Intention in Search Queries of Learners in Blended Learning Environments[C]// Proceedings of the 4th International Conference on Cybernetics, Cognition and Machine Learning Applications (ICCCMLA). Piscataway: IEEE, 2022: 308-315.
[24]	Wang Y Q, Wang S, Li Y Y, et al. Recognizing Medical Search Query Intent by Few-Shot Learning[C]// Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2022: 502-512.
[25]	Shah C, Hendahewa C, González-Ibáñez R. Rain or Shine? Forecasting Search Process Performance in Exploratory Search Tasks[J]. Journal of the Association for Information Science and Technology, 2016, 67(7): 1607-1623.
[26]	Etcheverry M, Moulin-Frier C, Oudeyer P Y. Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems[C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. New York: ACM, 2020: 4846-4859.
[27]	Ward A R, Capra R. OrgBox: Supporting Cognitive and Metacognitive Activities During Exploratory Search[C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2021: 2570-2574.
[28]	邢玉艳, 刘萍. 探索式搜索前后用户认知结构变化研究[J]. 图书情报工作, 2021, 65(22): 74-84. doi: 10.13266/j.issn.0252-3116.2021.22.008
[28]	(Xing Yuyan, Liu Ping. Research on Users' Cognitive Structure Changes Before and After Exploratory Search[J]. Library and Information Service, 2021, 65(22): 74-84.) doi: 10.13266/j.issn.0252-3116.2021.22.008
[29]	Athukorala K, Oulasvirta A, Głowacka D, et al. Narrow or Broad?: Estimating Subjective Specificity in Exploratory Search[C]// Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. New York: ACM, 2014: 819-828.
[30]	Athukorala K, Głowacka D, Jacucci G, et al. Is Exploratory Search Different? A Comparison of Information Search Behavior for Exploratory and Lookup Tasks[J]. Journal of the Association for Information Science and Technology, 2016, 67(11): 2635-2651.
[31]	袁红, 黄燕. 查询式搜索与探索式搜索行为比较研究——以健康信息为例[J]. 现代情报, 2019, 39(5): 48-56. doi: 10.3969/j.issn.1008-0821.2019.05.007
[31]	(Yuan Hong, Huang Yan. On the Comparation of Lookup and Exploratory Search Behavior—A Case Study of Health Information[J]. Journal of Modern Information, 2019, 39(5): 48-56.) doi: 10.3969/j.issn.1008-0821.2019.05.007
[32]	Agarwal M K, Sahu T. Lookup or Exploratory: What Is Your Search Intent?[OL]. arXiv Preprint, arXiv:2110.04640.
[33]	Khattak A, Habib A, Asghar M Z, et al. Applying Deep Neural Networks for User Intention Identification[J]. Soft Computing, 2021, 25(3): 2191-2220.
[34]	Hassan A, White R W, Dumais S T, et al. Struggling or Exploring?: Disambiguating Long Search Sessions[C]// Proceedings of the 7th ACM International Conference on Web Search and Data Mining. New York: ACM, 2014: 53-62.
[35]	魏丽, 张云秋. 探索式搜索行为的日志挖掘研究[J]. 图书情报工作, 2014, 58(11): 21-28. doi: 10.13266/j.issn.0252-3116.2014.11.003
[35]	(Wei Li, Zhang Yunqiu. A Study on Exploratory Search Behaviors Based on Log Mining[J]. Library and Information Service, 2014, 58(11): 21-28.) doi: 10.13266/j.issn.0252-3116.2014.11.003
[36]	Mikolov T, Sutskever I, Chen K, et al. Distributed Representations of Words and Phrases and Their Compositionality[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. New York: ACM, 2013: 3111-3119.
[37]	Kinley K, Tjondronegoro D, Partridge H, et al. Human-Computer Interaction: The Impact of Users’ Cognitive Styles on Query Reformulation Behaviour During Web Searching[C]// Proceedings of the 24th Australian Computer-Human Interaction Conference. New York: ACM, 2012: 299-307.
[38]	Kinley K, Tjondronegoro D, Partridge H, et al. Relationship between the Nature of the Search Task Types and Query Reformulation Behaviour[C]// Proceedings of the 17th Australasian Document Computing Symposium. New York: ACM, 2012: 39-46.
[39]	White R W, Huang J. Assessing the Scenic Route: Measuring the Value of Search Trails in Web Logs[C]// Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2010: 587-594.
[40]	马超. 面向探索式搜索过程的查询推荐方法研究[D]. 沈阳: 东北大学, 2018.
[40]	(Ma Chao. Research on Query Recommendation Methods Supporting Exploratory Search[D]. Shenyang: Northeastern University, 2018.)
[41]	White R W, Drucker S M. Investigating Behavioral Variability in Web Search[C]// Proceedings of the 16th International Conference on World Wide Web. New York: ACM, 2007: 21-30.
[42]	Kang I H, Kim G C. Query Type Classification for Web Document Retrieval[C]// Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2003: 64-71.
[43]	Ma C, Zhang B. A New Query Recommendation Method Supporting Exploratory Search Based on Search Goal Shift Graphs[J]. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(11): 2024-2036.
[44]	Craswell N, Campos D, Mitra B, et al. ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search[C]// Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York: ACM, 2020: 2983-2989.
[45]	Yin D, Tan J W, Zhang Z, et al. Learning to Generate Personalized Query Auto-Completions via a Multi-View Multi-Task Attentive Approach[C]// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2020: 2998-3007.
[46]	Li P F, Zhang Y, Zhang B. Understanding Query Combination Behavior in Exploratory Searches[J]. Applied Sciences, 2022, 12(2): 706-725.
[47]	Fleiss J L, Cohen J. The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability[J]. Educational & Psychological Measurement, 1973, 33(3):613-619.

[1]	Zhang Lanze, Zhao Xiaoliang, Liu Jintong, Gu Yijun. A Graph Convolutional Credit Fraud Detection Model with Spatial-Neighborhood Adaptability[J]. 数据分析与知识发现, 2024, 8(4): 137-151.
[2]	Feng Lizhou, Liu Furong, Wang Youwei. Detecting Rumor Based on Graph Convolution Network and Attention Mechanism[J]. 数据分析与知识发现, 2024, 8(4): 125-136.
[3]	He Guoxiu, Ren Jiayu, Li Zongyao, Lin Chenxi, Yu Haiyan. Revisiting Deep Learning-based Rumor Detection Models with Interpretable Tools[J]. 数据分析与知识发现, 2024, 8(4): 1-13.
[4]	Wang Xiwei, Wang Qiuyue, Cai Hongtian. Research on Recognition Model and Recognition Effect of Network Public Opinion Visual Emotion under Multi-Dimensional Attention Mechanism[J]. 数据分析与知识发现, 2024, 8(3): 156-167.
[5]	Zhou Changshun, Ying Wenhao, Zhong Shan, Gong Shengrong. Multi-Round Iterative Retrieval Algorithm for Parsing Question-Answering Process[J]. 数据分析与知识发现, 2024, 8(3): 120-131.
[6]	Ni Liang, Wu Peng, Zhou Xueqing. Topic Detecting on Multimodal News Data Based on Deep Learning[J]. 数据分析与知识发现, 2024, 8(3): 85-97.
[7]	Chen Zhuo, Jiang Xixi, Zhang Xiaojuan. Influence of Network Structure Changes on Co-word Network Link Prediction[J]. 数据分析与知识发现, 2024, 8(2): 114-130.
[8]	Xiang Shuxuan, Cao Yujie, Mao Jin. Computing Patent Similarity Based on Hierarchical Feature of Claims[J]. 数据分析与知识发现, 2024, 8(2): 33-43.
[9]	Shi Shuiqian, Jin Jing, Shen Gengyu, Wang Baojia, Ren Ni. Chinese Named Entity Disambiguation Based on Multivariate Similarity Fusion[J]. 数据分析与知识发现, 2024, 8(2): 56-64.
[10]	Zhang Xiongtao, Zhu Na, Guo Yuhui. A Survey on Session-Based Recommendation Methods with Graph Neural Network[J]. 数据分析与知识发现, 2024, 8(2): 1-16.
[11]	Liu Tong, Ren Xinru, Yin Jinhui, Ni Weijian. Knowledge Distillation with Few Labeled Samples[J]. 数据分析与知识发现, 2024, 8(1): 104-113.
[12]	He Yu, Zhang Xiaodong, Zheng Xin. Constructing Patent Knowledge Graph with SpERT-Aggcn Model[J]. 数据分析与知识发现, 2024, 8(1): 146-156.
[13]	Liu Tianchang, Wang Lei, Zhu Qinghua. Predicting User Churn of Smart Home-based Care Services Based on SHAP Interpretation[J]. 数据分析与知识发现, 2024, 8(1): 40-54.
[14]	Zhao Yiming, Chen Zhan, Zhang Fan. Identifying User Satisfaction Levels and Evolution Patterns in Exploratory Search[J]. 数据分析与知识发现, 2024, 8(1): 90-103.
[15]	Liu Rui, Chen Ye. Dynamic Movie Recommendation Considering Long-Term and Short-Term Interest and Its Evolution[J]. 数据分析与知识发现, 2024, 8(1): 80-89.

Viewed

Full text

Abstract

Cited

Shared

Discussed