Please wait a minute...
Data Analysis and Knowledge Discovery  2020, Vol. 4 Issue (11): 102-111    DOI: 10.11925/infotech.2096-3467.2020.0059
Current Issue | Archive | Adv Search |
Predicting Usefulness of Crowd Testing Reports with Deep Learning
Cai Jingxuan1,Wu Jiang1,2(),Wang Chengkun1
1School of Information Management, Wuhan University, Wuhan 430072, China
2Center for E-commerce Research and Development, Wuhan University, Wuhan 430072, China
Download: PDF (1461 KB)   HTML ( 23
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper tries to predict the usefulness of crowd testing reports with author attributes, text features, and image features. [Methods] First, we adopted deep learning techniques to extract text and image features from crowd testing reports. Then, we constrcuted a prediction model with full-connected neural network. Third, we trained the new model with 80% of samples and different input combinations. Finally, we examined our model’s performance with the remaining samples. [Results] With the help of text or image features, the prediction accuracy of the model increased by 4.24% and 5.21%, respectively. Using both the text and image features, our model’s prediction accuracy increased by 6.96%. [Limitations] The extracted features of texts and images were not understandable and interpretable. Therefore, we cannot identify specific features represented by each layer of neural network in the model. [Conclusions] The proposed model with text and image features can effectively predict the usefulness of crowd testing reports.

Key wordsCrowd Testing      Signal Theory      Deep Learning      Feature Extraction      Predictive Analysis     
Received: 15 January 2020      Published: 27 September 2020
ZTFLH:  G203  
Corresponding Authors: Wu Jiang     E-mail: jiangw@whu.edu.cn

Cite this article:

Cai Jingxuan,Wu Jiang,Wang Chengkun. Predicting Usefulness of Crowd Testing Reports with Deep Learning. Data Analysis and Knowledge Discovery, 2020, 4(11): 102-111.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2020.0059     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2020/V4/I11/102

维度 变量 说明 Obs Mean S.D. Max Min
因变量 Like 众测报告获得的点赞数 1 550 27.28 33.56 385 0
作者属性 Au_level 作者在平台上的等级 1 550 9.55 3.72 26 1
Au_article 作者在平台上发布的文章总数 1 550 24.57 31.18 246 1
Au_test 作者参与众测次数 1 550 5.10 4.82 114 1
Au_followed 作者关注数 1 550 911.74 1 486.37 10 620 0
Au_follower 作者粉丝数 1 550 57.02 66.68 1 225 0
产品属性 Prod_type 试用产品的类型(美妆护肤、美食、生活、服饰、电子产品、母婴) 1 550 - - - -
Prod_price 试用产品的价格 1 550 177.32 215.30 1 700 11.99
Prod_num 同时参与该商品众测人数 1 550 7.98 7.35 32 1
Variables Definitions and Descriptive Statistics
类别 内容 来源
用户词典 (1)产品及商家名称:美妆蛋、小白鞋、良品铺子等
(2)体验描述:沙漠皮、开箱、神器等
(3)网络用语及缩写:鸡冻、踩雷、轻奢等
(1)网站中对众测商品的描述
(2)高频未分词
停词词典 (1)标点符号
(2)虚词
(3)表情
(4)其他无意义词语:哈哈等
(1)哈尔滨工业大学停用词表
(2)高频无意义词
User Dictionary and Stop Words
Topic Perplexity of Crowd Testing Report
Extraction Process of Image Feature
Feature Integration (Model 4)
Model Construction
模型代码 特征组合 精确率 召回率 F1值 准确率
模型1 作者属性+产品属性 0.767 6 0.878 3 0.791 7 0.782 1
模型2 作者属性+产品属性+报告文本 0.862 4 0.840 3 0.809 9 0.824 5
模型3 作者属性+产品属性+报告图片 0.870 0 0.857 1 0.835 9 0.834 2
模型4 作者属性+产品属性+报告文本+报告图片 0.885 4 0.887 0 0.865 8 0.851 7
Model Performance
模型代码 图片预训练模型 精确率 召回率 F1值 准确率
模型3 VGG19 0.870 0 0.857 1 0.835 9 0.834 2
ResNet50 0.843 0 0.803 1 0.822 6 0.816 7
InceptionV3 0.504 1 0.941 9 0.670 1 0.512 3
NASNetMobile 0.795 2 0.792 3 0.779 0 0.768 8
模型4 VGG19 0.885 4 0.887 0 0.865 8 0.851 7
ResNet50 0.786 0 0.838 1 0.803 6 0.793 8
InceptionV3 0.859 8 0.806 7 0.831 6 0.831 3
NASNetMobile 0.885 0 0.745 5 0.801 1 0.810 5
Performance with Different Picture Pre-training Model
[1] Han Y, Zhang Z. Impact of Free Sampling on Product Diffusion Based on Bass Model[J]. Electronic Commerce Research, 2018,18(2):125-141.
doi: 10.1007/s10660-017-9264-9
[2] Koch O F, Benlian A. The Effect of Free Sampling Strategies on Freemium Conversion Rates[J]. Electronic Markets, 2017,27:67-76.
doi: 10.1007/s12525-016-0236-z
[3] Wu L, Deng S, Jiang X. Sampling and Pricing Strategy Under Competition[J]. Omega, 2018,80:192-208.
doi: 10.1016/j.omega.2018.01.002
[4] Hsu C L, Lin J C C, Chiang H S. The Effects of Blogger Recommendations on Customers’ Online Shopping Intentions[J]. Internet Research, 2013,23(1):69-88.
doi: 10.1108/10662241311295782
[5] Biswas D, Grewal D, Roggeveen A. How the Order of Sampled Experiential Products Affects Choice[J]. Journal of Marketing Research, 2010,47(3):508-519.
doi: 10.1509/jmkr.47.3.508
[6] Lin Z, Zhang Y, Tan Y. An Empirical Study of Free Product Sampling and Rating Bias[J]. Information Systems Research, 2019,30(1):260-275.
doi: 10.1287/isre.2018.0801
[7] Fehr E, Gächter S. Fairness and Retaliation: The Economics of Reciprocity[J]. The Journal of Economic Perspectives, 2000,14(3):159-181.
[8] Schmidhuber J. Deep Learning in Neural Networks: An Overview[J]. Neural Networks, 2015,61:85-117.
doi: 10.1016/j.neunet.2014.09.003 pmid: 25462637
[9] Ni F T, Zhang J, Noori M N. Deep Learning for Data Anomaly Detection and Data Compression of a Long-Span Suspension Bridge[J]. Computer-Aided Civil and Infrastructure Engineering, 2020,35(7):685-700.
doi: 10.1111/mice.v35.7
[10] Xu W H, Huang H, Zhang J, et al. CNN-based Skip-Gram Method for Improving Classification Accuracy of Chinese Text[J]. KSII Transactions on Internet and Information Systems, 2019,13(12):6080-6096.
[11] Yadav A, Vishwakarma D K. Sentiment Analysis Using Deep Learning Architectures: A Review[J]. Artificial Intelligence Review, 2020,53(6):4335-4385.
doi: 10.1007/s10462-019-09794-5
[12] Yang W M, Zhang X C, Tian Y P, et al. Deep Learning for Single Image Super-Resolution: A Brief Review[J]. IEEE Transactions on Multimedia, 2019,21(12):3106-3121.
doi: 10.1109/TMM.6046
[13] Dhillon A, Verma G K. Convolutional Neural Network: A Review of Models, Methodologies and Applications to Object Detection[J]. Progress in Artificial Intelligence, 2020,9:85-112.
doi: 10.1007/s13748-019-00203-0
[14] Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[OL]. arXiv Preprint, arXiv: 1409. 1556.
[15] Lakshmanan A, Krishnan H S. The A-ha! Experience: Insight and Discontinuous Learning in Product Usage[J]. Journal of Marketing, 2011,75(6):105-123.
[16] Kim J, Morris J D. The Power of Affective Response and Cognitive Structure In Product-Trial Attitude Formation[J]. Journal of Advertising, 2007,36(1):95-106.
doi: 10.2753/JOA0091-3367360107
[17] Chen H, Duan W, Zhou W. The Interplay Between Free Sampling and Word of Mouth in the Online Software Market[J]. Decision Support Systems, 2017,95:82-90.
doi: 10.1016/j.dss.2017.01.001
[18] Li F, Yi Z. Trial or No Trial: Supplying Costly Signals to Improve Profits[J]. Decision Sciences, 2016,48(4):795-827.
doi: 10.1111/deci.2017.48.issue-4
[19] Han Y, Zhang Z. Impact of Free Sampling on Product Diffusion Based on Bass Model[J]. Electronic Commerce Research, 2018,18(1):125-141.
doi: 10.1007/s10660-017-9264-9
[20] Connelly B, Certo T, Ireland R, et al. Signaling Theory: A Review and Assessment[J]. Journal of Management, 2011,37(1):39-67.
doi: 10.1177/0149206310388419
[21] Kirmani A, Rao A. No Pain, No Gain: A Critical Review of the Literature on Signaling Unobservable Product Quality[J]. Journal of Marketing, 2000,64(2):66-79.
doi: 10.1509/jmkg.64.2.66.18000
[22] Hampshire K, Hamill H, Mariwah S, et al. The Application of Signalling Theory to Health-Related Trust Problems: The Example of Herbal Clinics in Ghana and Tanzania[J]. Social Science & Medicine, 2017,188:109-118.
pmid: 28738317
[23] Mavlanova T, Benbunan-Fich R, Koufaris M. Signaling Theory and Information Asymmetry in Online Commerce[J]. Information & Management, 2012,49(5):240-247.
[24] Biswas D, Biswas A. The Diagnostic Role of Signals in the Context of Perceived Risks in Online Shopping: Do Signals Matter More on the Web?[J]. Journal of Interactive Marketing, 2004,18(3):30-45.
doi: 10.1002/dir.20010
[25] Spence M. Signaling in Retrospect and the Informational Structure of Markets[J]. American Economic Review, 2002,92(3):434-459.
doi: 10.1257/00028280260136200
[26] Donath J. Signals in Social Supernets[J]. Journal of Computer-Mediated Communication, 2007,13(1):231-251.
doi: 10.1111/j.1083-6101.2007.00394.x
[27] Jiang Z, Benbasat I. Virtual Product Experience: Effects of Visual and Functional Control of Products on Perceived Diagnosticity and Flow in Electronic Shopping[J]. Journal of Management Information Systems, 2005,21(3):111-148.
doi: 10.1080/07421222.2004.11045817
[28] Teng L, Ye N, Yu Y, et al. Effects of Culturally Verbal and Visual Congruency/Incongruency Across Cultures in a Competitive Advertising Context[J]. Journal of Business Research, 2014,67(3):288-294.
doi: 10.1016/j.jbusres.2013.05.015
[29] Evans J S B T. In Two Minds: Dual-Process Accounts of Reasoning[J]. Trends in Cognitive Sciences, 2003,7(10):454-459.
doi: 10.1016/j.tics.2003.08.012 pmid: 14550493
[30] Yoo J, Kim M. The Effects of Online Product Presentation on Consumer Responses: A Mental Imagery Perspective[J]. Journal of Business Research, 2014,67(11):2464-2472.
doi: 10.1016/j.jbusres.2014.03.006
[31] Macinnis D, Price L. The Role of Imagery in Information Processing: Review and Extensions[J]. Journal of Consumer Research, 1987,13(4):473-491.
doi: 10.1086/jcr.1987.13.issue-4
[32] Wang Q, Cui X, Huang L, et al. Seller Reputation or Product Presentation? An Empirical Investigation from Cue Utilization Perspective[J]. International Journal of Information Management, 2016,36(3):271-283.
doi: 10.1016/j.ijinfomgt.2015.12.006
[33] Zhang S, Lee D, Vir Singh P, et al. How Much is an Image Worth? Airbnb Property Demand Estimation Leveraging Large Scale Image Analytics[OL]. [2017-05-25]. https://ssrn.com/abstract=2976021.
[1] Zhou Zeyu,Wang Hao,Zhao Zibo,Li Yueyan,Zhang Xiaoqin. Construction and Application of GCN Model for Text Classification with Associated Information[J]. 数据分析与知识发现, 2021, 5(9): 31-41.
[2] Xu Yuemei, Wang Zihou, Wu Zixin. Predicting Stock Trends with CNN-BiLSTM Based Multi-Feature Integration Model[J]. 数据分析与知识发现, 2021, 5(7): 126-138.
[3] Zhao Danning,Mu Dongmei,Bai Sen. Automatically Extracting Structural Elements of Sci-Tech Literature Abstracts Based on Deep Learning[J]. 数据分析与知识发现, 2021, 5(7): 70-80.
[4] Huang Mingxuan,Jiang Caoqing,Lu Shoudong. Expanding Queries Based on Word Embedding and Expansion Terms[J]. 数据分析与知识发现, 2021, 5(6): 115-125.
[5] Zhong Jiawa,Liu Wei,Wang Sili,Yang Heng. Review of Methods and Applications of Text Sentiment Analysis[J]. 数据分析与知识发现, 2021, 5(6): 1-13.
[6] Zhang Guobiao,Li Jie. Detecting Social Media Fake News with Semantic Consistency Between Multi-model Contents[J]. 数据分析与知识发现, 2021, 5(5): 21-29.
[7] Chang Chengyang,Wang Xiaodong,Zhang Shenglei. Polarity Analysis of Dynamic Political Sentiments from Tweets with Deep Learning Method[J]. 数据分析与知识发现, 2021, 5(3): 121-131.
[8] Feng Yong,Liu Yang,Xu Hongyan,Wang Rongbing,Zhang Yonggang. Recommendation Model Incorporating Neighbor Reviews for GRU Products[J]. 数据分析与知识发现, 2021, 5(3): 78-87.
[9] Hu Haotian,Ji Jinfeng,Wang Dongbo,Deng Sanhong. An Integrated Platform for Food Safety Incident Entities Based on Deep Learning[J]. 数据分析与知识发现, 2021, 5(3): 12-24.
[10] Zhang Qi,Jiang Chuan,Ji Youshu,Feng Minxuan,Li Bin,Xu Chao,Liu Liu. Unified Model for Word Segmentation and POS Tagging of Multi-Domain Pre-Qin Literature[J]. 数据分析与知识发现, 2021, 5(3): 2-11.
[11] Lv Xueqiang,Luo Yixiong,Li Jiaquan,You Xindong. Review of Studies on Detecting Chinese Patent Infringements[J]. 数据分析与知识发现, 2021, 5(3): 60-68.
[12] Cheng Bin,Shi Shuicai,Du Yuncheng,Xiao Shibin. Keyword Extraction for Journals Based on Part-of-Speech and BiLSTM-CRF Combined Model[J]. 数据分析与知识发现, 2021, 5(3): 101-108.
[13] Li Danyang, Gan Mingxin. Music Recommendation Method Based on Multi-Source Information Fusion[J]. 数据分析与知识发现, 2021, 5(2): 94-105.
[14] Yu Chuanming, Zhang Zhengang, Kong Lingge. Comparing Knowledge Graph Representation Models for Link Prediction[J]. 数据分析与知识发现, 2021, 5(11): 29-44.
[15] Han Pu, Zhang Wei, Zhang Zhanpeng, Wang Yuxin, Fang Haoyu. Sentiment Analysis of Weibo Posts on Public Health Emergency with Feature Fusion and Multi-Channel[J]. 数据分析与知识发现, 2021, 5(11): 68-79.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn