Please wait a minute...
New Technology of Library and Information Service  2015, Vol. 31 Issue (5): 88-93    DOI: 10.11925/infotech.1003-3513.2015.05.12
Current Issue | Archive | Adv Search |
Using Logstash and ElasticSearch to Achieve Real-time Statistical Analysis of DSpace Logs
Chen He
Xiamen University Library, Xiamen 361005, China
Download: PDF(506 KB)   HTML  
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] The real-time statistical analysis system of DSpace logs is designed and implemented to meet the different needs of users, and to make up for lack of DSpace's statistical functions itself. [Context] For the design limitations, the DSpace's statistical functions are simple, rigid form of expression, and can not achieve interactive statistical analysis. [Methods] Use Logstash to collect and analyze DSpace logs, and use ElasticSearch to index the logs. Building QueryDSL to call ElasticSearch Java API to achieve different statistical functions, and show the graphical results with ECharts component. [Results] The real-time statistical analysis system of DSpace logs can get the browse rankings of items, collections and communities, get the download rankings of bitstreams, and get the regional rankings of website access, and so on. The statistics time can be customized by user, and the statistical result can be showed in different forms. [Conclusions] Using Logstash and ElasticSearch to achieve statistical analysis of DSpace logs has many excellences, just like no need to modify the code of DSpace, simple installation and deployment of the components, man-machine interactive query, fast and real-time, and rich forms to show the results.
Key words： Log analysis      DSpace      Logstash      ElasticSearch      ECharts     
Received: 25 November 2014      Published: 11 June 2015
:  G312  
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Chen He

Cite this article:

Chen He. Using Logstash and ElasticSearch to Achieve Real-time Statistical Analysis of DSpace Logs. New Technology of Library and Information Service, 2015, 31(5): 88-93.

URL:

http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.1003-3513.2015.05.12     OR     http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2015/V31/I5/88

[1] 顾立平. 机构知识库评价机制 [EB/OL]. [2014-10-06]. http://ir.las.ac.cn/handle/12502/6368. (Ku Liping. Evaluation for Institutional Repositories Development [EB/OL]. [2014-10-06]. http://ir.las.ac.cn/handle/12502/6368.)
[2] DSpace Statistics [EB/OL]. [2014-10-06]. https://wiki.duras-pace.org/display/DSDOC4x/DSpace+Statistics.
[3] DSpace Discovery [EB/OL]. [2014-10-06]. https://wiki.dura-
space.org/display/DSDOC4x/Discovery.
[4] 祝忠明, 马建霞, 卢利农, 等. 机构知识库开源软件DSpace的扩展开发与应用 [J]. 现代图书情报技术, 2009(7-8): 11-17. (Zhu Zhongming, Ma Jianxia, Lu Linong, et al. Developing an Institutional Repository Platform via Extending DSpace [J]. New Technology of Library and Information Service, 2009(7-8): 11-17.)
[5] 姚晓娜, 祝忠明. 基于分面搜索引擎Solr的机构知识库访问统计 [J]. 现代图书情报技术, 2011(7-8): 37-40. (Yao Xiaona, Zhu Zhongming. Usage Statistics of Institutional Repository Based on Faceted Search Engine Solr [J]. New Technology of Library and Information Service, 2011(7-8): 37-40.)
[6] Development of Usage Statistics for RepositóriUM [EB/OL]. [2014-10-06]. https://repositorium.sdum.uminho.pt/handle/1822/ 4803.
[7] ANU DSpace Statistics Installation Guide [EB/OL]. [2014-10-06]. http://sts.anu.edu.au/drs/downloads/dspace-stats/readme. html.
[8] Logstash [EB/OL]. [2014-10-06]. http://logstash.net/.
[9] Redis [EB/OL]. [2014-10-06]. http://redis.io/.
[10] ElasticSearch [EB/OL]. [2014-10-06]. http://www.elasticsear-ch.org/.
[11] QueryDSL [EB/OL]. [2014-10-06]. http://www.querydsl.com/.
[12] ECharts [EB/OL]. [2014-10-06]. http://echarts.baidu.com/.
[13] MaxMind [EB/OL]. [2014-10-06]. https://www.maxmind.com/.
[1] Xiang Zhou,Pengyi Zhang,Jun Wang. Impacts of Information Browsing Behaviors on Mobile Shopping: Case Study of Commerce APP Click Stream Analysis[J]. 数据分析与知识发现, 2018, 2(4): 1-9.
[2] Runwen Chen,Yong Qiu,Wenbin Huang,Jun Wang. Analyzing Private College Students’ Online Lifestyle with Web-logs[J]. 数据分析与知识发现, 2017, 1(8): 31-38.
[3] Zhao Ruixue, Du Ruopeng. Practice on Institutional Repository of Chinese Academy of Agricultural Sciences[J]. 现代图书情报技术, 2015, 31(2): 72-77.
[4] Chen Yong, Li Honglian, Lv Xueqiang. Analysis for the Search Behavior of Web Users[J]. 现代图书情报技术, 2014, 30(12): 10-17.
[5] Zeng Ting, Tu Feiping, Dong Li, Zou Rong. Study and Application of Event Mechanism for DSpace in the Extended Development[J]. 现代图书情报技术, 2012, 28(3): 73-77.
[6] Deng Hong, Ding Juntao, Tu Feiping. Design and Implementation of DSpace Resources Import Tool Using NoteExpress[J]. 现代图书情报技术, 2012, 28(1): 80-84.
[7] Zheng Lei, Zhu Zhongming. Research Progress of the Semantic Submission of Compound Digital Object[J]. 现代图书情报技术, 2011, 27(2): 1-7.
[8] Cui Yuhong. Research on Automatic Archiving System for Institutional Repositories[J]. 现代图书情报技术, 2010, 26(12): 76-80.
[9] Chen Xiaofeng, Zhang Zhiping, Bai Haiyan. The OAI-ORE’s Application and Implementation in Institutional Repository[J]. 现代图书情报技术, 2010, 26(11): 69-74.
[10] Tao Xinquan,Guo Guangchao,Yang Xiaoguang. Investigations on Data Migration of Heterogeneous Data Sources ——Taking the Library of Beihang University as an Example[J]. 现代图书情报技术, 2010, 26(1): 94-98.
[11] Zhu Zhongming,Ma Jianxia,Lu Linong,Li Fuqiang ,Liu Wei,Wu Denglu. Developing an Institutional Repository Platform via Extending DSpace[J]. 现代图书情报技术, 2009, 25(7-8): 11-17.
[12] Zou Rong,Zeng Ting,Jiang Airong,Guo Jing. Study and Implementation of DSpace-based Union Website[J]. 现代图书情报技术, 2009, 25(5): 67-71.
[13] Zou Rong,Fan Aihong,Jiang Airong. Construction of the Academic Papers Management System with DSpace[J]. 现代图书情报技术, 2009, (10): 90-94.
[14] Ma Jianxia, Zhu Zhongming, Tang Runhuan, LI Fuqiang, Wang Yuanming. A Try to Integrate Institutional Repository with Information Environment of Management of Scientific Research[J]. 现代图书情报技术, 2008, 24(2): 14-18.
[15] Wu Denglu ,Zhu Zhongming,Ma Jianxia,Han Ke. Design and Implementation of Embedded Retrieval Service for DSpace Repository System[J]. 现代图书情报技术, 2008, 24(10): 69-74.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn