Please wait a minute...
New Technology of Library and Information Service  2011, Vol. 27 Issue (7/8): 26-31    DOI: 10.11925/infotech.1003-3513.2011.07-08.05
Current Issue | Archive | Adv Search |
Targeted Websites Distributed and Precise Harvest System for Network Monitoring Technology
Xie Jing, Qu Yunpeng, Liu Jianhua
National Science Library, Chinese Academy of Sciences, Beijing 100190,China
Download: PDF(1923 KB)   HTML  
Export: BibTeX | EndNote (RIS)      
Abstract  By analyzing the existing open-source framework collection system, an accurate acquistition system is designed and developed based on Crawler4j. So the system can meet the real-time monitoring of collection of resources and accuracy requirements. And the paper introduces the design and implementation of the system.
Key wordsMonitoring      Distributed      Precise harvest     
Received: 05 May 2011      Published: 09 October 2011
: 

G250

 

Cite this article:

Xie Jing, Qu Yunpeng, Liu Jianhua. Targeted Websites Distributed and Precise Harvest System for Network Monitoring Technology. New Technology of Library and Information Service, 2011, 27(7/8): 26-31.

URL:

http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.1003-3513.2011.07-08.05     OR     http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2011/V27/I7/8/26

[1] Nutch.http://wiki.apache.org/nutch.

[2] Heritrix.http://crawler.archive.org/.

[3] Open Source Web Crawler for Java.http://code.google.com/p/crawler4j/.

[4] Trail:RMI.http://download.oracle.com/javase/tutorial/rmi/index.html.

[5] Cobra: Java HTML Renderer & Parser.http://lobobrowser.org/cobra.jsp.

[6] Regular Expression.http://en.wikipedia.org/wiki/Regular_expression.
[1] Quan Lu,Anqi Zhu,Jiyue Zhang,Jing Chen. Research on User Information Requirement in Chinese Network Health Community: Taking Tumor-forum Data of Qiuyi as an Example[J]. 数据分析与知识发现, 2019, 3(4): 22-32.
[2] Sili Wang,Wei Liu,Zhongming Zhu,Zhiqiang Wu,Jinping Wang. Tracking Scientific Information with CSpace Technology[J]. 数据分析与知识发现, 2017, 1(10): 85-93.
[3] Jianhua Hou,Shuang Guo. Analyzing Emerging Issues with Technology Entropy Method Based on Patents: Case Study of Carbon Capture[J]. 数据分析与知识发现, 2017, 1(1): 55-63.
[4] Yang Aidong,Liu Dongsu. Hadoop Based Public Opinion Monitoring System for Micro-blogs[J]. 现代图书情报技术, 2016, 32(5): 56-63.
[5] Yang Yang,Lin Hui,Hu Guangwei. Detecting Investment Risks of Photovoltaic Projects with Big Data: Case Study of Solarbao.com[J]. 现代图书情报技术, 2016, 32(11): 11-19.
[6] Zhuo Keqiu, Yu Wei, Su Xinning. Parallel Implementing Bursty Events Detection Using MapReduce[J]. 现代图书情报技术, 2015, 31(2): 46-54.
[7] Zhao Huaming. Research and Implementation of Textual Clustering in Distributed Environment[J]. 现代图书情报技术, 2015, 31(1): 82-88.
[8] Xiong Yongjun, Yuan Xiaoyi. Design and Implementation of Automatic Monitoring System about Library Document Database Running State[J]. 现代图书情报技术, 2014, 30(7): 127-132.
[9] Zhang Zhixiong, Liu Jianhua, Xie Jing, Qian Li, Zhang Min, Yu Gaihong. Design and Implementation of the Service Cloud for Strategic S&T Information Monitoring[J]. 现代图书情报技术, 2014, 30(6): 51-61.
[10] Yu Weiping, Yang Yufeng. Brand Scandal Spillover Monitor Index System Research Based on Micro-blog[J]. 现代图书情报技术, 2013, 29(2): 63-69.
[11] Zhu Yuqiang. Design of Monitoring Program to Detect the Browsable and Downloadable Status of Library’s Electronic Resources[J]. 现代图书情报技术, 2013, 29(11): 86-90.
[12] Xiao Qiang, Zhu Qinghua, Zheng Hua, Wu Kewen. Design and Implementation of Distributed Collaborative Filtering Algorithm on Hadoop[J]. 现代图书情报技术, 2013, 29(1): 83-89.
[13] Niu Yazhen, Zhu Zhongming. Overview about the Methods of Cross-system User Modeling for Personalization Service[J]. 现代图书情报技术, 2012, 28(5): 1-6.
[14] Wu Hong, Wang Fengying, Fu Xiuying. Design and Establishment of Legal Status Distributed Collection System Based on Patent Analysis[J]. 现代图书情报技术, 2012, (12): 66-71.
[15] Guo Xiaoqing, Ren Shougang, Xie Zhonghong. Research and Implementation of Drive-level Local User Behavior Monitoring System[J]. 现代图书情报技术, 2012, (10): 77-82.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn