Usenet-snatcher Based on Multithread and Mass-data Storage Supporting Breakpoint Transmission
Yang Daiqing1,2,3, Wang Zhiping1, Wang Xing1, Liu Minjian1, Chang Yingchun1
1. ;
2. National Science Library,Chinese Academy of Sciences,Beijing 100190,China;
3. Graduate University of Chinese Academy of Sciences, Beijing 100049, China
A usenet-snatcher is designed based on multithread to improve the download-speed and MIME-parsing-speed and a storage schema supporting breakpoint transmission is also proposed. Experiment shows that the usenet-snatcher can gather articles effectively and gathering-speed under multithread is faster than single thread.
杨代庆, 王志苹, 王星, 刘敏健, 常迎春. 一种断点续传的多线程新闻组抓取方法及存储结构[J]. 现代图书情报技术, 2011, 27(2): 29-33.
Yang Daiqing, Wang Zhiping, Wang Xing, Liu Minjian, Chang Yingchun. Usenet-snatcher Based on Multithread and Mass-data Storage Supporting Breakpoint Transmission. New Technology of Library and Information Service, 2011, 27(2): 29-33.