distributed crawler Latest Research Papers

Implementation of Efficient Distributed Crawler through Stepwise Crawling Node Allocation

JOURNAL OF ADVANCED INFORMATION TECHNOLOGY AND CONVERGENCE ◽

10.14801/jaitc.2020.10.2.15 ◽

2020 ◽

Vol 10 (2) ◽

pp. 15-31

Author(s):

Hyuntae Kim ◽

Junhyung Byun ◽

Yoseph Na ◽

Yuchul Jung

Keyword(s):

Distributed Crawler ◽

Node Allocation

Download Full-text

A Summary of Research on Web Data Acquisition Methods Based on Distributed Crawler

2020 IEEE 6th International Conference on Computer and Communications (ICCC) ◽

10.1109/iccc51575.2020.9345157 ◽

2020 ◽

Author(s):

Xuejiao Ren ◽

Hairong Wang ◽

Diwei Dai

Keyword(s):

Data Acquisition ◽

Web Data ◽

Distributed Crawler

Download Full-text

Implementation of Distributed Crawler System Based on Spark for Massive Data Mining

2020 5th International Conference on Computer and Communication Systems (ICCCS) ◽

10.1109/icccs49078.2020.9118442 ◽

2020 ◽

Author(s):

Feng Liu ◽

Wang Xin

Keyword(s):

Data Mining ◽

Massive Data ◽

Distributed Crawler

Download Full-text

Distance Education Management System Considering Load Balancing Strategy of Distributed Crawler System

2020 International Conference on Computer Engineering and Application (ICCEA) ◽

10.1109/iccea50009.2020.00075 ◽

2020 ◽

Author(s):

Yanqin Wang

Keyword(s):

Distance Education ◽

Load Balancing ◽

Management System ◽

Education Management ◽

Distributed Crawler ◽

Education Management System

Download Full-text

The Crawl and Analysis of Recruitment Data Based on the Distributed Crawler

Green Energy and Networking - Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering ◽

10.1007/978-3-030-62483-5_18 ◽

2020 ◽

pp. 162-168

Author(s):

Jiancai Wang ◽

Jianting Shi

Keyword(s):

Distributed Crawler

Download Full-text

A Load Balancing Algorithm for Distributed Crawler System

2019 IEEE 11th International Conference on Communication Software and Networks (ICCSN) ◽

10.1109/iccsn.2019.8905323 ◽

2019 ◽

Author(s):

Shutao Zhang ◽

Haibo Tan ◽

Liangfeng Chen ◽

Bo Lv

Keyword(s):

Load Balancing ◽

Load Balancing Algorithm ◽

Distributed Crawler

Download Full-text

Research on Scrapy-Based Distributed Crawler System for Crawling Semi-structure Information at High Speed

2018 IEEE 4th International Conference on Computer and Communications (ICCC) ◽

10.1109/compcomm.2018.8781062 ◽

2018 ◽

Cited By ~ 1

Author(s):

Fulian Yin ◽

Xiating He ◽

Zhixin Liu

Keyword(s):

High Speed ◽

Structure Information ◽

Distributed Crawler

Download Full-text

An Improved Bloom Filter in Distributed Crawler

2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) ◽

10.1109/smartworld.2018.00303 ◽

2018 ◽

Cited By ~ 1

Author(s):

Weipeng Zhou ◽

Pan Wang ◽

Xuejiao Chen ◽

Feng Ye

Keyword(s):

Bloom Filter ◽

Distributed Crawler

Download Full-text

Consistent Hashing Algorithm Based on Slice in Improving Scrapy-Redis Distributed Crawler Efficiency

2018 IEEE International Conference on Computer and Communication Engineering Technology (CCET) ◽

10.1109/ccet.2018.8542217 ◽

2018 ◽

Author(s):

Zhe-Yu Ying ◽

Feng-Li Zhang ◽

Qing-Yu Fan

Keyword(s):

Consistent Hashing ◽

Hashing Algorithm ◽

Distributed Crawler

Download Full-text

Anti-Crawler strategy and distributed crawler based on Hadoop

2018 IEEE 3rd International Conference on Big Data Analysis (ICBDA) ◽

10.1109/icbda.2018.8367682 ◽

2018 ◽

Author(s):

HongRu Wang ◽

ChunFang Li ◽

LingFei Zhang ◽

MinYong Shi

Keyword(s):

Distributed Crawler

Download Full-text

distributed crawler
Recently Published Documents

TOTAL DOCUMENTS

H-INDEX

Implementation of Efficient Distributed Crawler through Stepwise Crawling Node Allocation

A Summary of Research on Web Data Acquisition Methods Based on Distributed Crawler

Implementation of Distributed Crawler System Based on Spark for Massive Data Mining

Distance Education Management System Considering Load Balancing Strategy of Distributed Crawler System

The Crawl and Analysis of Recruitment Data Based on the Distributed Crawler

A Load Balancing Algorithm for Distributed Crawler System

Research on Scrapy-Based Distributed Crawler System for Crawling Semi-structure Information at High Speed

An Improved Bloom Filter in Distributed Crawler

Consistent Hashing Algorithm Based on Slice in Improving Scrapy-Redis Distributed Crawler Efficiency

Anti-Crawler strategy and distributed crawler based on Hadoop

Export Citation Format

distributed crawlerRecently Published Documents

TOTAL DOCUMENTS

H-INDEX

Implementation of Efficient Distributed Crawler through Stepwise Crawling Node Allocation

A Summary of Research on Web Data Acquisition Methods Based on Distributed Crawler

Implementation of Distributed Crawler System Based on Spark for Massive Data Mining

Distance Education Management System Considering Load Balancing Strategy of Distributed Crawler System

The Crawl and Analysis of Recruitment Data Based on the Distributed Crawler

A Load Balancing Algorithm for Distributed Crawler System

Research on Scrapy-Based Distributed Crawler System for Crawling Semi-structure Information at High Speed

An Improved Bloom Filter in Distributed Crawler

Consistent Hashing Algorithm Based on Slice in Improving Scrapy-Redis Distributed Crawler Efficiency

Anti-Crawler strategy and distributed crawler based on Hadoop

distributed crawler
Recently Published Documents