scholarly journals Distributed search engine for extraction of resume statistics using hadoop with combination of lucene indexing framework and the solr


Author(s):  
Nobuyoshi Sato ◽  
Minoru Udagawa ◽  
Minoru Uehara ◽  
Yoshifumi Sakai ◽  
Hideki Mori


2019 ◽  
Vol 1 (4) ◽  
pp. 333-349 ◽  
Author(s):  
Peilu Wang ◽  
Hao Jiang ◽  
Jingfang Xu ◽  
Qi Zhang

Knowledge graph (KG) has played an important role in enhancing the performance of many intelligent systems. In this paper, we introduce the solution of building a large-scale multi-source knowledge graph from scratch in Sogou Inc., including its architecture, technical implementation and applications. Unlike previous works that build knowledge graph with graph databases, we build the knowledge graph on top of SogouQdb, a distributed search engine developed by Sogou Web Search Department, which can be easily scaled to support petabytes of data. As a supplement to the search engine, we also introduce a series of models to support inference and graph based querying. Currently, the data of Sogou knowledge graph that are collected from 136 different websites and constantly updated consist of 54 million entities and over 600 million entity links. We also introduce three applications of knowledge graph in Sogou Inc.: entity detection and linking, knowledge based question answering and knowledge based dialog system. These applications have been used in Web search products to help user acquire information more efficiently.



2014 ◽  
Vol 519-520 ◽  
pp. 54-57
Author(s):  
Ai Ling Duan ◽  
Dan Cao ◽  
Hai Fang Si

Distributed search techniques of Hadoop are researched and analyzed. Combined with Lucene indexing objects, a search engine system IS successfully built. Efficiency of the system in both time and space is investigated. Merit of distributed processing architecture for a single architecture in data handling is verified. The access and update of file information in distributed search technology are further explored. The research plays a positive role in promoting study of related fields





IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 43001-43012
Author(s):  
Ali Raza ◽  
Kyunghyun Han ◽  
Seong Oun Hwang




Sign in / Sign up

Export Citation Format

Share Document