Research and Application on Optimization of Multi-Thread Download Technology for Enhanced Search Engine

2013 ◽  
Vol 756-759 ◽  
pp. 1008-1012
Author(s):  
Ya Juan Sun ◽  
Hong Lin ◽  
Bao Hui Wang
Keyword(s):  
Search Engine ◽  
Task Scheduling ◽  
Optimization Method ◽  
Acquisition System ◽  
Key Technology ◽  
Engine Efficiency ◽  
Content Acquisition ◽  
Protocol Adaptation

Multi-threaded file download as the key technology of content acquisition system for search engine, determines the efficiency and timeliness of content acquisition. In this paper, we do the research on optimization technologies which include multithreaded download based on P2SP, task scheduling based on MapReduce and download based on the protocol adaptation, designed to improve enhanced search engine efficiency. At last the result shows that the optimization method is successful for content acquisition.

scholarly journals An Efficient Task Scheduling Method for the Unified Interface Platform of the Electric Information Acquisition System

2018 ◽  
Vol 2 (1) ◽  
Author(s):  
Ye Fang Bin
Keyword(s):  
Task Scheduling ◽  
Information Acquisition ◽  
Optimal Solution ◽  
Minimum Variance ◽  
Acquisition System ◽  
Combination Method ◽  
Business Systems ◽  
Target Task ◽  
Static Data ◽  
Local Optimal Solution

Due to the large and frequent static data interaction between the Electric Information Acquisition System and the external business systems, researching on using limited server sources to do an efficient task scheduling is becoming one of the key technologies of the unified interface platform. The information interaction structure of the unified interface platform is introduced. Task scheduling has been decomposed into two stages, task decomposition and task combination, based on the features (various types and dispersed) of large static data. The principle of the minimum variance of the subtasks data quantity is used to do the target task resolving in the decomposition stage. The thought of the Greedy Algorithm is used in the taskcombination. Breaking the target task with large static data into serval composed tasks with roughly same data quantity is effectively realized. Meanwhile, to avoid the situation of the GA falling into the local optimal solution, an improved combination method has been put forward. Moreover, the new method creates more average composed tasks and making the task scheduling more effective. Ultimately, the effectiveness of the proposed method is verified by the experimental data.

scholarly journals Improving Search Engine Efficiency through Contextual Factor Selection

AI Magazine ◽  
2021 ◽  
Vol 42 (2) ◽  
pp. 50-58
Author(s):  
Anxiang Zeng ◽  
Han Yu ◽  
Qing Da ◽  
Yusen Zhan ◽  
Yang Yu ◽  
...  
Keyword(s):  
Search Engine ◽  
Large Scale ◽  
Learning To Rank ◽  
Contextual Factor ◽  
System Response ◽  
Search Query ◽  
Previous Approach ◽  
Engine Efficiency ◽  
Search Result ◽  
Selection Of

Learning to rank (LTR) is an important artificial intelligence (AI) approach supporting the operation of many search engines. In large-scale search systems, the ranking results are continually improved with the introduction of more factors to be considered by LTR. However, the more factors being considered, the more computation resources required, which in turn, results in increased system response latency. Therefore, removing redundant factors can significantly improve search engine efficiency. In this paper, we report on our experience incorporating our Contextual Factor Selection (CFS) deep reinforcement learning approach into the Taobao e-commerce platform to optimize the selection of factors based on the context of each search query to simultaneously maintaining search result quality while significantly reducing latency. Online deployment on Taobao.com demonstrated that CFS is able to reduce average search latency under everyday use scenarios by more than 40% compared to the previous approach with comparable search result quality. Under peak usage during the Single’s Day Shopping Festival (November 11th) in 2017, CFS reduced the average search latency by 20% compared to the previous approach.

Design and Construction of Distributed JavaScript Parsing System

2014 ◽  
Vol 6 (4) ◽  
pp. 1-14
Author(s):  
Bo Shen ◽  
Wei Huang ◽  
Xiaodi Li
Keyword(s):  
Search Engine ◽  
Distributed System ◽  
Task Scheduling ◽  
Information Acquisition ◽  
Scheduling Algorithm ◽  
Rapid Development ◽  
Internet Technology ◽  
The Internet ◽  
Computing Technology ◽  
Task Scheduling Algorithm

With the rapid development of the Internet technology, JS (short for JavaScript), as one of the representative of script languages, which is very powerful, is becoming more and more popular to the developers and users. But JS programming is more complex than usual static technology. In the field of search engine and information acquisition, it's very difficult to get the information hidden in script code. In this paper, the authors design a distributed system for parsing the JS code embedded in HTML file and retrieving the underling information. the authors describe how to extract JS codes from HTML file and parse them. Also, they introduce a task scheduling algorithm for the JS parsing system by employing Hadoop distributed computing technology. The experimental results indicate that the proposed algorithm and system can achieve a reasonable task scheduling efficiency and parse JS codes rapidly.

scholarly journals Resource Constrained Profit Optimization Method for Task Scheduling in Edge Cloud

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 118638-118652
Author(s):  
Liqiong Chen ◽  
Kun Guo ◽  
Guoqing Fan ◽  
Can Wang ◽  
Shilong Song
Keyword(s):  
Task Scheduling ◽  
Optimization Method ◽  
Resource Constrained ◽  
Profit Optimization

scholarly journals Effective task scheduling algorithm in cloud computing with quality of service alert bees and grey wolf optimization

2022 ◽  
Vol 25 (1) ◽  
pp. 550
Author(s):  
Nidhi Bansal ◽  
Ajay Kumar Singh
Keyword(s):  
Cloud Computing ◽  
Task Scheduling ◽  
Technology Development ◽  
Mathematical Proof ◽  
Scheduling Algorithm ◽  
Real Life ◽  
Optimization Method ◽  
Numerical Comparison ◽  
Scheduling Problems ◽  
Grey Wolf

Quality-based services are an indicative factor in providing a meaningful measure. These measures allow labeling for upcoming targets with a numerical comparison with a valid mathematical proof of reading and publications. By obtaining valid designs, organizations put this measure into the flow of technology development operations to generate higher profits. Since the conditions were met from the inception of cloud computing technology, the market was captured heavily by providing support through cloud computing. With the increase in the use of cloud computing, the complexity of data has also increased greatly. Applying natural theory to cloud technology makes it a fruit cream. Natural methods often come true, because survival depends on the live events and happenings, so using it in real life as well as any communication within technology will always be reliable. The numerical results are also showing a better value by comparing the optimization method. Finally, the paper introduces an adaptation theory with effective cloudsim coding of honey bees and grey wolf in conjunction with their natural life cycle for solving task scheduling problems. Using adapted bees improved the results by 50% compared with the original bees and secondly by honeybees and grey wolf improved 60%.

A New Smart Strategy for Web Searching of Commercial Products

2016 ◽  
Author(s):  
Lorenzo Cavalieri ◽  
Andrea Capitanelli ◽  
Silvia Ceccacci ◽  
Francesca Gullà ◽  
Alessandra Papetti ◽  
...  
Keyword(s):  
Search Engine ◽  
Search Strategy ◽  
Web Searching ◽  
Smart Object ◽  
Product Categories ◽  
Engine Efficiency ◽  
On Line ◽  
Essential Prerequisite ◽  
Average User

Search engine efficiency is an essential prerequisite to ensure a satisfactory on-line purchasing experience. Despite powerful tools available today, search engine is limited to a semantic elaboration of keywords and they do not allow users finding product categories that do not belong to their knowledge sphere. In this context, in order to make an effective search engine it is necessary to provide tools able to understand what the user is looking for and suggest the products that best satisfy their needs, regardless of users’ background. To this aim, this paper proposes an innovative smart search strategy, based on artificial intelligence technologies. In order to highlight the system potential, the smart object market case study has been considered. The SOs market is grown so quickly to disorient the average user and it offer a wide variety of products apparently similar, but that are characterized by different features that the average user fails to perceive.

Sign in / Sign up

Export Citation Format

Share Document