Cost to serve of large scale online systems

2016 ◽  
Author(s):  
Andres Paz Sampedro ◽  
Shantanu Srivastava
Keyword(s):  
Large Scale ◽  
Online Systems ◽  
Cost To Serve

scholarly journals Efficient Heterogeneous Collaborative Filtering without Negative Sampling for Recommendation

2020 ◽  
Vol 34 (01) ◽  
pp. 19-26 ◽  
Author(s):  
Chong Chen ◽  
Min Zhang ◽  
Yongfeng Zhang ◽  
Weizhi Ma ◽  
Yiqun Liu ◽  
...  
Keyword(s):  
Collaborative Filtering ◽  
Real World ◽  
Large Scale ◽  
State Of The Art ◽  
Heterogeneous Data ◽  
Model Parameters ◽  
Online Systems ◽  
Practical Applications ◽  
Real World Datasets ◽  
Primary Type

Recent studies on recommendation have largely focused on exploring state-of-the-art neural networks to improve the expressiveness of models, while typically apply the Negative Sampling (NS) strategy for efficient learning. Despite effectiveness, two important issues have not been well-considered in existing methods: 1) NS suffers from dramatic fluctuation, making sampling-based methods difficult to achieve the optimal ranking performance in practical applications; 2) although heterogeneous feedback (e.g., view, click, and purchase) is widespread in many online systems, most existing methods leverage only one primary type of user feedback such as purchase. In this work, we propose a novel non-sampling transfer learning solution, named Efficient Heterogeneous Collaborative Filtering (EHCF) for Top-N recommendation. It can not only model fine-grained user-item relations, but also efficiently learn model parameters from the whole heterogeneous data (including all unlabeled data) with a rather low time complexity. Extensive experiments on three real-world datasets show that EHCF significantly outperforms state-of-the-art recommendation methods in both traditional (single-behavior) and heterogeneous scenarios. Moreover, EHCF shows significant improvements in training efficiency, making it more applicable to real-world large-scale systems. Our implementation has been released 1 to facilitate further developments on efficient whole-data based neural methods.

IR Research and Innovation in Commercial Online Systems: An Exploratory Survey

2013 ◽  
Author(s):  
Sherry Koshman ◽  
Edie Rasmussen
Keyword(s):  
Information Retrieval ◽  
Large Scale ◽  
Interactive Systems ◽  
Information Industry ◽  
Online Systems ◽  
Research And Innovation ◽  
Retrieval Systems ◽  
Information Retrieval Systems ◽  
Operational Systems ◽  
Mcgill University

From the 1994 CAIS Conference: The Information Industry in Transition McGill University, Montreal, Quebec. May 25 - 27, 1994."Conventional" information retrieval systems (IRS), originating in the research of the 11950s and 1960s, are based on keyword matching and the application of Boolean operators to produce a set of retrieved documents from a database. In the ensuing years, research in information retrieval has identified a number of innovations (for example, automatic weighting of terms, ranked output, and relevance feedback) which have the potential to significantly enhance the performance of IRS, though commercial vendors have been slow to incorporate these changes into their systems. This was the situation in 1988 which led Radecki, in a special issue of Information Processing & Management, to examine the potential for improvements in conventional Boolean retrieval systems, and explore the reasons why these improvements had not been implemented in operational systems. Over the last five years, this position has begun to change as commercial vendors such as Dialog, Dow Jones, West Publishing, and Mead have implemented new, non-Boolean features in their systems, including natural language input, weighted keyword terms, and document ranking. This paper identifies some of the significant findings of IR research and compares them to the implementation of non-Boolean features in such systems. The preliminary survey of new features in commercial systems suggests the need for new methods of evaluation, including the development of evalutation measures appropriate to large-scale, interactive systems.

scholarly journals A Scalable and Hybrid Intrusion Detection System Based on the Convolutional-LSTM Network

Symmetry ◽  
2019 ◽  
Vol 11 (4) ◽  
pp. 583 ◽  
Author(s):  
Muhammad Ashfaq Khan ◽  
Md. Rezaul Karim ◽  
Yangwoo Kim
Keyword(s):  
Intrusion Detection ◽  
Information And Communication Technologies ◽  
Intrusion Detection System ◽  
Large Scale ◽  
Detection System ◽  
Security Threats ◽  
Online Systems ◽  
Network Intrusion ◽  
Lstm Network ◽  
Convolutional Lstm

With the rapid advancements of ubiquitous information and communication technologies, a large number of trustworthy online systems and services have been deployed. However, cybersecurity threats are still mounting. An intrusion detection (ID) system can play a significant role in detecting such security threats. Thus, developing an intelligent and accurate ID system is a non-trivial research problem. Existing ID systems that are typically used in traditional network intrusion detection system often fail and cannot detect many known and new security threats, largely because those approaches are based on classical machine learning methods that provide less focus on accurate feature selection and classification. Consequently, many known signatures from the attack traffic remain unidentifiable and become latent. Furthermore, since a massive network infrastructure can produce large-scale data, these approaches often fail to handle them flexibly, hence are not scalable. To address these issues and improve the accuracy and scalability, we propose a scalable and hybrid IDS, which is based on Spark ML and the convolutional-LSTM (Conv-LSTM) network. This IDS is a two-stage ID system: the first stage employs the anomaly detection module, which is based on Spark ML. The second stage acts as a misuse detection module, which is based on the Conv-LSTM network, such that both global and local latent threat signatures can be addressed. Evaluations of several baseline models in the ISCX-UNB dataset show that our hybrid IDS can identify network misuses accurately in 97.29% of cases and outperforms state-of-the-art approaches during 10-fold cross-validation tests.

New opportunities from information technology

1980 ◽  
Vol 2 (1) ◽  
pp. 29-36 ◽  
Author(s):  
P.W. Williams
Keyword(s):  
Information Technology ◽  
Information Systems ◽  
Large Scale ◽  
Personal Information ◽  
Post Office ◽  
Computing Systems ◽  
Online Systems ◽  
Medium Scale

Rapid developments in computing, telecommunications and microtechnology are expanding the ways in which information can be obtained. The online systems are now firmly established and the Post Office Prestel service is com mencing operations. There are several packages available for large-scale and medium-scale in-house computing systems and the advent of microcomputers means that complete informa tion systems for well under £ 10 000 will soon be available. Personal information systems can already be found for a few thousand pounds. The opportunities for the smaller informa tion unit are surveyed and the costs and difficulties of the possible options are reviewed.

Knowledge infrastructures for the Anthropocene

2016 ◽  
Vol 4 (1) ◽  
pp. 34-43 ◽  
Author(s):  
Paul N Edwards
Keyword(s):  
Climate Change ◽  
Large Scale ◽  
Practical Knowledge ◽  
Close Monitoring ◽  
Energy Materials ◽  
Intergovernmental Panel ◽  
Online Systems ◽  
Geophysical Processes ◽  
History Of

The technosphere metabolizes not only energy and materials, but information and knowledge as well. This article first examines the history of knowledge about large-scale, long-term, anthropogenic environmental change. In the 19th and 20th centuries, major systems were built for monitoring both the environment and human activity of all kinds, for modeling geophysical processes such as climate change, and for preserving and refining scientific memory, i.e. data about the planetary past. Despite many failures, these knowledge infrastructures also helped achieve notable successes such as the Limited Test Ban Treaty of 1963, the ozone depletion accords of the 1980s, and the Paris Agreement on climate change of 2015. The article’s second part proposes that knowledge infrastructures for the Anthropocene might not only monitor and model the technosphere’s metabolism of energy, materials and information, but also integrate those techniques with new accounting practices aimed at sustainability. Scientific examples include remarkable recent work on long-term socio-ecological research, and the assessment reports of the Intergovernmental Panel on Climate Change. In terms of practical knowledge, one key to effective accounting may be ‘recycling’ of the vast amounts of ‘waste’ data created by virtually all online systems today. Examples include dramatic environmental efficiency gains by Ikea and United Parcel Service, through improved logistics, self-provision of renewable energy, and feedback from close monitoring of delivery trucks. Blending social ‘data exhaust’ with physical and environmental information, an environmentally focused logistics might trim away excess energy and materials in production, find new ways to re-use or recycle waste, and generate new ideas for eliminating toxic byproducts, greenhouse gas emissions and other metabolites.

Some comments about observations and image processing of comet 29P/Schwassmann-Wachmann 1

1999 ◽  
Vol 173 ◽  
pp. 243-248
Author(s):  
D. Kubáček ◽  
A. Galád ◽  
A. Pravda
Keyword(s):  
Image Processing ◽  
Large Scale ◽  
Harvard College ◽  
Oak Ridge ◽  
Large Scale Structures ◽  
Short Period Comet ◽  
Short Period ◽  
Scale Structures ◽  
Harvard College Observatory ◽  
Period Comet

AbstractUnusual short-period comet 29P/Schwassmann-Wachmann 1 inspired many observers to explain its unpredictable outbursts. In this paper large scale structures and features from the inner part of the coma in time periods around outbursts are studied. CCD images were taken at Whipple Observatory, Mt. Hopkins, in 1989 and at Astronomical Observatory, Modra, from 1995 to 1998. Photographic plates of the comet were taken at Harvard College Observatory, Oak Ridge, from 1974 to 1982. The latter were digitized at first to apply the same techniques of image processing for optimizing the visibility of features in the coma during outbursts. Outbursts and coma structures show various shapes.

Evolution of Large-Scale Coronal Structures

1994 ◽  
Vol 144 ◽  
pp. 29-33
Author(s):  
P. Ambrož
Keyword(s):  
Magnetic Field ◽  
Field Line ◽  
Large Scale ◽  
Coronal Magnetic Field ◽  
Source Surface ◽  
Solar Cycles ◽  
Characteristic Relationship ◽  
The Magnetic Field ◽  
Coronal Structures

AbstractThe large-scale coronal structures observed during the sporadically visible solar eclipses were compared with the numerically extrapolated field-line structures of coronal magnetic field. A characteristic relationship between the observed structures of coronal plasma and the magnetic field line configurations was determined. The long-term evolution of large scale coronal structures inferred from photospheric magnetic observations in the course of 11- and 22-year solar cycles is described.Some known parameters, such as the source surface radius, or coronal rotation rate are discussed and actually interpreted. A relation between the large-scale photospheric magnetic field evolution and the coronal structure rearrangement is demonstrated.

Large-scale Motion of Solar Filaments

2000 ◽  
Vol 179 ◽  
pp. 205-208
Author(s):  
Pavel Ambrož ◽  
Alfred Schroll
Keyword(s):  
Magnetic Flux ◽  
Proper Motion ◽  
Time Scale ◽  
Large Scale ◽  
Accuracy Of Measurements ◽  
Solar Filaments ◽  
General Movement ◽  
Scale Motion ◽  
Velocity Scatter

AbstractPrecise measurements of heliographic position of solar filaments were used for determination of the proper motion of solar filaments on the time-scale of days. The filaments have a tendency to make a shaking or waving of the external structure and to make a general movement of whole filament body, coinciding with the transport of the magnetic flux in the photosphere. The velocity scatter of individual measured points is about one order higher than the accuracy of measurements.

Some applications of electron beam microanalysis techniques in VLSI process evaluation

1983 ◽  
Vol 41 ◽  
pp. 160-161
Author(s):  
Simon Thomas
Keyword(s):  
Integrated Circuits ◽  
Process Evaluation ◽  
Large Scale ◽  
Technology Development ◽  
Analytical Techniques ◽  
Successful Implementation ◽  
Analysis Of Failures ◽  
Characterization Of Materials ◽  
Circuits Vlsi

Trends in the technology development of very large scale integrated circuits (VLSI) have been in the direction of higher density of components with smaller dimensions. The scaling down of device dimensions has been not only laterally but also in depth. Such efforts in miniaturization bring with them new developments in materials and processing. Successful implementation of these efforts is, to a large extent, dependent on the proper understanding of the material properties, process technologies and reliability issues, through adequate analytical studies. The analytical instrumentation technology has, fortunately, kept pace with the basic requirements of devices with lateral dimensions in the micron/ submicron range and depths of the order of nonometers. Often, newer analytical techniques have emerged or the more conventional techniques have been adapted to meet the more stringent requirements. As such, a variety of analytical techniques are available today to aid an analyst in the efforts of VLSI process evaluation. Generally such analytical efforts are divided into the characterization of materials, evaluation of processing steps and the analysis of failures.

Sign in / Sign up

Export Citation Format

Share Document