Timely Reporting of Heavy Hitters Using External Memory

Given an input stream S of size N , a ɸ-heavy hitter is an item that occurs at least ɸN times in S . The problem of finding heavy-hitters is extensively studied in the database literature. We study a real-time heavy-hitters variant in which an element must be reported shortly after we see its T = ɸ N-th occurrence (and hence it becomes a heavy hitter). We call this the Timely Event Detection ( TED ) Problem. The TED problem models the needs of many real-world monitoring systems, which demand accurate (i.e., no false negatives) and timely reporting of all events from large, high-speed streams with a low reporting threshold (high sensitivity). Like the classic heavy-hitters problem, solving the TED problem without false-positives requires large space (Ω (N) words). Thus in-RAM heavy-hitters algorithms typically sacrifice accuracy (i.e., allow false positives), sensitivity, or timeliness (i.e., use multiple passes). We show how to adapt heavy-hitters algorithms to external memory to solve the TED problem on large high-speed streams while guaranteeing accuracy, sensitivity, and timeliness. Our data structures are limited only by I/O-bandwidth (not latency) and support a tunable tradeoff between reporting delay and I/O overhead. With a small bounded reporting delay, our algorithms incur only a logarithmic I/O overhead. We implement and validate our data structures empirically using the Firehose streaming benchmark. Multi-threaded versions of our structures can scale to process 11M observations per second before becoming CPU bound. In comparison, a naive adaptation of the standard heavy-hitters algorithm to external memory would be limited by the storage device’s random I/O throughput, i.e., ≈100K observations per second.

Data-Driven Multi-Asset Optimisation Under Uncertainty: A Case Study Using the New Indonesia's Fiscal Policy

Optimizing multiple assets under uncertain techno-economic conditions and tight government policies is challenging. Operator needs to establish flexible Plan of Development (POD)s and put priority in developing multiple fields. The complexity of production and the profit margin should be simultaneously evaluated. In this work, we present a new workflow to perform such a rigorous optimization under uncertainty using the case study of PHE ONWJ, Indonesia. We begin the workflow by identifying the uncertain parameters and their prior distributions. We classify the parameters into three main groups: operations-related (geological complexity, reserves, current recovery, surface facilities, and technologies), company-policies-related (future exploration plan, margin of profit, and oil/gas price), and government-related (taxes, incentives, and fiscal policies). A unique indexing technique is developed to allow numerical quantification and adapt with dynamic input. We then start the optimization process by constructing time-dependent surrogate model through training with Monte Carlo sampling. We then perform optimization under uncertainty with multiple scenarios. The objective function is the overall Net Present Value (NPV) obtained by developing multiple fields. This work emphasizes the importance of the use of time-dependent surrogate approach to account risk in the optimization process. The approach revises the prior distribution with narrow-variance distribution to make reliable decision. The Global Sensitivity Analysis (GSA) with Sobol decomposition on the posterior distribution and surrogate provides parameters’ ranking and list of heavy hitters. The first output from this workflow is the narrow-variance posterior distribution. This result helps to locate the sweet spots. By analyzing them, operator can address specific sectors, which are critical to the NPV. PHE ONWJ, as the biggest operator in Indonesia, has geologically scattered assets, therefore, this first output is essential. The second output is the list of heavy hitters from GSA. This list is a tool to cluster promising fields for future development and prioritize their development based on the impact towards NPV. Since all risks are carried by the operator under the current Gross Split Contract, this result is advantageous for decision-making process. We introduce a new approach to perform time-dependent, multi-asset optimization under uncertainty. This new workflow is impactful for operators to create robust decision after considering the associated risks.

Real-Time Quantification of Offshore Oil Spills and Environmental Damage During Blow-Out Accident

In the event of offshore oilfield blow-out, real-time quantification of both spilled volume, recovered oil and environmental damage is essential. It is due to costly recovery and restoration process. In order to develop a robust and accurate quantification, we need to consider numerous parameters, which are sometimes tricky to be identified and captured. In this work, we present a new modeling technique under uncertainty, which accommodates numerous parameters and interaction among them. We begin the model by identifying possible parameters that contributes to the process: grouped into (1) subsurface, (2) surface and (3) operations. Subsurface e.g. well and reservoir characteristics. Surface e.g. ocean, wind, soil. (3) Operations e.g. oil spill treatment blow-out rate, oil characteristics, reservoir characteristics, ocean current speed, meteorological aspects, soil properties, and oil-spill treatment (oil booms and skimmers). We assign prior distribution for each parameter based on available data to capture the uncertainties. Before progressing to uncertainty propagation, we construct objective response (amount of recovered oil) through mass conservation equation in data-driven and non-intrusive way, using design of experiment and regression-based method. We propagate uncertainties using Monte Carlo simulation approach, where the result is presented in a distribution form, summarized by P10, P50, and P90 values. This work shows how to robustly calculate the amount of recovered oil under uncertainty in the event of offshore blow out. There are several notable challenges within the approach: 1) determining the uncertainty range in blow-out rate in case of rupture occurs in the well, 2) obtaining data for wind and ocean current speed since there is an interplay between local and global climate, and 3) accuracy of capturing the shoreline geometry. Despite the challenges, the results are in-line with the physics and several recorded blow-out cases. Define what is blow out rate (important as has highest sensitivity). Through sensitivity analysis with Sobol decomposition (define this …), we can define the heavy hitters. These heavy hitters give us knowledge on which parameters should be aware of. In real-time quantification, this analysis can provide an insight on what treatment method should be performed to efficiently recover the spill. We also highlight about the sufficiency of the model to obtain several parameters’ range, for example blow-out rate. The model should at least capture the physics in high details and incorporate multiple scenarios. In the case of blow-out rate, we extensively model the well completion and consider leaking due to unprecedented fractures or crater formation around the wellbore. We introduce a new framework of modeling to perform real-time quantification of offshore oil spills. This framework allows inferring the causality of the process and illustrating the risk level.