scholarly journals Parallel Delay Multiply and Sum Algorithm for Microwave Medical Imaging Using Spark Big Data Framework

Algorithms ◽  
2021 ◽  
Vol 14 (5) ◽  
pp. 157
Author(s):  
Rahmat Ullah ◽  
Tughrul Arslan
Keyword(s):  
Big Data ◽  
High Performance ◽  
Programming Model ◽  
Low Cost ◽  
Microwave Imaging ◽  
Ultra Wideband ◽  
Reconstruction Process ◽  
Data Framework ◽  
Wideband Signals ◽  
Computational Part

Microwave imaging systems are currently being investigated for breast cancer, brain stroke and neurodegenerative disease detection due to their low cost, portable and wearable nature. At present, commonly used radar-based algorithms for microwave imaging are based on the delay and sum algorithm. These algorithms use ultra-wideband signals to reconstruct a 2D image of the targeted object or region. Delay multiply and sum is an extended version of the delay and sum algorithm. However, it is computationally expensive and time-consuming. In this paper, the delay multiply and sum algorithm is parallelised using a big data framework. The algorithm uses the Spark MapReduce programming model to improve its efficiency. The most computational part of the algorithm is pixel value calculation, where signals need to be multiplied in pairs and summed. The proposed algorithm broadcasts the input data and executes it in parallel in a distributed manner. The Spark-based parallel algorithm is compared with sequential and Python multiprocessing library implementation. The experimental results on both a standalone machine and a high-performance cluster show that Spark significantly accelerates the image reconstruction process without affecting its accuracy.

scholarly journals A high performance ultra-wideband low cost SMA-to-GCPW transition

2016 ◽  
Vol 13 (12) ◽  
pp. 20160290-20160290 ◽  
Author(s):  
Ding Xu ◽  
Zhengpeng Wang ◽  
Yi Wang ◽  
Jianhua Wu
Keyword(s):  
High Performance ◽  
Low Cost ◽  
Ultra Wideband

scholarly journals Task-based programming in COMPSs to converge from HPC to big data

2017 ◽  
Vol 32 (1) ◽  
pp. 45-60 ◽  
Author(s):  
Javier Conejero ◽  
Sandra Corella ◽  
Rosa M Badia ◽  
Jesus Labarta
Keyword(s):  
Big Data ◽  
High Performance ◽  
Programming Model ◽  
Good Alternative ◽  
Programming Models ◽  
Suitable Model ◽  
Advantages And Disadvantages ◽  
Big Data Applications ◽  
And Performance ◽  
The Right

Task-based programming has proven to be a suitable model for high-performance computing (HPC) applications. Different implementations have been good demonstrators of this fact and have promoted the acceptance of task-based programming in the OpenMP standard. Furthermore, in recent years, Apache Spark has gained wide popularity in business and research environments as a programming model for addressing emerging big data problems. COMP Superscalar (COMPSs) is a task-based environment that tackles distributed computing (including Clouds) and is a good alternative for a task-based programming model for big data applications. This article describes why we consider that task-based programming models are a good approach for big data applications. The article includes a comparison of Spark and COMPSs in terms of architecture, programming model, and performance. It focuses on the differences that both frameworks have in structural terms, on their programmability interface, and in terms of their efficiency by means of three widely known benchmarking kernels: Wordcount, Kmeans, and Terasort. These kernels enable the evaluation of the more important functionalities of both programming models and analyze different work flows and conditions. The main results achieved from this comparison are (1) COMPSs is able to extract the inherent parallelism from the user code with minimal coding effort as opposed to Spark, which requires the existing algorithms to be adapted and rewritten by explicitly using their predefined functions, (2) it is an improvement in terms of performance when compared with Spark, and (3) COMPSs has shown to scale better than Spark in most cases. Finally, we discuss the advantages and disadvantages of both frameworks, highlighting the differences that make them unique, thereby helping to choose the right framework for each particular objective.

scholarly journals Integrated High-Performance Platform for Fast Query Response in Big Data with Hive, Impala, and SparkSQL: A Performance Evaluation

2018 ◽  
Vol 8 (9) ◽  
pp. 1514 ◽  
Author(s):  
Bao Chang ◽  
Hsiu-Fen Tsai ◽  
Yun-Da Lee
Keyword(s):  
Big Data ◽  
Open Source Software ◽  
High Performance ◽  
Low Cost ◽  
Data Retrieval ◽  
Fast Response ◽  
Disk Cache ◽  
Memory Cache ◽  
Medium Enterprise ◽  
Effectiveness And Efficiency

This paper first integrates big data tools—Hive, Impala, and SparkSQL—which support SQL-like queries for rapid data retrieval in big data. The three introduced tools are not only suitable for operating in business intelligence to serve high-performance data retrieval, but they are also an open-source software solution with low cost for small-to-medium enterprise use. In practice, the proposed approach provides an in-memory cache and an in-disk cache to achieve a very fast response to a query if a cache hit occurs. Moreover, this paper develops so-called platform selection that is able to select the appropriate tool dealing with input query with effectiveness and efficiency. As a result, the speed of job execution of proposed approach using platform selection is 2.63 times faster than Hive in the Case 1 experiment, and 4.57 times faster in the Case 2 experiment.

HAMR: A dataflow-based real-time in-memory cluster computing engine

2016 ◽  
Vol 31 (5) ◽  
pp. 361-374 ◽  
Author(s):  
Yao Wu ◽  
Long Zheng ◽  
Brian Heilig ◽  
Guang R Gao
Keyword(s):  
Big Data ◽  
Memory Management ◽  
High Performance ◽  
Cluster Computing ◽  
Programming Model ◽  
Distributed Processing ◽  
Large Data ◽  
Computing System ◽  
Fine Grain ◽  
Execution Model

As the attention given to big data grows, cluster computing systems for distributed processing of large data sets become the mainstream and critical requirement in high performance distributed system research. One of the most successful systems is Hadoop, which uses MapReduce as a programming/execution model and takes disks as intermedia to process huge volumes of data. Spark, as an in-memory computing engine, can solve the iterative and interactive problems more efficiently. However, currently it is a consensus that they are not the final solutions to big data due to a MapReduce-like programming model, synchronous execution model and the constraint that only supports batch processing, and so on. A new solution, especially, a fundamental evolution is needed to bring big data solutions into a new era. In this paper, we introduce a new cluster computing system called HAMR which supports both batch and streaming processing. To achieve better performance, HAMR integrates high performance computing approaches, i.e. dataflow fundamental into a big data solution. With more specifications, HAMR is fully designed based on in-memory computing to reduce the unnecessary disk access overhead; task scheduling and memory management are in fine-grain manner to explore more parallelism; asynchronous execution improves efficiency of computation resource usage, and also makes workload balance across the whole cluster better. The experimental results show that HAMR can outperform Hadoop MapReduce and Spark by up to 19x and 7x respectively, in the same cluster environment. Furthermore, HAMR can handle scaling data size well beyond the capabilities of Spark.

scholarly journals A Big Data Framework for Quality Assurance and Validation

2019 ◽  
Vol 8 (2) ◽  
pp. 2490-2494
Keyword(s):  
Health Care ◽  
Big Data ◽  
Relational Databases ◽  
Programming Model ◽  
New Technology ◽  
Large Data ◽  
Quality Data ◽  
Post Processing ◽  
Data Framework ◽  
User Data

Big data is a new technology, which is defined by large amount of data, so it is possible to extract value from the capturing and analysis process. Large data faced many challenges due to various features such as volume, speed, variation, value, complexity and performance. Many organizations face challenges while facing test strategies for structured and unstructured data validation, establishing a proper testing environment, working with non relational databases and maintaining functional testing. These challenges have low quality data in production, delay in execution and increase in cost. Reduce the map for data intensive business and scientific applications Provides parallel and scalable programming model. To get the performance of big data applications, defined as response time, maximum online user data capacity size, and a certain maximum processing capacity. In proposed, to test the health care big data . In health care data contains text file, image file, audio file and video file. To test the big data document, by using two concepts such as big data preprocessing testing and post processing testing. To classify the data from unstructured format to structured format using SVM algorithm. In preprocessing testing test all the data, for the purpose data accuracy. In preprocessing testing such as file size testing, file extension testing and de-duplication testing. In Post Processing to implement the map reduce concept for the use of easily to fetch the data.

scholarly journals Upgrading a high performance computing environment for massive data processing

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Lucas M. Ponce ◽  
Walter dos Santos ◽  
Wagner Meira ◽  
Dorgival Guedes ◽  
Daniele Lezzi ◽  
...  
Keyword(s):  
Big Data ◽  
Data Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Data Access ◽  
Massive Data ◽  
Analysis Tool ◽  
Data Framework ◽  
Performance Computing ◽  
Massive Data Processing

Abstract High-performance computing (HPC) and massive data processing (Big Data) are two trends that are beginning to converge. In that process, aspects of hardware architectures, systems support and programming paradigms are being revisited from both perspectives. This paper presents our experience on this path of convergence with the proposal of a framework that addresses some of the programming issues derived from such integration. Our contribution is the development of an integrated environment that integretes (i) COMPSs, a programming framework for the development and execution of parallel applications for distributed infrastructures; (ii) Lemonade, a data mining and analysis tool; and (iii) HDFS, the most widely used distributed file system for Big Data systems. To validate our framework, we used Lemonade to create COMPSs applications that access data through HDFS, and compared them with equivalent applications built with Spark, a popular Big Data framework. The results show that the HDFS integration benefits COMPSs by simplifying data access and by rearranging data transfer, reducing execution time. The integration with Lemonade facilitates COMPSs’s use and may help its popularization in the Data Science community, by providing efficient algorithm implementations for experts from the data domain that want to develop applications with a higher level abstraction.

scholarly journals A Low Cost and Portable Microwave Imaging System for Breast Tumor Detection Using UWB Directional Antenna array

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
M. T. Islam ◽  
M. Z. Mahmud ◽  
M. Tarikul Islam ◽  
S. Kibria ◽  
M. Samsuzzaman
Keyword(s):  
Dielectric Properties ◽  
Breast Tumor ◽  
Breast Imaging ◽  
Imaging System ◽  
Low Cost ◽  
Microwave Imaging ◽  
Dielectric Constants ◽  
Ultra Wideband ◽  
Slot Antenna ◽  
Breast Phantom

Abstract Globally, breast cancer is a major reason for female mortality. Due to the limitations of current clinical imaging, the researchers are encouraged to explore alternative and complementary tools to available techniques to detect the breast tumor in an earlier stage. This article outlines a new, portable, and low-cost microwave imaging (MWI) system using an iterative enhancing technique for breast imaging. A compact side slotted tapered slot antenna is designed for microwave imaging. The radiating fins of tapered slot antenna are modified by etching nine rectangular side slots. The irregular slots on the radiating fins enhance the electrical length as well as produce strong directive radiation due to the suppression of induced surface currents that radiate vertically at the outer edges of the radiating arms with end-fire direction. It has remarkable effects on efficiency and gain. With the addition of slots, the side-lobe levels are reduced, the gain of the main-lobe is increased and corrects the squint effects simultaneously, thus improving the characteristics of the radiation. For experimental validation, a heterogeneous breast phantom was developed that contains dielectric properties identical to real breast tissues with the inclusion of tumors. An alternative PC controlled and microcontroller-based mechanical MWI system is designed and developed to collect the antenna scattering signal. The radiated backscattered signals from the targeted area of the human body are analyzed to reveal the changes in dielectric properties in tissues. The dielectric constants of tumorous cells are higher than that of normal tissues due to their higher water content. The remarkable deviation of the scattered field is processed by using newly proposed Iteratively Corrected Delay and Sum (IC-DAS) algorithm and the reconstruction of the image of the phantom interior is done. The developed UWB (Ultra-Wideband) antenna based MWI has been able to perform the detection of tumorous cells in breast phantom that can pave the way to saving lives.

scholarly journals Iot and Big Data Framework for Paddy Cultivation

2019 ◽  
Vol 8 (2) ◽  
pp. 6341-6348
Keyword(s):  
Big Data ◽  
Data Acquisition ◽  
Low Cost ◽  
Medium Size ◽  
Large Area ◽  
Size Category ◽  
Data Framework ◽  
Effective Manner ◽  
Contextual Framework ◽  
Paddy Cultivation

Agriculture needs agriculturists to adopt digital in terms of low cost data acquisition from Soil, Weather and water related resources through drones, satellites, sensors and weather stations where in data sources are different, unstructured, volume and veracity of data generated is also huge which poses as a big data problem to solved. Agriculture is the backbone of India and being the largest paddy producer in the world. TamilNadu alone contributes at 7% of the overall paddy cultivation. The key aspect of TamilNadu paddy cultivation is 90% of its farmers belong to the small and medium size category. It's important for farmers who are producing paddy to be equipped with the technology advancements in a simple and effective manner to manage better way of irrigation in terms of water management, improving yield and efficient use of fertilizer and pesticides. Due to inherent nature of large area involved and complex eco system involved paddy cultivation have been always posing challenges in new technology adoption in terms of data acquisition, processing and reporting and lacks easy to follow contextual framework technologists to adopt. This paper would discuss on the infrastructure, technology and big data aspects for paddy cultivation by qualitative research methods. The research would involve identification of the appropriate contextual framework through architectural means and algorithm which would help in sensor deployment strategy. At the end the paper would develop a framework for approaching the IoT and Big Data in paddy cultivation. The framework would outline the architecture components, protocols, communication interfaces which could be leveraged for paddy cultivation. Apart from this the framework also discusses the Wireless sensor network deployment and its key aspect such as coverage in the paddy fields

Design and analysis of CPW-fed compact planar antennas for ultra wideband communication systems

2016 ◽  
Author(s):  
◽  
Haitham Alsaif
Keyword(s):  
Communication Systems ◽  
High Performance ◽  
Low Cost ◽  
Single Layer ◽  
Ultra Wideband ◽  
University Of Missouri ◽  
Uwb Antennas ◽  
Operating Frequency Range ◽  
Performance Results ◽  
Compact Planar

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] In this research, three new designs of planar compact ultra-wideband (UWB) antennas have been studied, simulated, and experimentally measured. Their structures are not complicated in design, easy in fabrication with low cost. They are in different physical sizes and considered small compared to many recent published UWB antennas that have similar performance. The proposed antennas have ultra-wide bandwidth that cover the entire bandwidth allocated by FCC for such applications. They are made to be planar structure with a single layer in order to be easier in fabrication and for use in wireless devices and applications. The used feeding technique is coplanar wave-guide (CPW) in all of them due to the great advantages of this feeding methodology. Each design has certain more superiority over the others either in terms of operating frequency range, power gain, radiation pattern, or structure size. Although, all compact patch antennas demonstrate high performance results and are very suitable for ultra-wideband systems. Finally, since there are a variety of ultra-wideband applications with several characteristics requirements, the research is composed of three different sizes of compact planar single layers antennas. These antennas have similar or better performance than some other large size designs, which makes it suitable for very compact wireless gadgets. Thus, the ultra-wideband (UWB) systems designer will be able to select the most appropriate design for the application based on the antenna characterizes and size.

Sign in / Sign up

Export Citation Format

Share Document