Design and Development of the Mass Image Storage Platform Based on Hadoop

2015 ◽  
Vol 742 ◽  
pp. 721-725
Author(s):  
Xiao Qing Zhou ◽  
Jia Xiu Sun ◽  
Xing Xian Luo
Keyword(s):  
Data Storage ◽  
High Efficiency ◽  
Image Data ◽  
The Internet ◽  
Image Storage ◽  
New Thought ◽  
Management Efficiency ◽  
Internal Storage ◽  
Fast Development ◽  
Storage Ability

With fast development and deep appliance of the Internet, problem of mass image data storage stand out, so the problem of low management efficiency, low storage ability and high cost of traditional storage framework has appeared. The appearance of Hadoop provides a new thought. However, Hadoop itself is not suit for the handle of small files. This paper puts forward a storage framework of mass image files based on Hadoop, and solved the internal storage bottleneck of NameNode when small files are excessive through classification algorithm of preprocessing module and lead-in of high efficiency and first-level of index mechanism. The test manifests that the system is safe, easy to defend and has fine extension quality; as a result, it can reach to a fine effect.

Application of an image management system in microscopy

1992 ◽  
Vol 50 (2) ◽  
pp. 1052-1053
Author(s):  
Richard S. Chemock
Keyword(s):  
Data Storage ◽  
Cost Savings ◽  
Image Data ◽  
Analytical Techniques ◽  
Operational Mode ◽  
Fine Grained ◽  
Image Recording ◽  
Primary Output ◽  
Capacity Data ◽  
Image Management System

One of the most common tasks in a typical analysis lab is the recording of images. Many analytical techniques (TEM, SEM, and metallography for example) produce images as their primary output. Until recently, the most common method of recording images was by using film. Current PS/2R systems offer very large capacity data storage devices and high resolution displays, making it practical to work with analytical images on PS/2s, thereby sidestepping the traditional film and darkroom steps. This change in operational mode offers many benefits: cost savings, throughput, archiving and searching capabilities as well as direct incorporation of the image data into reports.The conventional way to record images involves film, either sheet film (with its associated wet chemistry) for TEM or PolaroidR film for SEM and light microscopy. Although film is inconvenient, it does have the highest quality of all available image recording techniques. The fine grained film used for TEM has a resolution that would exceed a 4096x4096x16 bit digital image.

New Trends in Digital Data Storage for the Internet of Things

2018 ◽  
Vol 6 (3) ◽  
pp. 359-363
Author(s):  
A. Saxena ◽  
◽  
S. Sharma ◽  
S. Dangi ◽  
A. Sharma ◽  
...  
Keyword(s):  
Internet Of Things ◽  
Data Storage ◽  
Digital Data ◽  
The Internet ◽  
Digital Data Storage ◽  
The Internet Of Things

Design of FPGA-Based Missile-Borne Image Acquisition and Storage Recorder

2015 ◽  
Vol 713-715 ◽  
pp. 1448-1451
Author(s):  
Lin Lu ◽  
Yan Feng Zhang ◽  
Xiao Feng Li
Keyword(s):  
Real Time ◽  
Data Storage ◽  
Storage System ◽  
Large Data ◽  
Engineering Practice ◽  
Image Storage ◽  
Video Images ◽  
Stable Performance ◽  
And Storage ◽  
Time Image

The high-altitude missile and other special application occasions have requirements on image storage system, such as small size, high storage speed, low temperature resistance, etc. Commonly used image storage system in the market cannot meet such requirement. In the paper, real-time image storage system solutions on missile based on FPGA should be proposed. The system mainly consists of acquisition module and memory reading module. The whole system adopts FPGA as main control chip for mainly completing real-time decoding and acquisition on one path of PAL format video images, reading and writing of NandFlash chipset, erasure, bad block management and so on. The solution has passed various environmental tests with stable performance, large data storage capacity and easy expansion, which has been used in engineering practice.

D-buffer: irregular image data storage made practical

2013 ◽  
Vol 21 (1) ◽  
Author(s):  
J. Lipowski
Keyword(s):  
Data Storage ◽  
Image Data ◽  
Development Aid ◽  
Graphics Hardware ◽  
Address Space ◽  
Output Data ◽  
Frame Buffer ◽  
New Methods ◽  
Hardware Architectures ◽  
Depth Buffer

AbstractModern hardware accelerated graphics pipelines are designed to operate on data in a so called streaming model. To process the data in this model one needs to impose some restrictions on input and output argument’s (most frequently represented by a two-dimensional frame buffer) memory structure. The output data regularity is obvious when we consider rasterizing hardware architecture, which draws 3D polygons using depth buffer to resolve the visible surface problem. But recently the user’s needs surpass those restrictions with increasing frequency. In this work we formulate and present new methods of irregular frame buffer storage and ordering. The so called deque buffer (or D-buffer) allows us to decrease the amount of memory used for storage as well as the memory latency cost by using pixel data ordering. Our findings are confirmed by experimental results that measure the processing time, which is up to four times shorter, when compared with previous work by other authors. We also include a detailed description of algorithms used for D-buffer construction on the last three consumer-grade graphics hardware architectures, as a guide for other researchers and a development aid for practitioners. The only theoretical requirement imposed by our method is the use of memory model with linear address space.

scholarly journals Realtime Data Traffic Analyser Locomotive of Big Data Analytics

2020 ◽  
Vol 8 (6) ◽  
pp. 5643-5646
Keyword(s):  
Communication Networks ◽  
Data Storage ◽  
Communication Systems ◽  
Mean Field ◽  
Big Data Analytics ◽  
The Internet ◽  
Data Traffic ◽  
Time Data ◽  
Storage Security ◽  
Internet Users

Since last decade, the exponential growth of the internet users and the size of data over the internet is increasing day by day, which lead to increase the complexity of the systems by implementing policies and security to avoid attacks on systems and networks. It is very important to understand and analyses the real time data traffic of the communication systems. The purpose of this paper to design a customized Java based application which enables analysts to capture the traffic at the bottleneck under the mean field communication environment where a large number of devices are communicating with each other. The sending data for further processing for analysis the trend to overcome vulnerabilities or to manage the effectiveness of the communication systems. The proposed application enables to capture 8 different types of protocol traffic such as HTTP, HTTPS, SMTP, UDP, TCP, ICMP and POP3. The application allows for analysis of the incoming/outgoing traffic in the visual to understand the nature of communication networks which lead to improve the performance of the networks with respect to hardware, software, data storage, security and reliability.

scholarly journals Payload Configurations for Efficient Image Acquisition – Indian Perspective

2014 ◽  
Vol XL-8 ◽  
pp. 1201-1206 ◽  
Author(s):  
D. R. M. Samudraiah ◽  
M. Saxena ◽  
S. Paul ◽  
P. Narayanababu ◽  
S. Kuriakose ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Data Acquisition ◽  
High Efficiency ◽  
Image Acquisition ◽  
Image Data ◽  
Space Research ◽  
Remotely Sensed Data ◽  
Data Utilization ◽  
Spectral Bands ◽  
The World

The world is increasingly depending on remotely sensed data. The data is regularly used for monitoring the earth resources and also for solving problems of the world like disasters, climate degradation, etc. Remotely sensed data has changed our perspective of understanding of other planets. With innovative approaches in data utilization, the demands of remote sensing data are ever increasing. More and more research and developments are taken up for data utilization. The satellite resources are scarce and each launch costs heavily. Each launch is also associated with large effort for developing the hardware prior to launch. It is also associated with large number of software elements and mathematical algorithms post-launch. The proliferation of low-earth and geostationary satellites has led to increased scarcity in the available orbital slots for the newer satellites. Indian Space Research Organization has always tried to maximize the utility of satellites. Multiple sensors are flown on each satellite. In each of the satellites, sensors are designed to cater to various spectral bands/frequencies, spatial and temporal resolutions. Bhaskara-1, the first experimental satellite started with 2 bands in electro-optical spectrum and 3 bands in microwave spectrum. The recent Resourcesat-2 incorporates very efficient image acquisition approach with multi-resolution (3 types of spatial resolution) multi-band (4 spectral bands) electro-optical sensors (LISS-4, LISS-3* and AWiFS). The system has been designed to provide data globally with various data reception stations and onboard data storage capabilities. Oceansat-2 satellite has unique sensor combination with 8 band electro-optical high sensitive ocean colour monitor (catering to ocean and land) along with Ku band scatterometer to acquire information on ocean winds. INSAT- 3D launched recently provides high resolution 6 band image data in visible, short-wave, mid-wave and long-wave infrared spectrum. It also has 19 band sounder for providing vertical profile of water vapour, temperature, etc. The same system has data relay transponders for acquiring data from weather stations. The payload configurations have gone through significant changes over the years to increase data rate per kilogram of payload. Future Indian remote sensing systems are planned with very high efficient ways of image acquisition. <br><br> This paper analyses the strides taken by ISRO (Indian Space research Organisation) in achieving high efficiency in remote sensing image data acquisition. Parameters related to efficiency of image data acquisition are defined and a methodology is worked out to compute the same. Some of the Indian payloads are analysed with respect to some of the system/ subsystem parameters that decide the configuration of payload. Based on the analysis, possible configuration approaches that can provide high efficiency are identified. A case study is carried out with improved configuration and the results of efficiency improvements are reported. This methodology may be used for assessing other electro-optical payloads or missions and can be extended to other types of payloads and missions.

scholarly journals The Role of Blockchain in Cyber Security

2021 ◽  
Vol 3 (1) ◽  
Author(s):  
Dylan Rafferty ◽  
Kevin Curran
Keyword(s):  
Data Storage ◽  
Cyber Security ◽  
Board Members ◽  
The Internet ◽  
Network Architectures ◽  
General Data Protection Regulation ◽  
Security Breaches ◽  
General Data ◽  
The Internet Of Things

Cyber security breaches are on the rise globally. Due to the introduction of legislation like the EU’s General Data Protection Regulation (GDPR), companies are now subject to further financial penalties if they fail to meet requirements in protecting user information.  In 2018, 75% of CEOs and board members considered cyber security and technology acquisitions among their top priorities, and blockchain based solutions were among the most considered options. Blockchain is a decentralised structure that offers multiple security benefits over traditional, centralised network architectures. These two approaches are compared in this chapter in areas such as data storage, the Internet of Things (IoT) and Domain Name System (DNS) in order to determine blockchain’s potential in the future of cyber security.

scholarly journals Analisa Real-Time Data log honeypot menggunakan Algoritma K-Means pada serangan Distributed Denial of Service

Repositor ◽  
2020 ◽  
Vol 2 (5) ◽  
pp. 541
Author(s):  
Denni Septian Hermawan ◽  
Syaifuddin Syaifuddin ◽  
Diah Risqiwati
Keyword(s):  
Real Time ◽  
Data Storage ◽  
Denial Of Service ◽  
The Internet ◽  
Distributed Denial Of Service ◽  
Time Data ◽  
Local Networks ◽  
Real Time Data ◽  
Internet Networks ◽  
Data Log

AbstrakJaringan internet yang saat ini di gunakan untuk penyimpanan data atau halaman informasi pada website menjadi rentan terhadap serangan, untuk meninkatkan keamanan website dan jaringannya, di butuhkan honeypot yang mampu menangkap serangan yang di lakukan pada jaringan lokal dan internet. Untuk memudahkan administrator mengatasi serangan digunakanlah pengelompokan serangan dengan metode K-Means untuk mengambil ip penyerang. Pembagian kelompok pada titik cluster akan menghasilkan output ip penyerang.serangan di ambil sercara realtime dari log yang di miliki honeypot dengan memanfaatkan MHN.Abstract The number of internet networks used for data storage or information pages on the website is vulnerable to attacks, to secure the security of their websites and networks, requiring honeypots that are capable of capturing attacks on local networks and the internet. To make it easier for administrators to tackle attacks in the use of attacking groupings with the K-Means method to retrieve the attacker ip. Group divisions at the cluster point will generate the ip output of the attacker. The strike is taken as realtime from the logs that have honeypot by utilizing the MHN.

scholarly journals TINJAUAN KETERSEDIAAN INFRASTRUKTUR TERKAIT IMPLEMENTASI KEBIJAKAN PENYERAHAN PEMBERITAHUAN PABEAN MELALUI INTERNET

2019 ◽  
Vol 12 (2) ◽  
pp. 44-58
Author(s):  
Agung Tri Safari
Keyword(s):  
Data Storage ◽  
Data Exchange ◽  
Secondary Data ◽  
Qualitative Approach ◽  
The Internet ◽  
Exchange System ◽  
Electronic Data ◽  
Full Implementation ◽  
Storage Media ◽  
Internet Network

Submission of Customs Declaration including matters regulated in the Customs Law. Furthermore, this matter is further elaborated based on the Minister of Finance Regulation. Customs Declaration can be submitted in writing on the form or in the form of Electronic Data. For those in the form of Electronic Data, it is delivered by submitting Electronic Data storage media in the form of diskettes or etc  to the Customs Office or through an Electronic Data exchange system. DJBC has issued a policy related to the full implementation (Mandatory) of the submission of customs declaration documents through the PDE system based on KEP-415 / BC / 2016 and began to be implemented effectively on January 1, 2019. The purpose of this study is to evaluate whether this policy can be applied in all regions of Indonesia This research is descriptive with a qualitative approach. Based on secondary data, not all regions of Indonesia are affordable to the internet network. The constraints on the availability of infrastructure are a necessity that must be anticipated. The results of this discussion can be input into the improvement of related policies.

Sign in / Sign up

Export Citation Format

Share Document