Graph Convolutional Networks for Cross-Modal Information Retrieval

In recent years, due to the wide application of deep learning and more modal research, the corresponding image retrieval system has gradually extended from traditional text retrieval to visual retrieval combined with images and has become the field of computer vision and natural language understanding and one of the important cross-research hotspots. This paper focuses on the research of graph convolutional networks for cross-modal information retrieval and has a general understanding of cross-modal information retrieval and the related theories of convolutional networks on the basis of literature data. Modal information retrieval is designed to combine high-level semantics with low-level visual capabilities in cross-modal information retrieval to improve the accuracy of information retrieval and then use experiments to verify the designed network model, and the result is that the model designed in this paper is more accurate than the traditional retrieval model, which is up to 90%.

COLOR IMAGE RETRIEVAL BASED ON FUZZY NEURAL NETWORK AND SWARM INTELLIGENCE TECHNIQUES

Image retrieval is an important system for retrieving similar images by searching and browsing in a large database. The image retrieval system can be a reliable tool for people to optimize the use of image accumulation, and finding efficient methods to retrieve images is very important. Recent decades have marked increased research interest in field image retrieval. To retrieve the images, an important set of features is used. In this work, a combination of methods was used to examine all the images and detect images in a database according to a query image. Linear Discriminant Analysis (LDA) was used for feature extraction of the images into the dataset. The images in the database were processed by extracting their important and robust features and storing them in the feature store. Likewise, the strong features were extracted for specific query images. By using some Meta Heuristic algorithms such as Cuckoo Search (CS), Ant Colony Optimization (ACO), and using an artificial neural network such as single-layer Perceptron Neural Network (PNN), similarity was evaluated. It also proposed a new two method by hybridized PNN and CS with fuzzy logic to produce a new method called Fuzzy Single Layer Perceptron Neural Network (FPNN), and Fuzzy Cuckoo Search to examine the similarity between features for query images and features for images in the database. The efficiency of the system methods was evaluated by calculating the precision recall value of the results. The proposed method of FCS outperformed other methods such as (PNN), (ACO), (CS), and (FPNN) in terms of precision and image recall. ABSTRAK: Imej dapatan semula adalah sistem penting bagi mendapatkan imej serupa melalui carian imej dan melayari pangkalan besar data. Sistem dapatan semula imej ini boleh dijadikan alat boleh percaya untuk orang mengoptimum penggunaan pengumpulan imej, dan kaedah pencarian yang berkesan bagi mendapatkan imej adalah sangat penting. Beberapa dekad yang lalu telah menunjukan banyak penyelidikan dalam bidang imej dapatan semula. Bagi mendapatkan imej-imej ini, ciri-ciri set penting telah digunakan. Kajian ini menggunakan beberapa kaedah bagi memeriksa semua imej dan mengesan imej dalam pangkalan data berdasarkan imej carian. Kami menggunakan Analisis Diskriminan Linear (LDA) bagi mengekstrak ciri imej ke dalam set data. Imej-imej dalam pangkalan data diproses dengan mengekstrak ciri-ciri penting dan berkesan daripadanya dan menyimpannya dalam simpanan ciri. Begitu juga, ciri-ciri penting ini diekstrak bagi imej carian tertentu. Persamaan dinilai melalui beberapa algoritma Meta Heuristik seperti Carian Cuckoo (CS), Pengoptimuman Koloni Semut (ACO), dan menggunakan lapisan tunggal rangkaian neural buatan seperti Rangkaian Neural Perseptron (PNN). Dua cadangan baru dengan kombinasi hibrid PNN dan CS bersama logik kabur bagi menghasilkan kaedah baru yang disebut Lapisan Tunggal Kabur Rangkaian Neural Perceptron (FPNN), dan Carian Cuckoo Kabur bagi mengkaji persamaan antara ciri carian imej dan imej pangkalan data. Nilai kecekapan kaedah sistem dinilai dengan mengira ketepatan mengingat pada dapatan hasil. Kaedah FCS yang dicadangkan ini mengatasi kaedah lain seperti (PNN), (ACO), (CS) dan (FPNN) dari segi ketepatan dan ingatan imej.

A Review on Content Based Image Retrieval System Features derived by Deep Learning Models

In a Content Based Image Retrieval (CBIR) System, the task is to retrieve similar images from a large database given a query image. The usual procedure is to extract some useful features from the query image, and retrieve images which have similar set of features. For this purpose, a suitable similarity measure is chosen, and images with high similarity scores are retrieved. Naturally the choice of these features play a very important role in the success of this system, and high level features are required to reduce the “semantic gap”. In this paper, we propose to use features derived from pre-trained network models from a deep- learning convolution network trained for a large image classification problem. This approach appears to produce vastly superior results for a variety of databases, and it outperforms many contemporary CBIR systems. We analyse the retrieval time of the method, and also propose a pre-clustering of the database based on the above-mentioned features which yields comparable results in a much shorter time in most of the cases. Keywords Content Based Image Retrieval Feature Selection Deep Learning Pre-trained Network Models Pre-clustering

A Novel Adaptive Feature Fusion Strategy for Image Retrieval

In the era of big data, it is challenging to efficiently retrieve the required images from the vast amount of data. Therefore, a content-based image retrieval system is an important research direction to address this problem. Furthermore, a multi-feature-based image retrieval system can compensate for the shortage of a single feature to a certain extent, which is essential for improving retrieval system performance. Feature selection and feature fusion strategies are critical in the study of multi-feature fusion image retrieval. This paper proposes a multi-feature fusion image retrieval strategy with adaptive features based on information entropy theory. Firstly, we extract the image features, construct the distance function to calculate the similarity using the information entropy proposed in this paper, and obtain the initial retrieval results. Then, we obtain the precision of single feature retrieval based on the correlation feedback as the retrieval trust and use the retrieval trust to select the effective features automatically. After that, we initialize the weights of selected features using the average weights, construct the probability transfer matrix, and use the PageRank algorithm to update the initialized feature weights to obtain the final weights. Finally, we calculate the comprehensive similarity based on the final weights and output the detection results. This has two advantages: (1) the proposed strategy uses multiple features for image retrieval, which has better performance and more substantial generalization than the retrieval strategy based on a single feature; (2) compared with the fixed-feature retrieval strategy, our method selects the best features for fusion in each query, which takes full advantages of each feature. The experimental results show that our proposed method outperforms other methods. In the datasets of Corel1k, UC Merced Land-Use, and RSSCN7, the top10 retrieval precision is 99.55%, 88.02%, and 88.28%, respectively. In the Holidays dataset, the mean average precision (mAP) was 92.46%.

Analysis of combined approaches of CBIR systems by clustering at varying precision levels

<span lang="EN-US">The image retrieving system is used to retrieve images from the image database. Two types of Image retrieval techniques are commonly used: content-based and text-based techniques. One of the well-known image retrieval techniques that extract the images in an unsupervised way, known as the cluster-based image retrieval technique. In this cluster-based image retrieval, all visual features of an image are combined to find a better retrieval rate and precisions. The objectives of the study were to develop a new model by combining the three traits i.e., color, shape, and texture of an image. The color-shape and color-texture models were compared to a threshold value with various precision levels. A union was formed of a newly developed model with a color-shape, and color-texture model to find the retrieval rate in terms of precisions of the image retrieval system. The results were experimented on on the COREL standard database and it was found that the union of three models gives better results than the image retrieval from the individual models. The newly developed model and the union of the given models also gives better results than the existing system named cluster-based retrieval of images by unsupervised learning (CLUE).</span>

Evaluation of a Novel Content-Based Image Retrieval System for the Differentiation of Interstitial Lung Diseases in CT Examinations

To evaluate the reader’s diagnostic performance against the ground truth with and without the help of a novel content-based image retrieval system (CBIR) that retrieves images with similar CT patterns from a database of 79 different interstitial lung diseases. We evaluated three novice readers’ and three resident physicians‘ (with at least three years of experience) diagnostic performance evaluating 50 different CTs featuring 10 different patterns (e.g., honeycombing, tree-in bud, ground glass, bronchiectasis, etc.) and 24 different diseases (sarcoidosis, UIP, NSIP, Aspergillosis, COVID-19 pneumonia etc.). The participants read the cases first without assistance (and without feedback regarding correctness), and with a 2-month interval in a random order with the assistance of the novel CBIR. To invoke the CBIR, a ROI is placed into the pathologic pattern by the reader and the system retrieves diseases with similar patterns. To further narrow the differential diagnosis, the readers can consult an integrated textbook and have the possibility of selecting high-level semantic features representing clinical information (chronic, infectious, smoking status, etc.). We analyzed readers’ accuracy without and with CBIR assistance and further tested the hypothesis that the CBIR would help to improve diagnostic performance utilizing Wilcoxon signed rank test. The novice readers demonstrated an unassisted accuracy of 18/28/44%, and an assisted accuracy of 84/82/90%, respectively. The resident physicians demonstrated an unassisted accuracy of 56/56/70%, and an assisted accuracy of 94/90/96%, respectively. For each reader, as well as overall, Sign test demonstrated statistically significant (p < 0.01) difference between the unassisted and the assisted reads. For students and physicians, Chi²-test and Mann-Whitney-U test demonstrated statistically significant (p < 0.01) difference for unassisted reads and statistically insignificant (p > 0.01) difference for assisted reads. The evaluated CBIR relying on pattern analysis and featuring the option to filter the results of the CBIR by predominant characteristics of the diseases via selecting high-level semantic features helped to drastically improve novices’ and resident physicians’ accuracy in diagnosing interstitial lung diseases in CT.