Erroneous pixel prediction for semantic image segmentation

We consider semantic image segmentation. Our method is inspired by Bayesian deep learning which improves image segmentation accuracy by modeling the uncertainty of the network output. In contrast to uncertainty, our method directly learns to predict the erroneous pixels of a segmentation network, which is modeled as a binary classification problem. It can speed up training comparing to the Monte Carlo integration often used in Bayesian deep learning. It also allows us to train a branch to correct the labels of erroneous pixels. Our method consists of three stages: (i) predict pixel-wise error probability of the initial result, (ii) redetermine new labels for pixels with high error probability, and (iii) fuse the initial result and the redetermined result with respect to the error probability. We formulate the error-pixel prediction problem as a classification task and employ an error-prediction branch in the network to predict pixel-wise error probabilities. We also introduce a detail branch to focus the training process on the erroneous pixels. We have experimentally validated our method on the Cityscapes and ADE20K datasets. Our model can be easily added to various advanced segmentation networks to improve their performance. Taking DeepLabv3+ as an example, our network can achieve 82.88% of mIoU on Cityscapes testing dataset and 45.73% on ADE20K validation dataset, improving corresponding DeepLabv3+ results by 0.74% and 0.13% respectively.

Vit-GAN: Image-to-image Translation with Vision Transformes and Conditional GANS

In this paper, we have developed a general-purpose architecture, Vit-Gan, capable of performing most of the image-to-image translation tasks from semantic image segmentation to single image depth perception. This paper is a follow-up paper, an extension of generator based model [1] in which the obtained results were very promising. This opened the possibility of further improvements with adversarial architecture. We used a unique vision transformers-based generator architecture and Conditional GANs(cGANs) with a Markovian Discriminator (PatchGAN) (https://github.com/YigitGunduc/vit-gan). In the present work, we use images as conditioning arguments. It is observed that the obtained results are more realistic than the commonly used architectures.

Vit-GAN: Image-to-image Translation with Vision Transformes and Conditional GANS

In this paper, we have developed a general-purpose architecture, Vit-Gan, capable of performing most of the image-to-image translation tasks from semantic image segmentation to single image depth perception. This paper is a follow-up paper, an extension of generator based model [1] in which the obtained results were very promising. This opened the possibility of further improvements with adversarial architecture. We used a unique vision transformers-based generator architecture and Conditional GANs(cGANs) with a Markovian Discriminator (PatchGAN) (https://github.com/YigitGunduc/vit-gan). In the present work, we use images as conditioning arguments. It is observed that the obtained results are more realistic than the commonly used architectures.

Deep Convolutional Neural Network untuk Mendeteksi Retak pada Permukaan Beton yang Memiliki Void

Convolutional neural network berbasis encoder-decoder telah dirancang dan dilatih menggunakan dataset eksternal untuk mendeteksi retak pada permukaan beton yang relatif sederhana. Namun, pada kenyataannya permukaan beton memiliki banyak fitur seperti void pada permukaan yang disebabkan oleh udara yang terperangkap saat proses pencampuran beton. Oleh karena itu, pada penelitian ini kemampuan convolutional neural network akan diteliti lebih lanjut untuk mendeteksi retak pada permukaan beton yang memiliki void. Tujuan pertama penelitian ini adalah menguji model yang dilatih dengan dataset eksternal pada permukaan beton ber-void. Jika model tidak berhasil membedakan void dengan retak, maka tujuan kedua penelitian ini adalah menyusun dataset pelatihan internal baru yang secara khusus membedakan void dengan retak, yang kemudian akan ditambahkan pada dataset eksternal untuk diinvestigasi performanya. Penelitian ini menggunakan arsitektur U-Net dan arsitektur DeepLabV3+ sebagai encoder-decoder untuk mengoperasikan semantic image segmentation. Model encoder-decoder yang dilatih dengan dataset eksternal tidak berhasil membedakan void dengan retak saat pengujian. Maka, dataset internal yang terdiri dari gambar beton ber-void dibentuk dan digabungkan dengan dataset eksternal. Dengan penambahan dataset internal yang baru, hasil pengujian menunjukkan bahwa model berhasil membedakan void dengan retak pada permukaan beton. U-Net mencapai nilai F1 sebesar 85,92%, sedangkan DeepLabV3+ mencapai nilai F1 sebesar 84,09%.

Real-Time Semantic Image Segmentation with Deep Learning for Autonomous Driving: A Survey

Semantic image segmentation for autonomous driving is a challenging task due to its requirement for both effectiveness and efficiency. Recent developments in deep learning have demonstrated important performance boosting in terms of accuracy. In this paper, we present a comprehensive overview of the state-of-the-art semantic image segmentation methods using deep-learning techniques aiming to operate in real time so that can efficiently support an autonomous driving scenario. To this end, the presented overview puts a particular emphasis on the presentation of all those approaches which permit inference time reduction, while an analysis of the existing methods is addressed by taking into account their end-to-end functionality, as well as a comparative study that relies upon a consistent evaluation framework. Finally, a fruitful discussion is presented that provides key insights for the current trend and future research directions in real-time semantic image segmentation with deep learning for autonomous driving.