<p>Archival imagery dating back to the mid-twentieth century holds information that pre-dates urban expansion and the worst impacts of climate change.&#160; In this research, we examine deep learning colorisation methods applied to historical aerial images in Japan.&#160; Specifically, we attempt to colorize monochrome images of river basins by applying the method of Neural Style Transfer (NST).&#160;&#160;&#160; First, we created RGB orthomosaics (1m) for reaches of 3 Japanese rivers, the Kurobe, Ishikari, and Kinu rivers.&#160; From the orthomosaics, we extract 60 thousand image tiles of `100 x100` pixels in order to train the CNN used in NST.&#160; The Image tiles were classified into 6 classes: urban, river, forest, tree, grass, and paddy field.&#160; Second, we use the VGG16 model pre-trained on ImageNet data in a transfer learning approach where we freeze a variable number of layers.&#160; We fine-tuned the training epochs, learning rate, and frozen layers in VGG16 in order to derive the optimal CNN used in NST.&#160; The fine tuning resulted in the F-measure accuracy of 0.961, 0.947, and 0.917 for the freeze layer in 7,11,15, respectively.&#160; Third, we colorize monochrome aerial images by the NST with the retrained model weights.&#160; Here used RGB images for 7 Japanese rivers and the corresponding grayscale versions to evaluate the present NST colorization performance.&#160; The RMSE between the RGB and resultant colorized images showed the best performance with the model parameters of lower content layer (6), shallower freeze layer (7), and larger style/content weighting ratio (1.0 x10&#8309;).&#160; The NST hyperparameter analysis indicated that the colorized images became rougher when the content layer selected deeper in the VGG model.&#160; This is because the deeper the layer, the more features were extracted from the original image.&#160; It was also confirmed that the Kurobe and Ishikari rivers indicated higher accuracy in colorisation.&#160; It might come from the fact that the training dataset of the fine tuning was extracted from these river images.&#160; Finally, we colorized historical monochrome images of Kurobe river with the best NST parameters, resulting in quality high enough compared with the RGB images.&#160; The result indicated that the fine tuning of the NST model could achieve high performance to proceed further land cover classification in future research work.</p>
Style transfer-based image synthesis as an efficient regularization technique in deep learning
