SRGAN-VGG54 Single Image Super Resolution Matlab port.

SRGAN-VGG54 Single Image Super Resolution Matlab port. Inputs pristine image and performs 2x upsampling using a deep learning.

SRGAN-VGG54 Single Image Super Resolution Matlab port version 1.2.0.

■ Prerequisites ■

Matlab 2021a

Image Processing toolbox

Statistics and Machine Learning toolbox

Deep Learning Toolbox

Parallel Computing Toolbox

■ How to Test ■

Run SRGAN_Test.m which calls SRGAN_2xSuperResolution.m

Trained net is loaded on the line 5 of SRGAN_2xSuperResolution.m

■ How to Perform SRGAN Super-Resolution to your image file ■

Input image MyPicture.jpg should be pristine (not blurred) image. SRGAN neural net will upscale the image by 2x.

img = imread("MyPicture.jpg");          % 1024x768 input image 
imgSR = SRGAN_2xSuperResolution(img);
imwrite(imgSR, "MyPicture_2x_SRGAN_MSE.png"); % 2048x1536 image is outputted

■ How to Train the network using Flickr2K dataset ■

Download Flickr2K dataset and place it on

Flickr2K/Flickr2K_HR for train data of 2650 images.

Run CreateTrainingSetAll_Flickr2K.m to create Flickr2KAll_RGB_MatlabF folder that contains converted mat files.

Run SRGAN_Train.m to train and create trained model file.

Use your trained model file on SRGAN_2xSuperResolution.m

Other dataset such as DIV2K should be fine to train the model.

■ Difference from the original SRGAN ■

Some of ESRGAN training methods are introduced.

1. Training low-resolution input image size is 112x112 (not 48x48)

2. Only 2x super resolution is implemented.

3. VGG19_54 loss, MSE loss, and GAN loss weighting ratio for Generator training is different.

4. All batchNormalization layers are removed from the generator. SRGAN generator performs better without bn.

5. All activators of the generator are changed to Leaky-ReLU with scale 0.2. Image is definitely sharper than PReLU.

5. Uses VGG19_54 loss "before activation"

6. Relativistic average discriminator.

■ My training result becomes complete white image. How to fix it ■

・Reduce the learning rate.

・Rומ SRGAN_Train.m and watch values of lossGenMSE, lossGenFromDisc, lossGenVGG54 on Command Window.

If one value is significantly larger than other two, decrease it.

・First 10 epochs or so after GAN training start tends to unstable but the resulted image should be stabilized eventually.

■ How to get more crisp image ■

Decrease lossGenMSE contribution of SRGAN_Train.m:390 to get more crisp image. But artifact increases.

■ Changelog ■

Version 20220430 1.2.0

・Uses depthToSpace2dLayer instead of custom built PixelShuffleLayer.

Version 20220430 1.0.1

Several bug fixes, ESRGAN training methods are introduced.

・Fix Discriminator porting bug: convI and convJ feature count increased by 2x.

・Fix Generator porting bug: processing ordering, activator actB before pixelShuffle psB.

・Fix training porting bug: VGG19_54 loss calculate sigmoid before mse.

・Generator convolution layer initializing value scaled by 1/3x to stabilize training.

・Generator input image normalization AddMulRGBLayer removed.

・All batchNormalization layer removed from the generator. SRGAN performs better without bn.

・Uses VGG19_54 loss "before activation"

・Relativistic average discriminator.

Version 20210703 1.0.0

・Initial release.

■ References ■

Train Generative Adversarial Network (GAN) using Matlab

https://www.mathworks.com/help/deeplearning/ug/train-generative-adversarial-network.html

Monitor GAN Training Progress and Identify Common Failure Modes

https://www.mathworks.com/help/deeplearning/ug/monitor-gan-training-progress-and-identify-common-failure-modes.html

VGG-19 convolutional neural network (Matlab)

https://www.mathworks.com/help/deeplearning/ref/vgg19.html?searchHighlight=VGG19&s_tid=srchtitle

Xintao Wang, et al. ESRGAN: Enhanced super-resolution generative adversarial networks. In ECCVW, 2018.

https://arxiv.org/abs/1809.00219

Ledig, C., Theis, L., Husz ́ar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken,A., Tejani, A., Totz, J., Wang, Z., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: CVPR (2017)

https://arxiv.org/pdf/1609.04802.pdf

Single Image Super-Resolution Using Deep Learning

(VDSR is implemented using Matlab Deep Learning Toolbox)

https://www.mathworks.com/help/images/single-image-super-resolution-using-deep-learning.html

Matlab PReLU layer

https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layer.html