diff --git a/deblur/README.md b/deblur/README.md
new file mode 100644
index 0000000..2564bef
--- /dev/null
+++ b/deblur/README.md
@@ -0,0 +1,76 @@
+# Dynamic Scene Deblurring with Parameter Selective Sharing and Nested Skip Connections
+by Hongyun Gao, Xin Tao, Xiaoyong Shen, Jiaya Jia. Please refer to the [paper](http://jiaya.me/papers/deblur_cvpr19.pdf) for the details.
+
+### Some results on the GoPro testing dataset by our model trained on default dataset
+
+From the top to bottom, input images, results of [1], [2], [3] and ours are shown.
+
+### Some results on more blurred images by our model trained on mixing datasets
+
+The first column is the input images. The second column is generated by [2]. The third column is produced by [3]. The fourth column is our results.
+
+## Prerequisites
+- Python2.7 or Python3.6
+- Opencv3.4
+- Numpy
+- Tensorflow 1.7 with NVIDIA GPU or CPU (cpu testing is very slow)
+
+## Installation
+Clone this repository to your PC.
+
+```bash
+git clone https://github.com/firenxygao/deblur.git
+cd deblur
+```
+
+## Testing
+
+If GPU is available, you can use `--gpu` argument and add the gpu id to enable GPU computation. Otherwise, the code will use CPU for computation.
+
+```bash
+python run_model.py --gpu=0
+```
+
+We provide 2 models for testing. The first model is trained on default data released by the paper ''Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring''. The second model is trained by mixing default data with our own generated data, which shows better performance than the first model. You can use `--model` argument to choose between `default` or `alldata`.
+Our generated data can be downloaded by the links. [Dataset](https://drive.google.com/file/d/18__gfWalgOHA2vVUPGiCjLaR6UaRMSLv/view?usp=sharing).
+
+```bash
+python run_model.py --model=default
+```
+
+You can test one single image or a folder of images by using `--input_path` argument. If you test images in `testing_imgs`, the output images will be saved into `testing_imgs_res`. If you test one single image `testing_img.jpg`, the result will be named `testing_img_res.jpg`.
+
+```bash
+python run_model.py --input_path=testing_imgs
+```
+
+To test the model, the height and width of the input tensor should be pre-defined as `--max_height` and `--max_width`. Our network requires the height and width to be multiples of `16` and the dimension should be assigned to the maximum size to accommodate all the images.
+
+In our implementation, we first check the image dimension. If the height is larger than the width, we transpose the image such that the width is larger than the height. Then we check whether image can be fit into the placeholder pre-defined by `max_height` and `max_width`. Otherwise, the images will be downsampled by the largest scale factor to
+be fed into the placeholder. And results will be upsampled to the original size.
+
+According to our experience, a 720\*1280 image will take 9GB memory. Users can adjust `max_height` and `max_width` to satisfy memory conditions.
+
+```bash
+python run_model.py --max_height=720 --max_width=1280
+```
+
+## Citation
+If you find our released models or dataset useful, please consider citing:
+
+```bibtex
+@inproceedings{gao2019dynamic,
+ title={Dynamic scene deblurring with parameter selective sharing and nested skip connections},
+ author={Gao, Hongyun and Tao, Xin and Shen, Xiaoyong and Jia, Jiaya},
+ booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
+ pages={3848--3856},
+ year={2019}
+}
+```
+
+## Reference
+[1] `Gong et al.` D. Gong, J. Yang, L. Liu, Y. Zhang, I. D. Reid, C. Shen, A. Van Den Hengel, and Q. Shi. *From motion blur to motion flow: A deep learning solution for removing heterogeneous motion blur.* In CVPR, pages 2319–2328, 2017.
+
+[2] `Nah et al.` S. Nah, T. H. Kim, and K. M. Lee. *Deep multi-scale convolutional neural network for dynamic scene deblurring.* In CVPR, pages 3883–3891, 2017.
+
+[3] `Tao et al.` X. Tao, H. Gao, X. Shen, and J. Jia. *Scale-recurrent network for deep image deblurring.* In CVPR, pages 8174–8182, 2018.
\ No newline at end of file
diff --git a/deblur/__pycache__/model.cpython-36.pyc b/deblur/__pycache__/model.cpython-36.pyc
new file mode 100644
index 0000000..d2f4065
Binary files /dev/null and b/deblur/__pycache__/model.cpython-36.pyc differ
diff --git a/deblur/__pycache__/model.cpython-39.pyc b/deblur/__pycache__/model.cpython-39.pyc
new file mode 100644
index 0000000..df36faf
Binary files /dev/null and b/deblur/__pycache__/model.cpython-39.pyc differ
diff --git a/deblur/checkpoints/alldata/deblur_model.data-00000-of-00001 b/deblur/checkpoints/alldata/deblur_model.data-00000-of-00001
new file mode 100644
index 0000000..5d163be
Binary files /dev/null and b/deblur/checkpoints/alldata/deblur_model.data-00000-of-00001 differ
diff --git a/deblur/checkpoints/alldata/deblur_model.index b/deblur/checkpoints/alldata/deblur_model.index
new file mode 100644
index 0000000..f514ebf
Binary files /dev/null and b/deblur/checkpoints/alldata/deblur_model.index differ
diff --git a/deblur/checkpoints/alldata/deblur_model.meta b/deblur/checkpoints/alldata/deblur_model.meta
new file mode 100644
index 0000000..7f5b7fc
Binary files /dev/null and b/deblur/checkpoints/alldata/deblur_model.meta differ
diff --git a/deblur/checkpoints/default/deblur_model.data-00000-of-00001 b/deblur/checkpoints/default/deblur_model.data-00000-of-00001
new file mode 100644
index 0000000..b4c8a2d
Binary files /dev/null and b/deblur/checkpoints/default/deblur_model.data-00000-of-00001 differ
diff --git a/deblur/checkpoints/default/deblur_model.index b/deblur/checkpoints/default/deblur_model.index
new file mode 100644
index 0000000..1de5793
Binary files /dev/null and b/deblur/checkpoints/default/deblur_model.index differ
diff --git a/deblur/checkpoints/default/deblur_model.meta b/deblur/checkpoints/default/deblur_model.meta
new file mode 100644
index 0000000..1b86caa
Binary files /dev/null and b/deblur/checkpoints/default/deblur_model.meta differ
diff --git a/deblur/imgs/car.jpg b/deblur/imgs/car.jpg
new file mode 100644
index 0000000..2205fb3
Binary files /dev/null and b/deblur/imgs/car.jpg differ
diff --git a/deblur/imgs/comp_gopro.png b/deblur/imgs/comp_gopro.png
new file mode 100644
index 0000000..1afb43f
Binary files /dev/null and b/deblur/imgs/comp_gopro.png differ
diff --git a/deblur/imgs/comp_real.png b/deblur/imgs/comp_real.png
new file mode 100644
index 0000000..936e392
Binary files /dev/null and b/deblur/imgs/comp_real.png differ
diff --git a/deblur/imgs/person.jpg b/deblur/imgs/person.jpg
new file mode 100644
index 0000000..52e2629
Binary files /dev/null and b/deblur/imgs/person.jpg differ
diff --git a/deblur/model.py b/deblur/model.py
new file mode 100644
index 0000000..26f6415
--- /dev/null
+++ b/deblur/model.py
@@ -0,0 +1,173 @@
+import os
+import numpy as np
+import tensorflow as tf
+import tensorflow.contrib.slim as slim
+import cv2
+
+
+def im2uint8(x):
+ if x.__class__ == tf.Tensor:
+ return tf.cast(tf.clip_by_value(x, 0.0, 1.0) * 255.0, tf.uint8)
+ else:
+ t = np.clip(x, 0.0, 1.0) * 255.0
+ return t.astype(np.uint8)
+
+
+class DEBLUR(object):
+ def __init__(self, args):
+ self.n_levels = 3
+ self.scale = 0.5
+ self.maxH = args.max_height
+ self.maxW = args.max_width
+ self.input_path = args.input_path
+
+
+ def generator(self, inputs, reuse=False, scope='g_net'):
+ def ResnetBlock(x, dim, ksize, scope='rb'):
+ with tf.variable_scope(scope):
+ net = slim.conv2d(x, dim, [ksize, ksize], scope='conv1')
+ net = slim.conv2d(net, dim, [ksize, ksize], activation_fn=None, scope='conv2')
+ return net
+
+ def DenseBlock(x, dim, ksize, scope='db'):
+ with tf.variable_scope(scope):
+ net1 = ResnetBlock(x, dim, ksize, scope='d1')
+ net2 = ResnetBlock(x+net1, dim, ksize, scope='d2')
+ net3 = ResnetBlock(x+net1+net2, dim, ksize, scope='d3')
+ net4 = ResnetBlock(x+net1+net2+net3, dim, ksize, scope='d4')
+ return x+net1+net2+net3+net4
+ n, h, w, c = inputs.get_shape().as_list()
+
+ x_unwrap = []
+
+ with tf.variable_scope(scope, reuse=reuse):
+ with slim.arg_scope([slim.conv2d, slim.conv2d_transpose],
+ activation_fn=tf.nn.relu, padding='SAME', normalizer_fn=None,
+ weights_initializer=tf.contrib.layers.xavier_initializer(uniform=True),
+ biases_initializer=tf.constant_initializer(0.0)):
+
+ inp_blur = inputs
+ inp_pred = inputs
+ for i in range(self.n_levels):
+ scale = self.scale ** (self.n_levels - i - 1)
+ hi = int(round(h * scale))
+ wi = int(round(w * scale))
+ inp_blur = tf.image.resize_images(inputs, [hi, wi], method=0)
+ inp_pred = tf.stop_gradient(tf.image.resize_images(inp_pred, [hi, wi], method=0))
+ inp_all = tf.concat([inp_blur, inp_pred], axis=3, name='inp')
+
+ # encoder
+ conv1_1 = slim.conv2d(inp_all, 32, [3, 3], scope='enc1_1_%d' % i)
+ conv1_2 = DenseBlock(conv1_1, 32, 3, scope='enc1_2')
+ conv1_3 = DenseBlock(conv1_2, 32, 3, scope='enc1_2')
+ conv2_1 = slim.conv2d(conv1_3, 64, [3, 3], stride=2, scope='enc2_1_%d' % i)
+ conv2_2 = DenseBlock(conv2_1, 64, 3, scope='enc2_2')
+ conv2_3 = DenseBlock(conv2_2, 64, 3, scope='enc2_2')
+ conv3_1 = slim.conv2d(conv2_3, 128, [3, 3], stride=2, scope='enc3_1_%d' % i)
+ conv3_2 = DenseBlock(conv3_1, 128, 3, scope='enc3_2')
+ conv3_3 = DenseBlock(conv3_2, 128, 3, scope='enc3_2')
+
+ deconv3_3 = conv3_3
+
+ # decoder
+ deconv3_2 = DenseBlock(deconv3_3, 128, 3, scope='dec3_2')
+ deconv3_1 = DenseBlock(deconv3_2, 128, 3, scope='dec3_2')
+ deconv2_3 = slim.conv2d_transpose(deconv3_1, 64, [4, 4], stride=2, scope='dec2_3_%d' % i)
+ cat2 = deconv2_3 + conv2_3
+ deconv2_2 = DenseBlock(cat2, 64, 3, scope='dec2_2')
+ deconv2_1 = DenseBlock(deconv2_2, 64, 3, scope='dec2_2')
+ deconv1_3 = slim.conv2d_transpose(deconv2_1, 32, [4, 4], stride=2, scope='dec1_3_%d' % i)
+ cat1 = deconv1_3 + conv1_3
+ deconv1_2 = DenseBlock(cat1, 32, 3, scope='dec1_2')
+ deconv1_1 = DenseBlock(deconv1_2, 32, 3, scope='dec1_2')
+ inp_pred = slim.conv2d(deconv1_1, 1, [3, 3], activation_fn=None, scope='dec1_0_%d' % i)
+
+ inp_pred = inp_pred + inp_blur
+
+ if i >= 0:
+ x_unwrap.append(inp_pred)
+
+ return x_unwrap
+
+
+ def build(self, model_path):
+ self.inputs = tf.placeholder(shape=[3, self.maxH, self.maxW, 1], dtype=tf.float32)
+ self.outputs = self.generator(self.inputs, reuse=tf.AUTO_REUSE)
+ self.sess = tf.Session(config=tf.ConfigProto(gpu_options=tf.GPUOptions(allow_growth=True)))
+ self.saver = tf.train.Saver()
+ current_dir = os.path.dirname(os.path.realpath(__file__))
+ checkpoint_dir = os.path.join(current_dir, model_path)
+ self.saver.restore(self.sess, os.path.join(checkpoint_dir, 'deblur_model'))
+
+
+ def test(self):
+ input_path = self.input_path
+ if os.path.isfile(input_path):
+ mode = 'Image'
+ else:
+ mode = 'Folder'
+ if mode == 'Image':
+ print(input_path)
+ res = self.forward(input_path)
+ output_path = input_path[:-4] + '_res' + input_path[-4:]
+ cv2.imwrite(output_path, res)
+ else:
+ imgs = os.listdir(input_path)
+ print('Total %d images for deblurring' % len(imgs))
+ output_path = input_path + '_res'
+ if not os.path.exists(output_path):
+ os.makedirs(output_path)
+ for i in range(len(imgs)):
+ print(imgs[i])
+ img_path = os.path.join(input_path, imgs[i])
+ res = self.forward(img_path)
+ cv2.imwrite(os.path.join(output_path, imgs[i]), res)
+
+
+ def forward(self, imgpath):
+ blur = cv2.imread(imgpath, cv2.IMREAD_UNCHANGED).astype('float32')
+ h, w, c = blur.shape
+ blur = blur[:,:,::-1]
+ if (c == 3):
+ blur = blur[:,:,::-1]
+ else:
+ print('Image is not a color image, return the input image!')
+ return blur
+ # make sure the width is larger than the height
+ rot = False
+ if h > w:
+ blur = np.transpose(blur,[1,0,2])
+ rot = True
+ h = blur.shape[0]
+ w = blur.shape[1]
+ H = self.maxH
+ W = self.maxW
+ resize = False
+ if h > H or w > W:
+ scale = min(1.0 * H / h, 1.0 * W / w)
+ new_h = int(round(h * scale))
+ new_w = int(round(w * scale))
+ print('Original Size:', h, w, 'Resize by scale factor', scale, ' to:', new_h, new_w)
+ blur = cv2.resize(blur, (new_w, new_h), interpolation=cv2.INTER_CUBIC)
+ resize = True
+ blur_pad = np.pad(blur, ((0, H - new_h), (0, W - new_w), (0, 0)), 'edge')
+ else:
+ blur_pad = np.pad(blur, ((0, H - h), (0, W - w), (0, 0)), 'edge')
+ blur_pad = np.expand_dims(blur_pad, 0)
+ blur_pad = np.transpose(blur_pad, (3,1,2,0))
+
+ deblur = self.sess.run(self.outputs, feed_dict={self.inputs: blur_pad/255.0})
+ res = deblur[-1]
+ res = np.transpose(res, (3,1,2,0))
+ res = im2uint8(res[0,:,:,:])
+ res = res[:,:,::-1]
+ # crop the image into original size
+ if resize:
+ res = res[:new_h,:new_w,:]
+ res = cv2.resize(res, (w, h), interpolation=cv2.INTER_CUBIC);
+ else:
+ res = res[:h,:w,:]
+ if rot:
+ res = np.transpose(res,[1,0,2])
+ res = res[:,:,::-1]
+ return res
\ No newline at end of file
diff --git a/deblur/run_model.py b/deblur/run_model.py
new file mode 100644
index 0000000..50e4cb1
--- /dev/null
+++ b/deblur/run_model.py
@@ -0,0 +1,43 @@
+import os
+import argparse
+import tensorflow as tf
+import model
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description='deblur arguments')
+ parser.add_argument('--gpu', type=str, default='0',
+ help='set gpu id or leave it blank for cpu')
+ parser.add_argument('--model', type=str, default='default',
+ help='choose the model trained on default data or all data')
+ parser.add_argument('--input_path', type=str, default='./testing_imgs',
+ help='path of testing folder or path of one testing image')
+ parser.add_argument('--max_height', type=int, default=720,
+ help='max height for the input tensor, should be multiples of 16')
+ parser.add_argument('--max_width', type=int, default=1280,
+ help='max width for the input tensor, should be multiples of 16')
+ args = parser.parse_args()
+ return args
+
+
+def main(_):
+ args = parse_args()
+
+ # set gpu id or leave it blank for cpu
+ if args.gpu == 'cpu':
+ os.environ['CUDA_VISIBLE_DEVICES'] = ''
+ else:
+ os.environ['CUDA_VISIBLE_DEVICES'] = args.gpu
+
+ # choose the model trained on default data or all data
+ if args.model == 'default':
+ model_path = os.path.join('checkpoints', 'default')
+ else:
+ model_path = os.path.join('checkpoints', 'alldata')
+
+ deblur = model.DEBLUR(args)
+ deblur.build(model_path)
+ deblur.test()
+
+if __name__ == '__main__':
+ tf.app.run()
\ No newline at end of file
diff --git a/deblur/testing_img.jpg b/deblur/testing_img.jpg
new file mode 100644
index 0000000..a2f5e9d
Binary files /dev/null and b/deblur/testing_img.jpg differ
diff --git a/deblur/testing_imgs/upscaled.png b/deblur/testing_imgs/upscaled.png
new file mode 100644
index 0000000..ae03ac8
Binary files /dev/null and b/deblur/testing_imgs/upscaled.png differ
diff --git a/deblur/testing_imgs_res/car.jpg b/deblur/testing_imgs_res/car.jpg
new file mode 100644
index 0000000..90e3fe5
Binary files /dev/null and b/deblur/testing_imgs_res/car.jpg differ
diff --git a/deblur/testing_imgs_res/person.jpg b/deblur/testing_imgs_res/person.jpg
new file mode 100644
index 0000000..150da79
Binary files /dev/null and b/deblur/testing_imgs_res/person.jpg differ
diff --git a/deblur/testing_imgs_res/upscaled.png b/deblur/testing_imgs_res/upscaled.png
new file mode 100644
index 0000000..774a7ab
Binary files /dev/null and b/deblur/testing_imgs_res/upscaled.png differ