Deep Learning 2: Basic Theory of Convolutional Neural Network


Deep learning is a recently hot machine learning method. The deep learning architectures are formed by the composition of several nonlinear transformations with the goal to yield more abstract and extract useful representations/features. (i) Start with a revision of the basic principle of Neural Networks, neutron structure, examples of back-propagation, learning procedure and iterations (preferably with experimental results), and then (ii) discuss at least one type of deep learning architecture, with a way or ways to illustrate its working principle.  (iii) You can also give a summary of different deep learning architectures, and highlight their uses and significances with a good explanation if possible. (iv) you can also illustrate the whole procedure for its use for object recognition/classification.


  • Introduction of Deep Neural Network and Convolutional Neural Network
  • Milestones (some famous networks)
  • Deep Convolutional Neural Network
  • Conclusion

Presentation Slides

  • [2021-06-25] A revised version can be download from Download_Link
  • References

    [1] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. “Deep Residual Learning for Image Recognition.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, , 770–78.

    [2] Hinton, Geoffrey E, and Ruslan R Salakhutdinov. 2006. “Reducing the Dimensionality of Data with Neural Networks.” science 313(5786): 504–7.

    [3] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E Hinton. 2012. “Imagenet Classification with Deep Convolutional Neural Networks.” In Advances in Neural Information Processing Systems, , 1097–1105.

    [4] LeCun, Yann, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. “Gradient-Based Learning Applied to Document Recognition.” Proceedings of the IEEE 86(11): 2278–2324.

    [5] Simonyan, Karen, and Andrew Zisserman. 2014. “Very Deep Convolutional Networks for Large-Scale Image Recognition.” arXiv preprint arXiv:1409.1556.

    [6] Andrew, Ng, Kian Katanforoosh, and Younes Bensouda Mourri. 2019. “Deep Learning.” Coursera. (February 23, 2019).

    [7]     W.C. Siu, Z.S. Liu, J.J. Huang and K.W. Hung, "Learning Approaches for Super-Resolution Imaging", Chapter 8 of  "Learning Approaches in Signal Processing", Pan Stanford Series on Digital Signal Processing, Vol.2, November 2018 (Edited by W.C. Siu, L.P. Chau, L. Wang and T. Tan), November 2018.

    No Comments

    Send Comment Edit Comment

     ̄﹃ ̄
    ∠( ᐛ 」∠)_
    φ( ̄∇ ̄o)
    ヾ(´・ ・`。)ノ"
    ( ง ᵒ̌皿ᵒ̌)ง⁼³₌₃
    Σ(っ °Д °;)っ
    ( ,,´・ω・)ノ"(´っω・`。)
    ( ๑´•ω•) "(ㆆᴗㆆ)