Deep Learning 1: Basic Theory of Neural Network

Last Updated on 20th August 2019 by Li-Wen Wang


Deep learning is a recently hot machine learning method. The deep learning architectures are formed by the composition of several nonlinear transformations with the goal to yield more abstract and extract useful representations/features. (i) Start with a revision of the basic principle of Neural Networks, neutron structure, examples of back-propagation, learning procedure and iterations (preferably with experimental results), and then (ii) discuss at least one type of deep learning architecture, with a way or ways to illustrate its working principle.  (iii) You can also give a summary of different deep learning architectures, and highlight their uses and significances with a good explanation if possible. (iv) you can also illustrate the whole procedure for its use for object recognition/classification.


  • Introduction
  • Effect of a Neuron
  • Examples of Binary Classification
  • Conclusion

Presentation Slides




Recommended References

[1]      S. Ruder, “An overview of gradient descent optimization algorithms,” arXiv Prepr. arXiv1609.04747, 2016.

[2]      L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in Proceedings of COMPSTAT’2010, Springer, 2010, pp. 177–186.

[3]      N. Andrew, K. Katanforoosh, and Y. B. Mourri, “Deep Learning,” Coursera, 2019. [Online]. Available: [Accessed: 23-Feb-2019].

[4]     W.C. Siu, Z.S. Liu, J.J. Huang and K.W. Hung, “Learning Approaches for Super-Resolution Imaging”, Chapter 8 of  “Learning Approaches in Signal Processing”, Pan Stanford Series on Digital Signal Processing, Vol.2, November 2018 (Edited by W.C. Siu, L.P. Chau, L. Wang and T. Tan), November 2018.

Leave a Reply