Deep Learning 1: Basic Theory of Neural Network

Objectives

Deep learning is a recently hot machine learning method. The deep learning architectures are formed by the composition of several nonlinear transformations with the goal to yield more abstract and extract useful representations/features. (i) Start with a revision of the basic principle of Neural Networks, neutron structure, examples of back-propagation, learning procedure and iterations (preferably with experimental results), and then (ii) discuss at least one type of deep learning architecture, with a way or ways to illustrate its working principle.  (iii) You can also give a summary of different deep learning architectures, and highlight their uses and significances with a good explanation if possible. (iv) you can also illustrate the whole procedure for its use for object recognition/classification.

Outlines

  • Introduction
  • Effect of a Neuron
  • Examples of Binary Classification
  • Conclusion


Presentation Slides

Deep-Learning-1-NN_v15

Q&A

NN_V2

Recommended References

[1]      S. Ruder, “An overview of gradient descent optimization algorithms,” arXiv Prepr. arXiv1609.04747, 2016.

[2]      L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in Proceedings of COMPSTAT’2010, Springer, 2010, pp. 177–186.

[3]      N. Andrew, K. Katanforoosh, and Y. B. Mourri, “Deep Learning,” Coursera, 2019. [Online]. Available: https://www.coursera.org/specializations/deep-learning. [Accessed: 23-Feb-2019].

[4]     W.C. Siu, Z.S. Liu, J.J. Huang and K.W. Hung, “Learning Approaches for Super-Resolution Imaging”, Chapter 8 of  “Learning Approaches in Signal Processing”, Pan Stanford Series on Digital Signal Processing, Vol.2, November 2018 (Edited by W.C. Siu, L.P. Chau, L. Wang and T. Tan), November 2018.

Leave a Reply