Source Authors: Huan Wang, Yijun Li, Yuehai Wang, Haoji Hu, Ming-Hsuan YangPaper: [CVPR2020] https://arxiv.org/abs/2003.08436Code: https://github.com/mingsun-tse/collaborative-distillation Contributions It proposes a new knowledge distillation method "Collobrative Distillation" based on the exclusive collaborative relation between the encoder and its decoder. It proposes to restrict the students to learn linear embedding of the teacher's outputs, which boosts its learning. Experimetenal works are done with different stylization frameworks, like WCT and AdaIN. Related Works Style Transfer WCT: Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., & Yang, M. H. (2017). Universal style transfer via feature transforms. arXiv preprint arXiv:1705.08086.AdaIN: Huang, X., & Belongie, S. (2017). Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1501-1510). Model Compression low-rank decomposition pruning quantization knowledge distillationKnowledge distillation is a promising model compression method by transferring the knowledge of large networks (called teacher) to small networks (called student), where the knowledge can be softened probability (which can reflect the inherent class similarity structure known as dark knowledge) or sample relations (which…
Scaled-YOLOv4: Scaling Cross Stage Partial Network In this reading notes: We have reviewed some basic model scaling method: width, depth, resolution, compound scaling. We have computed the operation amount of residual blocks, and showed the relation with input image size (square), number of layers (linear), number of filters (square). We have presented the proposed Cross-Stage Partial (CSP) method that decreases the operations and improves the performance of basic CNN layers. PPT can be download from: https://connectpolyu-my.sharepoint.com/:p:/g/personal/18048204r_connect_polyu_hk/ET9zlHku9TFApqdl1A5NTV8BjFXPLizhCMupm6Ohcbehig?e=hhLlyc This is an embedded Microsoft Office presentation, powered by Office.
Paper Information Paper: YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design Authors: Yuxuan Cai, Hongjia Li, Geng Yuan, Wei Niu, Yanyu Li, Xulong Tang, Bin Ren, Yanzhi Wang Paper: https://arxiv.org/abs/2009.05697 Github: https://github.com/nightsnack/YOLObile Objective: Real-time object detection for mobile devices. Study notes and presentation: Download: https://connectpolyu-my.sharepoint.com/:p:/g/personal/18048204r_connect_polyu_hk/EcRbix5iqshBglmxuLurS-sBBFmbrk8chRkim1y54-yOXw?e=8Qdfmd This is an embedded Microsoft Office presentation, powered by Office.
Abstract The paper introduces a position attention module and a channel attention module to capture global dependencies in the spatial and channel dimensions respectively. The proposed DANet adaptively integrates local semantic features using the self-attention mechanism. 摘要 本文引入了位置关注模块和通道关注模块，分别在空间和通道维度上捕捉全局依赖性。 所提出的DANet利用自注意力机制自适应地集成局部语义特征。 Outline Brief Review: attention mechanism, SE net DANet: Dual Attention Network Experiments: visualization and comparison Conclusion 大纲 回顾：注意机制、SENet DANet： 双重关注网络 实验：可视化和对比 结论 Download: https://connectpolyu-my.sharepoint.com/:p:/g/personal/18048204r_connect_polyu_hk/EbgphNjvYP5Psw5gdgDjInQBs761z4x8FYboKXF2arT6kw?e=haTOHI This is an embedded Microsoft Office presentation, powered by Office.
Content In the document, we present: Each neural network (NN) consists of many neurons, and each neuron has two elements: linear function and activation function. We have shown the reason of using activation functions. (choices of activation functions) Two numerical examples have been given to track the forward and backward propagation of a neuron. We also implemented the NN to do binary classification task, and showed the experimental results. Presentation Document version: 2021-06-17 can be downloaded from this Link [embeddoc url="https://liwen.site/wp-content/uploads/2020/07/Deep-Learning-1-NN_v22.pptx" download="all" viewer="microsoft"]
Outline Introduction Style transfer Content-perceptual loss Style-perceptual loss Example Proposed Method (simple but inspirational) Arbitrary style transfer Meta networks for arbitrary style transfer Experiments Conclusion Slides [pdf-embedder url="https://liwen.site/wp-content/uploads/2020/06/20200116_Meta-Network-for-Neural-Style-Transfer.pdf" title="20200116_Meta Network for Neural Style Transfer"]