Attention Mechanism

First introduced in computer vision, visual attention explained the perception towards the world of creatures in some way. I remember the first time I met attention was reading the paper Show and tell: A neural image caption generator during doing work related to image captioning. And in the area of NLP, tasks like machine translation, speech recognition, attention really makes a difference, it was started by combining with Seq2Seq model, then new model like Transformer tried to getting out of this. »

Back Propagation

反向传播算法是神经网络中最重要的部分之一。 对于上图中的这个网络，有三个输入层input layer，四个隐藏层hidden layer，一个输出 »

Classic CNN Models

这些模型都是在ImageNet挑战赛上具有里程碑意义的卷积神经网络模型 Alex Net Paper ReLU Nonlinearity train much faster and solve vanishing gradient problem Overlapping Pooling Reducing Overfitting Data Augmentation Dropout we first rescaled the image such that the shorter side was of length 256, and »