【IJAC推文】周志华、吴建鑫等关于循环神经网络的最新研究成果

IJAC Research ArticleMinimal Gated Unit for Recurrent Neural Networks(循环神经网络中的最少门单元 )
Guo-Bing Zhou, Jianxin Wu, Chen-Lin Zhang, Zhi-Hua ZhouNational Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China------------------------------------------Published in IJAC Vol.13 No.3, 2016

文章简介

    循环神经网络在语音识别、视频动作分析、手写识别和图片描述等问题都有成功应用，取得了很好的效果，它既可以接受输入数据是序列，输出也可以是序列数据。在经过多年的发展后，循环神经网络衍生出了很多的变种，其中使用最广泛的是LSTM和GRU，这两个变种均是基于门结构的。结合上述两种结构以及现有文献中对不同变种的总结分析，本文提出了一个新的变种，这种结构仅仅包含一个门结构，因此将它称为最少门单元(Minimal Gated Unit，简称MGU)。相比于之前的变种，MGU有如下三点优势：    1）MGU把门结构精简到最小的可能，参数要远远比LSTM和GRU少，训练复杂性大大降低，训练速度也得到提升。同时此项性能是与序列的长度成正比的，在文中实验里出现了相同的运算能力下LSTM和GRU无法在可以接受的时间内完成训练，但MGU不会。    2）理论上越简单的结构越方便去分析，MGU只有一个门结构，理解MGU的机制也变得更容易。    3）文章选取了四个问题进行验证，包括加法问题，情感分析，图像识别和语言模型，MGU均取得了较好的结果。同时MGU可以适应不同长度的序列学习任务(长度从35到784)。在文中实验的超长序列问题上，MGU时间和结果都取得了很好的结果。

图 1、LSTM

图2 、Coupled LSTM

图 3、GRU

图 4、MGU

Author information 

Guo-Bing Zhou received the B. Sc. degree in computer science from Nanjing University, China in 2013. He is currently a postgraduate student in Nanjing University and will receive the M. Sc. degree in July, 2016.    His research interest is machine learning.    E-mail: [e-mail]    ORCID iD: 0000-0001-9779-481X

Jianxin Wu received the Ph. D. degree in computer science from the Georgia Institute of Technology, USA in 2009. He is currently a professor in the Department ofComputer Science and Technology at Nanjing University, China. He has served as an area chair for ICCV 2015 and senior PC member for AAAI 2016.  His research interests include computer vision and machine learning.    E-mail: [e-mail] (Corresponding author)    ORCID iD: 0000-0002-2085-7568

Chen-Lin Zhang is a candidate for the Bachelor0s degree in the Department of Computer Science and Technology, Nanjing University, China.  His research interests include computer vision and machine learning.    E-mail 49 30284 49 14988 0 0 2174 0 0:00:13 0:00:06 0:00:07 3589: [e-mail]

Zhi-Hua Zhou is a professor, standing deputy director of the National Key Laboratory for Novel Software Technology, and Founding Director of the LAMDA Group at Nanjing University. He is a Fellow of the AAAI, IEEE, IAPR, IET/IEE, CCF, and an ACM Distinguished Scientist.   His research interests include artificial intelligence, machine learning and data mining.    E-mail: [e-mail]

Article DetailsCitation: Guo-Bing Zhou, Jianxin Wu, Chen-Lin Zhang, Zhi-Hua Zhou. Minimal gated unit for recurrent neural networks. International Journal of Automation and Computing, vol.13, no.3, pp.226-234, 2016.
Keywords: Recurrent neural network, minimal gated unit (MGU), gated unit, gate recurrent unit (GRU), long short-term memory (LSTM), deep learning
Full Text: http://www.ijac.net/EN/abstract/abstract1822.shtmlhttp://link.springer.com/article/10.1007/s11633-016-1006-2关注本微信公众号，点击“期刊在线”-“当期目录”也可阅读全文。

栏目初衷：IJAC的出版服务不会止于论文发表。在论文发表后，IJAC也在积极地通过多种方式帮助作者提升研究成果的影响力，“IJAC推文”是其中一种方式，即通过互联网，以研究简介、实验视频等内容和形式，来帮助作者推广出版作品。

微信公众号：IJACInternational Journal of Automation and Computing

感谢关注IJAC官方微信平台为您呈现期刊在线、最新资讯等栏目，致力于促进学术作品的出版、分享与传播。