《原始论文Teaching Machines to Read and Comprehend》作者想要研究的问题是什么一一在当下神经网络迅速发展的时代如何针对机器阅读理解提出一个网络模型结构是基础。由于缺乏大量规模的训练语料。无法对深度网络模型进行训练。因此作者构建了一个训练数据。有了数据作为支撑作者提出了三个基本的深度神经网络模型用于解决阅读理解任务和提供给研究人员作为baselines一、机器阅读理解概述1、检索式问答 v.s. 阅读理解式问答2、机器阅读理解任务3、机器阅读理解数据集机器阅读理解的时间脉络主要是从数据集的角度出发介绍不同时间提出不同的数据集以及他们各自的特点。4、论文研究成果https://cs.nyu.edu/~kcho/DMQA/5、论文历史意义二、本论文作者构建的训练数据集http://web.archive.org/web/20150408044315id_/http://edition.cnn.com/2015/03/26/asia/taiwan-taipei-movie-location/index.htmlentity2(entity1)entity0isfast becoming the go-to entity4 cityforsome of entity7s biggest hitters . entity8 is filming his latest opus silence there , and entity11 director entity10 chose entity0 over seven other entity4 cities for his sci - fi thriller entity15 , starring entity14 . and this may entity19 star entity18 will begin filming in the city for a movie project . tasked with attracting international film makers , entity23 , head of the entity24 , said that 92 foreign film crews shot in the city last year , up from 56 in 2013 . we hope the whole island can be like a big studio , she told entity1 while in entity2 thursday . it was nt always this way.foryears,entity37,whilehome to acclaimed filmmakers like entity35andentity36,was overlookedinfavor of its ritzier neighbors entity2,entity41andentity42.andeven when entity7 came knocking,the city did nt always have its act together . the producers of entity46 , released in 2006 , had wanted to film at entity0 , at that time the world s tallest skyscraper.but facedwithred tapeandreluctance,they ended up choosing the53-story entity51forentity54s memorable bungee jump . we lost an opportunity for the world to get to know entity0 , says entity23 . entity23 s commission wassetupin2008to court international film makers but it was nt until director entity61 , who was born in entity37 , filmed the entity66 - winning hit entity65 on the island that it began to earn a reputation as an accommodating and affordable place to shoot . entity61 filmed the memorable and technically difficult scenes of a shipwrecked boy and a tiger at a purpose - built facility at abandoned airport in the entity37 city of entity73 . with its relatively unknown cityscape , entity37 can also function as a generic entity4 backdrop . the island , which was a entity77 colony , is already being used as a stand - in for entity77 . entity77 director entity79 used entity37 s high-speed rail systemina bullet-strewn action sequenceforcrime drama entity82 , which competed at the entity83.entity77 rail authorities turned him away.andentity8s silence , due to release in 2016 , is a historic drama about two entity87 priests who travel to entity77 . entity37 is also a popular alternative to entity90 , where there are many restrictions on filmmakers -- authorities can censor scripts considered politically sensitive or obscene . entity2 director entity96 used both entity37 and entity90 as locations in entity98 . dubbed entity90 s titanic , it focuses on a ship that sank when the entity102 government fled entity90forentity37in1949asthe entity103 took over--a sensitive periodinentity90 history.entity37andentity90 are still governed separately.entity90 authorities asked entity96 to tone down the heroics of a entity102 soldier,according to the entity109--notsomething that entity37 would ever require,says entity23.the commission offers incentivesforinternational film crews.up to $2millionisavailable per movie--half of thatasa cash subsidy.but justasimportantisthe islands versatility as a location , says entity23 . while many of the film crews are from neighboring entity4 countries , the city has hosted crews from entity126 and entity127 , while the entity128 shot some of its newly released drama entity132 in the city . it s a small island.within half an hour,you can gofromthe streets to the mountains to the sea.placeholderisshooting his latest movieinthe cityentity8entity23:Jaoentity24:Taipei Film Commissionentity36:Hou Hsiao-hsienentity82:Shield of Strawentity83:Cannes International Film Festivalentity87:Jesuitentity132:XYentity2:Hong Kongentity1:CNNentity0:Taipeientity7:Hollywoodentity11:Frenchentity4:Asianentity8:Martin Scorseseentity51:Shanghai Bank of China Towerentity79:Takashi Miikeentity54:Tom Cruiseentity77:Japanentity73:Taichungentity15:Lucyentity14:Scarlett Johanssonentity35:Edward Yangentity37:Taiwanentity10:Luc Bessonentity19:The Walking Deadentity18:Andrew Lincolnentity102:Nationalistentity103:Communistsentity128:BBCentity98:The Crossingentity126:Latviaentity96:Wooentity127:Germanyentity90:Chinaentity46:Mission Impossible IIIentity41:Shanghaientity42:Tokyoentity66:Oscarentity65:Life of Pientity109:South China Morning Postentity61:Lee三、机器阅读理解模型架构1、Deep LSTM Reader发表当前论文时2015年Transformer还没有问世。LSTM仍然是文本序列建模的主流模型所以作者使用LSTM作为特征提取器。输入问题和文档拼接输入到LSTM模型中输出两层LSTM的最后隐状态的拼接然后分类选出一个答案实体每一个样例都有一个候选实体列表2、Attention ReaderLSTMAttention【问题整体与文档中的每个token进行Attention计算“相关”权重】2.1 输入输入问题、文档分别用独立不同的LSTM进行特征提取2.2 交互问题整体与文档中的各个token进行Attention文档document中第t tt个token的隐层向量为y d ( t ) y → d ( t ) ∣ ∣ y ← d ( t ) y_d(t)\overrightarrow{y}_d(t)||\overleftarrow{y}_d(t)yd​(t)y​d​(t)∣∣y​d​(t)查询问题query整体的隐层向量为u y → q ( q ) ∣ ∣ y ← q ( 1 ) u\overrightarrow{y}_q(q)||\overleftarrow{y}_q(1)uy​q​(q)∣∣y​q​(1)即将第一个词的隐层向量后向LSTM的最后得到的隐层向量与最后一个词的隐层向量前向LSTM的最后得到的隐层向量进行相加得到问题的隐层向量计算Attention使用加法Attention2.3 输出输出经过Attention之后分类Softmax线性层选出一个答案实体3、The Impatient ReaderLSTM改进Attention【问题中的每个token与文档中的每个token进行Attention计算“相关”权重】2.1 输入输入问题、文档分别用独立不同的LSTM进行特征提取2.2 交互文档document中第t tt个token的隐层向量为与Attention Reader一致y d ( t ) y → d ( t ) ∣ ∣ y ← d ( t ) y_d(t)\overrightarrow{y}_d(t)||\overleftarrow{y}_d(t)yd​(t)y​d​(t)∣∣y​d​(t)查询问题query整体的隐层向量为与Attention Reader一致u y → q ( q ) ∣ ∣ y ← q ( 1 ) u\overrightarrow{y}_q(q)||\overleftarrow{y}_q(1)uy​q​(q)∣∣y​q​(1)查询问题query中第i ii个token的隐层向量为y q ( i ) y → q ( i ) ∣ ∣ y ← q ( i ) y_q(i)\overrightarrow{y}_q(i)||\overleftarrow{y}_q(i)yq​(i)y​q​(i)∣∣y​q​(i)其中∣ q ∣ |q|∣q∣表示 问题的长度r ( ∣ q ∣ ) r(|q|)r(∣q∣)表示r rr最终的向量表示2.3 输出输出经过复杂的Attention之后分类Softmax线性层选出一个答案实体四、实验结果1、模型对比传统机器阅读理解方法Maximum frequency选择最常出现的实体作为答案Exclusive frequency问题中未出现文本中最常出现的实体作为答案Frame-semantic model:语义解析Word distance model编辑距离等神经网络机器阅读理解方法神经网络的方法在大规模的数据集上效果优于传统的方法通过增加复杂的attention机制能够提升模型的表现对于不同的数据集不同的attention的方式效果会不同2、案例研究问题谁为moms创建了一场秋季时装秀?通过对模型中的attention进行可视化 可以发现对ent63的关注度最高通过阅读文章模型的关注度和人类回答问题时的关注度一致五、论文总结关键点如何构建一个大规模的语料提供给神经网络训练如何设计模型对文章和问题进行交互创新点首次提出了一个大规模的训练数据便于后续的研究提出了三个基本的神经网络模型以及较为复杂的attention机制启发点这一篇文章是针对完形填空的阅读理解。如何去构造不同的任务类型是后续可研究的地方在模型对实体进行分类的时候可解释性较差并为涉及到因果推理等关键问题数据集较为简单。文章只是简单的对问题和篇章进行了串联。如何更好的进行问题和篇章的交互后续也提出了一大批模型构建问题和篇章的交互。