序
前言
常用符號錶
第一部分 機器學習基礎
第1章 緒論3
1.1人工智能...............................4
1.1.1人工智能的發展曆史....................5
1.1.2人工智能的流派.......................7
1.2機器學習...............................7
1.3錶示學習...............................8
1.3.1局部錶示和分布式錶示...................9
1.3.2錶示學習...........................11
1.4深度學習...............................11
1.4.1端到端學習..........................12
1.5神經網絡...............................13
1.5.1人腦神經網絡........................13
1.5.2人工神經網絡........................14
1.5.3神經網絡的發展曆史....................15
1.6本書的知識體係...........................17
1.7常用的深度學習框架.........................18
1.8總結和深入閱讀...........................20
第2章 機器學習概述23
2.1基本概念...............................24
2.2機器學習的三個基本要素......................26
2.2.1模型..............................26
2.2.2學習準則...........................27
2.2.3優化算法...........................30
2.3機器學習的簡單示例——綫性迴歸.................33
2.3.1參數學習...........................34
2.4偏差-方差分解............................38
2.5機器學習算法的類型.........................41
2.6數據的特徵錶示...........................43
2.6.1傳統的特徵學習.......................44
2.6.2深度學習方法........................46
2.7評價指標...............................46
2.8理論和定理..............................49
2.8.1PAC學習理論........................49
2.8.2沒有免費午餐定理......................50
2.8.3奧卡姆剃刀原理.......................50
2.8.4醜小鴨定理..........................51
2.8.5歸納偏置...........................51
2.9總結和深入閱讀...........................51
第3章 綫性模型
3.1綫性判彆函數和決策邊界......................56
3.1.1二分類............................56
3.1.2多分類............................58
3.2Logistic迴歸.............................59
3.2.1參數學習...........................60
3.3Softmax迴歸.............................61
3.3.1參數學習...........................62
3.4感知器.................................64
3.4.1參數學習...........................64
3.4.2感知器的收斂性.......................66
3.4.3參數平均感知器.......................67
3.4.4擴展到多分類........................69
3.5支持嚮量機..............................71
3.5.1參數學習...........................73
3.5.2核函數............................74
3.5.3軟間隔............................74
3.6損失函數對比.............................75
3.7總結和深入閱讀...........................76
第二部分 基礎模型
第4章 前饋神經網絡81
4.1神經元.................................82
4.1.1Sigmoid型函數.......................83
4.1.2ReLU函數..........................86
4.1.3Swish函數..........................88
4.1.4GELU函數..........................89
4.1.5Maxout單元.........................89
4.2網絡結構...............................90
4.2.1前饋網絡...........................90
4.2.2記憶網絡...........................90
4.2.3圖網絡............................90
4.3前饋神經網絡.............................91
4.3.1通用近似定理........................93
4.3.2應用到機器學習.......................94
4.3.3參數學習...........................95
4.4反嚮傳播算法.............................95
4.5自動梯度計算.............................98
4.5.1數值微分...........................99
4.5.2符號微分...........................99
4.5.3自動微分...........................100
4.6優化問題...............................103
4.6.1非凸優化問題........................103
4.6.2梯度消失問題........................104
4.7總結和深入閱讀...........................104
第5章 捲積神經網絡109
5.1捲積..................................110
5.1.1捲積的定義..........................110
5.1.2互相關............................112
5.1.3捲積的變種..........................113
5.1.4捲積的數學性質.......................114
5.2捲積神經網絡.............................115
5.2.1用捲積來代替全連接....................115
5.2.2捲積層............................116
5.2.3匯聚層............................118
5.2.4捲積網絡的整體結構....................119
5.3參數學習...............................120
5.3.1捲積神經網絡的反嚮傳播算法...............120
5.4幾種典型的捲積神經網絡......................121
5.4.1LeNet-5............................122
5.4.2AlexNet...........................123
5.4.3Inception網絡........................125
5.4.4殘差網絡...........................126
5.5其他捲積方式.............................127
5.5.1轉置捲積...........................127
5.5.2空洞捲積...........................129
5.6總結和深入閱讀...........................130
第6章 循環神經網絡133
6.1給網絡增加記憶能力.........................134
6.1.1延時神經網絡........................134
6.1.2有外部輸入的非綫性自迴歸模型..............134
6.1.3循環神經網絡........................135
6.2簡單循環網絡.............................135
6.2.1循環神經網絡的計算能力..................136
6.3應用到機器學習...........................138
6.3.1序列到類彆模式.......................138
6.3.2同步的序列到序列模式...................139
6.3.3異步的序列到序列模式...................139
6.4參數學習...............................140
6.4.1隨時間反嚮傳播算法....................141
6.4.2實時循環學習算法......................142
6.5長程依賴問題.............................143
6.5.1改進方案...........................144
6.6基於門控的循環神經網絡......................145
6.6.1長短期記憶網絡.......................145
6.6.2LSTM網絡的各種變體...................147
6.6.3門控循環單元網絡......................148
6.7深層循環神經網絡..........................149
6.7.1堆疊循環神經網絡......................150
6.7.2雙嚮循環神經網絡......................150
6.8擴展到圖結構.............................151
6.8.1遞歸神經網絡........................151
6.8.2圖神經網絡..........................152
6.9總結和深入閱讀...........................153
第7章 網絡優化與正則化157
7.1網絡優化...............................157
7.1.1網絡結構多樣性.......................158
7.1.2高維變量的非凸優化....................158
7.1.3神經網絡優化的改善方法..................160
7.2優化算法...............................160
7.2.1小批量梯度下降.......................160
7.2.2批量大小選擇........................161
7.2.3學習率調整..........................162
7.2.4梯度估計修正........................167
7.2.5優化算法小結........................170
7.3參數初始化..............................171
7.3.1基於固定方差的參數初始化.................172
7.3.2基於方差縮放的參數初始化.................173
7.3.3正交初始化..........................175
7.4數據預處理..............................176
7.5逐層歸一化..............................178
7.5.1批量歸一化..........................179
7.5.2層歸一化...........................181
7.5.3權重歸一化..........................182
7.5.4局部響應歸一化.......................182
7.6超參數優化..............................183
7.6.1網格搜索...........................183
7.6.2隨機搜索...........................184
7.6.3貝葉斯優化..........................184
7.6.4動態資源分配........................185
7.6.5神經架構搜索........................186
7.7網絡正則化..............................186
7.7.1?1和?2正則化........................187
7.7.2權重衰減...........................188
7.7.3提前停止...........................188
7.7.4丟棄法............................189
7.7.5數據增強...........................191
7.7.6標簽平滑...........................191
7.8總結和深入閱讀...........................192
第8章 注意力機製與外部記憶197
8.1認知神經學中的注意力.......................198
8.2注意力機製..............................199
8.2.1注意力機製的變體......................201
8.3自注意力模型.............................203
8.4人腦中的記憶.............................205
8.5記憶增強神經網絡..........................207
8.5.1端到端記憶網絡.......................208
8.5.2神經圖靈機..........................210
8.6基於神經動力學的聯想記憶.....................211
8.6.1Hopfiel網絡........................212
8.6.2使用聯想記憶增加網絡容量.................215
8.7總結和深入閱讀...........................215
第9章 無監督學習219
9.1無監督特徵學習...........................220
9.1.1主成分分析..........................220
9.1.2稀疏編碼...........................222
9.1.3自編碼器...........................224
9.1.4稀疏自編碼器........................225
9.1.5堆疊自編碼器........................226
9.1.6降噪自編碼器........................226
9.2概率密度估計.............................227
9.2.1參數密度估計........................227
9.2.2非參數密度估計.......................229
9.3總結和深入閱讀...........................232
第10章 模型獨立的學習方式235
10.1集成學習...............................235
10.1.1AdaBoost算法........................237
10.2自訓練和協同訓練..........................240
10.2.1自訓練............................240
10.2.2協同訓練...........................240
10.3多任務學習..............................242
10.4遷移學習...............................245
10.4.1歸納遷移學習........................246
10.4.2轉導遷移學習........................247
10.5終身學習...............................249
10.6元學習.................................252
10.6.1基於優化器的元學習....................253
10.6.2模型無關的元學習......................254
10.7總結和深入閱讀...........................255
第三部分 進階模型
第11章 概率圖模型261
11.1模型錶示...............................262
11.1.1有嚮圖模型..........................263
11.1.2常見的有嚮圖模型......................264
11.1.3無嚮圖模型..........................267
11.1.4無嚮圖模型的概率分解...................267
11.1.5常見的無嚮圖模型......................269
11.1.6有嚮圖和無嚮圖之間的轉換.................270
11.2學習..................................271
11.2.1不含隱變量的參數估計...................271
11.2.2含隱變量的參數估計....................273
11.3推斷..................................279
11.3.1精確推斷...........................279
11.3.2近似推斷...........................282
11.4變分推斷...............................283
11.5基於采樣法的近似推斷.......................285
11.5.1采樣法............................285
11.5.2拒絕采樣...........................287
11.5.3重要性采樣..........................288
11.5.4馬爾可夫鏈濛特卡羅方法..................289
11.6總結和深入閱讀...........................292
第12章 深度信念網絡297
12.1玻爾茲曼機..............................297
12.1.1生成模型...........................299
12.1.2能量最小化與模擬退火...................301
12.1.3參數學習...........................302
12.2受限玻爾茲曼機...........................304
12.2.1生成模型...........................305
12.2.2參數學習...........................307
12.2.3受限玻爾茲曼機的類型...................308
12.3深度信念網絡.............................309
12.3.1生成模型...........................310
12.3.2參數學習...........................310
12.4總結和深入閱讀...........................313
第13章 深度生成模型317
13.1概率生成模型.............................318
13.1.1密度估計...........................318
13.1.2生成樣本...........................319
13.1.3應用於監督學習.......................319
13.2變分自編碼器.............................319
13.2.1含隱變量的生成模型....................319
13.2.2推斷網絡...........................321
13.2.3生成網絡...........................323
13.2.4模型匯總...........................323
13.2.5再參數化...........................325
13.2.6訓練..............................325
13.3生成對抗網絡.............................327
13.3.1顯式密度模型和隱式密度模型...............327
13.3.2網絡分解...........................327
13.3.3訓練..............................329
13.3.4一個生成對抗網絡的具體實現：DCGAN..........330
13.3.5模型分析...........................330
13.3.6改進模型...........................333
13.4總結和深入閱讀...........................336
第14章 深度強化學習339
14.1強化學習問題.............................340
14.1.1典型例子...........................340
14.1.2強化學習定義........................340
14.1.3馬爾可夫決策過程......................341
14.1.4強化學習的目標函數....................343
14.1.5值函數............................344
14.1.6深度強化學習........................345
14.2基於值函數的學習方法.......................346
14.2.1動態規劃算法........................346
14.2.2濛特卡羅方法........................349
14.2.3時序差分學習方法......................350
14.2.4深度Q網絡..........................353
14.3基於策略函數的學習方法......................354
14.3.1REINFORCE算法......................356
14.3.2帶基準綫的REINFORCE算法...............356
14.4演員-評論員算法...........................358
14.5總結和深入閱讀...........................360
第15章 序列生成模型365
15.1序列概率模型.............................366
15.1.1序列生成...........................367
15.2N元統計模型.............................368
15.3深度序列模型.............................370
15.3.1模型結構...........................370
15.3.2參數學習...........................373
15.4評價方法...............................373
15.4.1睏惑度............................373
15.4.2BLEU算法..........................374
15.4.3ROUGE算法.........................375
15.5序列生成模型中的學習問題.....................375
15.5.1曝光偏差問題........................376
15.5.2訓練目標不一緻問題....................377
15.5.3計算效率問題........................377
15.6序列到序列模型...........................385
15.6.1基於循環神經網絡的序列到序列模型...........386
15.6.2基於注意力的序列到序列模型...............387
15.6.3基於自注意力的序列到序列模型..............388
15.7總結和深入閱讀...........................390
附錄數學基礎 393
附錄A 綫性代數 394
附錄B 微積分 404
附錄C 數學優化 413
附錄D 概率論 420
附錄E 信息論 433
索引 439
· · · · · · (收起)