We develop a general duality between neural networks and compositionalkernels, striving towards a better understanding of deep learning. We show thatinitial representations generated by common random initializations aresufficiently rich to express all functions in the dual kernel space. Hence,though the training objective is hard to optimize in the worst case, theinitial weights form a good starting point for optimization. Our dual view alsoreveals a pragmatic and aesthetic perspective of neural networks andunderscores their expressive power.
translated by 谷歌翻译