南方科技大学翁文康团队提出低深度高鲁棒性的光学神经网络
FUTURE | 远见 闵青云 选编
近日，南方科技大学的翁文康教授与香港城市大学的张笑鸣博士合作，提出了一种低深度、高鲁棒性的光学神经网络硬件结构，该硬件结构在计算精度上具有指数提升。成果「Low-depth optical neural network」发表于全球新锐综合性研究期刊Chip，并被遴选为首刊封面文章。
光子芯片(photonic chip)是下一代高性能计算机的候选平台之一。基于它的相干光计算在机器学习领域的应用，也是最近的一个研究热点。光学神经网络（optical neural network）已在图像、语音识别等问题上实现了应用，展示出了其广阔的应用前景。然而，在通向大规模光计算工业应用的道路上，仍然存在着一些技术障碍。其中之一就是集成光子线路对噪声的鲁棒性问题。在中光子线路中，基本操作的误差会逐渐积累，影响最终计算结果的精度。光计算的精度会随着线路深度的增加而渐降低。以线性变换为例，传统的相干光学实现方法是基于奇异值分解，其线路深度随着数据维度线性增加。当输入数据足够大时，输出数据的准确性将无法被确保。因此，如何设计低深度的线路结构，是实现大规模、高精度的光计算，及其在机器学习领域应用的关键问题之一。
近日，南方科技大学的翁文康教授与香港城市大学的张笑鸣博士合作，提出了一种低深度、高鲁棒性的光学神经网络硬件结构。该硬件结构基于「光学点积计算单元」，每一个计算单元包含一个由干涉仪组成的二叉树，其线路深度随着输入数据的维度仅以对数形式增长。线路深度的降低可以极大地提高对于误差的鲁棒性。以线性变换的计算为例，对比于传统的奇异值分解方法，该硬件结构在计算精度上具有指数提升。作者通过解析和数值方法印证了该结果。
基于低深度的点积计算单元，该工作还设计了两类机器学习应用。第一类是光学线性感知器（optical linear perceptron）。通过训练，仅使用一个点积计算单元，便可以在高维数据中找到一个超平面，并对输入数据集进行线性二分类。第二类是光学神经网络。它由大量点积计算单元与光学非线性变换组合而成，理论上可以等效为任何前馈神经网络。因此，能够表达复杂的非线性函数关系。作者在鸢尾花卉(Iris)以及字符识别(letter recognition)数据集上进行了数值模拟，验证了算法的有效性，结果表明在相同的噪声模型影响下，相较于传统基于奇异值分解方法，基于低深度的光学机器学习算法都有更加优越的鲁棒性。
成果「Low-depth optical neural network」发表于杂志Chip。
Low-depth and Robust architecture for optical neuralnetworks
Photonic chip is a promising platform for the next generation of high-speed computing, and recently, many efforts have been made in developing on-chip optical neural networks for machine-learning applications. The rapid progress of photoniccomputation for machine learning shows its great potential in the tasks of image and speech recognition, clustering, etc.
Toachieve scalable optical computation and optical neural networks for industrialapplications, one of the main challenges is the robustness of large-scaleoptical devices in the presence of noise. The noise robustness is mainlydetermined by the depth of photonic circuits, because the error that occurs atany part of the circuit will accumulate to the final output results. Take linear transformations as an example,the conventional method is based on optical singular-value decomposition, whosecircuit depth increases linearly with the data dimension. In other words, thecomputation accuracy reduces significantly as the data size increase.Therefore, an outstanding question is whether there are schemes with betternoise robustness.
In recent work, researchers from Southern University of Science and Technology and City University of Hong Kong proposeda novel 「low-depth」 architecture for photonic computation and optical neural networks. In their framework, calculations are based on a computation unit for dot products, which contains a set of interferometers connected as a binarytree structure. In this way, dot products can be calculated with a circuit depth increasing only logarithmic with the data dimension. The low-depth nature of the architecture provides an exponential improvement on the noise robustness. For linear transformation, the authors provide analytic and numerical evidences that, under the same noise level, the computation error rate has an exponential reduction compared to the conventional approaches based on singular-value decomposition.
The authors also developed two machine-learning applications based on their architecture. The first one is the optical linear perceptron with a single computation unit. The well-trained linear perceptron can output a hyperplane separating two classes of data with different labels. The second one is the low-depth optical neural network (LD-ONN), which is the combination of the low-depth linear transformation scheme and the non-linear optical activations. Similar to the standard feed-forward neural networks, it can in principle represent any functions to arbitrary accuracy. Numerical results show that compared to conventional methods based on optical singular-value decomposition, the low-depth schemes have much better classification accuracies under the same noise level.
This proposal opens a new avenue toward scalable photonic computation for machine-learning applications. This work was published in Chip.
关于 Chip
Chip是全球唯一聚焦芯片类研究的综合性国际期刊，已入选由中国科协、教育部、科技部、中科院等单位联合实施的「中国科技期刊卓越行动计划高起点新刊项目」，为科技部鼓励发表「三类高质量论文」期刊之一。
Chip期刊由上海交通大学与Elsevier集团合作出版，并与多家国内外知名学术组织展开合作，为学术会议提供高质量交流平台。
Chip秉承创刊理念: All About Chip，旨在发表与芯片相关的各科研领域尖端突破，助力未来芯片科技发展。迄今为止，Chip已在其编委会汇集了来自12个国家的57名世界知名专家学者，其中包括多名中外院士及IEEE、ACM等知名国际协会终身会士（Fellow）。
Chip首刊将于2022年3月以完全开放获取形式发布。尽请期待！
文章预印版：
https://www.sciencedirect.com/science/article/pii/S2709472321000022?via%3Dihub
End