过去十年中机器学习的快速改进开始产生深远的影响。对于通信,具有有限的domainexpertise的工程师现在可以使用现成的学习包来设计基于模拟的高性能系统。在机器学习的当前革命之前,大多数通信工程师都非常清楚可以使用随机梯度下降来学习系统参数(例如滤波器系数)。然而,一点也不清楚,系统架构中更复杂的部分也可以学习。在本文中,我们讨论了机器学习技术在双通信问题中的应用,并着重于从结果系统中学到的东西。我们惊喜地发现,在一个例子中观察到的收益有一个简单的解释,事后才明白。从本质上讲,深度学习发现了一种简单而有效的策略,这种策略以前没有被考虑过。
translated by 谷歌翻译
The problem of low complexity, close to optimal, channel decoding of linearcodes with short to moderate block length is considered. It is shown that deeplearning methods can be used to improve a standard belief propagation decoder,despite the large example space. Similar improvements are obtained for themin-sum algorithm. It is also shown that tying the parameters of the decodersacross iterations, so as to form a recurrent neural network architecture, canbe implemented with comparable results. The advantage is that significantlyless parameters are required. We also introduce a recurrent neural decoderarchitecture based on the method of successive relaxation. Improvements overstandard belief propagation are also observed on sparser Tanner graphrepresentations of the codes. Furthermore, we demonstrate that the neuralbelief propagation decoder can be used to improve the performance, oralternatively reduce the computational complexity, of a close to optimaldecoder of short BCH codes.
translated by 谷歌翻译
在本文中,我们提出了一种基于置信传播(BP)和深度学习的极性稀疏神经网络解码器(SNND)。首先,将极性BP解码的常规因子图转换为类似于低密度奇偶校验(LDPC)码的bipartiteTanner图。然后展开Tannergraph并将其翻译成神经网络(DNN)的图形表示。复数和积算法(SPA)是具有低复杂度的经修改的tomin-sum(MS)近似。我们通过使用单一权重来参数化网络,从而显着减少重量。通过深度学习的训练技术进行优化,提出的SNND实现了SPA的比较解码性能,并且在($ 128,64 $)和($ 256,128 $)代码上获得了大约0.5美元的MS解码增益。此外,降低了60美元\%的复杂性,并且解码延迟明显低于传统的极地BP。
translated by 谷歌翻译
A novel deep learning method for improving the belief propagation algorithmis proposed. The method generalizes the standard belief propagation algorithmby assigning weights to the edges of the Tanner graph. These edges are thentrained using deep learning techniques. A well-known property of the beliefpropagation algorithm is the independence of the performance on the transmittedcodeword. A crucial property of our new method is that our decoder preservedthis property. Furthermore, this property allows us to learn only a singlecodeword instead of exponential number of code-words. Improvements over thebelief propagation algorithm are demonstrated for various high density paritycheck codes.
translated by 谷歌翻译
A novel and efficient neural decoder algorithm is proposed. The proposed decoder is based on the neural Belief Propagation algorithm and the Automorphism Group. By combining neural belief propagation with permutations from the Automorphism Group we achieve near maximum likelihood performance for High Density Parity Check codes. Moreover, the proposed decoder significantly improves the decoding complexity, compared to our earlier work on the topic. We also investigate the training process and show how it can be accelerated. Simulations of the hessian and the condition number show why the learning process is accelerated. We demonstrate the decoding algorithm for various linear block codes of length up to 63 bits.
translated by 谷歌翻译
在统计上明确定义的信道上可靠地进行通信的代码设计是一项重要的工作,涉及深入的数学研究和广泛的实际应用。在这项工作中,我们提出了通过深度学习获得的第一个代码家族,这些代码在几十年的研究中设计了显着的代码。正在考虑的通信信道是带有反馈的高斯噪声信道,其研究由Shannon发起;理论上已经知道反馈以提高通信的可靠性,但是没有成功地构建这样做的实用代码。我们通过将信息理论洞察力与基于循环神经网络的编码器和解码器相结合来创建新的代码,从而创造出超过已知代码3个数量级的可靠性。我们还展示了代码的几个理想特性:(a)对较大块长度的推广,(b)与已知代码的可组合性,(c)对实际约束的适应性。这一结果对编码理论也有更广泛的影响:即使信道具有明确的数学模型,加深学习方法与信道特定信息理论见解相结合,也有可能击败数十年数学研究所构建的最先进的代码。
translated by 谷歌翻译
在本文中,我们介绍了综合症丢失,这是一种基于综合症放松的神经纠错解码器的替代损失函数。综合症丢失惩罚解码器以产生不对应于有效码字的输出。我们表明,对于许多短码代码而言,使用该综合症的训练会使解码器具有始终较低的帧错误率,在训练期间几乎没有额外的成本,并且没有额外的成本推断。所提出的方法不依赖于传输的码字的知识,使其成为用于在线自适应改变信道条件的有前途的工具。
translated by 谷歌翻译
The training complexity of deep learning-based channel decoders scales exponentially with the codebook size and therefore with the number of information bits. Thus, neural network decoding (NND) is currently only feasible for very short block lengths. In this work, we show that the conventional iterative decoding algorithm for polar codes can be enhanced when sub-blocks of the decoder are replaced by neural network (NN) based components. Thus, we partition the encoding graph into smaller sub-blocks and train them individually, closely approaching maximum a posteriori (MAP) performance per sub-block. These blocks are then connected via the remaining conventional belief propagation decoding stage(s). The resulting decoding algorithm is non-iterative and inherently enables a high-level of parallelization, while showing a competitive bit error rate (BER) performance. We examine the degradation through partitioning and compare the resulting decoder to state-of-the-art polar decoders such as successive cancellation list and belief propagation decoding.
translated by 谷歌翻译
We revisit the idea of using deep neural networks for one-shot decoding of random and structured codes, such as polar codes. Although it is possible to achieve maximum a posteriori (MAP) bit error rate (BER) performance for both code families and for short codeword lengths, we observe that (i) structured codes are easier to learn and (ii) the neural network is able to generalize to codewords that it has never seen during training for structured, but not for random codes. These results provide some evidence that neural networks can learn a form of decoding algorithm, rather than only a simple classifier. We introduce the metric normalized validation error (NVE) in order to further investigate the potential and limitations of deep learning-based decoding with respect to performance and complexity.
translated by 谷歌翻译
我们提出并讨论了物理层深度学习的几种新应用。通过将通信系统解释为自动编码器,我们开发了一种将通信系统设计视为端到端重建任务的基本新方法,该任务旨在在单个过程中联合优化发送器和接收器组件。我们展示了如何将这种想法扩展到多个发射器和接收器的网络,并将无线电变压器网络的概念作为在机器学习模型中结合专家领域知识的手段。最后,我们展示了卷积神经网络在原始IQ样本上的应用,用于调制分类,相对于依赖于专家特征的传统方案,它实现了竞争准确性。本文最后讨论了未来调查的开放性挑战和领域。
translated by 谷歌翻译
Capacity-achieving polar codes have gained significant attention in recent years. In general, polar codes can be decoded by either successive cancellation (SC) or the belief propagation (BP) algorithm. However, unlike SC decoders, performance optimizations for BP decoders have not been fully explored yet. In this paper, we explore novel early stopping criteria for polar BP decoding to significantly reduce energy dissipation and decoding latency. First, we propose two detection-type novel early stopping criteria for detecting valid outputs. For polar (1024, 512) codes, these two stopping criteria can reduce the number of iterations by up to 42.5% at 3.5 dB. Then, we propose a novel channel condition estimation approach, which can help select different stopping criteria in different SNR regions. Furthermore, the hardware archi-tectures of polar BP decoders with the proposed stopping criteria are presented and developed. Synthesis results show that with the use of the proposed stopping criteria, the energy dissipation, and average latency of polar (1024, 512) BP decoder can be reduced by 10 30 with 2 5 hardware overhead, and average throughput can be increased by 20 55 .
translated by 谷歌翻译
Coding theory is a central discipline underpinning wireline and wireless modems that are the workhorses of the information age. Progress in coding theory is largely driven by individual human ingenuity with sporadic breakthroughs over the past century. In this paper we study whether it is possible to automate the discovery of decoding algorithms via deep learning. We study a family of sequential codes parametrized by recurrent neural network (RNN) architectures. We show that creatively designed and trained RNN architectures can decode well known sequential codes such as the convolutional and turbo codes with close to optimal performance on the additive white Gaussian noise (AWGN) channel, which itself is achieved by breakthrough algorithms of our times (Viterbi and BCJR decoders, representing dynamic programing and forward-backward algorithms). We show strong generalizations, i.e., we train at a specific signal to noise ratio and block length but test at a wide range of these quantities, as well as robustness and adaptivity to deviations from the AWGN setting.
translated by 谷歌翻译
在低延迟约束下设计信道代码是5G标准中最苛刻的要求之一。然而,传统代码的性能的清晰表征仅在大块长度限制中可用。代码设计由那些渐近分析引导,并且需要大块长度和长延迟以实现期望的错误率。此外,当为一个信道(例如,加法高斯白噪声(AWGN)信道)设计的代码用于另一个信道(例如非AWGN信道)时,启发式必须实现任何非平凡的性能 - 其中严重缺乏不稳定性和适应性。通过联合设计基于回归神经网络(RNN)的编码和解码器获得,我们提出了一种端到端学习神经码,其在块设置下优于经典卷积码。凭借这种设计新型神经阻滞码的经验,我们提出了一类新的码延迟约束 - 低延迟高效自适应鲁棒神经(LEARN)码,其性能优于现有技术的低延迟码以及禁止码。稳健性和适应性。学习代码显示了为现代深度学习的未来通信技术和通信工程见解设计新的通用和通用代码的潜力。
translated by 谷歌翻译
本文提出了深度学习辅助迭代检测算法形成超载多输入多输出(MIMO)系统,其中发射天线的数量$ n(\ gg 1)$大于接收天线$ m $的数量。由于所提出的算法基于具有可训练参数的投影梯度下降方法,因此将其命名为可训练的投影梯度检测器(TPG-检测器)。可以使用标准深度学习技术(例如后向传播和随机梯度下降算法)来优化诸如步长参数之类的可训练内部参数。这种方法被认为是数据驱动的调整带来了所提出的方案的显着优点,例如快速收敛。 TPG检测器的主要迭代过程包括矩阵向量乘积运算,需要$ O(m n)$ - 时间的迭代步骤。此外,TPG探测器中可训练参数的数量与天线数$ n $和$ m $无关。 TPG探测器的这些特征导致其快速和稳定的训练过程以及对大型系统的合理可扩展性。数值模拟表明,所提出的检测器实现了与用于大规模过载MIMO信道的已知算法(例如,最先进的IW-SOAV检测器)的检测性能相当,具有较低的计算成本。
translated by 谷歌翻译
We study two families of error-correcting codes defined in terms of very sparse matrices. "MN" (MacKay-Neal) codes are recently invented, and "Gallager codes" were first investigated in 1962, but appear to have been largely forgotten, in spite of their excellent properties. The decoding of both codes can be tackled with a practical sum-product algorithm. We prove that these codes are "very good," in that sequences of codes exist which, when optimally decoded, achieve information rates up to the Shannon limit. This result holds not only for the binary-symmetric channel but also for any channel with symmetric stationary ergodic noise. We give experimental results for binary-symmetric channels and Gaussian channels demonstrating that practical performance substantially better than that of standard convolutional and concatenated codes can be achieved; indeed, the performance of Gallager codes is almost as close to the Shannon limit as that of turbo codes.
translated by 谷歌翻译
深度学习领域的最新发展促使许多研究人员将这些方法应用于量子信息中的问题。 Torlaiand Melko首先提出了一种基于神经网络的表面码解码器。从那时起,许多其他研究人员已经应用神经网络来研究解码环境中的各种问题。这方面的一个重要发展是由于Varsamopoulos等人。谁提出了使用神经网络的两步解码器。 Maskara等人的后续工作。使用相同的概念来解码各种噪声模型。我们提出了一种类似的两步神经解码器,它使用逆奇偶校验矩阵用于拓扑颜色代码。我们表明它在2D六边形颜色代码上优于非神经解码器的独立Pauli误差噪声模型的最先进性能。我们的最终解码器独立于噪声模型,达到了10美元/%的门槛。我们的结果与Maskara等人最近关于量子误差校正的神经解码器的工作相当。我们的解码器在培训成本方面具有显着的优势与Maskara等人相比,网络的复杂程度更高。我们提出的方法也可以扩展到任意维度和其他稳定器代码。
translated by 谷歌翻译
本文提出了深度学习辅助迭代检测算法形成超载MIMO信道。该算法基于用于稀疏信号恢复的迭代软阈值算法。所提出的方案的显着特征是检测器具有相当低的计算成本并且包含可训练的参数,其可以利用标准深度学习技术进行优化。可训练参数的数量与通道大小不变,这促进了检测器的快速和稳定的训练过程。数值模拟表明,对于大规模过载MIMO信道,所提出的检测具有与最先进的IW-SOAV检测器相当的检测性能。
translated by 谷歌翻译
我们为加性高斯白噪声(AWGN)信道提出了一种新的极性码构造框架(即,选择冷冻比特位置),为给定的解码算法量身定制,而不是基于连续消除(SC)的(非必要的最佳)假设。解码。所提出的框架基于遗传算法(GenAlg),其中信息集的人口(即集合)基于其各自的错误率性能通过进化变换连续演化。这些人群朝向适合解码行为的信息集收敛。使用我们提出的算法,我们构造一个长度为2048的极性码,码率为0.5,没有CRC辅助,适合于普通的连续消除列表(SCL)解码,实现与CRC辅助SCL解码相同的误码率性能,并导致BER为10 ^ { - 6} $时的编码增益为1 dB。此外,信念传播(BP) - 尾部极性代码在不对解码算法本身进行任何修改的情况下接近SCL错误率性能。
translated by 谷歌翻译
Starting from Shannon's celebrated 1948 channel coding theorem, we trace the evolution of channel coding from Hamming codes to capacity-approaching codes. We focus on the contributions that have led to the most significant improvements in performance vs. complexity for practical applications, particularly on the additive white Gaussian noise (AWGN) channel. We discuss algebraic block codes, and why they did not prove to be the way to get to the Shannon limit. We trace the antecedents of today's capacity-approaching codes: convolutional codes, concatenated codes, and other probabilistic coding schemes. Finally, we sketch some of the practical applications of these codes.
translated by 谷歌翻译
在本文中,神经网络辅助比特交织编码调制(NN-BICM)接收机被设计用于减轻LDPC编码的直流偏置光正交频分复用(DCOOFDM)系统中的非线性削波失真。将交叉熵作为损失函数,通过反向传播算法训练前馈网络,通过softmax激活函数输出条件概率,从而辅助改进的对数似然比(LLR)改进。为了减少复杂性,该前馈网络利用单个符号和相应的高斯方差简化了输入层,而不是关注多个子载波之间的载波间干扰。在神经网络辅助BICM和灰色标记的基础上,我们提出了一种具有迭代解码(NN-BICMID)的比特交织编码调制的新型堆叠网络架构。通过在最后一次迭代中从LDPC解码器中的外部LLR导出的先验概率计算条件概率,在每个迭代时间分别定制神经网络检测器的费用,其性能得到进一步改善。利用最佳DC偏置作为动态区域的中点,仿真结果表明NN-BICM和NN-BICM-IDchems都比其他同类产品获得了显着的性能提升,其中NN-BICM-ID明显优于NN-BICM。调制和编码方案。
translated by 谷歌翻译