Currently, the field of application of AI and neural networks is expanding significantly. In particular, neural networks are actively used in the field of computer vision, improving the quality of images and videos, and text analysis. In this case, neural networks are used in most cases as a "black box" without any understanding of the internal mechanisms of their work, which is simply unacceptable in the context of solving critical tasks by neural networks. This direction is devoted to the analysis of training neural networks using information theory methods. The idea is to build estimates of two mutual information: between the output of the hidden layer of the neural network and the class label (this estimate shows how well the hidden layer has learned) and between the output of the hidden layer and the input dataset (this estimate shows how well the network has learned discard insignificant signs). Analysis of these values shows how well the neural network is trained. The main difficulty lies in the assessment of mutual information between vectors of large dimension.

Let's return to the decoding problem, which is a classification problem. An obvious and significant difference between this problem and traditional classification problems is the exponentially large (with a long code) number of classes. Indeed, after all, already a linear code with a length of 100 bits and a rate of 0.5 contains 250 codewords. Even this number of words is too large to fully train the neural network (in other words, show it all the words). Thus, the only way to defeat the curse of dimensionality is through a combination of existing decoding techniques and machine learning algorithms, resulting in ad-hoc neural network architectures.

It is known that with the advent of a quantum computer, encryption algorithms such as RSA and El-Gamal will cease to provide secrecy, as the problems of factorization and discrete logarithm will cease to be complicated (they are solved on a quantum computer in polynomial time). Thus, the problem arises of developing methods of post-quantum cryptography, including code cryptography and cryptography on lattices. Another important area is to ensure privacy and anonymity when training neural networks (differential privacy).