Information theory and 5G/6G networks

Alexey Frolov, Associate Professor, CDISE

Meet Alexey Frolov, Skoltech Associate Professor at the Center for Computational and Data-Intensive Science and Engineering, and Applied Information Theory Group Lead. Alexey is passionate about information theory and its application and works on making it more available for solving everyday challenges. Enjoy his story!

research interests

My main area of research is information theory. The main argument in choosing this field was that information theory combines beautiful mathematics (probabilistic method, algebra, graph theory, etc.) and at the same time practical applications, such as the development of algorithms for wireless networks 5G / 6G. In life, such a combination is quite rare.

The problems of information theory are pretty close to the problems of machine learning and AI. Here's a simple example: consider the problem of reliable transmission of information. It is required to add redundancy to the transmitted data (having received the codeword) so that the receiver can recover the transmitted information. It is easy to see that this is a classification problem: the channel output must be assigned to one of the classes (codewords).

The problems of information theory are pretty close to the problems of machine learning and AI. Here's a simple example: consider the problem of reliable transmission of information. It is required to add redundancy to the transmitted data (having received the codeword) so that the receiver can recover the transmitted information. It is easy to see that this is a classification problem: the channel output must be assigned to one of the classes (codewords).

We conduct the scientist research on the border of information theory, communications and machine learning with a special focus on application of the research results in communications, including engineering and industry problems. Learn more.

main research areas of the group

At Skoltech, I am leading the Applied Information Theory group. We conduct scientific research at the intersection of information theory, telecommunications and machine learning. The main areas of research can be divided into three groups:

Currently, the field of application of AI and neural networks is expanding significantly. In particular, neural networks are actively used in the field of computer vision, improving the quality of images and videos, and text analysis. In this case, neural networks are used in most cases as a "black box" without any understanding of the internal mechanisms of their work, which is simply unacceptable in the context of solving critical tasks by neural networks. This direction is devoted to the analysis of training neural networks using information theory methods. The idea is to build estimates of two mutual information: between the output of the hidden layer of the neural network and the class label (this estimate shows how well the hidden layer has learned) and between the output of the hidden layer and the input dataset (this estimate shows how well the network has learned discard insignificant signs). Analysis of these values shows how well the neural network is trained. The main difficulty lies in the assessment of mutual information between vectors of large dimension.

Let's return to the decoding problem, which is a classification problem. An obvious and significant difference between this problem and traditional classification problems is the exponentially large (with a long code) number of classes. Indeed, after all, already a linear code with a length of 100 bits and a rate of 0.5 contains 250 codewords. Even this number of words is too large to fully train the neural network (in other words, show it all the words). Thus, the only way to defeat the curse of dimensionality is through a combination of existing decoding techniques and machine learning algorithms, resulting in ad-hoc neural network architectures.

It is known that with the advent of a quantum computer, encryption algorithms such as RSA and El-Gamal will cease to provide secrecy, as the problems of factorization and discrete logarithm will cease to be complicated (they are solved on a quantum computer in polynomial time). Thus, the problem arises of developing methods of post-quantum cryptography, including code cryptography and cryptography on lattices. Another important area is to ensure privacy and anonymity when training neural networks (differential privacy).

the future of AI in 5G/6G

The use of AI at physical level can significantly improve signal detection and decoding algorithms as well as the channel estimation algorithms. Another important application of AI is resource allocation and scheduling. While the use of AI can significantly improve the capacity of 5G networks, the system designers should pay attention to security and privacy issues: (a) it is known that AI algorithms are vulnerable to adversarial attacks, (b) collecting the network data should fulfill the privacy requirements (private user information should to be disclosed).

and now about Skoltech

We are Skoltech – a new international English-speaking STEM university that was founded by the group of world-renowned scientists in 2011 in Moscow, Russia. In just 8 years, we united dozens of researchers and globally renowned professors, built a stunning campus, set up world-class labs and made it to the top 100 young universities in the Nature Index. Read more >>

Discover Skoltech in less than 3 minutes