Interview with Tomáš Mikolov, Joining CIIRC CTU and RICAIP Centre Since April 2020
We took this opportunity to ask Tomáš Mikolov what decided him in favour of CIIRC CTU and what areas of research he will continue to focus on.
What led you to the idea of returning to the Czech Republic?
I’ve been abroad with the odd break since 2010. I began working full time in California in 2010, later in New York and also for a year in Paris. So, I’ve been gone for the past decade. It was time to go back to the Czech Republic. I had the impression I’d tried everything I needed in my field and saw what needed to be seen. I managed to work with a number of scientists who are the most cited experts in their fields. For example, I had the opportunity at various times to work with scientists who have won the Turing Prize, which is the equivalent of the Nobel Prize in IT (note: the ACM A.M. Turing Award is awarded annually to individuals for contributions of lasting and major technical importance to the computer field by the Association of Computing Machinery). So, a further stay abroad was no longer the main motivation for me. There are smart people everywhere in the world, including the Czech Republic. A scientist’s success is often decided by a combination of different factors, including chance – some can be influenced, others can’t. Anyway, great research can be done more or less anywhere, especially when it comes to basic research. You just need inspiration and good ideas for basic research in our field.
But why come back to the Czech Republic?
It’s something greatly underestimated in the Czech Republic, but when you look around the world, you realise that things work well here and it’s a good place to live. We don’t have earthquakes or major natural disasters. People often only realise that when they spend a few years abroad and come back. I honestly don’t understand questions like why would you move to Prague from New York. New York is a nice city, but, for example, public transport is much better in Prague. What’s more, people are less aggressive here. In terms of tourist numbers, that will probably be a draw. When I started comparing, I realised I could do the same research here. I was never interested in going to a prestigious university like MIT or Stanford. It would probably look better on my CV, but I was never that bothered. I never aimed to be a rich and famous scientist. Things just turned out like that on their own (laughs).
What was decisive about the work at CIIRC CTU?
It wasn’t a clear decision from the start; it was a combination of factors. I also considered other workplaces. However, CIIRC CTU impressed me as one of the most progressive and attractive in terms of location and the new building. What’s more, I found the ecosystem of people already working at CIIRC interesting. There are a number of people here who, like me, have worked abroad for several years in top research and, like me, have returned to the Czech Republic. They have similar experience and quite possibly similar views on research or education. The three most prominent scientists I’ve been communicating with since the beginning are Jan Šedivý, who invited me here first, as well as Josef Urban and Josef Šivic. It’s important for me to do things that make sense to me with people I enjoy working with.
You mentioned that you will continue to do basic research. What do you see as its main purpose?
The concept of basic research is often difficult to grasp. Scientists work on ideas that often appear abstract. For outsiders, it’s something that has no concrete results. It can take a long time for an idea to be put into practice. But in the end, it can all happen faster than people imagine: who would have thought a hundred years ago that people would land on the moon in a few decades? Basic research is a bit like roulette, you don’t know where the ball will drop. Sometimes it’s chance or luck – you try twenty or thirty ideas, it turns out that four work, but only one of them is ultimately put into practice. The probability of making a fundamental discovery is quite small, but its effect and impact can be enormous. It moves us forward, we are inventing completely new things, new machines, medicines. That’s what basic research is about – we have to try and move forward. Innovation wouldn’t be possible without basic research. That’s often forgotten. We frequently only see the icing on the cake when the result has already been put into practice. At the same time, when we look at successful technology companies today, whether its Google, Facebook, Microsoft or Apple, these companies are often based on discoveries made by scientists some 20-30 or more years ago. These companies got rich on the internet, but neither Google nor Facebook invented the internet. In order to have new things and for that system to work, we need the support of basic research and subsequently to support putting the results of basic research into practice. Then, if this result is commercially successful, no further support is required.
How do you see current research in artificial intelligence?
In the past, research on artificial intelligence focused on scientists trying to copy some part of the human decision-making process. This can be well demonstrated, for example, using chess. They said to themselves – in order for a computer to play chess like a human, it must be at least as clever and intelligent as a human in order to understand such a complex game. It’s a relatively complex intellectual activity. Gradually, however, it turned out that it doesn’t have to be such a complex intellectual activity. The computer can go through millions of combinations of moves, try them out and judge which of these combinations will give it the best result. It can thus play chess well and does not have to understand the game at all.
In the past, artificial intelligence was about scientists looking for shortcuts to achieve a particular result. We define one concrete, narrowly specified task. If the computer can solve it adequately, it may be possible to come up with techniques that can be generalised and applied to other tasks. That was the idea, but it could not be entirely accomplished. Personally, I don’t think it will happen in the foreseeable future.
Couldn’t the research of neural networks that you’ve been doing so far help in this?
Although neural networks can be used for a number of tasks, it’s always based on the principle of supervised learning. That means that you always have to tell the computer what to do. If you show it a million examples of what a cat and dog look like, the computer will be able to distinguish a cat from a dog perhaps 99.9% of the time. It appears to be working. But then if you show it a kitten or a puppy, where the pictures look a little different, you will find that you have to start from the very beginning. Teach it again. That’s inefficient. Transfer between tasks is negligible compared to human abilities. It’s the same with chess or go games. When you change one rule of the game or place the pieces differently, the algorithms completely fall apart and it doesn’t work. You have to start building the solution from scratch.
So, how are complex systems different?
Complex systems could lead to general or strong artificial intelligence, as the scientific community calls it. Algorithms should have the potential for independent intelligence like humans. But no one knows that today, no one can build it. Even world-renowned scientists, such as Geoffrey Hinton, Yoshua Bengio, or Yann Lecun, with whom I have spoken about it, have any idea how to create truly strong artificial intelligence. When I was at Google Brain, Facebook AI research, I visited dozens of other research groups. I don’t know anyone who currently knows how to move forward.
That’s why we have to try new ideas. Many people are doing applied research where they take what works. For example, they can teach a classifier that can recognise cats and dogs to recognise something else. So, they take another lot of data from another domain. But that’s still just another application of something that was already invented, for example, ten or twenty years ago.
So, is there another solution?
Complex systems are systems with lots of simple elements that can interact with each other. Intelligence in complex systems is not something we define explicitly. We don’t put it in the system directly. On the contrary, we try to create systems where a kind of gradual artificial evolution creates the complexity – intelligence. It’s similar to natural evolution on Earth. Here, too, the complexity of life gradually increased over millions of years. Millions or billions of years ago, life would simply have been unable to solve the tasks that we can and must solve now as humans.
Is there something to build on in the field of complex systems or it is a completely new field?
Ideas about complex adaptive systems are nothing new. Scientists who built the first computers were already thinking about how to make smart computers as early as the 1950s. Evolution was one of the main interesting ideas. The situation now is similar to that of neural networks. Mathematical models of neurons and the brain can be traced back to the 1940s. A number of scientists were already working on neural networks in the 1980s, when there was a period of about ten years that artificial neural networks were relatively popular among scientists. However, they fell out of favour in the 1990s, as it became clear that simpler models could handle similar tasks just as well as neural networks. No one thought they could work better. I saw the potential of neural networks, I believed they could be used in more areas. That we could use the same model to solve language, translations, speech recognition, image recognition. The idea of neural networks was definitely not new, but rather a question of how to get it across the finish line, how to get it going, to make it work – and it was my generation that succeeded.
Does that mean you now want to get complex systems off the ground?
You could say that. You can compare it to the idea of building a plane. People had the idea 500 years ago. But it took a few more centuries to learn how to make it work. How to get the plane off the ground. My colleagues and I managed to get neural networks off the ground about ten years ago.
Now I think it could work with complex systems. Again, this is not a new idea that no one’s had before. But today’s complex systems are not used where they could actually work. They are used in areas that don’t make sense. They are often replaced by simpler models.
I think there are currently no good ideas on how to get complex systems off the proverbial ground. It may be a bit of a lottery, because no one knows what will and won’t work and how long such research will take. But it’s a very creative activity.
Is anyone else helping you in your efforts?
I’m already building a small team of students and doctoral students at CIIRC. I have a student from France and one from Charles University in my team. I think our group will have around five people in a year or two. My goal is to have several people around me with similar scientific interests, who are not afraid to try to invent something new, basically from scratch.