A revolution in the profiling world
Psychologists at Moscow State University, Lomonosov, have created a computer program that reads micromovements on a person’s face without including emotions in the analysis, better than professional profilers. The purpose of the “digital profiler” is to notice all changes (including very short durations) in the face associated with different human emotions: anxiety, fear, contempt, surprise, etc. According to scientists, it is this approach, and not the definition of emotions by subjective impression, that makes it possible to predict the true thoughts of a person. This program can be used with a polygraph, a device used to detect lies.
MV Lomonosov, a professor at the Department of Personality Psychology at the Faculty of Psychology at Moscow State University, Aleksey Gusev, Doctor of Psychology. – Since the world has not yet developed suitable algorithms for accurate facial analysis, we had to develop our own original computer vision algorithms. They analyze the change in the incident light on the face surface. And then these changes can be translated into meaningful information. And this is done with the help of multi-layered logical rules created by our scientific group. We have six levels of such rules.”
Alexander Kremlev, an employee of the Department of General Psychology at the Faculty of Psychology at Moscow State University, part of a scientific group of three, developed a new algorithm that analyzes the distribution of light on the face surface. For analysis, the right and left half of the face is divided into 14 zones, which in turn are divided into microzones – there are more than 300.
In each region, the algorithm searches for information about facial skin movement according to the international standard Facial Action Coding System (FACS), created by Paul Ekman, a renowned psychologist and consultant on the popular television series Lie to me. At the same time, the main character, Dr. It is the prototype of Lightman.
This is the only currently recognized system in the world for identifying facial movements.
Callous
An important point in the work of the “digital profiler” from Moscow State University is that the concept of emotion in facial analysis is not used in this way.
“We are not looking for emotions. We are looking for eyebrow shifting, squinting, mouth extension vertically, horizontally. We learned to measure 22 basic motor units that can be used to make any facial expression, including emotional expressions,” explains Mikhail Baev, a psychotherapist, facial analysis specialist, one of the developers of the “digital profiler.” expression.
FACS motor units are individually identifiable minimal facial movements. For example: lifting the inner part of the eyebrows, lifting the outer part of the eyebrows, the appearance of nasolabial folds, stretching the lower eyelid, stretching the mouth, etc.
A computer program “collects” expressions of various emotions, based on the selection of such mimic movements.
“It’s also important that facial movements can be very fast, between 200 and 500 milliseconds. Not every person can notice it. The neural networks we refuse to use also fail to notice this because they are trained on samples marked by experts. And experts only work on photos without micro-movements,” stressed Professor Gusev. .
Mixed feelings
Why not train the neural networks on a new dataset that will consist of videos, not photos?
It turns out that you need about 500,000 flagged video samples just to identify one emotion of anger. Also, there are not many experts in the world who understand facial expressions and their work is rather slow. The best expert evaluates one minute of video recording against ten minutes of work. It turned out that collecting such a dataset would be very costly, both in terms of money and time.
Additionally, according to the scientists, the neural network approach has already shown low efficiency in terms of replacing expert face coders.
“The appearance of expressions of true emotions on a person’s face is a rare occurrence, no more than 1-2 times an hour. Baev explains that the rest of the time we saw a fragmented facial expression. – Facial expressions are much more complex and not just emotions. After analysis, neural networks give the result as a percentage, for example: 10% anger, 20% contempt, and 5% joy. But what conclusion can be drawn from such an analysis? It goes against the concept of expressing emotions.”
In addition, you often need to analyze videos that, in principle, do not contain specific emotions: these are interviews before being hired, interrogation by a researcher, etc.
In this case, the only possible way is to analyze the entire spectrum of facial movements, including micro expressions.
After using the “digital profiler” from Moscow State University, the results are similar to the results of using a polygraph. In other words, algorithms recognize excitement, stress, tension in response to a specific question. However, since the polygraph wires are not connected to the person, the result can be interpreted more accurately. In a calm environment, he does not suspect that in parallel there is an analysis of micro-movements on his face.
The Future of Digital Profiler
At the output, experts get a lot of data because each participant is evaluated against hundreds of parameters. Since it is impossible to systematize such a large amount of data manually, a computer is also used to process them.
Experts are also trying to extract information about hidden emotions from the audiotape. Sound analysis is performed by changing more than 80 physical parameters. Layered Voice Analysis, a technology developed by the Israeli company Nemesysco, is used. This technology is called a voice lie detector.
“Digital Profiler” is currently involved in various projects of private companies. And its future application looks quite diverse.
The program can be used in medicine: to diagnose mental illness, to evaluate heart patients, to calculate the most successful face correction options in aesthetic medicine.
Undoubtedly, the “digital profiler” is indispensable in the work of researchers and HR professionals. Therefore, on the basis of this system, scientists plan to create various methods of work in the field of personnel selection and forensic psychological examination.
In addition, the “digital profiler” can effectively capture deep fake images, even in videos that are often impossible to see with your eyes. The program will do this by analyzing a person’s individual mimic motor profile. This profile is individual for everyone and it is extremely difficult, if not impossible, to fake it.
The system can also be applied in the field of animation and creation of robots. It will help to create completely natural facial expressions of the character without the use of acting and motion capture technology.
“It’s interesting that, despite all the advancement of technology, the problem of facial analysis has yet to be solved,” says Baev. “Understanding human facial expressions remains challenging for both machines and humans. But the trend to abandon neural networks in the analysis of emotions is evident in countries that have developed profiling.”
In parallel, scientists from Japan and England are working in the same direction.