Before smartphones were measured? Cores, megapixels, gigahertz. Now everything has rushed forward so much in terms of technology that everyone has optimal values for these indicators, and what they are trying to push above the norm is just a marketing ploy.
However, now there is one position on which smartphones are starting to stick out themselves as cooler - this is artificial intelligence.
Remember the modern advertising slogans: "processor with a neuromodule", "camera with AI", "smart algorithms" and other "magic" phrases that push us to buy a new device.
But how does it work, and most importantly, what is the use of artificial intelligence?
How did AI end up in a smartphone?
An ordinary program solves a problem according to a certain algorithm, which was written by a person. And if there are many such tasks, or if the initial data and conditions are constantly changing? A person cannot process tons of information and write thousands of algorithms. So let the soulless machine do it!
The main thing is to set it up correctly, and then it will start looking for the optimal solution on its own. This is how artificial intelligence and its subsection machine learning appeared. One of the machine learning algorithms is neural networks that you've probably heard of.
Promotional video:
We have been using AI in our smartphone for a long time. Plotting a route taking into account traffic jams, searching for cats among photos in the "cloud", communicating with a voice assistant - all this would be impossible without smart algorithms. These tasks are solved remotely: the request is sent to the server, processed, and the response comes back.
Why not put AI right inside the smartphone? He has already learned a lot "out of the box", he learns constantly with us. The capacities of modern devices allow. We immediately get two advantages:
- it's fast: no need to waste time and traffic on data transmission;
- it's safe: all data is stored inside the smartphone, you do not share it with third-party services.
Manufacturers have even allocated a separate unit for AI inside the mobile processor - NPU (Neural Processing Unit, neural module). This is an accelerator of machine learning algorithms, with it the smartphone solves problems even faster and requires less energy.
The premium version of the Mate 20 Pro with 512 GB of memory and leather inserts - Porsche Design Huawei Mate 20 RS.
The first processor with NPU was Huawei Kirin 970. It is installed, for example, in smartphones Huawei P20 Pro and nova 3. And in 2018 the world's first processor with dual neuromodules Kirin 980 was released. NPU performance increased by 226%, and energy efficiency - by 182 %. The flagships Huawei Mate 20 Pro and Mate 20 are equipped with such a processor.
So why does a smartphone need artificial intelligence?
Face recognition
Almost all modern smartphones support face unlocking. In inexpensive models, the technology is simple: the front camera takes a picture of the face, and special algorithms compare it with a given pattern using key points. Coincided - your own, did not coincide - a stranger.
This technology does not work well in the dark, and there is a problem with security: if a smartphone is shown a photograph of the owner or a similar person, there is a chance of a coincidence.
More expensive smartphones use a different method. For example, in the Huawei nova 3, two front-facing cameras form a 3D image, and an IR light helps in the dark. The flagship Huawei Mate 20 Pro generally projects 30 thousand points onto the face, builds a 3D model, and then compares it with the original model.
And if you complicate the task? Wear glasses or a hat, grow a beard or put on makeup? It turns out a completely different face. Then machine learning comes to the rescue. AI notices such changes, makes adjustments to the model and realizes that in front of him is the owner of the smartphone. Thanks to the NPU, these operations take a fraction of a second.
Improving autonomy
Neural networks can also predict the future. Cope with this better than psychics. AI in a smartphone analyzes our habits, notices behavioral scenarios (launching applications at a certain time, inactivity during sleep, etc.) and itself understands which applications are needed at a certain moment and which are not.
Important programs and processes will be loaded into memory in advance so that they open faster, and unnecessary ones will be deleted so as not to consume power. Check your email first thing at work? The app is already waiting for you. Do Maps and Instagram work at night? Out of memory, now is not the time.
Digging even deeper, AI predicts processor load based on frame rate. As a result, dynamic scenes seem to us smooth, without "twitching". This is especially important in games.
Optimizing performance
Artificial intelligence even drives performance. Let's explain using the Kirin 980 as an example: the main processor (CPU) has only 8 cores, of which 2 work at high frequency, 2 at medium and 4 at high. Clearly - in the picture (the higher the column - the higher the power):
Reading books and listening to music in the background doesn't require a lot of resources. Why load all cores? Let only low frequencies work. Facebook, Vkontakte are "heavier" - then the "average" are connected to work.
But tough hacks in PUBG or epic battles in WoT at maximum settings, video rendering are resource-intensive tasks, then everyone dances: the CPU and GPU (graphics processing unit) work at maximum power. AI notices all this and distributes power optimally.
Improving sound quality
Images can be more than just visual. AI is also responsible for sound quality.
When you talk on the phone, the AI immediately "separates" the voice from extraneous sounds and amplifies it, and the background noise mutes it. As a result, you can be heard even when a strong wind is blowing, the subway cars are thundering, or your favorite band is annealing at a concert.
Blurring the background and highlighting the color
Films with special effects on a smartphone are also related to neural networks. Modern flagships of Huawei are able to shoot video with a blurred background: the subject remains clear, and the foreground and background are blurred in bokeh. This effect is usually achieved with expensive high-aperture lenses.
Another trick is color highlighting: the AI understands who you are shooting (it must be a person) and keeps it in focus. Predictive (read - predictive movement) focusing helps. Then the AI paints only the subject, and leaves the surrounding world in black and white. Remember the movie Sin City? There is the same effect.
Offline translator
There is such a translator for Android and iOS - Microsoft Translator. In 2018, he learned how to translate texts without an internet connection, thanks in large part to the artificial intelligence system. And in partnership with Huawei, Microsoft has refined Translator for the Kirin 980 processor.
What came out of it: on smartphones of the Huawei Mate 20 line, machine translation accuracy increased by 23% and came close to human translation. At the same time, text processing speed is 3 times higher than on other flagship smartphones, and language packs "weigh" 2 times less.
Photo enhancement
Perhaps the most noticeable influence of AI we see in photography. The camera algorithms determine what exactly we are shooting and in what conditions, and they themselves select the best settings: contrast, saturation, white balance.
For example, smartphones with the Kirin 980 processor recognize up to 1,500 scenarios from 25 categories - even before you press the shutter button. Neural networks distinguish cats from dogs, snow from leaves, sea from sky. And there are different settings for everyone.
AI also understands which mode is best to shoot. Point the camera at the document - the "Scanner" mode is activated, which selects only the area with text. The Kremlin is in the frame - the "Architecture" mode is turned on. Aim at a person - turns on portrait mode with blur.
Portraits are generally open to AI. You can twirl bokeh with different strengths and shapes, apply makeup and change the type of lighting around. A smartphone cannot replace a DSLR with professional optics, but making a gorgeous portrait on an avatar is easy.
Image segmentation
Pattern recognition and categorization are the main tasks that machine learning solves. AI arranges and combines photos on a smartphone based on similarities.
Photos of gamburegres, salads, fruits are food, they have a separate folder. AI puts pictures of cars in another folder, buildings in a third. If neural networks find someone's face in different images, they will also be in folders, but inside the “faces” category.
Do you want to take away all the photos with your mother or grandmother? Here are all the shots where they got into the frame. Have you photographed an important document long ago, but forgot when? Here are all the frames that have text. Instead of manually enumerating hundreds, thousands of files, 10-15 are left to check. It's comfortable.
Smart stabilization
Popular image stabilization technologies in smartphones are OIS and EIS, optical and electronic, respectively. Huawei has developed AIS - an AI-based stabilizer. It stands for AI Image Stabilization.
AIS is best at shooting night photos. As you can see: press the button for 4-6 seconds (without a tripod!), And you get a sharp picture with excellent detail and a minimum of noise.
How the smartphone sees it:
- first 0.2 seconds AIS determines the position of the camera in space. Accuracy - up to 98%;
- then the camera takes many photos with different parameters;
- AI selects the best shots. If there was a shift, it combines them, after which it processes each section and removes noise.