The possibilities of artificial intelligence of an unmanned vehicle on the roads are close to human ones, although in some cases they are insignificantly, but surpass them, a study by Cognitive Technologies has shown.
As the company told RIA Novosti, the purpose of the study was to determine who, at the current stage of creating unmanned vehicles, is better at understanding individual road situations - artificial intelligence (AI) or natural (EI) - that is, a person. "In other words: is the artificial human mind already winning in the task of controlling the drone or not?" - noted in the company
Cognitive Technologies specialists compared the capabilities that AI has already achieved. 17 volunteers took part in tests to assess the quality and speed of detection (detection, identification - Ed.) Of road signs, cars and pedestrians, carried out on public roads in Moscow and the Moscow region.
A number of tests were carried out in difficult weather conditions (dark time of day, rain), and tests were also carried out in such difficult-to-recognize conditions as the transition from light to shade and blinding by the sun. The volunteer, in accordance with the task, voiced certain objects of the road scene, noticed by him, in parallel, the results of recognition of the AI of the same line of objects were recorded. At the end of the trip, the data was processed and entered into the general test table.
The studies were carried out in September - November 2017, using one video camera (mono camera).
The tests were prepared in such a way that the number of objects that appear simultaneously did not exceed three, since otherwise it would be technically difficult for a person to voice the results of detection. On each route, a person was given no more than 30–35 objects to recognize.
In total, testing was carried out on 27 different routes in settlements at speeds of 50-60 kilometers per hour.
Robot versus human
Promotional video:
The study showed that the capabilities of artificial intelligence are already close to natural, in a number of cases the robot "defeated" a person, albeit by a small margin.
The first result of the study - determining the quality of detection of road scene objects in good weather and road conditions - were approximately equal results for EI and AI - both in speed and accuracy. According to the latest indicator for road signs, cars and pedestrians, they exceeded 99%.
AI performed better in tests when individual objects were not fully visible (behind trees, parked cars and other obstacles).
The advantage of AI was not in the quality of recognition, but in the speed of detection. For example, in tests on road signs, the AI was able to see partially obstructed objects for a split second earlier. “The test showed that in more difficult conditions, the subjects (volunteers - Ed.) Often noticed road signs momentarily later than the AI did. However, this time gives the control system an additional advantage for processing and analyzing information about the traffic situation in general,”said Yuri Minkin, head of the department of unmanned vehicles at Cognitive Technologies.
The exact reaction time of AI and humans under the conditions of this experiment was not recorded, its value fluctuated from about 0.5 seconds and did not exceed 1 second. The recognition accuracy of UM and AI in these tests exceeded 99%.
In tests with partially obscured cars and pedestrians, the AI advantage in speed was preserved, but it could already be considered as a statistical error (of the order of 0.5 seconds or less). The recognition accuracy of UM and AI was also more than 99%.
An insignificant advantage of AI in the speed and quality of detection was also noted in rain conditions when recognizing road signs - it was 98.3% versus 97% for EI. When recognizing cars, the results were approximately equal (about 98% with a difference of less than 0.5% in favor of AI). “This result can be explained by the greater resistance of the AI to this kind of interference when detecting objects of relatively small size,” the company explained.
In addition, the AI won in borderline situations: in situations of transition from light to shadow and blinding, the AI was faster at coping with the task of recognition. The human reaction time was longer in almost all tests, but did not exceed 1 second. The recognition accuracy of both UI and AI was more than 98%.
The tests showed that the person did a little better at recognizing pedestrians in difficult conditions (night, rain). The UM recognition accuracy was approximately 99.2% versus 98% for AI - both at night and in the rain. “A person is a rather difficult object to recognize: pedestrians do not have a constant shape, they can walk embracing, holding hands, carrying a load, and so on. And if in good conditions AI did not yield to humans, in difficult conditions the capabilities of AI have turned out to be slightly better so far,”Minkin noted.
conclusions
The main result of the study was the approximation of the capabilities of AI to EI. Moreover, as the test results showed, the advantage of AI manifested itself when the road and weather conditions became more complicated, the company noted. “We can expect that with an increase in computing power, as well as in the quality of sensors and software, the advantage of AI will become obvious, as it was in chess,” said a representative of Cognitive Technologies.
“Of course, many aspects of organizing and conducting research cannot claim to be perfect, and we will work together with the expert community on the development of both the methodology and the tests themselves. But you need to understand that this is the first attempt in history to compare the capabilities of AI and humans. And as a first approximation, we got real results that can and should be reckoned with, on the basis of which it is possible to predict the development of the direction of creating AI for autobots, said Olga Uskova, President of Cognitive Technologies.
According to her, the company expects to conduct the following tests, analyzing the quality of understanding of various road situations by humans and AI.
The company noted that the initial goal was not to compare the capabilities of artificial intelligence and humans.
“Initially, we were not going to compare the capabilities of humans and AI. We planned to test our own AI system for unmanned vehicles C-Pilot. The idea to compare AI with a human came already during the tests. But we got fired up with this idea, it became insanely interesting for us to get an assessment of the comparison of artificial intelligence with a person,”said Yuri Minkin, head of the department of unmanned vehicles at Cognitive Technologies.