Focusing on building big data, including advanced detailed maps and real-time sensing data.
Dr. Cha Won-yong conducts a detailed analysis of 110 Google self-driving car patents.

It was analyzed that patents for Google's self-driving cars are mainly concentrated in sensor systems, self-driving computer systems, and authentication systems between users and autonomous vehicles.

Dr. Cha Won-yong, a leading Korean futurist and the CEO of ASPAC Future Technology Management Research Institute, said this in his recently published book, “Intensive Patent Analysis of Google AI-Based Autonomous Cars,” and said, “Most of Google’s patents are aligned with Level (L) 3 and 4 as defined by the US SAE.” He added, “These are patents related to software design capabilities related to artificial intelligence (AI)-machine learning (ML)-deep learning (DL), which are the core of autonomous vehicles, and the situations, responses, and learning that occur during actual road driving tests based on this.”

Patents are concentrated on sensor systems, autonomous driving computer systems, and user-autonomous vehicle authentication systems.

Dr. Cha analyzed 110 of Google's important patents from 2009 to December 31, 2015, in relation to Google's Self-Driving Car (SDC) project, which is a semi-autonomous car (Self-Autonomous Car) and an autonomous car (Autonomous Car). Most of these patents are related to sensor systems, which are the core of autonomous vehicles, autonomous driving computer systems, and authentication systems between users and autonomous vehicles when commercialized.

Google's acquisition of these patents was a hidden effort, and it's ongoing. From 2009 to January 31, 2016, Google conducted road tests with 55 semi-autonomous vehicles in two cities across two states. Since beginning testing in California with a Toyota Prius in 2009, the company has driven a total of 3.85 million kilometers (2,408,597 miles), of which 2.27 million kilometers were in autonomous mode. This represents approximately 60% of the total, and Google increased this figure to 80% last year.


▲ Google conducted road tests with 55 semi-autonomous vehicles in two cities across two states from 2009 to January 31, 2016. (Photo: Google)

So, does the fact that autonomous driving isn't even 90% accurate mean there are still many technical flaws? Dr. Cha Won-yong bluntly responds, "That's also part of the learning process." He pointed out that “most people only know about Google’s real-world road tests, but if there were a very powerful simulator that could allow semi-autonomous cars to learn before they even leave the garage, they could learn in the garage,” and that “in these simulators’ virtual labs, Google’s semi-autonomous cars are learning by driving 3 million miles (4.8 million km) of virtual roads a day.”

In this regard, he added that the passage explicitly mentioning the disabling of autonomous driving functions should also be noted. Google's self-driving car has disabling its functions 341 times, most of which (304 times) occurred on public roads. In contrast, only 32 instances occurred on highways. The reasons for disabling autonomous driving mode were weather conditions and unexpected situations that could occur on the road, such as jaywalking. Dr. Cha stated, "The important point is that Google has identified the causes of these function disablings and trained its self-driving cars in a virtual lab, and these results are reflected in the patent application."

Focus on fusion sensor systems and fusion algorithms

What does Google's patent analysis reveal about sensor systems, autonomous driving computer systems, and user-autonomous vehicle authentication systems? In sensor technology, which is one of the core technologies of autonomous vehicles, the fusion sensor system and fusion algorithm (Sensor Fusion Algorithm) composed of sensors such as cameras, radar, lidar, ultrasound, and audio installed on the roof, mirrors, front and rear bumpers, side doors, or under the body are drawing attention.


▲ Google Self-Driving Car Project (Photo Google)

The next crucial factor is driving data. In other words, autonomous driving requires a combination of a detailed prior map based on big data and detailed real-time sensing data. This means conducting driving tests in manual mode, where the driver drives the vehicle, scanning road types, lane widths, lane shapes (dotted/solid lines, etc.), shoulders, signs, guardrails, trees, obstacles, intersection types, and traffic signal types. Based on this data, a new, precise and detailed map must be created.

This advanced detailed map is stored in the data of the autonomous driving computer system memory and can be retrieved when driving in autonomous mode and compared and analyzed with the sensor's real-time sensing map (Detailed Real-Time Map). Dr. Cha said, “This kind of comparative analysis requires building up big data in advance, which is why Google has conducted 3.85 million kilometers of actual driving tests so far.”


▲Last year, the Ministry of Trade, Industry and Energy held a self-driving car contest at the Daegu Intelligent Automobile Parts Testing Center .

Finally, a highly detailed model of traffic patterns must be created based on the pre-detailed map and real-time sensing data. Depending on the location of the pre-detailed map, the model includes other vehicle attributes or properties of moving objects, such as the distribution of typical or expected speeds of the autonomous vehicle, trajectories along lanes, and where it accelerated or de-accelerated (speed-regulated).

Through continuous testing, the vehicle can detect and model the driving patterns of these other vehicles to predetermine threshold values. When a vehicle exceeds these thresholds, the vehicle alerts the driver to take control of the steering wheel and switches from autonomous mode to manual mode. If other vehicles drive normally within these thresholds, the autonomous vehicle automatically switches to autonomous mode.

Urgent need to build an ecosystem through domestic companies' platforms
One way is to focus on the sensor fusion system field.

Dr. Cha pointed out that, with Google developing a fully autonomous vehicle utilizing artificial intelligence, it is urgent to build an ecosystem through a platform of domestic large corporations, small and medium-sized enterprises, and venture companies. He brought up the concept of autonomous vehicles, explaining, "According to the definition of autonomous vehicles stipulated in the California law permitting autonomous driving testing, an autonomous vehicle is not a vehicle equipped with one or more collision avoidance systems... (omitted) An autonomous vehicle is a vehicle that has the ability to drive autonomously or comprehensively without the active control or monitoring of a human operator."

He analyzed that if we focus our capabilities on areas where we excel, such as sensor fusion systems, we will be able to dominate the global market. In the book's preface, he stated, "Although Google's patents show a significant lead, analyzing patents from other industries could provide a strategy to circumvent this." He added, "Furthermore, like the United States, Korea needs to invest in autonomous vehicles and road infrastructure with a long-term perspective. While autonomous vehicle research and development is crucial, we must not forget that existing road infrastructure, including advanced detailed maps, must be reorganized to enable autonomous driving."