Research on Driver Intentional Reasoning for Intelligent Vehicles

The team of Professor Dongpu Cao from the University of Waterloo in Canada proposed to further clarify and model the driver’s brain intention generation mechanism based on driver behavior analysis, hoping to obtain accurate driving intention judgments before the driver performs specific vehicle operations.

The team of Professor Dongpu Cao from the University of Waterloo in Canada proposed to further clarify and model the driver’s brain intention generation mechanism based on driver behavior analysis, hoping to obtain accurate driving intention judgments before the driver performs specific vehicle operations.

Significance

The development of smart cars and the popularization of advanced driver assistance systems have brought great guarantees to vehicle safety, and further improved the driving safety and comfort of vehicles. However, with the development of vehicle intelligence, the intelligent control unit and the driver are increasingly sharing the underlying control of the vehicle. It is inevitable that the intelligent car will “seize Power” the driver or interfere with the driver at important moments. Make a control strategy that is beneficial to the driver’s own interests, which in turn will cause safety hazards. Therefore, smart cars cannot ignore the understanding and perception of the highest decision maker of the vehicle-the driver. The advanced driver assistance system at this stage has preliminary monitoring functions of the driver’s behavior, such as eye recognition to prevent fatigue driving, action recognition to prevent distraction, and driver emotion recognition. However, from the perspective of smart car co-driving, only detecting the facial features and behavior of the driver is still difficult to meet the needs of smart driving of the vehicle. Therefore, the team of Professor Dongpu Cao from the University of Waterloo in Canada proposed to further clarify and model the driver’s brain intention generation mechanism based on driver behavior analysis, hoping to obtain accurate driving intention judgment before the driver performs specific vehicle operations.

The reasoning of driving intention can make the driver assistance system understand and assist the driver’s driving tasks more comprehensively. Taking lane-changing intention as an example, the prediction of lane-changing intention helps the vehicle environment perception system to detect the driver’s driving area of ​​interest early and realize blind spot detection and driving warning. At the same time, it can also avoid the problem of conflicts with the road maintenance system on the control of the vehicle caused by the driver’s irregular lane changing operations, such as not turning on the turn signal. The team is improving the human brain intention recognition model, and at the same time applying this model to decision-making and planning problems of high-level unmanned vehicles. By learning the intention generation and decision-making mechanism of skilled drivers, it can guide future unmanned vehicles in the anthropomorphic decision-making problems in similar road environments. Driver’s intentional reasoning can also promote the research of driver’s situation awareness (situation awareness), by learning the driver’s driving characteristics, driving knowledge, and vehicle interaction methods to establish a more complete driver’s cognitive model.

Intentional reasoning

Accurately distinguishing the driver’s intention level is the premise of driving intention reasoning. Driver intentions can be classified from many aspects, such as time scale, direction of movement, and the number of intention prediction tasks. From the time scale, it can be divided into strategic level (Strategical Level), task level (Tactical Level), and control level (Operational Level), as shown in Figure 1. The strategic intent at the top level is the overall planning of the current driving task, such as choosing a relevant route, driving strategy, destination, and so on. The time scale is the longest, usually in minutes or hours. Task-level driving intention is the focus of research, which includes various common driving behaviors, such as lane changing, steering, and braking. Due to the randomness of the road environment, this part of the driving intention cannot be judged as accurately as the strategy-level intention, and can only be reversed by relying on the sequential driver’s behavior characteristics. This level of driving intention is usually at the minute or second level. The lowest control level intention is the concrete manifestation of the task level intention, such as the driver’s horizontal and vertical control of the vehicle. The control-level intent is faster than the first two levels of intent, usually at the second or millisecond level. Each task-level intent is usually composed of a series of manipulation-level intents. Therefore, inferences about task-level intentions can be derived by identifying driver-related manipulation behaviors. Other driver intention classification methods include horizontal and vertical intention classification methods based on the direction of vehicle movement, and intention prediction tasks, such as a single intention and multi-intent fusion prediction method.

Research on Driver Intentional Reasoning for Intelligent Vehicles
figure 1.Driver Intent Time Scale Classification

Intentional reasoning

The method of driver intention reasoning mainly focuses on the modeling methods of machine learning, such as generative models (hidden Markov model, dynamic Bayesian network) and discriminative models (support vector machine, feedforward neural network, decision tree) Wait. Discriminant models mostly use non-time-series feature data as model input and the model is equivalent to a classification network for driving intention judgment. The generative model, taking the Hidden Markov Model as an example, allows short-term time series data to be modeled. However, due to the volume and depth of the model, it is usually difficult to capture the dynamic characteristics of long-term time series data, and the predictive ability and accuracy of the model are low. According to the UCSD Trivedi team, the generative model is more effective for the task of multi-intention reasoning than the discriminative model. In recent years, with the development of deep learning technology, time-series deep neural networks have gradually been used in the decision-making and planning of drivers or smart vehicles. As the current mainstream time series neural network, deep loop neural network is also used for driver’s intention reasoning. Combined with long and short-term memory networks, deep cyclic neural networks have a deeper network structure, which can save driver behavior characteristics for a longer period of time, which is conducive to obtaining the dependencies between long-term sequential driving behaviors and establishing a more accurate intention reasoning model.

In addition to learning-based methods, Drexel University’s D. Salvucci and others also developed a cognitive process representation method based on a mathematical model from the perspective of cognitive psychology, which can explain some of the mechanisms and mechanisms of intention generation. However, it is difficult to effectively use driver behavior based on a clear mathematical model. Data cannot fully consider the impact of other mental factors such as driving style, distraction or fatigue on driving intentions. At the same time, smart vehicles can have hundreds of sensors at every turn, and mathematical representation methods are difficult to make full use of time-series vehicle dynamics to accurately predict driving intentions. The common driving intention modeling method is shown in Figure 2.

Research on Driver Intentional Reasoning for Intelligent Vehicles
figure 2.Classification of Driver Intention Modeling Methods

Intentional reasoning evaluation index

The evaluation index of intentional reasoning can be judged from two aspects: accuracy and advancement. Taking lane-changing intention as an example, the lane-changing intention can be simply divided into straight line keeping, left-lane change, and right-lane change based on the driver’s lane-changing behavior. Through supervised learning to make the model produce corresponding prediction results, combined with receiver operating characteristic (ROC) curve and other classification accuracy judgment methods, the prediction accuracy of driving intention can be measured. The evaluation of advancement is relatively complicated. Taking the important time of lane change shown in Fig. 3 as an example, time T1 represents the driver’s intention to change lanes. Due to the unpredictability of implicit lane changing intentions, it is difficult to accurately measure the specific intention generation time. T2 means that the driver starts to perform the lane-changing operation and the vehicle crosses the lane line of the current lane at T3, and finally completes the complete lane-changing behavior at T4. Therefore, the purpose of lane change intention reasoning is to determine the lane change behavior of the vehicle before T3. More strictly, it is necessary to predict the current driving intention before the driver starts to perform the lane change operation at T2. With the shortening of observation time, the difficulty of intent reasoning is also increasing.

Research on Driver Intentional Reasoning for Intelligent Vehicles
image 3.Important moments in the lane change process

Outlook

The current research on the driver’s intention is mainly focused on a single intention reasoning method, which usually requires the assumption that the driver will complete the lane-changing action and only use the successful lane-changing data in the model building process. This is usually difficult to obtain satisfactory prediction accuracy in complex traffic scenarios. At the same time, the relationship between the driver’s intention and the state of other drivers has not yet been established. For example, different levels of attention or fatigue can produce different driving behaviors. In the future, it is necessary to comprehensively combine driver state analysis and driving environment analysis data to establish a robust driver state adaptive model to accurately predict driving intentions.

The Links:   https://www.slw-ele.com/lt121s1-153.html“> LT121S1-153 LTM190E1-L03 IGBTMODULE