Recent commercial vehicles are being equipped with a higher level of automated driving systems (ADS). These ADS, such as lane centering assist (LCA), adaptive cruise control (ACC), and automatic emergency braking (AEB), enable vehicles to drive autonomously without the intervention of driver under certain circumstances (Alrefaie et al., 2019; NHTSA, 2013; SAE International, 2018). In conventional driving without ADS, drivers are not recommended to perform non-driving related task (NDRT) because it can cause distraction and safety issues (Liang et al., 2021). Thus, performing NDTR has been considered secondary and distracting tasks. However, the increase in automation level enables drivers to immerse diverse tasks even if it is not related to driving.

The Society of Automotive Engineers (SAE) classified driving automation into six levels, from level 0 (no automation) to level 5 (full automation) (SAE International, 2018). It is expected that the driver's role in vehicles will completely change when we get to the era of fully autonomous vehicles. Human intervention will be required in different tasks depending on the level of automation. Therefore, research on how drivers interact with different levels of automation is important to ensure safe driving (Köhn et al., 2019). In particular, it is important to focus on how driver's behavior changes due to the level of automation and the new type of tasks demanded by drivers for each level. For instance, vehicles of level 3 (conditional automation) and level 4 (high automation) automation will be featured with automated systems that will assist most of the driving journeys depending on the environment (NHTSA, 2013). Thus, drivers will be only required to intervene in some emergency situations in these levels (Yoon et al., 2021; Eriksson and Stanton, 2017).

The intervention of drivers to regain control of the vehicle in highly automated vehicles is defined as the takeover process. The most important task for drivers in a highly automated driving situation is to be able to regain control of the vehicle, it is critical to take into consideration driver's state before the takeover request is issued. For instance, drivers can interact with their smartphones, watch a video, or read a book during automated driving mode and then be required to regain control of the vehicle. These changes in driver behavior and states are considered important in studies of highly automated vehicles. In this study, we focus on assessing and evaluating the NDRT used in previous research and how these tasks influence the takeover process. Hence, this will enable to understand the changes in task characteristics, as well as takeover performance for safe driving in highly automated vehicles. This study aims to analyze the effects of NDRT on the takeover process in highly automated driving by applying a systematic literature review (SLR) method. The following research questions were addressed to achieve the objective:

RQ1: What types of NDRT were used to evaluate drivers' takeover performance in highly automated driving?

RQ2: What measures were used to evaluate the takeover process in relation to NDRT in highly automated driving?

RQ3: How did NDRT influence takeover process in highly automated driving?

We applied the SLR method to achieve the research goal and answer the RQs (Booth et al., 2016). Among the various methods, we followed the preferred reporting items for systematic reviews and meta-analysis (PRISMA) (Moher et al., 2009) which includes four stages: identification, screening, eligibility, and final selection. Figure 1 shows the flow of included and excluded articles based on the four stages. During the search process, we defined three keyword groups. The first group included 'autonomous', 'automated', and 'self-driving'. The second group of keywords included 'takeover' and 'transition of control'. The final group consisted of 'non-driving related task', 'physical task', 'cognitive task', and 'visual task'. We made combinations of three keywords from each group and searched articles with the "AND" Boolean; for instance: 'automated + takeover + non-driving related task'. A total of 24 keyword combinations were used to collect the review articles. Science Direct, Web of Science, ACM Digital Library were selected as databases. The search process was conducted in December 2021. In the first stage, identification, we collected 265 articles (Science Direct: 128, Web of Science: 78, and ACM Digital Library: 59). After eliminating the redundant articles, a total of 146 articles were left. In the screening stage, we reviewed the articles based on the following criteria: 1) include full papers or full proceedings papers, 2) include articles published after 2010, and 3) exclude abstract-only proceedings, book chapters, thesis, and technical report. At this stage, a total of 66 articles were left. In the eligibility stage, we reviewed the paper based on the title and abstract considering the following criteria: 1) include articles from the perspective of Human Factors and Human-Computer Interaction and 2) exclude articles that were purely technical, developmental, electrical/mechanical/computer engineering relevant. In the final selection stage, we selected articles that were related to autonomous vehicles and highly automated vehicles. After completing this procedure, 27 articles were selected for the final analysis. All the selected papers were fully reviewed in terms of RQs.

Figure 1. Flow of the systematic review process

Modify Delete

3.1 Types of NDRT for evaluating drivers' takeover performance in highly automated driving

The NDRTs used in autonomous vehicle research can be classified into two categories: experimental tasks and natural tasks. The experimental tasks include activities that are conducted in the controlled environment of a laboratory to observe human behaviors. These tasks are characterized as easily controllable by the researchers. On the other hand, natural tasks refer to real activities that can be performed in real driving contexts. The experiment can secure ecological validity with natural tasks compared to using experimental tasks. However, it is more difficult to control experimental task conditions.

Table 1 shows a summary of the experimental tasks used to evaluate driver behavior in highly autonomous driving research. Experimental tasks mostly consisted of activities previously used in the psychological research area to observe the mental and cognitive behaviors of humans. For instance, the experimental NDRT included the Modified Psychology Experiment Building Language visual search task, Little Man task, Surrogate Reference Task, Peripheral Detection Task, Reading Span Task, and n-back tasks. These tasks were selected to provide drivers with different levels of cognitive load, engagement on the NDRT, or type of visual load during automated driving. It is possible to notice that studies using experimental tasks could control the psychological state of each of the participants and therefore were able to focus on determining how driver's psychological condition influences the takeover process.

As presented in Table 2, naturalistic tasks include the no-task condition (not doing anything), phone conversation, reading, conversing, watching a video, interacting with an in-vehicle infotainment system (IVIS), drinking, using an electronic device, using a smartphone, playing mobile games, texting on the phone, listening to music, and grooming. As we can see from the result of the description of the naturalistic tasks, these tasks can be considered secondary tasks in conventional driving and therefore as commonly performed activities by the drivers. From the results of the analysis of previous studies, we encountered one important characteristic of these tasks. In previous studies using natural tasks, the experimenter found it difficult to control the way in which each of the participants was engaged in the naturalistic task. That is to say, the same natural task can vary between experiments and even among participants in the same research. This is because naturalistic tasks were asked to be performed by the driver; however, the driver can perform the same task in different ways. For instance, in a phone conversation task, the topic of conversation (level of cognitive demand, engagement, attention to the conversation), the way of interaction (hands-free or private phone call), and hands in use (left hand, right hand, no hands in use) are not fixed. Therefore, naturalistic tasks were performed in a less controlled environment, presenting different drivers' levels of demand regarding the driver's physical, cognitive, and visual resources in use.

The result of the analysis of NDRT helps us classify tasks into two main groups, depending on how and what aspects of driver behavior are going to be evaluated or assessed. Both approaches have pros and cons depending on the research objectives. Therefore, it is possible to suggest that the main difference between the experimental and naturalistic tasks is the trade-off between the ecological validity of NDRT and the controlled NDRT conditions for the research. The selection of NDRT for future autonomous vehicle research is highly related to the research objective, hypothesis, and the desired quality of the results. For the stage in which takeover in highly automated driving research is being conducted, it is possible to notice that most articles opt to use natural tasks as an initial study of the influence of NDRT on the takeover process.

3.2 Evaluation measurement for takeover in relation to NDRT

There are several measurements used to evaluate the transition of control in highly automated driving for drivers engaged in NDRT. Based on the analysis of previous research, performance measures can be divided into two main categories. The first category consists of measures that evaluate the process involved in the transition of control between the vehicle and the drivers. This process consists of measuring and evaluating the performance of drivers with regards to the allocation of visual attention on the road, the physical allocation of hands and feet to drive, and cognitive resource allocation to gain situational awareness. These measures mostly evaluate the performance of the drivers to regain control of the vehicle and they can be defined as takeover measures. The second category consists mainly of the evaluation of driver's state after the takeover process. That is, it measures and evaluates the quality of driving performance after the transition of control was performed. This second category can be named post-takeover measures. Table 3 shows a summary of the measures used in previous research to evaluate the effect of NDRT in the transition of control in highly automated driving.

Takeover measures assess the performance of the takeover by analyzing the whole takeover task and by looking up in detail each of the sub-process involved. Based on previous articles, the takeover task is analyzed by assessing the performance by evaluating and gathering data on the takeover difficulty, response time, takeover readiness, and the transition method selected to regain control of the vehicle. Also, it includes measures that assess the sub-process of takeover process which can be divided depending on the resources used during the re-engagement task. Thus, measures are classified into motoric, visual, and cognitive performance. For the motoric measure, most studies focused on analyzing the time taken by drivers to react to the takeover request by assessing the time taken to allocate drivers' hands on the wheel. Visual measures focus on the time taken for drivers to first gaze or fix their visual attention on the road. For the cognitive evaluation, situation awareness and workload of drivers when regaining control of the vehicle after being engaged in an NDRT, are assessed.

Post-takeover measures evaluate driving performance after the transition of control. These measures are used to identify the results of how safely drivers were able to regain control of the vehicle after the transition. Measures include the time of collision with an obstacle and the number of collisions due to unsuccessful transition of control. According to previous studies, the engagement of drivers with NDRT can influence the success and failure of a takeover which is related to the level of safety and quality of the takeover.

3.3 Statistical results of NDRT effect in takeover process

After literature review, we can conclude that the influence of NDRT in the takeover process varies between research. From Table 4, we can see that the NDRT type used in each of the experiments as well as the NDRT task condition given to each of the participants, were different. These differences in the statistical research analysis was because the NDRT selected for each of the experiments were different. That is, depending on the purpose of the research, the experimenter selected a different type of NDRT, taking into consideration the characteristics of the NDRT. Therefore, it is possible to demonstrate that NDRT is an important element that must be considered in research focusing in highly automated driving.

Table 5 shows a summary of the results of experimental research analysis on the influence of NDRT in takeover for highly automated driving. The results of previous studies on the influence of NDRT varied between and within measures. Takeover measures that did not have a significant effect on NDRT were takeover difficulty (Alrefaie et al., 2019), control transition method (Baek et al., 2019), and time to collision (Borojeni et al., 2018). Conversely, NDRT showed to have a significant effect on takeover readiness (Du et al., 2020) and situation awareness (Liang et al., 2021). Nevertheless, there were measures showing a significant effect of NDRT on the transition of control as well as research showing no significant effect. For instance, the different NDRTs and their engagement was found to have a significant effect on response time whereas, some studies showed no significant effect. Similar results were found for hands-on-wheel time and eyes-on-road reaction time.

Although NDRT selected for the experiment looked similar between various studies, task characteristics varied within each NDRT. That is, for some experiments, 'watching video' task consisted of watching a video on the smartphone, while for others, it consisted of watching a video on a mounted display on the center console of the vehicle. Therefore, although they seem to be the same task, the manner in which the driver engages in the task differs. Therefore, it is important to consider aspects of the task that are important for the takeover context such as hand position, visual distraction, cognitive load, and engagement.

For the experimental task, the tasks are defined as those where the users had to employ their visual and cognitive resources. However, for the natural task, the task specification does not seem to be of as much significance as in the experimental task. Hence, existing studies mostly focus on the difference in equipment involved rather than on the task itself. In view of this, we can focus on characteristics such as, the location of task (a hand-held personal equipment or a device mounted on the vehicle); extent of physical engagement in that task (using both hands, one hand in use, or no hand in use); and visual and cognitive aspects of the task.

From the evaluation of NDRT used in previous studies, it is possible to justify why they showed different results on the effect of NDRT for the same measures. This is because, although different studies seem to use the same tasks such as smartphone interaction, the condition in which the NDRT was performed differs between studies and also within a study. Therefore, it is important to analyze in detail the various resources used for the NDRT, such as the visual, motoric or physical, and mental resources, that might influence the transition of control (Yoon et al., 2021).

For the measures used to evaluate the transition of control in highly automated driving, it was found that existing studies implicitly consider the influence of NDRT. That is, the main focus of the measure is on how the transition of control occurred rather than on how the NDRT influenced the transition of control. However, cognitive measures such as workload and situation awareness not implicitly considered the NDRT in the evaluation process. Furthermore, the results of this research enable us to understand the importance of the state of drivers before the transition of control, which in this case is characterized by drivers involvement in NDRT. However, further research is needed to analyze and classify the aspects of the drivers that can impact the transition of control in highly automated driving.