본문바로가기

Multiple Measures of Driver Workload according to Takeover Request Lead Time in Conditionally Autonomous Driving

Abstract

Objective: The aim of this study is to investigate the effect of Takeover request lead time (TORlt) on several measures of workload and dissociation among the measures.

Background: Autonomous driving has received considerable attention and takeover request (TOR) is one of the major issues in autonomous driving. A human driver must be ready to transfer the control of the vehicle when TOR is given. Therefore, it is important how and when to request driver to transfer the control. In this regard, TORlt have received great attention and there are many existing scholarly works about effect of TORlt on takeover performance. Inappropriate TORlt make driver overload or underload and the abnormal workload deteriorate driver performance in takeover situation.

Method: Experiment with driving simulator was conducted. TORlt was controlled in 7 levels (3s, 7s, 10s, 15s, 30s, 45s, 60s) and driver workload was measured by both subjective and physiological methods. Driving Activity Load Index (DALI) was conducted to measure subjective workload and respiration rate (RSP), heart rate (HR), and galvanic skin response (GSR) was used to measure objective workload.

Results: TORlt had a significant effect on subjective workload. The result of subjective workload showed that workload and TORlt has a negative relationship. There was also a significant effect of TORlt on GSR. GSR was increased as TORlt increased because of frustration that made by long TORlt. On the other hand, there were no significant effects of TORlt on respiration rate and heart rate.

Conclusion: As the workload changes according to TORlt, TORlt is an important factor to be considered in takeover situation. Moreover, multiple measures of workload should be used to investigate workload in takeover situation because there is dissociation among measures.

Application: Workload could be used to evaluate takeover performance of drivers. The results of this paper suggest multiple measures including subjective methods and physiological signals be used to assess driver performance in takeover situation.



Keywords



Conditionally Autonomous driving (CAD) Takeover Request (TOR) Takeover Request lead time (TORlt) Subjective workload Physiological measures



1. Introduction

Autonomous driving is one of the most interesting technology in recent years with expectation of solving problems caused by automobiles such as accidents, pollution, and traffic jams (Cho et al., 2013; Singh, 2018; Schrank et al., 2012). However, current autonomous vehicles cannot handle all driving situations. Therefore, it requires drivers to intervene in certain situations. SAE international defined these levels of autonomous driving as partial (level 2) and conditional (level 3) autonomous driving (SAE International, 2016).

In level 3 autonomous driving, drivers are not required to monitor the driving situations and they may perform non-driving related tasks (NDRTs). However, drivers must pay attention to driving situation and make an appropriate reaction when takeover request (TOR) occurs. In order for the driver to make an appropriate reaction, how drivers are informed about takeover situation becomes crucial. Therefore, many existing scholarly works investigated characteristics of TOR such as takeover request lead time (TORlt; Gold et al., 2013; Gold et al., 2018; Eriksson and Stanton, 2017; Mok et al., 2015; Mok et al., 2017) and modalities (Bazilinskyy et al., 2018; Petermeijer et al., 2017; Wan and Wu, 2018; Yun et al., 2018).

Takeover request lead time (TORlt), also called time budget, is one of the main factors affecting driving performance in a takeover situation. In a manual driving situation, if sufficient time is given before the reaction, drivers can interpret the situation and make appropriate decisions (Abe and Richardson, 2004; Abe and Richardson, 2005), vice versa. This is applicable in autonomous driving domain as well. In autonomous driving domain, if insufficient TORlt is given to drivers, they cannot handle the takeover situation appropriately (Gold et al., 2013; Gold et al., 2018; Mok et al., 2015; Mok et al., 2017; Payre et al., 2016; Van den Beukel and Van der Voort, 2013; Wan and Wu, 2018; Zhang et al., 2018). On the other hand, longer TORlt allows drivers to react appropriately. However, TOR with excessively long lead time can be interpreted as a false alarm. Therefore, drivers can be distracted and delay their response until the situation become critical (Wan and Wu, 2018; Wilde, 1982). Therefore, it is necessary to design a TOR with an optimal range of time budget for safer performance while transition of control.

Besides its impact on takeover performance, TORlt also has an effect on driver workload (Eriksson and Stanton, 2017; Wan and Wu, 2018). Driver workload has a considerable influence on driving performance and can either overload or underload it. Such overload or underload can cause deterioration in driving performance (Brookhuis and de Waard, 2010; da Silva, 2014; de Waard and Brookhuis, 1997). In parallel with the manual driving domain, many researchers investigated driver workload in autonomous driving domain (de Winter et al., 2014; Eriksson and Stanton, 2017; Hajek et al., 2013; Wan and Wu, 2018). However, these studies either did not investigate workload induced by TOR or measure driver workload in a subjective method.

This study focused on workload induced by TOR. Many researches have focused on overall workload or workload while performing NDRT alone because they considered that demanding NDRTs make takeover performance (De Winter et al., 2014; Eriksson and Stanton, 2017). However, according to Wan and Wu (2018), takeover request without sufficient time budget may increase driver workload and generate erratic driver response. Workload increased by TOR is a critical factor that affects takeover performance. Therefore, it is necessary to investigate workload induced by workload.

Additionally, a few researches that assessed workload induced by TOR used subjective methods. Workload is measured through a survey at the end of experiments. Therefore, it is not a suitable for measuring workload in dynamic situations such as driving. Unlike subjective measures, workload quantified by physiological signals can be measured and analyzed in real time, which makes workload more applicable in driving situations. Also, there could be dissociation among workload measures (Wickens et al., 2015). Each measure is sensitive to different aspect of the task. Hence, physiological signals as well as subjective measure were used to measure driver workload.

The objective of this study is to investigate the effects of TORlt on driver workload in takeover situation. This study hypothesizes that workload would increase as TORlt is too short or too long. To investigate the hypothesis, an experiment was conducted with driving simulator and workload was measured by subjective and objective method.

2. Method

2.1 Participants

Twenty-eight participants (23 males, 5 females) between the age of 20 and 31 years old (M = 25.32, SD = 2.67) were recruited from Korea University. All participants had valid driver's licenses and had normal or corrected-to-normal vision. Before the experiment, all participants signed written informed consent and received instructions. Participants were compensated with $18 for participation.

2.2 Apparatus

The experimental environment was designed using OpenDS 4.5 driving simulator and presented on a 45-inch Samsung TV monitor. Participants controlled the driving simulator through Logitech G25 steering wheel, pedal, controller and performed an NDRT via iPad mini during autonomous driving. Autonomous driving system was operated with the button on the controller. Participants could turn the autonomous driving system on and off according to the driving situation. Participants' physiological signals were recorded with MP100 from Biopac.

2.3 Driving scenario

The experiment was simulated in two-lane highway environment (see Figure 1). The ego-vehicle was driving in the second lane and there were three vehicles in the first lane and no vehicle in the second lane. In the takeover situation, concurrent with starting point of TOR, an obstacle appears in the driver's lane. Drivers must take a lane change into the first lane in order to avoid the collision. After the lane change, drivers return to the second lane and drive until they are stabilized. When drivers are stabilized, they turn off autonomous driving system and complete the subjective workload rating.

Figure 1. Schematic illustration of the sequence of the takeover process. The orange car is the participants' car. After the TOR, the participant monitors the situation and responds to the request. Then participant passes the obstacle by changing the lane manual (Yun and Myung, 2019)

2.4 Experiment design

TORlt was set as an independent variable to investigate the effects of TORlt on driver workload in conditional autonomous driving. The levels of TORlt was chosen according to existing literature about TORlt (Eriksson and Stanton, 2017; Gold et al., 2013). Most of research sets TORlt between 3s to 10s. However, in this study, we also set longer TORlts to investigate driver behavior with enough time to take over. Therefore, TORlt was controlled in 7 levels (3s, 7s, 10s, 15s, 30s, 45s, 60s). Each session of the experiment dealt with one TORlt level and was conducted in random sequence.

A dependent variable is driver workload measured by a subjective measure and physiological measures. Driving Activity Load Index (DALI; Pauzié, 2008) was conducted to measure subjective workload. DALI is a modified version of NASA-TLX to measure workload in driving situation. DALI has six dimensions including effort of attention, visual demand, auditory demand, temporal demand, interruption and situational stress (see Table 1) while NASA-TLX includes physical demand, mental demand, temporal demand, effort, performance, and frustration. Because the experiment is conducted in driving situation, DALI was performed in this study.

Sub-dimensions

Description

Effort of attention

To evaluate the attention required by the activity - to think about, to decide, to choose, to look for and so on

Visual demand

To evaluate the visual demand necessary for the activity

Auditory demand

To evaluate the auditory demand necessary for the activity

Temporal demand

To evaluate the specific constraint owing to timing demand when running the activity

Interruption

To evaluate the possible disturbance when running the driving activity simultaneously with any other supplementary task such as phoning, using systems or radio and so on

Situational stress

To evaluate the level of constraints/stress while conducting the activity such as fatigue, insecure feeling, irritation, discouragement and so on

Table 1. DALI sub-dimensions and description (Pauzié, 2008)

In addition, physiological measures were conducted to measure objective workload. Respiration rate (RSP), Heart rate (HR), and Galvanic skin response (GSR) were selected as physiological signals to measure objective workload. RSP, HR, and GSR were recorded with MP100 from Biopac, and sampled at 200HZ. According to Kramer (1991), RSP and HR represent general workload while GSR is sensitive to emotion and arousal. Through the experiment, workload from different measures were compared and analyzed.

In a session of the experiment, participants had to perform the NDRT during autonomous driving. Participants were required to touch a numbered ball in the ascending order as NDRT as shown in Figure 2. The visually demanding task was given in order to simulate conditionally autonomous driving, which drivers are not necessary to monitor the driving situation. Performance of the NDRT was not collected.

Figure 2. Capture of driving simulator display (left) and the NDRT display (right) (Yun and Myung, 2019)
3. Results

3.1 Subjective workload measured by DALI

The mean and standard deviation of the subjective workload measured by DALI are shown in Figure 3. To investigate the effect of TORlt on subjective workload, Analysis of Variance (ANOVA) was conducted using SAS 9.4. The result showed a significant effect of TORlt on subjective workload (F(6, 189) = 16.24, p < 0.0001). Also, LSD multiple comparison test was conducted, and the result suggests that shorter TORlt imposes a higher workload on the driver as shown in Figure 3. Specifically, subjective workload was significantly higher at 3s than other TORlt (p < 0.0001 for each pair-wise comparison); significantly higher at 7s than 30s (p < 0.0001), 45s (p = 0.0008), and 60s (p = 0.0006); significantly higher at 10s than 30s (p = 0.0098).

Figure 3. Result of subjective workload (DALI)

3.2 Objective workload measured by physiological measures

Driver workload after TOR was measured by physiological signals. Physiological signals were collected and analyzed until drivers avoid the obstacle. Mean of respiration rate (Mean RSP), Mean of Heart rate (Mean HR), Mean of galvanic skin response (Mean GSR) were used to assess driver workload. Physiological data were normalized in order to eliminate individual differences. To investigate the effect of TORlt on physiological measures, ANOVA was conducted using SAS 9.4.

3.2.1 Mean RSP

The mean and standard deviation of the objective workload measured by mean RSP are shown in Figure 4. ANOVA result indicated that there was no significant effect of TORlt on mean RSP (F(6,189) = 1.23, p = 0.2935).

Figure 4. Result of mean respiration rate

3.2.2 Mean HR

The mean and standard deviation of the objective workload measured by mean HR are shown in Figure 5. ANOVA result indicated that there was no significant effect of TORlt on mean HR (F(6,189) = 0.77, p = 0.5935).

Figure 5. Result of mean heart rate

3.2.3 Mean GSR

The mean and standard deviation of the objective workload measured by mean GSR are shown in Figure 6. The ANOVA results showed there was significant effect of TORlt on mean GSR (F(6,189) = 10.51, p < 0.0001) and as TORlt increases, mean GSR shows an increasing trend. According to LSD multiple comparison test, Mean GSR was significantly higher at 30s than others (p < 0.001) except 10s (p = 0.0546) and 60s (p = 0.4741).

Figure 6. Result of mean galvanic skin response
4. Discussion

This study investigated the effect of TORlt on subjective and objective workload of drivers in takeover situation. In autonomous driving domain, many existing works investigated TORlt and workload. However, previous works have either focused on workload before TOR occur or used subjective measure for assess driver workload. However, his study focused on workload induced by TOR and presented various methods for measuring the driver workload.

Subjective workload is measured by Driver Activity Load Index. There was a significant effect of TORlt on the DALI score and subjective workload increases as TORlt become shorter. This result is consistent with the result of Wan and Wu (2018) which stated that driver's performance, such as crash rate and minimum time to collision, is deteriorated as TORlt increases. In this study, driver workload was increased in short TORlt as expected. Drivers, who were given the TOR with short lead time, did not have sufficient time to perceive and comprehend the driving situation and make an appropriate decision. As a result, drivers could not cope with the takeover situation and their workload increased. However, driver workload was not increased in excessively long TORlt. Long TORlt was expected to increase driver workload because driver could assume long TORlt as a false alarm or feel distraction (Wan and Wu, 2018). However, there was no significant difference in long TORlts (30s, 45s, 60s) in this study. Participants, who considered long TORlt as false alarm, would respond that their workload is high because they felt frustration and checked driving situation several time whether an obstacle is presented in the road. On the other hand, because the experiment is not real driving, some participants regarded the driving situation as a relaxed situation and their workload was low as TORlt increases. As a result, the different trends of workload are canceled out and there was no significant difference in subjective workload among long TORlts.

In addition, objective workload is measured by respiration rate (RSP), heart rate (HR), and galvanic skin response (GSR). There was no significant effect of TORlt on mean RSP and mean HR. The reason is that participants regarded driving situation after the takeover as simple driving task. In the experiment, participants just drove straight freeway after they passed the obstacle. This situation is too simple, so it is possible that even short TORlt could not impose workload on drivers. Unlike mean RSP and mean HR, TORlt had a significant effect on mean GSR. According to Kramer (1991), Physiological signals are sensitive to different resource demands (Kramer, 1991; Ryu and Myung, 2005). In this study, excessive temporal demand because of short TORlt and distraction caused by long TORlt were demands imposed on the participants. Hence, GSR which is sensitive to emotion and frustration (Kramer, 1991) was influenced by TORlt. In future researches, it is necessary to measure physiological signals while controlling various resource demand to investigate effect of TORlt on physiological signals extensively.

Finally, there is a limitation in this study. There is lack of diversity in participants. Participants are recruited from SNS, and university website, so participants were at a young age and had little driving experience. Different results would have been produced if the experiment was conducted to drivers with older age and more driving experience. Therefore, future research should include participants with a wide range of age and driving experience.

5. Conclusion

This study examined effect of TORlt on driver workload. Driver workload was measured by DALI subjective, and Physiological signals such as respiration, heart rate and galvanic skin response. We assumed that there would be quadratic or curvilinear relationship between TORlt and workload. However, the result showed that TORlt and workload did not have quadratic relationship. Subjective workload increases as TORlt decreases. Unlike subjective workload, physiological signals presented different relationship between TORlt. There were no significant effects of TORlt on respiration rate and heart rate while TORlt had a significant effect on galvanic skin response. Because each measure of workload is sensitive to different aspect of the situation, different workload was obtained depending on workload measures. Therefore, various workload measures are necessary for investigate workload in takeover situation.



References


1. Abe, G. and Richardson, J., The effect of alarm timing on driver behaviour: an investigation of differences in driver trust and response to alarms according to alarm timing. Transportation Research Part F: Traffic Psychology and Behaviour, 7(4-5), 307-322, 2004.
Google Scholar 

2. Abe, G. and Richardson, J., The influence of alarm timing on braking response and driver trust in low speed driving. Safety Science, 43(9), 639-654, 2005.
Google Scholar 

3. Bazilinskyy, P., Petermeijer, S.M., Petrovych, V., Dodou, D. and de Winter, J.C., Take-over requests in highly automated driving: A crowdsourcing survey on auditory, vibrotactile, and visual displays. Transportation Research Part F: Traffic Psychology and Behaviour, 56, 82-98, 2018.
Google Scholar 

4. Brookhuis, K.A. and de Waard, D., Monitoring drivers' mental workload in driving simulators using physiological measures. Accident Analysis & Prevention, 42(3), 898-903, 2010.
Google Scholar 

5. Cho, H.S., Lee, D.M. and Kim, Y.C., 2010 Traffic Congestion Costs: Estimation and Trend Analysis, The Korea Transport Institute (KOTI) Research Report, 1-102, 2013.


6. da Silva, F.P., Mental workload, task demand and driving performance: what relation?. Procedia-Social and Behavioral Sciences, 162, 310-319, 2014.
Crossref  Google Scholar 

7. de Waard, D. and Brookhuis, A., Behavioural adaptation of drivers to warning and tutoring messages: Results from an on-the-road and. Heavy Vehicle Systems, 4(2-4), 222, 1997.
Google Scholar 

8. de Winter, J.C., Happee, R., Martens, M.H. and Stanton, N.A., Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence. Transportation Research Part F: Traffic Psychology and Behaviour, 27, 196-217, 2014.
Google Scholar 

9. Eriksson, A. and Stanton, N.A., Takeover time in highly automated vehicles: noncritical transitions to and from manual control. Human Factors, 59(4), 689-705, 2017.
Google Scholar 

10. Gold, C., Damböck, D., Lorenz, L. and Bengler, K., "Take over!" How long does it take to get the driver back into the loop?. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 57, No. 1, pp. 1938-1942). Sage CA: Los Angeles, CA: SAGE Publications, 2013.
Google Scholar 

11. Gold, C., Happee, R. and Bengler, K., Modeling take-over performance in level 3 conditionally automated vehicles. Accident Analysis & Prevention, 116, 3-13, 2018.
Google Scholar 

12. Hajek, W., Gaponova, I., Fleischer, K.H. and Krems, J., Workload-adaptive cruise control-A new generation of advanced driver assistance systems. Transportation Research Part F: Traffic Psychology and Behaviour, 20, 108-120, 2013.
Google Scholar 

13. Kramer, A.F., Physiological metrics of mental workload: A review of recent progress. Multiple-task Performance, 279-328, 1991.
Crossref  Google Scholar 

14. Mok, B., Johns, M., Lee, K.J., Miller, D., Sirkin, D., Ive, P. and Ju, W., Emergency, automation off: Unstructured transition timing for distracted drivers of automated vehicles. In 2015 IEEE 18th International Conference on Intelligent Transportation Systems (pp. 2458-2464). IEEE. 2015.
Google Scholar 

15. Mok, B., Johns, M., Miller, D. and Ju, W., Tunneled in: Drivers with active secondary tasks need more time to transition from automation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 2840-2844). ACM, 2017.
Google Scholar 

16. Pauzié, A., A method to assess the driver mental workload: The driving activity load index(DALI). IET Intelligent Transport Systems, 2(4), 315-322, 2008.
Google Scholar 

17. Payre, W., Cestac, J. and Delhomme, P., Fully automated driving: Impact of trust and practice on manual control recovery. Human Factors, 58(2), 229-241, 2016.
Google Scholar  PubMed 

18. Petermeijer, S., Bazilinskyy, P., Bengler, K. and de Winter, J., Take-over again: Investigating multimodal and directional TORs to get the driver back into the loop. Applied Ergonomics, 62, 204-215, 2017.
Google Scholar 

19. Ryu, K. and Myung, R., Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. International Journal of Industrial Ergonomics, 35(11), 991-1009, 2005.
Google Scholar 

20. SAE International., Taxonomy and definitions for terms related to driving automationsystems for on-road motor vehicles. (J3016_201609). Warrendale, PA: Author, 2016.


21. Schrank, D., Eisele, B. and Lomax, T., TTI's 2012 urban mobility report. Texas A&M Transportation Institute. The Texas A&M University System, 4, 2012.
Google Scholar 

22. Singh, S., Critical reasons for crashes investigated in the National Motor Vehicle Crash Causation Survey (Traffic Safety Facts Crash · Stats. Report No. DOT HS 812 506). Washington, DC: National Highway Traffic Safety Administration. 2018.
Google Scholar 

23. Van den Beukel, A.P. and Van der Voort, M.C., The influence of time-criticality on Situation Awareness when retrieving human control after automated driving. In 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013) (pp. 2000-2005). IEEE. 2013.
Google Scholar 

24. Wan, J. and Wu, C., The Effects of Lead Time of Take-Over Request and Nondriving Tasks on Taking-Over Control of Automated Vehicles. IEEE Transactions on Human-Machine Systems, (99), 1-10, 2018.
Google Scholar 

25. Wickens, C.D., Hollands, J.G., Banbury, S. and Parasuraman, R., Engineering psychology and human performance. Psychology Press, 2015.
Google Scholar 

26. Wilde, G.J., The theory of risk homeostasis: implications for safety and health. Risk Analysis, 2(4), 209-225, 1982.
Google Scholar 

27. Yun, H., Lee, J.W., Yang, H.D. and Yang, J.H., Experimental Design for Multi-modal Take-over Request for Automated Driving. In International Conference on Human-Computer Interaction (pp. 418-425). Springer, Cham, 2018.
Google Scholar 

28. Yun, Y.D. and Myung, R., Effects of Takeover Request Lead Time to Driver Workload in Autonomous Driving. Journal of the Ergonomics Society of Korea, 38(2), 61-71, 2019.


29. Zhang, B., de Winter, J., Varotto, S., Happee, R. and Martens, M. Determinants of take-over time from automated driving: A meta-analysis of 129 studies. Transportation Research Part F: Traffic Psychology and Behaviour, 64, 285-307, 2019.
Google Scholar 

PIDS App ServiceClick here!

Download this article