JESK Journal of the Ergonomics Society of Korea

Multimodal Driver Drowsiness Detection in Level 3 Automated Driving: Effect of Time Window Length Using Eye-Tracking and HRV

Eun Hye Jang , Woojin Kim , Mi Chang , Jiwoo Han , Daesub Yoon
10.5143/JESK.2026.45.5.479 Epub 2026 November 06

0: Cited By

Abstract

Objective: This study aims to develop a multimodal drowsiness detection framework by integrating eye-tracking–based behavioral features and ECG-derived heart rate variability (HRV), and to systematically investigate the impact of time window length on feature correspondence with the 300-s reference and classification performance.

Background: Driver drowsiness is a major contributing factor to traffic accidents, particularly in conditionally automated driving environments where drivers must remain fallback-ready and respond appropriately to takeover requests. Existing approaches have limitations in capturing both physiological and behavioral aspects of drowsiness, and the effect of time window length on detection performance remains insufficiently explored.

Method: Data from 97 participants with synchronized KSS, eye-tracking, and ECG records were analyzed. Observations with KSS=7 were excluded as an ambiguous state. The final binary dataset comprised 448 observations (296 alert, 152 drowsy). Six nested windows ranging from 10 to 300s were compared using the same 45 valid common features, Random Forest, and participant-level stratified group 5-fold cross-validation. Participant-cluster bootstrap resampling provided 95% confidence intervals for ROC-AUC and paired differences from the 300-s reference.

Results: The 300-s window achieved ROC-AUC=0.813 (95% CI 0.751~0.868). Among shortened windows, 40s showed the highest ROC-AUC [0.778 (0.714~0.834)] and F1-score (0.624); its paired difference from 30s was -0.036 (95% CI -0.076~0.004). The 10-s window performed worst (ROC-AUC=0.695). Feature selection and importance analyses repeatedly identified pupil variability, fixation duration, and eyelid-related features as major contributors, whereas HRV features showed less consistent importance across window lengths.

Conclusion: Window length materially affected drowsiness classification. Although the 300-s window provided the highest performance, the 40-s window showed the most favorable trade-off among shortened windows and should be prioritized for prospective validation. Because a non-inferiority margin was not prespecified, the 40-s window cannot be considered a confirmed replacement for the 300-s reference. The findings further suggest that eye-tracking features, especially pupil variability, fixation duration, and eyelid-related measures, are useful candidates for drowsiness detection, whereas the incremental role of HRV features requires further validation.

Application: A 300-s window remains a useful analytical reference, while 40s should be prioritized for prospective validation of a more responsive driver-monitoring system. Future studies should prespecify an acceptable performance-loss margin and evaluate external generalizability and detection delay.

Keywords

Driver drowsiness Multimodal detection Eye-tracking Heart rate variability Time window optimization

References

1. Abe, T., PERCLOS-based technologies for detecting drowsiness: current evidence and future directions, Sleep Advances, 4(1), zpad006, 2023. doi:10.1093/sleepadvances/zpad006
Google Scholar