For securing the safety of socio-technical systems such as nuclear power plants and petroleum process industries, probabilistic safety analysis (PSA) has been used to support risk-informed decision-making processes. Because human error management is a significant contributor of system safety, human reliability analysis (HRA), which systematically identifies important human failure events (HFEs) and quantifies performance or reliability of human operators, has been recognized as an essential part of PSA.

Several kinds of HRA methods have been developed. The widely employed HRA methods are Technique for human error rate prediction (THERP; Swain and Guttmann, 1983), Accident sequence evaluation program (ASEP; Swain, 1987), Human Cognitive Reliability/Operator Reliability Experiments (HCR/ORE; Parry et al., 1992), Cause-Based Decision Tree Method (CBDTM; Parry et al., 1992), Standardized Plant Analysis Risk Human Reliability Analysis (SPAR-H; Gertman et al., 2005), A Technique for Human Event Analysis (ATHEANA; Barriere et al., 2000), Cognitive Reliability and Error Analysis Method (CREAM; Hollnagel, 1998), and Korean Human Reliability Analysis Method (K-HRA; Jung et al., 2005). These methods provide different processes in defining HFEs for given system events, identifying significant human tasks or actions, evaluating the states of performance shaping factors (PSFs) such as procedure quality and task complexity, and quantifying human error probabilities (HEPs) of each HFE with consideration of the evaluated states.

It has been reported that there are several issues needing to be resolved for the realistic application of HRAs (Forester et al., 2006; Bladh et al., 2014; Forester et al., 2014; Bowie et al., 2015; Forester et al., 2016). For example, there should be reasonable prediction of human reliability based on human information processing or cognitive mechanisms (Parry et al., 2013). From the psychological underpinning of HRA, human failure mechanisms can be clearly identified. Second, the inter-analyst variability and intra-analyst variability of HRA applications needs to be reduced. It is necessary for detailed guidance and clear analysis processes to be established for this issue. In addition, quantitative information of HRA methods such as nominal HEPs or PSF effects on HEPs needs to be supported by empirical data. Owing to insufficient data of human reliability, estimation of various quantitative information has relied on expert judgment as opposed to necessary empirical data.

The NRC (Nuclear Regulatory Commission) and the EPRI (Electric Power Research Institute) recently developed a new HRA method of IDHEAS (Integrated Decision-Tree Human Event Analysis System) to tackle these issues (Xing et al., 2017). Based on the understanding of macro-cognitions and related PSFs, a coherent and traceable process of HRA application was produced. However, the validation results of the IDHEAS showed that it has still considerable variability of its applications (Liao et al., 2017). Moreover, results of in some cases indicated that the HEPs generated by the IDHEAS method may be incompatible with estimates from empirical data in some cases.

In this paper, we suggest four directions of HRA method improvements for practically assessing HEPs. Among the HRA issues that have been addressed in several literatures (Kim et al., 2017b), these suggestions mainly deal with how to decrease application variability and adhere the HRA results to the empirical evidence. The issues regarding variability and empirical evidence have been recently emphasized in the comparative studies of HRA methods (Bladh et al., 2014; Forester et al., 2014; Bowie et al., 2015; Forester et al., 2016). To depict concrete examples of HRA processes, we assumed that the HRA based on IDHEAS method was implemented for reliability prediction of control room operators during internal at-power events of nuclear power plants. Because most tasks in emergency situations of nuclear power plants are proceduralized, it is also assumed that the human reliability is assessed by investigating how crews appropriately follow relevant procedures.

2.1 HRA methods

The prospective HRA basically aims to predict a HEP for a specific task or event through qualitative analysis and quantification. The qualitative analysis (1) understands the event and/or scenarios of interest, (2) identifies HFEs potentially contributable to the risk, (3) derives significant tasks for HFEs, (4) examines PSFs influencing reliability of the tasks, and (5) evaluates states of the PSFs in the specific contexts. The quantification involves applying information obtained through the qualitative analysis to generate the reliability information about HFEs. The (1) HEPs, (2) their recovery failure probabilities, (3) uncertainties of the estimates, and (4) dependencies between HFEs are calculated and integrated into the system's risk model.

HRA techniques are generally divided into first- and second- generation methods (Pasquale et al., 2013; Bell and Holroyd, 2009). The first-generation approach views HEP quantification from the perspective of component reliability quantification. Hence, it is perceived that human tasks can be deconstructed into primary actions; with the possibilities of success or failure representing the reliability of action. In this case, internal cognitive mechanism of human is not considered for reliability estimation. The well-known first-generation methods are THERP, ASEP, HCR/ORE, CBDTM, and SPAR-H.

First-generation HRA methods perceive human error from a behavioristic viewpoint. Contrastingly, the second-generation approach attempts to identify cognitive mechanisms of human errors and the way PSFs are associated with each cognitive mechanism. Hence, second-generation HRA methods emphasize the qualitative analysis of human errors based on cognitive models. Examples include ATHEANA, CREAM, MisDiagnosis Tree Analysis (MDTA; Kim et al., 2008), Human Reliability Management System (HRMS; Kirwan, 1997), and Méthode d'Evaluation de la Réalisation des Missions Opérateur pour la Sûreté (MERMOS; Le Bot et al., 1999). Nevertheless, quantitative results of these methods are not supported by empirical evidence. Thus, flexibility of the qualitative analysis process and the resultantly high variability between/within analyzers is expected (Pasquale et al., 2013).

2.2 IDHEAS

To easily understand HRA method, this paper briefly describes an HRA method recently developed. Under the historical background of HRA method development, the NRC and EPRI developed the IDHEAS method to tackle the unresolved issues of established HRA methods (Xing et al., 2017). Figure 1 shows the main process of IDHEAS method. After evaluating an accidental scenario of interest and identification of HFEs, critical tasks in HFEs are analyzed according to the development of crew response diagrams (CRDs). A CRD represents the paths where operators are expected to follow relevant procedures for the given events. After crew failure paths are outlined in CRDs, crew failure modes (CFMs) possibly appearing during the performance of each task, are identified. These CFMs can be seen as atomic failure modes or error types of the operators. There are 15 types of CFMs based on three kinds of cognitive response phases (Table 1). For calculating the HEP of each CFM, states of related PSFs are evaluated along with the decision trees (DTs) provided in this method. After the HEPs of all potential CFMs are determined, the total HEP of a HFE for a PSA scenario (S) are quantified by the addition of the HEPs in CFMs (equation 1).

       (1)

Here, Prob (CFM | CRD path, S) is an HEP of CFM belonging to a CRD path for a PSA scenario S.

Based on the comprehensive foundations of cognitive psychology, this method provided the theoretical framework to identify cognitive failures of human operators and quantitative links between failures and contextual factors of PSFs (Liao et al., 2015). Also, CRDs supported the structured task analyses of significant HFEs by graphically representing the paths of successful responses, failure responses, or recoveries of the failures based on procedural flows. In addition, the structured HRA process enhanced assessment traceability and result reasonableness.

However, previous studies have reported that key issues related to the practical and realistic application of HRAs are still present in the IDHEAS method. Liao et al. (2017) presented an HRA test result by comparing HEP predictions using IDHEAS with other methods and empirical HEP estimations. In this test study, considerable variability between analysts were observed during construction of CRD, HEP quantification of CFMs, and evaluation of PSF effects on HEPs. In some cases, the HEPs predicted by the proposed method did not coincide with probabilities estimated from the empirical data. Compared to high resources of assessment efforts for HRA, the accuracy or consistency of HEP prediction was not ensured. In addition, data needed for IDHEAS quantification was insufficient. Therefore, a few HEPs are presented via expert judgments (Liao et al., 2015). Although several HRA databases have been developing in many countries, there are gaps between current available data and the data required for HRAs.

Figure 1. The HEP quantification framework of IDHEAS (Liao et al., 2017)

Modify Delete

3.1 Plant-specific guideline for task analysis

The second-generation HRA method such as the IDHEAS method is flexible to develop a CRD and select CFMs significant to critical paths in CRDs. However, this flexibility also means potential variability of HRA practitioners and requires practitioners to have a high understanding of task structure and human cognition. Although collaboration between HRA specialists and PSA experts is a well-known practice for good HRA application, HRA activities can be resource-intensive in many cases.

Operators need to follow highly formalized operating procedures during emergency situations; hence, CRD development, which decomposes the HFE into emergency tasks along with the procedures, can be guided by some rules. For example, when a Combustion Engineering (CE)-type plant like OPR (optimized power reactor)-1000 plant or APR (Advanced power reactor)-1400 plant is tripped, the operator should perform the standard post-trip action (SPTA) procedure. After the initial responses according to the SPTA, the operator determines the cause of the situation based on the diagnostic action (DA) procedure. The operator then transfers to a procedure that the DA procedure recommends.

With regard to the structure of emergency operating procedures, it is possible to identify essential branches of CRDs. For example, if the HFE of interest is associated with the diagnosis of emergency situations, the acquisition of plant information for the DA procedure and the decision making activity from the DA procedure should be counted in the CRD branches. Likewise, it is possible to predefine how a crew can transfer procedures to enter a procedure to be performed for the desired action. Additionally, human actions significant to plant safety can be derived for each procedure. If the significant actions are not modeled as HFEs, the analyzer can add the actions into the CRD paths. Table 2 shows examples of the procedure information mentioned previously. If an analyzer develops the CRD using the information like that presented in Table 2, the important tasks will be entailed in the task analysis with relatively less resources. It is also possible to develop an expert system that recommends important human actions to be considered for a specific procedure based on information presented in Table 2.

3.2 Distinctive human error classification

Another reason that elevates variability of HRA application is ambiguity in selection of CFMs that are contributable to a reliability of each CRD path. For example, although two CFMs, critical data incorrectly processed and miscommunication can be distinguished theoretically, it is difficult to identify which CFM is crucial to a specific task in a practical manner. This is because various types of cognitive activities are mixedly employed to the task. When an operator inappropriately reported plant information to another operator, it is attributable to operator's misunderstanding of plant situations. However, this kind of human error may have also occurred due to minor mistake in speech. At times, these cases may not dis-ambiguously discriminated by the analyzer, but also by the operator who made the mistake.

The classification scheme of human error at the level of cognitive response phase such as plant status assessment, response planning, and execution can facilitate the human error causes, because the roles of operators in a crew team are different along with the classification of the cognitive response phases (Kim et al., 2017a). Using the command-and-control protocol, board operators including the reactor operator, turbine operator, and electric operator, obtain plant information and report it to a shift supervisor. The shift supervisor makes a decision based on the information delivered by the board operators, follows relevant procedures, and gives directions to the other operators. After, the board operators manipulate plant components according the direction given by the shift supervisor.

Table 3 shows an example of a human error classification proposed by Kim et al. (2017a). The cognitive activity of this taxonomy was defined with an assumption that an action of any operators in a crew team can be noted as a cognitive activity of an individual human. The cognitive activities and task types are basically distinguished based on characteristics of the tasks that the procedure sentences require. The tasks types or cognitive activities can hence be distinctively classified. In addition, the success or failure of each task or cognitive activity can also be counted evidently. An error of omission (EOO) implies that a given task is not performed when the task was required, whilst an error of commission (EOC) indicates that the task was performed in an inappropriate manner.

3.3 Empirical data

Most HRA methods are insufficiently supported by empirical data; rather, it relies on expert judgment. For instance, the IDHEAS method assigned the HEPs in DTs for the CFMs by information mainly estimated by an expert elicitation process. However, it is well-recognized that empirical human reliability data is a key to improve HRA quality (Chang et al., 2014; Groth et al., 2014; Kim et al., 2018a). This problem can be more evident when it is attempted to develop an HRA method for digitalized control rooms, because we have less experience of such control rooms to estimate quantitative information. There have been many efforts to generate HEPs based on empirical data (e.g., Chang et al., 2014; Reece et al., 1994; Kirwan et al., 1997; Preischl and Hellmich, 2013). However, due to differences in the theological basis and/or environmental settings of data collection processes, the meanings of each estimates and effects of relevant PSFs can be different (Liao et al., 2015). Researchers need to be careful when developing or updating HRA methods from such statistics.

To use the same human error taxonomy for HRA with the taxonomy employed for data collection allows minimizing the subjectivity of HEP quantification and ensuring transparent HRA results. It is because a different taxonomy demands subjective interpretation of the HEP estimates from data and another technical basis for the HEP adjustment. For example, Korea Atomic Energy Research Institute (KAERI) produced HEPs based on the taxonomy described in Table 3 (Kim et al., 2017a). This taxonomy was developed according to the tasks indicated in procedure sentences; therefore, it is possible to use this for evaluating reliability of procedure-driven human behaviors. However, it is required to examine suitability of the taxonomy when human behaviors not strictly guided by procedures are modeled.

3.4 Clarification of PSF definition

For a more reliable HRA analysis, Forester et al. (2014) carried out a comparative study for HRA methods and they recommended to provide a clear guideline of PSF rating or evaluation. As an example of ambiguous definitions, during evaluation of a PSF level such as procedure quality, an analyst may consider task complexity required by the procedure whilst another analyst can focus on the existence of ambiguous sentences in the procedural step. Hence, it is useful for HRA practitioners to define PSFs with observable or measurable metrics. The Human Reliability Evaluator for Computer-based Control Room Actions (HuRECA) method establishes several rules or anchor points to help PSF assessors determine levels or states of PSFs (NEA, 2012). Specific PSF variables (e.g., number of manipulations required) rather than abstract PSF concepts (e.g., task complexity) were employed for DTs of HEP calculation.

Using abstract factors about PSFs has an advantage of that wide range of features can be evaluated. However, detailed definitions and features determinant to the PSF levels should be provided together to facilitate analysts' understanding of the meaning of PSF ratings. If too many checkpoints are used, resource usage necessary for PSF analysis can be escalated. Alternatively, another set of PSF variables can be used for PSF evaluation. Although, this strategy can improve analysis usability, it is required to reveal that PSF variables included in HRA methods are significant to human reliability while the other variables are not.

A hypothetical case study was conducted to demonstrate applied results of the suggestions. The information shown in Table 2 was employed as a plant-specific guideline for the task analysis. The human error taxonomy proposed by Kim et al. (2017a) was used for classifying the CFMs in the selected HFE. The HEPs estimates presented by Kim et al. (2018a) was also considered as an empirical basis of HRA in this study. Lastly, the PSF variables identified in the study of Kim et al. (2018a) and their definitions were used again in this case study.

To employ estimates from empirical data regarding both HEPs and PSF effects on the HEPs, we assumed the estimates produced by an empirical study of Kim et al. (2018a). In that study, nominal HEPs for each type of task and the quantitative relations between PSFs and the nominal HEPs are predicted from the full-scope simulation data using a logistic regression technique. Although there are some issues impeding a fact that those estimates are realistic, those estimates were used in this study for the feasibility of the suggestions. Figure 2 shows a DT representing the relation between PSF and the HEPs. This figure explains an EOC HEP during a response planning task can increase according to (1) whether the reactor was tripped or not, (2) whether the task was required in a contingency action part or expected part of procedure, and (3) whether the procedure provides component codes to be manipulated or not. The present study described how an HEP of the HFE is quantified using DTs in all relevant tasks and their PSF levels evaluated. The performance time uncertainty and recovery factors or dependency analysis of the HEP were excluded in this study.

Figure 2. A DT determining an HEP of EOC during a response planning task (Kim et al., 2018a)

Modify Delete

The HFE selected for this case study is the event of feed-and-bleed operation during loss of all feedwater. The feed-and-bleed operation implies opening a pressurizer safety valve for intentionally releasing the reactor coolant system pressure when any heat removal in the secondary system is not possible. For this event, the operator using CE-type procedure follows the SPTA procedure to check the important status of plant systems, diagnoses the ongoing events based on the DA procedure, and responds to the situations of loss of all feedwater. The procedure for loss of all feedwater asks to verify that the level of steam generator is not sufficient. If the control of steam generator level is not possible, the operator transfer to the feed-and-bleed procedure.

From Table 2, it was found that the tasks in two procedure steps including procedure transitions and tasks in other two procedure steps including safety valve open were important to the feed-and-bleed event. The tasks in the three important step can be decomposed by the classification scheme presented by Kim et al. (2017a). Table 4 shows how the procedure sentences in the important steps can be described in terms of the classification scheme and the relevant HEPs are generated using the DTs in (Kim et al., 2018a). The total HEP, sum of all HEPs in the selected tasks, was 7.37E-02.

In this paper, some improvements for a more practical and realistic application of HRA are suggested with an example of an HRA method recently developed. Specifically, analysis variability reduction amongst analysts based on empirical data was emphasized for such enhancements. The NRC is developing a new method, called as IDHEAS ECA (Event and condition assessment), which is a simplified version of the current IDHEAS model for extending HRA application (Chang et al., 2016). KAERI is also developing an HRA method for computer-based control rooms (Kim et al., 2017b). With data collection about human performance and reliability in a digitalized control room, a new method based on empirical data will be established (Kim et al., 2018b). Through such efforts, we believe that HRA methods can be more practical and realistic based on empirical findings about human cognitive theories.

As mentioned in the introductory section, we assumed that all emergency events are responded by dedicated procedures. If we have sufficient human reliability data, it is possible to predict an HEP of any event or task formalized by specific procedures using the proposed approach. However, some procedures of other safety-critical systems may not involve fully-described instructions. Many nuclear power plants also provide highly-abstract guidelines rather than specific procedures to crews in case of severe accidents. To quantify reliabilities of the behaviors following such kind of guidelines, different processes for task analysis or HEP quantification are required to establish. With the improvement of HRA methods that we recommended, the new task analysis and quantification process can elaborate HRA methods for severe accidents or other industries.