China
Africa
Explore chapters and articles related to this topic
Applying Aerial Photography to Map Marsh Species in the Wetland of Lake Okeechobee
Published in Caiyun Zhang, Multi-sensor System Applications in the Everglades Ecosystem, 2020
The experimental analyses showed that the dataset including texture measures produced a high accuracy for marsh species mapping in the lake area. Landis and Koch (1977) suggested that a Kappa value larger than 0.80 indicates a strong agreement or accuracy. All three classifications using the dataset with texture measures produced an OA equal to or more than 80%, and thus marsh species maps were produced using all three classifiers, as shown in Figure 4.6. In general, the classified maps showed a consistent spatial distribution of marshes for the selected study site, which was dominated by graminoid, sawgrass, cordgrass, and swamp shrubland. Cattail and smartweed were sparse; common reed and willow were moderate and shown as bigger patches. An error matrix was also constructed for the ANN classified map, which showed the highest accuracy among three classifiers (OA=81%), as shown in Table 4.3. The user’s accuracies ranged from 50.0% (class 4: cattail) to 100.0% (class 1: willow; class 9: mud). The producer’s accuracies varied from 33.3% (class 4: cattail) to 100% (class 5: common reed). Cattail, an invasive species in the Greater Everglades, was the most difficult marsh species to classify, while common reed, a giant grass considered a looming threat to the Greater Everglades, was the easiest to identify for the study site.
Beyond Gut Feelings…
Published in Douglas A. Wiegmann, Scott A. Shappell, A Human Error Approach to Aviation Accident Analysis, 2017
Douglas A. Wiegmann, Scott A. Shappell
To control for this, a more conservative statistical measure of inter-rater reliability, known as Cohen’s Kappa, is typically used (Primavara et al., 1996). Cohen’s Kappa measures the level of agreement between raters in excess of the agreement that would have been obtained simply by chance. The value of the kappa coefficient ranges from one, if there is perfect agreement, to zero, if all agreements occurred by chance alone. In general, a Kappa value of 0.60 to 0.74 is considered “good” with values in excess of 0.75 viewed as “excellent” levels of agreement (Fleiss, 1981). At a minimum then, the goal of any classification system should be 0.60 or better.
Inter-rater consensus in safety management
Published in John Davies, Alastair Ross, Brendan Wallace, Linda Wright, Safety Management, 2003
John Davies, Alastair Ross, Brendan Wallace, Linda Wright
The Kappa coefficient (Cohen 1960) has been widely used, principally in the medical field after a review of inconsistency in clinical methods by Koran (1975). In keeping with the distinction outlined above between correlation and consensus on individual cases, Kappa is used precisely because it can be interpreted ‘as a measure of the amount of agreement (as opposed to correlation or association) between two raters ...’ (Spitznagel and Helzer 1985emphasis added). However, Kappa has been extensively critiqued, and its use in this context is questioned here.
Advancing natural language processing (NLP) applications of morphologically rich languages with bidirectional encoder representations from transformers (BERT): an empirical case study for Turkish
Published in Automatika, 2021
Akın Özçift, Kamil Akarsu, Fatma Yumuk, Cevhernur Söylemez
Kappa, i.e. κ, is a statistic metric which measures inter-rater reliability for categorical items. Inter-rater reliability is defined as the degree of agreement among the raters. This statistical score measures the consensus degree based on the decisions of predictors. In other words, it measures the agreement between two predictors who each classify N items into exclusive categories and κ is defined as follows: where is given below and it is also identical to accuracy. In Equation (7), Pe is defined as the number of times a rater i predicted category k with N observations and it is given as and it is also calculated with the following relation in terms of confusion matrix terms. The overall score of κ varies between −1 and 1. The obtained score is a statistical measure of how the obtained results far from occurring by chance. More empirically, while values smaller than 0.40 show fair agreement, the values between 0.40 and 0.60 show moderate agreements. Simply, for a confident classification performance evaluation, we should obtain 0.60–0.80 for good agreement and higher than 0.80 for the perfect agreement [51]. Therefore, we calculated κ metric in the experiments to evaluate statistical confidence of obtained results.
Incremental to radical ideas: paradigm-relatedness metrics for investigating ideation creativity and diversity
Published in International Journal of Design Creativity and Innovation, 2019
Eli M. Silk, Shanna R. Daly, Kathryn W. Jablokow, Seda McKilligan
The first step in evaluating the usefulness of the alternative paradigm-relatedness metrics was determining to what extent the two coders reliably applied the metrics. An agreement percentage and Cohen’s kappa (Cohen, 1960) for each independent code are reported in Table 4. Percent agreement was calculated as the total number of ideas coded identically by both raters divided by the total number of ideas coded, then multiplied by 100%. Cohen’s kappa was calculated according to the following formula: κ = [P(a) − P(e)] / [1 − P(e)], where P(a) represents the proportion of ideas coded identically by the two raters, and P(e) represents the proportion of ideas for which agreement between the raters was expected by chance. The chance agreement was based on the observed frequencies of the codes in the data-set. The Cohen’s kappa value represents the proportion of agreement between raters after chance agreement has been removed, and so is a stricter measure of agreement than percent agreement. We evaluated the strength of agreement between the raters using qualitative benchmarks as defined by Landis and Koch (1977).
Determining suitable machine learning classifier technique for prediction of malaria incidents attributed to climate of Odisha
Published in International Journal of Environmental Health Research, 2022
Pallavi Mohapatra, N. K. Tripathi, Indrajit Pal, Sangam Shrestha
Kappa is the measurement of the inter-rater reliability, representing the extent to which the data collected in the study are correct representations of the variables measured (McHugh 2012). Kappa can be represented as: