Construct validity of the Bar-On modelDate of publication: 08/07/2007 This section is still being developed but will eventually contain complete summaries of studies that have examined the construct validity of the Bar-On model of emotional-social intelligence. In the meantime, I have summarized the key studies that I am aware of. Should you wish to share findings from a study that you have conducted or have detailed information on studies that others have conducted focusing on this topic, please use the template provided above for summarizing this study and email it to us (info@reuvenbaron.org). You are invited to provide results that confirm or refute these findings and help us understand this area better. In order to demonstrate that a concept is robust, one must first show that it is actually describing what it was designed to describe. This is usually done by examining its construct validity. There are a number of basic approaches to examining the construct validity of psychometric and conceptual models [Anastasi, 1988]. The approach that I have adopted was to simply demonstrate that the EQ-i correlates higher with other measures of emotional-social intelligence than with measures of other constructs such as cognitive intelligence and personality. The findings confirm that the EQ-i has the least amount of overlap with tests of cognitive tests. This is followed by findings indicating a greater degree of overlap with personality tests. And the greatest degree of domain overlap exists between the EQ-i and other EI measures.
In an effort to examine the divergent construct validity of the Bar-On model, the EQ-i has been concomitantly administered with various measures of cognitive intelligence (including the Wechsler Adult Intelligence Scale, Progressive Raven Matrix, and the General Adult Mental Ability Scale) to a total of 4,218 individuals in six studies [Bar-On, 2004]. The results indicate that there is only minimal overlap between the EQ-i and tests of cognitive (academic) intelligence, which was expected in that this instrument was not designed or intended to assess this type of performance. This finding is also confirmed by David Van Rooy and his colleagues [Van Rooy & Viswesvaran, 2004; Van Rooy et al., 2004; D. L. Van Rooy, personal communication from April 2003], who suggests that no more than 4% of the variance of the EQ-i can be explained by cognitive intelligence according to a recent meta-analysis including 10 studies (n>5,000). In addition to shedding light on the construct validity of the Bar-On model and measure of emotional-social intelligence (i.e., what it is and is not describing), these findings indicate that emotional-social intelligence and cognitive intelligence are not strongly related and are most likely separate constructs. Not only is this assumption statistically supported by findings presented by me and others [Bar-On, 2004; Van Rooy & Viswesvaran, 2004; Van Rooy et al., 2004], but there is also neurological evidence suggesting that the neural centers governing emotional-social intelligence and those governing cognitive intelligence are located in different areas of the brain. More succinctly, the right amygdala, right somatosensory and insular cortices, ventromedial prefrontal cortex and anterior cingulate appear to be governing basic aspects of emotional-social intelligence [Bar-On et al., 2003; Bechara & Bar-On, 2006; Bechara, Damasio & Bar-On, 2006; Lane & McRae, 2004], while the dorsolateral prefrontal cortex is thought to govern key aspects of cognitive functioning [Duncan, 2001].
Subsequent to submitting their pioneering meta-analysis of emotional intelligence for publication in December 2002, Van Rooy and Viswesvaran expanded the number of studies in their original analysis of the construct validity of emotional intelligence. Their most recent meta-analysis suggests that the degree of overlap between the EQ-i and personality tests is no more than 15% based on 8 studies in which more than 1,700 individuals participated [D. L. Van Rooy, personal communication from April 2003]. This overlap is smaller than was previously thought and strongly suggests that the EQ-i must be measuring something else other than personality traits. It also makes sense that the EQ-i is not measuring personality traits, because the 15 emotional and social competencies, skills and facilitators that it measures (a) increase almost continuously from childhood to the end of the fourth decade of life as was previously mentioned, and (b) they can also be significantly increased within a matter of a few weeks as a result of training [Bar-On, 2003, 2004]; personality traits are simply not as malleable as these factors appear to be. When this small degree of overlap with personality is coupled with the even smaller degree of overlap with cognitive intelligence, the large unexplained variance that remains logically suggests that the EQ-i is measuring something else other than these constructs; and based on what is presented below, I argue that a substantial amount of this unexplained variance in the Bar-On model and measure can be explained by a larger domain overlap which is observed when the EQ-i is correlated with other measures of emotional-social intelligence. More precisely, the degree of significant overlap between the EQ-i and these other EI measures is nearly twice as high as that explained by personality and cognitive intelligence combined.
In order to examine the convergent construct validity of the Bar-On model and measure, the correlation between the EQ-i and other EI instruments was evaluated. In another publication [Bar-On, 2004], I have summarized the major findings related to the convergent construct validity of the EQ-i based on 13 studies in which a total of 2,417 individuals participated. These findings indicate that the degree of domain overlap between the EQ-i and other EI measures is about 36%, which is substantial when evaluating construct validity [Anastasi, 1988]. When compared with a 4% overlap with IQ tests and a 15% overlap with personality tests, it is obvious that the EQ-i is measuring what these other EI measures are measuring (i.e., emotional-social intelligence) rather than cognitive intelligence or personality traits.
The above findings suggest that EQ-i possesses good construct validity – i.e., for the most part, this instrument is measuring what it was designed to measure. This suggests that the Bar-On model is a valid concept of emotional-social intelligence in that it is describing key aspects of emotional-social intelligence rather than other psychological constructs such as cognitive intelligence or personality. Empirically demonstrating this point [Bar-On, 2004] is thought to dispel what some psychologists have assumed regarding the Bar-On conceptual and psychometric model and have prematurely concluded based on less extensive and conclusive findings [e.g., Brackett & Mayer, 2003; Matthews et al., 2002; Newsome et al., 2000].
Other EI measures, such as the ECI and MSCEIT, have not yet examined construct validity as robustly as has been done with the EQ-i on larger and more diverse samples. Unfortunately, very few published studies have yet examined the degree of correlation between the MSCEIT and other EI measures; and most of the existing publications present primarily divergent evidence for the MSCEIT’s construct validity. However, it is insufficient to assess the construct validity of a measure by examining only its divergent construct validity (i.e., what it is not measuring); one must logically present convergent construct validity as well (i.e., what it is measuring). In order to establish that a particular measure of a psychological construct is psychometrically sound, it is axiomatic in test construction to examine and compare both divergent as well as convergent evidence [Anastasi, 1988; Campbell & Fiske, 1959].
When the findings related to the EQ-i are compared with the actual degree of domain overlap between ability-based measures of emotional-social intelligence and tests of cognitive intelligence as well as personality [Van Rooy & Viswesvaran, 2004; Van Rooy et al., 2004], the accuracy, meaningfulness and usefulness of dichotomously describing these measures as either “mixed” or (non-mixed) “ability” models come into question. On the one hand, the EQ-i overlaps with cognitive intelligence and personality tests no more than 20% while the degree of overlap between the MSCEIT and these types of tests does not exceed 15% [Bar-On, 2004; Van Rooy & Viswesvaran, 2004; Van Rooy et al., 2004; D. L. Van Rooy, personal communication from April 2003]. In other words, the vast majority of the variance of both conceptual and psychometric models (80% and 85% respectively) is not explained by personality and/or by cognitive intelligence. Therefore, the “mixed” characteristic used by some [Mayer et al., 2000] to describe some of these models, exists in all such models and measures in that they all overlap with personality traits and cognitive intelligence to some extent, but the actual difference between them within this small degree of overlap does not justify using descriptors such as “mixed” versus “abilities” as a meaningful way of categorizing these models and measures. All models of human behavior are influenced at least to some extent by a “mixed” cross-section of bio-psycho-social predictors and facilitators including biomedical predispositions and conditions, cognitive intelligence, personality, motivation and environmental influences. This argument has been made in psychology more than a quarter of century ago [Bem & Allen, 1974]; and more than half a century ago, David Wechsler specifically argued that part of this “mix” impacts intelligent behavior [Wechsler, 1940, 1943].
|
|
