ABOUT MEASUREMENT

The tale begins with an assortment of semi-mythical early scientists, who agreed in their observations that lead only melts when it is very hot—much hotter than the temperature at which ice melts, and quite a bit cooler than the temperature at which iron melts. These observations, made repeatedly, resulted in the hypothesis that lead melts at a particular temperature. To test this theory it was necessary to develop a standard for measuring temperature. A variety of early thermometers were developed and implemented. Partly because these early temperature-measuring devices were poorly calibrated, and partly because different temperature-measuring devices employed different scales, the temperature at which lead melted seemed to vary from device to device and context to context. Scientists divided into a number of ‘camps’. One group argued that there were multiple pathways toward melting, which explained why the melting seemed to occur at different temperatures. Another group argued that the melting of lead could not be understood apart from the context in which the melting occurs.

Only when a measure of temperature had been adequately developed and widely accepted did it become possible to observe that lead consistently melts at about 327º C. Armed with this knowledge, scientists asked what it is about lead that causes it to melt at this particular temperature. They then developed hypotheses about the factors contributing to this phenomenon, observing that changes in altitude or air pressure seemed to result in small differences in its melting temperature. So, context did seem to play a role! In order to observe these differences more accurately, the measurement of temperature was further refined. The resulting observations provided information that ultimately contributed to an understanding of lead’s and other elements’ molecular structure.

While parts of this story are fictional, it is true that the thermometer has greatly contributed to our understanding of the properties of lead. Interestingly, the thermometer, like all other measures, emerged from what were originally qualitative observations about the effects of different amounts of heat that were quantified over time. The value of the thermometer, as we all know, extends far beyond its use as a measure of the melting temperature of lead. The thermometer is a measure of temperature in general, meaning that it can be employed to measure temperature in an almost limitless range of substances and contexts. It is this generality, in the end, that makes it possible to investigate the impact of context on the melting temperature of a substance, or to compare the relative melting temperatures of a range of elemental substances. This generality (or context-independence) is one of the primary features of a good measure.

 

1. the identification of a unidimensional, content and context-independent trait (temperature, length, time);
   

2. a system for assessing the amount of the trait;
   

3. determinations of the reliability and validity of the assessments; and finally
   

4. the calibration of a measure.

A good thermometer has all of the qualities of a good measure. It is a well-calibrated instrument that can be employed to accurately and reliably measure a general, unidimensional trait across a wide range of contexts.

 

In a sense, good measures make scientific progress possible by ensuring that scientists in a given field are speaking a common language. What if cognitive scientists had access to an accurate, valid, and reliable general measure of cognitive development, one that spanned the developmental continuum from birth through adulthood? What might be some of the implications for cognitive research and education?

In the 19th century, James Mark Baldwin described a series of developmental levels in children’s and adolescents’ reasoning abilities. He saw each of these levels as changes in the way individuals thought, not just what they thought. In the 20th century, scientists like Jean Piaget expanded upon these insights, describing several different ways of thinking that built upon one another over the course of childhood and adolescence. During the 1970’s and 1980’s, researchers like Karen Kitchener, Patricia King, Lawrence Kohlberg, Robert Kegan, and Kurt Fischer documented similar changes in adulthood. In the 1990’s Theo Dawson undertook the task of translating their qualitative descriptions of developmental levels into a quantitative measure of cognitive development. The result is a developmental metric called the Lectical Assessment System (LAS). Like the thermometer, the LAS is can be employed to accurately and reliably measure a general, unidimensional trait across a wide range of contexts.

 

First, the LAS, because it is content independent, can be employed to investigate conceptual development in any knowledge domain—just as a thermometer can be used to check the temperature of any substance. Once reasoning performances are assigned to their place on the developmental dimension, they can be subjected to a variety of content analyses. Matrices of conceptual content by developmental level reveal patterns of conceptual change that are very difficult to expose with conventional methods. While individual growth can only be studied longitudinally (Singer & Willett, 2003), a developmental metric makes it possible to meaningfully examine inter-individual developmental trends in cross-sectional data, putting an end to the questionable practice of using age as a proxy for development (Dawson, Commons, & Wilson, 2005).

Second, the LAS helps to eliminate the effects of sample bias in accounts of conceptual development. Conventional accounts of conceptual development are generally constructed by examining the behavior of individuals in small longitudinal samples. These accounts then form the basis for developmental assessment systems that often confound developmental level and conceptual content, such that particular concepts come to be overly identified with a given developmental level (Dawson, in press; Dawson & Gabrielian, 2003; Dawson et al., 2003). Because the LAS allows us to specify an individual’s place on the developmental continuum without reference to the particular conceptual content of his or her reasoning performance, we are able to examine the empirical relation between particular conceptual content and a given developmental level, making it possible to interpret that relation as part of an independent analysis.

Third, the LAS can be employed to describe conceptual development across the entire developmental continuum, producing seamless accounts of development that can be employed to inform our understanding of developmental processes as well as curriculum design, instruction, and assessment. Though they have contributed importantly to our understanding of conceptual development in science, current accounts of conceptual development in the sciences are generally piecemeal, either because the research targets a particular age-group, or the developmental model being employed—such as the novice/expert model—dictates the comparison of two extreme groups. Attempts to tie together isolated results are complicated by the lack of a strong and coherent developmental theory. The strand maps presented in the Atlas of Science Literacy (2001), represent an important effort to define learning sequences based on the general notion that development moves from concrete to abstract or from simple to complex. An accurate and reliable developmental metric would lend much greater specificity to efforts of this kind.

 

Fourth, the LAS makes it possible to meaningfully compare developmental progress across knowledge domains. This means we can create a developmental report card like that shown in the figure below, in which an employee’s developmental progress in multiple skill areas is traced over time. This figure includes developmental zones, indicating the relation between the task demands of each management level and developmental (skill) levels.

Employees can use information about their reasoning skills to improve their performance and prepare themselves for advancement. For example, the figure above shows the developmental progress made by a manager (now in a Senior level 1 position) over the course of twelve years in four critical skill areas. At year fourteen she displays a wide range of competence across these four skill areas. Her scores on decision making and ethical reasoning are in the Senior level 1 range, and her score on leadership reasoning is in the Senior level 2 range, indicating that in these areas, she has strong skills that match or exceed the difficulty level of her Senior 1 management position. However, her reflective judgment skills are in the Mid-level management range. Equipped with this information, she can optimize her development as a manager by engaging in learning activities that will move her reflective judgment skills from unelaborated abstract systems to highly elaborated abstract systems. Once all of her skills are in the highly elaborated abstract systems range, she can more readily make the transition into single principles reasoning, which will prepare her to meet the task demands of Senior 2 management.

 

 

Fifth, the LAS can be employed longitudinally to compare the developmental trajectories and pathways of individuals. Accounting for the developmental dimension would reduce the danger of conflating developmental and other effects. For example, much cross-cultural research employs age or school grade as a proxy for development. When differences in conceptions are found, these are often attributed to cultural differences. This practice is highly questionable, because, though age and developmental level are correlated in childhood and adolescence, they are far from an identity (Armon & Dawson, 1997; Dawson et al., 2005; Fischer & Bidell, 1998).

Sixth, the LAS can be employed to link results from existing research, providing a more coherent picture of conceptual development. The LAS acts as a single developmental dimension along which existing research findings can be arranged and reassessed.

Seventh, the LAS can be employed to inform curriculum development. The link between students’ cognitive developmental level and the likelihood that they will profit from instruction is already well-established (Case & Okamoto, 1996; Cavallo, 1996; Germann, 1994; Lawson, Alkhoury, Benford, Clark, & Falconer, 2000; Lawson & Renner, 1975; Lawson & Thompson, 1988; Lawson & Weser, 1990; Renner & Marek, 1990; Shayer & Adey, 1993). By employing the LAS to study conceptual development, we are able to specify the relation between cognitive development and conceptual learning in a particular content domain. This provides support for a cognitive developmental pedagogy in the form of concrete knowledge about the way in which particular concepts develop (typically, idiosyncratically, and optimally). This knowledge can be used to design learning and performance standards as well as curricula that reflect these standards.

Finally, The LAS can be employed in assessment and evaluation. Research, assessment, evaluation, and curricula can be coordinated around a single developmental agenda in which the same metric that informs curriculum development, is employed to evaluate classroom learning, compare the effectiveness of different curricula, and provide the basis for practical scoring rubrics for educators.