Research & Publications

Quantile Maps

Fact Sheets and Brochures


  • Novel Interpretations of Academic Growth
    by Gary L. Williamson, Ph.D.; November, 2016
    Featured in: JAEPR, Vol 2, No. 2.
    Integrating a construct theory with Rasch measurement not only places persons and tasks on a common scale, but it also resolves the indeterminacy of scale location and unit size when the scale is anchored in an operationalized task continuum based on the construct theory. Such an approach has several advantages for understanding academic growth as evidenced in a series of empirical examples, which demonstrate how to: a) conjointly interpret student reading growth in the context of reading materials concomitantly used during instruction; b) interpret a reading growth trajectory in light of future (e.g., postsecondary) reading requirements; c) forecast individual reading comprehension rates relative to both contemporary and future text complexity requirements; and d) create growth velocity norms for average academic growth in reading or mathematics achievement.
  • An Investigation of Dimensionality Across Grade Levels and Effects on Vertical Linking for Elementary Grade Mathematics Achievement Tests (PDF)
    by Samantha S. Burg, Ph.D.; NCME; March, 2008
    It is a widely held belief that mathematical content strands reflect different constructs which produce multidimensionality in mathematical achievement tests for Grade 3-8. This study analyzes the dimensional structure of mathematical achievement tests aligned to NCTM content strands using four different methods for assessing dimensionality. The effect of including off-grade linking items as a potential source of dimensionality was also considered. The results indicate that although mathematical achievement tests for Grades 3-8 are complex and exhibit some multidimensionality, the sources of dimensionality are not related to the content strands or the inclusion of several off-grade linking items. The complexity of the data structure along with the known overlap of mathematical skills suggest that mathematical achievement tests could represent a fundamentally unidimensional construct.
  • The Quantile® Framework for Mathematics Quantifies the Mathematics Ability Needed for College and Career Readiness (PDF)
    by Gary L. Williamson, Ph.D.; Eleanor Sanford-Moore, Ph.D.; Lisa Bickel; MetaMetrics®, July, 2016
    The objective of this research is to answer the question, “What mathematics must a student be capable of performing to be ready for college or a career?” To address the question we analyzed mathematical concepts and skills that students may encounter as they begin their postsecondary education and/or enter the workplace. The answer is predicated on two perspectives: (a) mathematical readiness for college implies being ready for instruction in advanced mathematics courses associated with the beginning of the postsecondary educational experience; and, (b) readiness for the mathematical demands of careers implies, at a minimum, sufficient mathematical ability to perform well on the mathematics content required for a high school diploma. To answer the key question, we analyzed the difficulty of mathematical skills and concepts incorporated into the mathematics lessons found in mathematics texts commonly used in the United States. The Quantile® Framework for Mathematics provides the measurement foundation to place on a common scale both student mathematics ability and the difficulty of mathematical skills and concepts. Thus, we infer requisite student ability from the observed difficulty of mathematical skills and concepts contained in mathematics lessons presented in mathematics textbooks. We regard mathematics ability as an individual, malleable attribute, which improves with instruction and practice.
  • Initial Validation of Theory of Task Difficulty and Creation of Item Families (PDF)
    by Mary Ann Simpson, Jeff Elmore, Lisa Bickel, and Ruth Price, MetaMetrics®, April 15-19, 2015
    Initial validity results concerning a unified theory of mathematics task difficulty are reported. Data examined contained student responses to automatically generated computational fluency problems in addition, subtraction, multiplication, and division. Multilevel linear modeling produced results largely congruent with past research. Results concerning the construction of theory-based item families were mixed. Implications of the results are discussed.
  • Construction of a Dynamic Item Generator for K-12 Mathematics (PDF)
    by Mark Kellogg, Steve Rauch, Ryan Leathers, Mary Ann Simpson, David Lines, Lisa Bickel, and Jeff Elmore, MetaMetrics®, April 15-19, 2015
    MetaMetrics researchers developing a theory of task difficulty in mathematics needed a tool to generate large quantities of mathematics items with specific feature criteria. The “math item generator,” or MIG project, was undertaken in order to make affordable the creation of mathematics items at scale. This paper describes three approaches to item generation – steps, step-sets, and orchestrations – along with constraints that can be combined to produce mathematics items. The last section of the paper talks about the integration and implementation of the MIG.
  • Family Group Generation Theory - Large Scale Implementation (PDF)
    by Lisa Bickel, Mary Ann Simpson, Eleanor Sanford-Moore, Ruth Price, Lela Durakovic, Sandra Totten, and Audra Kosh, MetaMetrics®, April 15-19, 2015
    The difficulty of a mathematical task depends on its key features. Although the constructs underlying task difficulty may be similar across topics in K-12 mathematics, actual task features generally differ. The genesis of the Family Group concept was a semantic and syntactic theory of mathematical task difficulty. The semantic components (components that affect meaning) considered in the theory are underlying concepts, number types, vocabulary, symbols, task types, and object types. The syntactic component (component that considers the arrangement of elements) is the number of distinct steps needed to solve a problem (the number of sub-tasks). The plans and progress of a work group engaged in identifying and quantifying such features is described.
  • A Unified Theory of Task Difficulty in K-12 Mathematics (PDF)
    by A. Jackson Stenner, M. A. Simpson, W. P. Fisher, Jr., and D. S. Burdick, MetaMetrics®, April 15-19, 2015
    The authors present a unified theory of task difficulty in K-12 mathematics. In a framework of text complexity, the difficulty of English text may be measured by the text’s syntactic complexity and semantic difficulty (Stenner, Burdick, Sanford, & Burdick, 2006). Just as natural language is a sign system, so is mathematics. The authors endeavored to identify potential syntactic and semantic elements that can be used to describe mathematics task difficulty. Informal review of the research literature in education, cognitive and developmental psychology, and cognitive psychometrics as well as consultation with subject matter experts suggested feature classes for the specification of the semantic and syntactic elements of task difficulty. Practical and philosophical benefits, including the facilitation of automatic item generation, are discussed.
  • A Quantitative Task Continuum for K-12 Mathematics (PDF)
    by Eleanor E. Sanford-Moore, Ph.D., Gary L. Williamson, Ph.D., Lisa Bickel, Heather Koons, Ph.D., Robert F. Baker, Ph.D., Ruth Price, MetaMetrics®, November 12, 2014 
    This research quantifies the difficulty of mathematics lessons drawn from mathematics textbooks commonly used in the United States. It also documents the mathematical complexity of textbook lessons within and across grades. Lessons were extracted from selected textbooks used in grades K-12 in the United States and analyzed. Textbooks aligned with the Common Core State Standards for Mathematics (CCSSM) and those not aligned with CCSSM were measured using the Quantile Framework for Mathematics, and each lesson was assigned a Quantile measure to represent its mathematical difficulty. The results of this research show that the median mathematical difficulty of textbook lessons consistently increases with grade, and that within grades, lessons vary in their mathematical complexity.

  • The Quantile Framework for Mathematics: Linking Assessment with Instruction (PDF)
  • Weaving Mathematical Connections from Counting to Calculus: Knowledge Clusters and The Quantile Framework for Mathematics(PDF) by Bethany S. Hudnutt
  • A Mathematics Problem: How to Help Students Achieve Success in Mathematics Through College and Beyond (PDF) by Malbert Smith III, Ph.D and Jason Turner
  • Supporting Differentiated Math Instruction in a Common Core World (PDF) by Malbert Smith III, Ph.D and Jason Turner
  • The Need for Differentiating Mathematics Instruction (PDF) by Malbert Smith III, Ph.D
  • The Quantile Framework for Mathematics in the Home (PDF) by Ruth Price and Jill Cassone
  • Using Chilldren's Literature to Teach Mathematics (PDF) by Ruth Price and Colleen Lennon