Key Takeaway: 

Common training procedures proved effective for training preschool teachers to collect progress monitoring data, an essential skill for teachers, especially those working within the implementation of multi-tiered support systems. While teachers acquired, maintained, and generalized data collection procedures, they did not implement these procedures outside study sessions, highlighting the importance of top-down scaffolds to ensure regular implementation of progress monitoring. – Ashley Parnell

Progress Monitoring

“Progress-monitoring is an essential skill for teachers serving children for whom the general curriculum is insufficient.” “The reason for progress monitoring is simple: it gives you data to measure whether the intervention or instruction a student is receiving is actually helping them close gaps in their learning”.1  

Increased adoption of multi-tiered systems of supports (MTSS) across education programs heightens the need for feasible, routine, and reliable data collection systems. In turn, identifying effective methods for training teachers to implement data collection is crucial. 

The Study 

In response to this need, the current study evaluated the effects of a training package on teacher implementation of data collection procedures in inclusive preschool classrooms.

Two teachers (one lead and one assistant) per class in two different classrooms were trained to implement a teacher-directed behavioral observation (TDBO) data collection, which involved:

  1. Securing student’s attention
  2. Presentation of demand or question
  3. Absence of any form of prompting
  4. Provision of specific praise of correct response or ignoring/responding neutrally to errors.
  5. Scoring of each trial as correct or incorrect.

Each teacher was assigned three children in their respective classroom. All children scored in the bottom 25% of their class on the Assessment, Evaluation, and Programming System (AEPS), a criterion-based assessment, in one of the following areas: concepts, premath, or phonological awareness and emergent reading. Using the AEPS assessment data, one behavior/skill and 5-10 related targets were identified for each child (e.g., labeling of letter sounds, letter A-J; counting objects, numbers 1-5). 

Teacher training package consisted of: 1) a video-based multimedia presentation, 2) a single-page hand-out describing each TDBO procedure, and 3) structured feedback following each training session. Sessions took place in the classroom setting 4 days per week (3 sessions per day with each child assigned to them) for 3 months. Teachers selected the materials and activity in which they collected data on the child’s target skill. Generalization data were collected throughout the study, with the exception of participants that mastered the material during the skill acquisition phase. 

Results and Implications

Results suggest:

  • Commonly used teacher training practices are functionally related to teacher mastery of TDBO procedures for collecting data on child progress within inclusive preschool classrooms. 
  • Generalization probes indicated that teachers may be able to implement and maintain the procedures with fidelity across children and skill types. 
  • Teachers reported never using the data collection procedures outside of study sessions.

Implications to practice include:

  • Lack of data collection outside sessions highlights the need for stakeholders within MTSS models to consider a systems-level approach to development and implementation, as training alone will likely be insufficient in ensuring implementation if there is no oversight.
  • Initial lectures in teacher training are almost always conducted in person. Technology can be leveraged successfully to improve common teacher training practices and reduce in-person training time, promoting accessibility, and flexibility (i.e., video-based multimedia presentation).

Summarized Article:

Shepley, C., Grisham-Brown, J., Lane, J. D., & Ault, M. J. (2020). Training Teachers in Inclusive Classrooms to Collect Data on Individualized Child Goals. Topics in Early Childhood Special Education, 0271121420915770.

Summary by: Ashley M. Parnell — Ashley strives to apply the MARIO Framework to build evidence-based learning environments that support student engagement, empowerment, and passion, and is working with a team of educators to grow and share this framework with other educators.

Academic researcher Collin Shepley participated in the final version of this summary.

Additional References:

  1. Spruell, F. M. (2021, June). Progress monitoring for MTSS at the secondary level. Branching minds. https://www.branchingminds.com/blog/progress-monitoring-secondary-level-mtss

Key Takeaway

Despite decades of growth in the identification of special educational needs and provision of services, academic performance in students with learning disabilities remains lower than their neurotypical peers. Therefore, we must ask if special education services improve academic outcomes for students with disabilities? Results suggest that entering special education early has the biggest impact on students who have been classified with specific learning disabilities. —Frankie Garbutt

Analyzing Public Data on Students with Learning Disabilities and Academic Performance

In this article, Amy Ellen Schwartz (Syracuse University), Bryant Gregory Hopkins (Michigan State University), and Leanna Stiefel (New York University) analyze the “New York City public school data of the 44,000 students with specific learning disabilities over a seven-year period” to investigate the effects of special education services on academic performance.

The researchers point out that “there is surprisingly little evidence to guide special education policy and answer the question of whether services work.” They argue that there is a growing wealth of literature on the effects of school policy on general education students yet a lack of research into the efficiency of special education services. 


The study focuses on students with specific learning disabilities (LDs) “in particular for two reasons. First, LDs are the largest group of students with disabilities (SWD) in 2015, representing 35 percent of SWDs nationally and 40 percent in NYC.1 Second, because the majority of LDs are classified after school entry (typically grades 3 through 8), we observe outcomes both before and after classification.”

The study thoroughly examines “the quantitative literature on the effectiveness of special education,” “background regarding the special education classification process and the nature of learning disabilities,” and “data and models, respectively” before discussing results and conclusions.

Recommendations and Limitations


Overall the research indicates “that special education works to improve outcomes for students with learning disabilities.” As a result, the authors suggest earlier identification for special needs services to ensure gains in students’ performances. Moreover, it is suggested that findings could be generalized for students with learning disabilities, yet “more work investigating the differential effects of alternative service settings could be useful for policymakers.” 

However, it is acknowledged that there are several limitations, i.e. the lack of data analyzed for high school students or kindergarten as the study focused on elementary and middle school students. Similarly, authors could not “distinguish severity of disability within LDs and it is possible that effects differ with severity.”

The paper highlights the positive effect of special education on students’ academic performance and paves the way for methods to evaluate data on the efficacy of policies and practices, as well as starting to build the evidence base to improve special education for all in the USA. 

Summarized Article:  

Schwartz, A. E., Hopkins, B. G., & Stiefel, L. (2021). The effects of special education on the academic performance of students with learning disabilities. Journal of Policy Analysis and Management, 40(2), 480–520. https://doi.org/10.1002/pam.22282 

Summary by: Frankie Garbutt – Frankie believes that the MARIO Framework encourages students to become reflective, independent learners who progress at their own rate.

Additional References:

  1. U.S. Department of Education (USDOE), National Center for Education Statistics (NCES). (2018). Digest of Education Statistics, 2016. NCES 2017–094, Chapter 2.

Key Takeaway

How can we make certain that the technological tools available to educators are effective in engaging and ensuring learning for students who have learning difficulties? Love and Ewoldt suggest having a guide to help educators evaluate the digital resources that help our neurodiverse learners be successful during online learning. —Nika Espinosa

A Guide to Gauge Online Learning Tools

As we continue to navigate online learning as a response not only to education evolving but the worldwide pandemic as well, it is imperative that we scrutinize the different platforms we use to ensure that all our students are engaged, supported, and learning appropriate content. “However, information related to how students with learning disabilities (LDs) access online learning environments has proven difficult to ascertain.”1 There are so many platforms out there; how do we gauge the most effective ones? As our learners are neurodiverse, it is highly unlikely that special educators will find a standardized platform that will suit our learners. Love and Ewoldt instead propose, through the lens of universal design for learning, a guide to gauge these platforms for our students.

“In supporting students with LDs in asynchronous environments, the process for evaluating, implementing, and supplementing asynchronous instructional materials should be systematic in nature.” The authors, in their article, proposed the following guidelines:

Alignment with Standards 

Schools and educators work in environments where academic standards are expected to be achieved. It is important that the digital resources being utilized allow for opportunities, if not certainty, for standards to be met. As special educators, we need to make those clear connections between individualized educational goals and academic standards. However, the authors encourage going beyond just alignment. Once learning targets are determined, educators then need to determine if the instructional digital resources can support our neurodiverse learners.

Addressing Student Needs 

Love & Ewoldt suggest that educators consider the following steps:

  • ensure that an organizer is present for the task;
  • activate background knowledge and make material relevant by connecting to previous learning;
  • clearly establish learning targets;
  • help students in organizing themselves;
  • ensure explicit instruction that includes academic language, key concepts, and opportunities that allow for student application;2 
  • and provide opportunities for students to give and receive feedback.

These points can evaluate whether the instructional materials will be able to reach learning targets and, at the same time, ensure that our student accommodations or scaffolds are met. 

Course Navigation 

Materials used need to be scrutinized on how well they can support our learners’ navigation of both platform and content, presentation of academic language, and how they address student accommodations and modifications. “Significant evidence supports the idea that when information is presented clearly through the material given to students with LDs via digital interfaces, gains in new knowledge and skills can be made in a variety of academic areas.”3 Multimedia used should also present information in such a way that it doesn’t distract the learner from the learning engagement. 

Assessment 

“Finally, special educators should ensure that adequate measures of student progress are available within the tools they are evaluating.” This can look like feedback on how students are responding to the technological resources, but at the same time, it should also include independent tasks that show learning. Special educators can scrutinize whether these resources are able to reflect student learning. 

There are many fantastic digital platforms available to educators to support and enhance learning. “However, given the rapidly evolving nature of research into the technology necessary for delivering online instruction, it is important that guidance be provided to practitioners for establishing and implementing effective online learning environments.” The guidelines proposed by Love and & Ewoldt can help us ensure that our neurodiverse learners will be supported and are learning along with their peers. 

Summarized Article:

Love, M. L., & Ewoldt, K. B. (2021). Implementing Asynchronous Instructional Materials for Students With Learning Disabilities. Intervention in School and Clinic, 105345122110018. https://doi.org/10.1177/10534512211001847

Summary by: Nika Espinosa – Nika believes that personalized learning is at the heart of special education and strives to collaborate with educators in providing a holistic, personalized approach to supporting all learners through the MARIO Framework.

Additional References:

  1. Basham, J. D., Carter, R. A., Rice, M. F., & Ortiz, K. (2016). Emerging state policy in online special education. Journal of Special Education Leadership, 29(2), 70–78.
  2. Hughes, C. A., Morris, J. R., Therrien, W. J., & Benson, S. K. (2017). Explicit instruction: Historical and contemporary contexts. Learning Disabilities Research & Practice, 32(3), 140–148. https://doi.org/10.1111/ldrp.12142
  3. Kennedy, M. J., Thomas, C. N., Meyer, J. P., Alves, K. D., & Lloyd, J. W. (2014). Using evidence-based multimedia to improve vocabulary performance of adolescents with LD: A UDL approach. Learning Disability Quarterly, 37(2), 71–86. https://doi.org/10.1177/0731948713507262

Key Takeaway

The process of identifying reading disabilities and the interpretation of data around the identification of reading disabilities can be inconsistent and conflicting, as they will depend on who is in charge of the process and their training. Since there is considerable variation in the process of identifying reading disabilities, learning support teachers need to not only use current evidence-based and comprehensive assessments to identify and diagnose reading disabilities in a timely manner, but they also need to administer the appropriate interventions for learners to achieve educational success. —Michael Ho

“An estimated 10% to 15% of U.S. school-age children are identified with reading disabilities. Without consistent identification approaches, practitioners may lack a shared understanding of what constitutes RDs and, consequently, how to address areas of challenge in education plans.” On the other hand, a shared understanding of what leads to RDs can lead to effective instruction.

Al Dahhan, Mesite, Feller, Christodoulou (2021) administered a survey across the United States to identify current practices associated with the identification of reading disabilities (RDs). They specifically examined three areas: (a) who identifies and/or diagnoses RDs and what their roles are in this process, (b) the training that these practitioners have received relevant to this process, and (c) the current processes used by practitioners in educational and clinical settings to identify/diagnose RDs.

965 practitioners, including classroom teachers, special educators, reading specialists, school psychologists, and speech-language pathologists were invited to participate in the Reading Diagnostics Survey, and their responses were analyzed.

Variations in Approaches to Identifying RDs

Across school districts and states, there is a range of different definitions, eligibility criteria, diagnostic processes, guidelines, and policies for identifying RDs in both school settings. There are also differences among these features between school and clinical settings. 

Dahhan et al. (2021) refer to Mellard et al. (2009)1 and Scruggs and Mastropieri (2002)2—“Few studies reported on the variability in choice of reading assessments, cutoff points for test scores, pre-referral and/or progress monitoring approaches, magnitude of discrepancies between scores (when applicable), definition of adequate progress, and use of professional judgments.”

Additionally, there are inconsistencies among school districts and states on the use of the IQ/Achievement discrepancy criteria, use of Response to Intervention (RTI) and Multi-Tiered System of Supports (MTSS) models, and use of Personal Support Worker (PSW).

Practitioner Roles in Identifying RDs

In response to the first area of ‘Practitioner Roles in Identifying RDs’, participation reported the following practitioners, from most to least, directly assessing students for suspected RDs: school psychologists, speech and language pathologists, special educators, reading specialists, and classroom teachers. 

Multiple professionals conduct reading and writing assessments to identify RDs, while cognitive and language assessments tend to be conducted by school psychologists and speech and language pathologists.

Practitioner Training

In response to the second area of ‘Practitioner Training’, speech and language pathologists generally reported receiving less graduate training, while school psychologists frequently reported more graduate training on identifying RDs than those in other professions.

Practitioners in clinical settings and those with more training on this topic report higher levels of confidence compared to practitioners in school settings.

Measures and Procedures used to Identify RDs

In response to the third area of ‘Measures and Procedures used to Identify RDs”, more than 75% of participants indicated that they always evaluate word reading, reading comprehension, and reading fluency.

Practitioners in clinical settings less frequently indicated that they select measures based on accessibility and more frequently indicated that they select measures based on their validity and reliability. On the other hand, school-based practitioners primarily use measures available in their setting that they have been trained to use. 

The most commonly reported criteria included: failure to respond to intervention, an IQ/Achievement Discrepancy, and scoring a standard deviation or more below the population mean. 

The differences show that the criteria used to identify specific learning disabilities in reading vary across, and sometimes within, school settings.

Limitations

The limitations in this study are mainly related to the recruitment process.

Given the nonrandom sampling approach, these results cannot be expected to generalize to all practitioners across school and clinical settings in the United States. Moreover, participants from Massachusetts were oversampled and

medical professionals with roles in diagnosing RDs were underrepresented.

It is recommended in future studies to recruit more representative samples of practitioners, conduct qualitative evaluations that include practitioner

Interviews, and explore the role of student-level characteristics and contextual factors.

Summarized Article:

Al Dahhan, N. Z., Mesite, L., Feller, M. J., & Christodoulou, J. A. (2021). Identifying Reading Disabilities: A Survey of Practitioners. Learning Disability Quarterly, 44(4), 235–247. https://doi.org/10.1177/0731948721998707

Summary by: Michael Ho—Michael supports the MARIO Framework because it empowers learners to take full control of their personalized learning journey, ensuring an impactful and meaningful experience.

Additional References:

  1. Mellard, D. F., McKnight, M., & Woods, K. (2009). Response to Intervention screening and progress-monitoring practices in 41 local schools. Learning Disabilities Research & Practice, 24(4), 186–195. https://doi.org/10.1111/j.1540-5826.2009.00292.x
  2. Scruggs, T. E., & Mastropieri, M. A. (2002). On babies and bath-water: Addressing the problems of identification of learning disabilities. Learning Disability Quarterly, 25(3), 155–168. https://doi.org/10.2307/1511299

Key Takeaway: A number of factors affect the perception of key stakeholders in relation to the fairness of assessment practices for students with learning differences. Elements such as student disability, existing assessment processes, the socio-emotional environment, stakeholders’ conceptions of fairness, and contextual facilitators and barriers to inclusive practices interact to influence the overall fairness factor of classroom assessment. Having an awareness of this multidimensional conceptualization of fairness is helpful in evaluating whether assessment practices are offering equal opportunities to demonstrate learning, and also scaffolds students’ ability to self-advocate for their needs. -Akane Yoshida

“Creating inclusive classrooms has been a justice movement in education,” say Rasooli et. al., and in this paper they seek to fill the void they find in current literature regarding fairness in assessment practices by adding the voices of students with learning differences, their parents, and their teachers to the mix. 

Their paper contributes a framework for fairness in assessment as “a multidimensional concept that is negotiated and navigated in the cyclical and dynamic interactions with classroom teaching and interactions.” According to the authors, this conceptualization is “closely tied with the sociocultural theories of assessment that recognise the social, cultural and economic milieu within which teachers and students interpret and enact fairness in assessment.”

The study methodology describes a process by which data was pulled from open-ended surveys submitted by teachers, students, and their parents from 19 secondary schools across Australia. The questionnaires included such queries as “How was the assessment adjusted for you?” for the student survey, “Do you think this adjustment better allowed [your child] to demonstrate what [they] knew or could do?” for the parent survey, and “Do you think you would adjust assessment differently in the future for this student? If yes, please comment on what changes you would make.” for the teacher survey. Inductive and thematic coding was used by the researchers to identify themes in the responses. Through this analysis, four larger themes emerged: “conceptions of fairness, fair classroom assessment practices, fair socio-emotional environment and contextual barriers and facilitators of fair practices.”

Summarized below are the findings in relation to each theme:

  1. Overall conceptions of fairness: Participants expressed equal accessibility for all students as being the greatest determinant of fairness in assessment. Adjustments to assessment practices were thought to be fair when they offered students with learning differences optimal opportunity for success in line with mainstream expectations.
  1. Fair classroom practices: Three sub-themes emerged from the responses as factors that can support or hinder fairness in assessment:
  • Differentiation of the assessment preparation process and design (accessibility of the mode of assessment, clarity in the task format and expectations, as well as the opportunity to prepare for the assessment)
  • Differentiation of assessment settings and environment (provision of a quiet space, additional time and breaks) 
  • Differentiation of assessment scheduling (ensuring that multiple assessments do not occur within a short period of time)
  1. Fair socio-emotional environment: Three sub-themes emerged here as well:
  • Student self-concept 
  • Impact of the learning difference on the socio-emotional environment
  • Relationships with teachers and peers
  1. Contextual barriers and facilitators of fair practices: Participants identified school and national-level policies, teacher experience, availability of paraprofessionals and other human resources, class size and parent influence as being the most influential factors in fair assessment.

While the study drew upon participants from a variety of grade levels and learning differences, it concedes that future research involving a larger sample size from a wider range of educational systems would be necessary in order to lend greater credibility to its conclusions. 

Summarized Article:

Rasooli, A., Razmjoee, M., Cumming, J., Dickson, E., & Webster, A. (2021). Conceptualising a Fairness Framework for Assessment Adjusted Practices for Students with Disability: An Empirical Study. Assessment in Education: Principles, Policy & Practice, 1-21.

Summary by: Akane Yoshida—Akane believes that developing supportive and nurturing relationships with students is key to helping them to attain their personal benchmarks for success. She loves how the MARIO Framework operationalizes this process and utilizes systematic measurement of student learning and teacher effectiveness to guide interventions.

Key Takeaway: This article highlights whether the use of a teacher evaluation tool encourages instruction that responds to the needs of students with learning disabilities. The authors suggest a tool that cultivates the teaching of skills to learners in sequences to allow for practice and reflection, consequently leading to mastery, and the inclusion of direct and explicit instruction to allow educators to react and adapt to the individual learner’s needs as they progress through their learning journey. —Frankie Garbutt

Danielson’s Framework for Teaching

“All teachers are evaluated using the same tool, regardless of the teacher’s role, suggesting that the instructional approach supported by one tool would meet the needs of all students,” say Hannah Morris- Matthews, Kristabel Stark and Nathan Jones (Boston University), Mary Brownell ( University of Florida) and Courtney Bell (Educational Testing Service, Princeton) in this Journal of Learning Disabilities article. The authors investigated whether Danielson’s Framework for Teaching (FFT) is a teacher evaluation tool which encourages instruction that adequately responds to the needs of students with learning disabilities.

Review of the Literature

The use of observation tools, to evaluate teacher practices and identify professional development needs, can be “agnostic and universal,” thus creating the assumption that one instructional approach benefits all learners. The academics rooted their research in the Load Reduction Theory (LRI) because previous studies suggest that students with learning disabilities benefit from direct and explicit instruction. LRI practices “avoid overburdening the working memory, and facilitate productive interaction between long-term and working memory.” This is achieved through using the pillars of LRI teaching practices (intensive, explicit, systematic and individualized instruction) in order to support learners with cognitive disabilities.

The Findings

The study addressed two questions: 

1. What assumptions about instructional quality are present in Danielson’s FFT? 

2. To what extent does Danielson’ FFT make practices associated with LRI visible?

The methodology of the study looked at the language of the observation tool due to its role in “defining, evaluating and developing good teaching.” Their analysis found that practices that reduce cognitive load are rare and instead favour practices which are student-driven and focused on making sense of complex content. They concluded that “observers using FFT would direct these teachers to practices that would likely serve as barriers to equitable and efficient learning opportunities.”

Limitations

Nonetheless, the limitations of this study were acknowledged as it had not investigated “how FFT might operate in practice” because the analysis “does not provide insight into the ways that raters make use of the tool.” Moreover, the focus “foregrounds the needs of students whose disability influences cognitive processing,” and thus, “does not explicitly speak to the needs of all students with disabilities.” Consequently, it was suggested that further research is required into how the framework is used as a whole and not just segments of the teacher evaluation tool. The authors suggest that future researchers “query how observers use and make sense of the rubrics to better understand the processes through which they arrive at ratings and determine directions for professional development.”

Conclusion

The conclusion was that FFT as an instrument “may not be an appropriate mechanism through which to support a continuum of effective instruction for students with learning disabilities and other struggling learners.” The researchers proposed to root observation tools in the cognitive load theory, recognising the need for a diverse tool box with practices that can respond to learners’ needs.

Article Summarized:

Morris-Mathews, H., Stark, K. R., Jones, N. D., Brownell, M. T., & Bell, C. A. (2020). Danielson’s Framework for Teaching: Convergence and Divergence With Conceptions of Effectiveness in Special Education. Journal of Learning Disabilities, 54(1), 66–78. https://doi.org/10.1177/0022219420941804

Summary By: Frankie Garbutt- Frankie believes that the MARIO Framework encourages students to become reflective, independent learners who progress at their own rate.

Article Abstract

Evidence-based practices (EBPs) emerge as inherent to the successful implementation of a comprehensive and combined multi-tiered system of support (MTSS) model. The intended result of multi-tiered intervention progression combined with EBP is a validated, data-based approach to understanding students’ needs along with a description of what promotes or inhibits their academic and social–emotional and behavioral performance. The purpose of this chapter is to present a combined research- and practice-based framework for integrating a comprehensive MTSS model with EBP, and thus, optimize the results stemming from school improvement efforts. Toward this goal, EBPs and strategies are reviewed to address concerns in the academic and social–emotional and behavioral domains along with recommendations for their application within MTSS.

Schools are in the midst of intensive educational reform. A comprehensive multi-tiered model to address a full range of academic, behavioral, and social needs among children and youth is envisioned as a preferable practice model in response to reform initiatives. To attain the goal of successful multi-tiered intervention implementation, however, the model must incorporate evidence-based practices (EBPs). The chapter begins with an overview of EBPs and the rationale for incorporating them within a multi-tiered system of support (MTSS) model. Next, applications of MTSS and EBP in academic domains and social–emotional and behavioral domains are provided. These sections highlight specific strategies for successful implementation at tier 1, tier 2, and tier 3. Key questions that have emerged regarding EBP approaches within MTSS are addressed. Finally, a summary of the current status of EBP and MTSS is presented along with future research and practice implications.

MARIO Connections

Stoiber and Gettinger’s study connects to the MTSS thread woven throughout the MARIO Framework. In particular, this study drives our work around evidence-based practices and how we measure the success of the interventions we utilize.