Sub-Badge 1: Evaluate Instructional and Non-Instructional Interventions
Evaluation and Implementation
Challenge 1: Implement formative evaluation plans
Criteria for successful completion of this challenge: Evidence of implementing a formative evaluation plan to provide information that can be used to make adjustments and improvements in the design. Evidence must show a formative evaluation plan (expert review, one-to-one evaluation, small group, and field trial). Reflection must address: Which phase(s) of formative evaluation did you conduct? Which data did you collect (e.g., clarity and accuracy of instruction, general attitudes, procedural issues, etc.)? What were the results of formative evaluation and how did it affect your design?
Examples: Evaluation Plan (EDCI 528), Design Documents (EDCI 572), Learning Module (EDCI 575), eLearning Project (EDCI 569), artifacts showing strategies for implementation of an evaluation plan (design, performance, workplace, educational, other).
Reflection
The competency I am addressing is “Implement formative evaluation plans.” The artifact I am submitting is my Food Photography Composition Basics module developed in Articulate Rise for EDCI 569, along with the storyboard and design document created as part of the planning process. These materials document both the course content and the formative evaluation process I conducted through informal one-to-one testing. This evaluation took place during the development phase and directly impacted the instructional decisions I made prior to submission.
To evaluate clarity, pacing, and instructional alignment, I asked my mother to complete the module as a representative learner and provide real-time feedback. I used a custom checklist to gather her impressions of navigation, visual layout, and clarity of instructions. Her feedback pointed out that some instructions were too text-heavy and that certain buttons weren’t clearly labeled. Based on this, I revised several activity prompts, added image labels to improve usability, and restructured the placement of the quiz at the end to improve flow. This implementation of formative evaluation helped ensure that the course design was not only theoretically sound but also accessible and intuitive for real learners.
My prior experience in user experience and visual communication contributed to how I approached this evaluation. I knew that content clarity and visual flow were critical, especially for learners new to photography or online learning. This feedback process helped validate many of the design choices I had made while revealing small usability issues that might have been overlooked otherwise. In past projects, I often relied more heavily on peer reviews, but this one-to-one approach gave me a closer look at how a learner might experience the course independently.
This artifact works well for this challenge because it shows how feedback, even when gathered informally, can lead to meaningful design adjustments. I learned the importance of collecting feedback from a real learner early in the process and testing usability just as much as instructional accuracy. In future projects, I plan to formalize this process by using pre- and post-surveys or field trials with small learner groups to further validate instructional effectiveness and learner engagement.
Challenge 2: Implement summative evaluation plans
Criteria for successful completion of this challenge: Evidence of implementing a summative evaluation plan to evaluate the effectiveness of the instruction and decide whether to continue to use instruction. Evidence must show an evaluation plan (e.g., Kirkpatrick’s Four Levels of evaluation). Reflection must address: If the implementation of the summative evaluation met your expectations. What were the results of the summative evaluation (did you continue with program/instruction, did you cancel it, did you modify it)?
Examples: The following assignments are applicable if implemented: Evaluation Plan (EDCI 528), Evaluation Plan (EDCI 577), artifacts showing implementation of an evaluation plan (design, performance, workplace, educational, other).
Reflection
For the challenge of implementing summative evaluation plans, I developed a detailed evaluation plan for the "Generative AI for Educators" course at the University of Texas at Austin. This plan employed Kirkpatrick’s Four Levels of Evaluation, tailored to assess the effectiveness of the instructional interventions applied throughout the course.
This artifact exemplifies my competency in creating structured evaluation frameworks that align with educational objectives and learning outcomes. It demonstrates a strategic approach to evaluation, focusing on not just immediate learning outcomes but also on long-term behavioral changes and results, which are crucial for assessing the true impact of educational interventions.
The formulation of this summative evaluation plan was significantly informed by my previous academic pursuits and practical experiences in instructional design and evaluation. Drawing on my knowledge from courses like EDCI 577, where I learned about various evaluation models, this plan reflects a synthesis of theoretical understanding and practical application, ensuring a rigorous and effective evaluation process.
Through this exercise, I enhanced my ability to critically apply theoretical models in real-world settings, underscoring the importance of evaluation in educational design. This project reinforced the value of thoughtful, ongoing assessment and taught me the importance of adapting evaluation strategies to meet specific educational needs and contexts. Moving forward, I intend to further develop my skills in dynamic evaluation, focusing on real-time feedback mechanisms to adapt and improve instructional materials continuously.
Artifact: Summative Evaluation Plan for “Generative AI for Educators” - EDCI 577
This document details the summative evaluation plan developed for assessing the long-term effectiveness of the "Generative AI for Educators" course, using Kirkpatrick’s Four Levels of Evaluation to ensure comprehensive assessment of educational outcomes.