Week 3

• Chapters in Section III discuss evaluation in instructional design and provide you with three evaluation models: CIPP, Five-Domain, and Kirkpatrick’s Training Evaluation model. Search for at least two other models used for evaluation and summarize these models. Describe how you would use them to evaluate your instruction.

According to the textbook, Evaluation is the process of determining that merit, worth, and value of things, and evaluations are the products of that process. 

Brinkerhoff's Success Case Method is focused on finding what about a training for other organizational intervention worked. Questions that can be answered in the SCM are; what information, strategies, skills, worked when, how, with whom and where, what successful results were achieved, what good did it do, what factors helped produce the successful result, and what additional suggestions are there for improvement.  The focus of Brinkerhoff's Success Case Method is on locating and understanding program success so that success can become more widespread.The Success Case Method follows five steps: 

1) Focus and plan the study

2) Construct a visual impact model - listing the outcomes that are hoped for or are expected to result from this study

3) Conduct a survey research study to identify the best and worst cases. 

4) Conduct in-depth interviews with multiple success cases. The purpose of this step is to gain detailed information to document.

5) Write up and communicate the findings of the evaluation. 

This evaluation theory reminded me of several things. First of all, every class I have ever taken in college had a course evaluation. Most of the questions are to gather information about what is working in the class. Second of all, it reminded me of going to retailers and restaurants now days and getting a receipt with a survey code on it wanting your feedback. They are asking for what is working and what is not. I always delight in taking those surveys to report both my happiness or my displeasure with the organization. I would use this evaluation method to make a Survey Monkey or a fun feedback form for my students after doing projects. I think we are too often focused on the grade students make and not the student's opinion of the project or what they actually gained from it. I think this again gives the students freedom to express themselves and gives them choice in what they want to do for projects to come. The more the students feel they are involved in their learning process the more they value it and the better the work products will be. 

Patton's Utilization-Focused Evaluation (U-FE)

The U-FE is done for and with specific intended users for specific intended uses. Patton stressed the importance that evaluation findings are used, rather than filed away. The evaluator's goal is to stay with the program from beginning to end and make sure that the evaluation results continue to be used after the evaluator leaves. The thought behind U-FE is that an evaluation done but not used is of little use to the business/organization. The U-FE has 9 major steps

1)Conduct a Readiness assessment to see if the people involved are ready and able to commit to the U-FE

2) Identify the primary intend uses and develop working relationships with them.

3) Conduct a situational analysis - identify what problems or supports you will encounter.

4) Identify the primary intended users for when the evaluator is finished and has left

5) Focus the evaluation - what is the high priority for the organization?

6) Design the evaluation so that it is feasible and will produce results that can have action taken 

7)Collect, analyze, and interpret the evaluation data 

8) Continually facilitate evaluation use - show how to use the evaluation during the evaluation period and not just after.

9) Conduct a metaevaluation (an evaluation of the evaluation) The evaluation is only successful if the findings are used effectively.

In my class instruction I would use the U-FE method to "take the class' temperature" during a project. How are we doing? What is working? What can we change to get better results? I think that doing this will clear up many problems seen at the end of projects or any misunderstandings students have. It again gives the students voice and the ability to take control of their learning. I think the results of a project with this kind of evaluation would be much better than one where it is assigned and then an evaluation is done after. I think facilitating the evaluation during the project would be more beneficial to me as a teacher, and as a student. 

• Reflect on what other questions that instructional design evaluation should address besides whether the instructional design leads to comparable amounts of learning and learner satisfaction as traditional methods. 

I think that instructional design evaluation could address other ideas students have that may help their satisfaction as a learner. An open response part of an evaluation that may ask what other ways would you like to learn this subject matter or what other ideas do you have for projects about this topic? I think that teachers may be able to tap into students' interests through that type of question.  

*Should return on investment and management of resources also be considered in evaluations of instructional programs? What other measures should/could be considered? 

I think if you spend a lot of time on a project that does not help your students become a master of the subject or help them in other grade levels or, dare I say it, a state test, then you are not getting a return on investment. I helped a new teacher this year who insisted on spending a lot of time making foldables, but the students were not getting the information that they needed to get to pass the grade level state exam and so it was wasted effort. I tried to explain that to her, but she did not take kindly to my suggestions. I just feel like you have to have bang for your buck. If you are spinning wheels on projects that are not helping your students grow, then you are not benefiting them in the long run.                                                                     

• Section IV focuses on human performance, performance support systems, knowledge management systems, and the concept of informal learning. Not all problems in learning and/or performance require an instructional one. Many times a non-instructional approach is a more appropriate solution. Identify a performance problem in your area of work and identify non-instructional solutions that may help solve the problem. Would better performance support systems, knowledge management systems, or opportunities for informal learning solve the problem?

Performance Improvement- I think the biggest problem I have right now in my area of work with special education students is shifting the responsibility to the learner. I work with learning disabled students. They are very used to and comfortable with being told what to do and how to do it than thinking for themselves. This has been a big problem since we switched from TAKS to STAAR. With TAKS there were simple ways to solve questions. If it was a summary question, you looked for the answer choice with the BME - beginning, middle, and end. When STAAR came around they asked questions with summary of certain paragraphs and using paraphrased language, this threw off our students who were looking for the quick TAKS fix. In elementary school we are very used to helping students in need. Special ed students NEED our help! However, they will never spread their wings if they are not allowed to have the responsibility of their learning shifted to them.