Section 3: Evaluating, Implementing and Managing Instructional Programs and Projects

Evaluation in Instructional Design

CIRO Model

The CIRO Model is an evaluation approach that is described in the work of P. Warr, M. Bird and N. Rackham. It is based on four measurement categories but differs from the Kirkpatrick Model in several respects. The four categories are:

Context evaluation - measures the context within which a program takes place. It takes into account the way performance needs were identified, learning objectives were established, and the way the objectives link to and support the necessary competencies.

Input evaluation - measures a number of inputs and focuses on the resources needed to meet performance needs (e.g. staff, facilities, equipment, budget)

Reaction evaluation - measures how the participants reacted to the program. Draws on the participants' opinions of the program and how it might be improved.

Outcome evaluation - measures the training and development outcomes against the objectives.

By using this model I would first take into consideration the context within which the instruction takes place (in my case it would be a library setting). I would ask myself if the needs had been identified, if the objectives had been clearly stated, and whether the objectives supported the competencies. Next, I would need to determine if the resources were adequate to meet the needs. Were the staff, facilities, equipment, budget adequate to get the job done? Then I would reflect upon how the partipants had responded to the program. What did they think about the instruction? Were they actively engaged? What would they change? Finally, I would measure outcome of the instruction? Were the objectives effectively taught? Was there growth? Did the outcome meet the objectives?

Connoisseurship Model

Eisner's Connoisseurship Model involves a connoisseur or expert in a field of study estimating the worth of a new innovation. According to Eisner (1998), connoisseurship is the "art of appreciation" and educational connoisseurs are those who have a distintive awareness of qualitites in different settings. He thinks connoisseurs evaluate the issues by looking at their qualities and illuminate a situation so that it can be seen or appreciated by others as well. The connoisseurship model has two major implications: holistic approach to the anlysis and interpretation of data and multiple perspectives in the evaluative tasks.

I would employ this method to evaluate my instruction by taking into account the entire process of instruction, the reaction of learners, data collection, and outcomes of the instruction in order to determine qualities to be highlighted. I would then need to be able to reflect on action, engage with feelings, and be able to make informed judgements.

Technological Innovation

Rather than going to the expense of buying Smartboards, our elementary campus recently purchased Mimio Interactive devices for all teachers. The compact Mimio bar attaches to any standard whiteboard converting it into an interactive teaching medium that can manipulate anything on a computer screen right from the whiteboard. I was excited to see that most of our teachers, especially the younger digital natives, embraced this new technical innovation with enthusiasm and curiosity. There were those teachers, however, that were more hesitant to try anything new. They felt overwhelmed by all of their responsibilities and by their fear of new technology. After they had been trained on this new technology and after they realized it was relatively easy to use (complexity) most of the teachers on our campus were using the Mimio on a daily basis. The relative advantage of this new technology is that it offers an innovative way for teachers to create engaging, student-centered, interactive lessons to encourage active learning. Mimio technology is compatible with the needs of teachers and learners. It offers teachers ready-to-use lesson content, enlivens lessons with audio and video, records lessons to be reused whenever needed, or enables them to import existing lessons from Powerpoint or Adobe Acrobat. Mimio offers a range of training resources to fit the needs of users. It is easy to use right out of the box but they offer a flexible range of training programs if help is needed. As far as I know there is no free trial. The way I handled trialability on our campus was to purchase one and allow teachers the opportunity to come for onsight training and "try it out". Observable benefits of this new technology on our campus have been rapid adoption, widespread use, and active envolvement from learners.

Situational Leadership

As educators, we all understand the term "scarcity" when it comes to resources. With budgets being cut, staff being reduced and demands on our time stretched to the limit, these chapters hit home for me. Due to a reduction in staff, I no longer have a full-time library aide; however, my job requirements are increasing, and I often feel overwhemed by the responsibilities of my job. Personally, it would terrify me to find that I was responsible for professional development for technology use. When I stop and think about it, though, I am already responsible for some of it. I am responsible for the purchase of equipment which often includes training. I am proud to say that we have significantly increased the amount of technology being implemented in our classrooms since I assumed my position as librarian/media specialist almost 5 years ago. The phases involved in Situational Leadership are employed as follows:

Phase 1: When we first acquire new technology, the program must be supervised closely. Users must be trained with detailed information on how to use the equipment or program.

Phase 2: As the users gain confidence, my role as leader is to continue to explain and clarify use of the technology by answering questions and monitoring in order to help the learning process flow smoothly.

Phase 3: Now is the time to make sure that positive results are being produced and users are rewarded for their effort. These rewards might be intrinsic (feeling of pride at their accomplishments) or extrinsic (opportunity to share what they've accomplished with cohorts). The objective here is to make certain users feel informed and successful in using the new technology.

Phase 4: Once the learning has taken place, my role is to monitor the program and give users the opportunity to work on their own to produce a product.