Program Evaluation

Program Evaluation

Group 3

Ball State University

Group Members	Roles	Commented on
Alissa Howe	Find evaluator, compile group reflection, student response	Group 1 and Group 4
Eliseo Garcia	Table 4, student response
Emily Sheperd	Compile paper, student response	Group 4
Laura Weyand	Find evaluator, student response	Group 2 and 4

Professionals’ Evaluations

Please click on the names below to view the evaluations. The copy and paste link will be posted in the first comment of this post.

• Teresa Kelly Evaluation
• TJ Evaluation

Student’s Responses

Alissa Howe

Alissa agrees with TJ that the group work portion will be valuable in putting what they have learned into practice. It will give the adjuncts time to work on a real life “problem” and they will be able to ask questions of the trainers as they encounter them. TJ brought up the question of whether or not there would be enough time for the adjuncts to learn the information thoroughly enough to be able to be on their own with the information after the session. She recommended scheduling more sessions or having a person available to answer questions after the session. Alissa agrees that this would be a good addition to the program and that it would be beneficial for the adjuncts to have support after the session. It will also be beneficial to the trainers’ as it will enable them to evaluate the effectiveness of the training and provide support to those who need it even after the session, or sessions, are over.

        Alissa also agrees with Teresa Kelly that having a program based on theory is important. Also, that prior experience is used, as one of Knowles’s (1968) assumption states. Alissa also agrees that it is important to be able to evaluate whether or not the adjuncts have learned the information and are able to use it. It could also be helpful for them to evaluate each other’s plans so that they can learn from each other and collaborate. Alissa would agree that this can be added to the program – a portion where they look over and evaluate one another’s lesson plans.

Eliseo Garcia

Both reviews provide valid points and recommendations that Eliseo Garcia agrees with based on his personal experience.  Similar to Teresa Kelly views, Eliseo agrees that it is a good idea that we are assessing each participant's prior knowledge at the beginning of the session  and making adjustments to the program to accommodate their needs.  Eliseo also agreed that working in groups and providing them with hand-on activities will provide an enjoyable experience for the participants.

However, both reviewers offered excellent questions or recommendations.  Such as, should we provide a follow up survey, should we provide a follow up session and should we offer the information to a contact person for further questions?  These are all valid question that Eliseo felt should be addressed.  

Eliseo would like to focus on a comment TJ made.  TJ stated she believes the time it would take participants to grasp PBL may be more that we are allotting for.  Eliseo would agree with her assessment based on his prior knowledge.  Eliseo participated in PBL training and it was a three day training session.  It is Eliseo’s belief that group 3 may not be able to complete a PBL training session in 90 minutes.  Based on other training sessions Eliseo has participated in, he would recommend providing participants with an introductory video that participants must watch prior to attending the training session.  Eliseo has personally attended training sessions where this was a requirement.  Therefore, the training group can cut out this time from the session and focus more on the application during the training session.

Emily Sheperd

Emily does agree with Ms. Kelly’s evaluation; the major item she critiqued was the long-term efficacy of the program. She suggested looking at creating a long-term review of the adjuncts implementation of the PBL model. One way to do this would be similar to how the program we reviewed, PROGRAM, did it through using completing workshops (like the one our group proposed) paired with reviews of their work in order to qualify for tenure. By pairing a long-term review and reflection with our workshops, it would cause our participants to be more invested in their learning, fitting closer with Knowles’s third assumption ready to learn.

Emily also agreed with Ms. TJ’s evaluation; her major critique was how would we ensure participants have enough time to learn the material and put it into practice. In the above paragraph, I describe the idea of using a long-term evaluation for tenure as a way to ensure the participants utilize the material. Pairing with that idea, the participants can have an “instructor and evaluator” they meet with regularly over their first few years while working towards tenure that provides them with help if they have questions about the PBL model and also prepares them for the evaluations they will do for tenure.

Laura Weyand

I appreciated and agreed with many of the suggestions given in the evaluations of our program design.  I thought both Ms. Kelly and Ms. TJ provided practical suggestions to improve the program and increase its effectiveness. It was evident that these suggestions were based on their own experiences. Ms. TJ’s suggested allowing more time for lesson plan work and providing a follow up session to allow more time for reflection and practice. She also recommended providing contact information to the participants for further questions. Ms. Kelly suggested the idea of evaluating the participants learning during workshop to see that the concepts were actually understood. She also was curious if we could somehow monitor that the participants were implementing the information gained in their courses after the workshop.

To improve our program design I would extend the lesson plan activity thus extending the length of the workshop slightly.  In groups participants will look at actual examples of PBL lesson plans first and discuss the components. Then they will apply these examples to rework their own lesson plans to include components of PBL. After the workshop participants will be given a reflection sheet. Participants will be encouraged to put this lesson plan into practice in one of their courses and utilize the reflection sheet to evaluate how it went in their own classroom.  I would want workshop instructors to follow up with participants later through email.

Group Reflection

Highlights

Upon reflection of our program, our group identified some highlights.  One of these highlights was getting feedback about the program from those currently working in the adult education field.  It offered different points of view from those of just the group members.  It allowed us to reflect on ways that we could improve the program that we had not originally considered.  Another highlight that was identified was that participants will leave the program with experience in a skill that they can immediately apply to their current practice.  In addition to leaving the program with some experience, participants will have the chance, during the program, to collaborate with peers and to learn from one another.  The final identified highlight was that participants will leave the program having seen Knowles’s (1968) assumptions for adult learners in practice in addition to leaving with a basic working knowledge of Problem-Based Learning.

Process

Our group followed the detailed group plan that we wrote to begin the semester.  We communicated via email, Skype, and Google Docs to share information, make plans, and review assignments. Some of the tips we recommend are to stay organized, and follow the group plan, and ask group members for clarification when confusion arises.  Facilitating consistent and effective communication was key to our group being able to work in a productive and cohesive manner.  Timely response to emails and inquiries helped the group to stay on track.

Table 4. Summary of Program Evaluation

Evaluator	Ideas for Improvement	Revisions/Responses
Teresa Kelly	Send participants a follow-up evaluation form.  Check on implementation and progress of PBL strategies.	Develop a long-term evaluation for tenure as a way to ensure the participants utilize the material.
TJ	Provide follow up sessions between instructor and participants. Provide participants the contact information to presenter for follow up questions.	Provide evaluators the email address and telephone number of the instructor for follow-up questions. Have instructor and evaluator meet regularly over the first few years while working towards tenure that provides them with help if they have questions about the PBL model.