Table 1 Questions used to evaluate CIPP components and data collection methods*
CIPP components | Evaluation questions | Data collection method |
|---|---|---|
Context | What is necessary or useful: in other words, what are the educational needs? What are the impediments to meeting necessary or useful needs? What pertinent expertise, services, or other assets are available? What relevant opportunities (e.g., funding opportunities, administrative support) exist? | - Document review - Literature review - Demographic data analysis - Surveys - Records analysis (e.g. test results, learner performance data) - Focus groups - Advisory group |
Input | What are the potential approaches to meeting the identified educational need? How feasible is each of the identified approaches, given the specific educational context of the need? How cost-effective is each identified approach, given the specific educational context of the need? | - Literature review - Expert consultants - Inviting proposals from persons interested in addressing the identified needs - Pilot trials to assess available human and material resources to evaluate the work plan and strategy for relevance, feasibility, cost, and economy |
Process | How was the programme actually implemented, compared to the plan? Are/were programme activities on schedule? If not, why? Is/was the programme running on budget? If it is/was over or under the planned budget, why? Is/was the programme running efficiently? If not, why? What do/did participants and observers think about the quality of the process? | - Participant observers - Document review - Open-ended survey questions provided to the participants (learners, operators, instructor) - Periodic exchange of information with project leaders and staff to monitor and provide feedback on the process and record the actual process |
Product | What positive outcomes of the programme can be identified? What negative outcomes of the programme can be identified? Were the intended outcomes of the programme realised? Were there unintended outcomes, either positive or negative? What are the short-term implications of programme outcomes? What are the longer-term implications of programme outcomes? How effective was the program? How sustainable are the intended and positive programme outcomes? | - Stakeholders’ judgments of the project or programme (Evaluation from the education and training review committee) - Comparative studies of outcomes with those of similar projects or programmes (Including expert evaluation) - Assessment of achievement of programme objectives (Usefulness at work, academic achievement, etc.) - Surveys (Level of satisfaction) - Participant reports of project effects (Self-evaluation) - Comparing outcomes to assessed needs (Comparative Studies of outcome with assess needs) |