Subject:  Evaluating Program Effectiveness notes |
Author:
Ken- CT
|
[
Next Thread |
Previous Thread |
Next Message |
Previous Message
]
Date Posted: 13:03:56 09/19/03 Fri
Evaluating Program effectiveness
3-4 years ago, Mass’s new strategic plan includes a new priority on evaluation of programs. ½ M grant for this project. Half went to endowment of the position.
4 initiatives:
Public survey- telephone
3 Different outside evaluations of program areas
Computer based registration/evaluation process
Develop a evaluation system for the whole org to use- offering fellowships to program staff to work on developing the system.
What we want from session:
• How effective is env ed across the country on social change? Or are we just hoping for an impact? What does impact mean? Who are your audiences? Where do you want to see your impact?
• Looking at types of programs- how can we use evaluation to prioritize our programming?
• How do we interpret the evaluations we get back?
• Evaluating vs. surveying?
• What is it you want to know?
• How to compare apples to apples- a systematic collection of data.
Can the Alliance pool together good questions and share?
• What is a successful program- reaching a large enough audience- what are you trying to accomplish?
What do want to question:
• A quality program- do they come back year after year?
o How do we measure the impact on the participants? We have limited resources for analysis?
o Look for a final product.
o Start with program objectives- be right up front. What do we want participants to walk away with increased knowledge or beliefs?
Ex- name 3 things living in the pond
o How do you measure increased appreciation of nature?
• Set benchmarks- what do we wan to accomplish? Spend time to plan this before jumping into designing activity.
o Specific outcomes
• Budgetary success? Balance program income with relation of mission
o High mission value, but low revenue can be balanced with more high income, less mission oriented programs
• Have we struck an emotional or spiritual impact?
• How do we record the qualitative impacts (emotional)?
Writing, tell a story
This is very time consuming too
Environmental ed is behind on overall evaluation of effectiveness, but if we begin to share our data to remain relevant.
Very important for funding. Need letters from kids, etc, but not always hard data from evaluations- want to see complete impact.
Other types of tools:
Anecdotal data to be meaningful needs to be kept. Set up a system (paper on wall, daily journal, formalize with daily/program reports) Look through all this for trends. What messages are repeating?
Survey to rate the value of the program, match to curriculum for field trips. Comment section for program weaknesses.
Underlying questions of this tool:
Effectiveness of presenter
Program content
Cost value
How do you get evaluations back-
Self addressed stamped envelope- got many back, more than without stamp but not really measured (20%-40% return)
Get them back before group leaves works well- not as thorough (85%-90%)
Some school groups trying to do pre and post tests
Post program naturalist self-evaluation
Gimmick to get returns- boat to Monhegan had a drawing for free trip if you fill out survey
Fill in post test and get a candy bar
Informal interviews are legitimate, taping is the best way, otherwise, writing down as quickly as possible. To formalize interview, write out questions before hand. (exit interview) Kasha uses this for pre-school programs, ask the parent how the child liked the class.
Can we send teachers a 2nd form with more details, to get more information to evaluate programming for that school you work with more often.
Needs assessment to teachers- program descriptions, “learning standards supported” Send out pre-program planner- survey to answer- “what part of program description do you find most meaningful” Return is 65% or so. Helps staff to set their objectives for that day, and meet halfway. Schedule program whenever they will call (atleast 2 weeks in advance) Include this in pre-packet
[
Next Thread |
Previous Thread |
Next Message |
Previous Message
]
| |
Feedback: 