Using online surveys to evaluate the success of an education project

[Justin Sharpe]

I think that it is important from a child/youth perspective that they are seen as contributors to disaster prevention rather than passive observers who are 'taught what to do'. I have been running a scheme of work with more than 300 students and 6 weeks after they completed their unit on 'dangerous geography' they have been responding to an online survey where over 178 completed surveys have been collected that indicate how successful they educational resources and teaching was at engaging students and their parents in disaster preparedness and planning. So far 175 respondents have an emergency kit at home made by students, whereas only 38% of parents have talked to their children about making a family emergency kit! I'm now going to liaise with the local authority to get simple information out to these families about how to prepare...but this all started with well thought out and tested educational materials put together but with children evaluating the materials while providing their teacher (me) with an idea if what they had learned and retained as well as what they engaged with most (making their own films) as well as what was the most useful knowledge (protective behaviour and making up emergency kits for their family, thereby feeling useful members if their community!

This has worked well for me and I have will be using this in the future. All too often projects are delivered but honest non biased evaluation is harder to obtain. Is this a possible answer?

[Roly Hermans]

We used an online survey tool (Survey Monkey) when we were consulting with teachers on what changes should be made to "What's the Plan Stan" (the New Zealand DRR resource for primary/intermediate schools) during its revision this year.

We sent the survey to all teachers who had participated in teacher workshops at the time that WTPS was originally launched a couple of years ago. The information received was then presented to a couple of small working groups of teachers and emergency management officials to begin the WTPS revision process.

Overall, my feeling is that online surveys are a good tool, but need to be supported by a strategy to motivate sufficient responses to make the collated data valid. We offered a prize to entice teachers to respond. We also made sure there were not too many questions. Another thing to think about is to ensure the questions can be answered on the spot (having to go and find something they need before they can answer might put potential responders off).

[Justin Sharpe]

How many surveys did you get back? My surveys were filled in by the end users - i.e. the students! I think too many education programmes don't get feedback from the students and plough on regardless which means that resources don't get changed or adapted and enthusiasm and understanding can wane, or worse still students switch off from the subject.

[Roly Hermans]

Our return rate, from memory was around 20/25%.

In the original design of the What's the Plan Stan resource, extensive student views and feedback were sought and incorporated. This current revision, however, is intended mainly to align the existing resource to the new national curriculum that comes into play here in New Zealand in 2010. So the objective of the online survey was to find out from teachers how the resource could be improved in that regard. One of the outcomes of the survey, for instance, was a stronger emphasis on inquiry learning, so that is one of the improvements being added to the revised What's the Plan Stan resource.

[Yolanda (Yoli) McGlinchey]

I also use Survey Monkey to get feedback from Emergency Operations Center staff on how the performance was during any incident. We've had three fires in the last year and need as much input as we can get from staff.

Also, in teaching disaster preparedness and management I always want input from the students/participants so that I can adjust my teachings as needed. It is always good to get as much information as possible to create curriculum that is thought provoking and timely.