What’s the flavor of your training?

This forum is a melting pot for those that are passionate about instructional design. Oftentimes learning can be bland, with the right spices you can cook up a more flavorful learning experience. We hope that by sharing our experiences, we are stirring the pot so that we can swap recipes for developing training solutions. We are providing basic ingredients from which you can select to make your own great learning recipe.



Tuesday, March 25, 2008

To Knead or Not to Knead? That is the Assessment.

We've posted a few of our recipes that focus on starting a project, and now we want to pose a question to you. One core component that is often overlooked when starting a project is the Needs Assessment.

As we begin to write our blog on finding the adequate banquet facility (by performing the Needs Assessment), we want to get a take on what people are doing or have done to ensure that the party they have planned will meet party goers expectations, as well as meet your party goals.

We would like to know if you have performed a Needs Assessment. Was it a small scale assessment or an elaborate one? In other words, what techniques did you use to gather information (e.g., surveys, focus groups, online polls, etc.)? What conclusions did you draw from your collection of information? Were all project stakeholders on board? Or did you do your own research to ensure the goals of the project were on task? If you have not performed a Needs Assessment, please elaborate on the circumstances and/or reasons why one was not conducted.

If you have ideas to share, good advice to lend, or want to share a bad experience so that we can all learn from it, please respond to this blog post. In responding please provide the following information:

  • How many were you cooking for (target audience)?
    • How elaborate was your party?
  • Was your party a buffet style or sit down service (delivery method)?
  • What lessons did you learn?
  • How did the Needs Assessment help you to plan a successful party? Or how did the lack of a Needs Assessment affect your party planning and outcome?
    • Was your facility too small for the party?
    • Was the party poorly planned?

We look forward to reviewing your recipes!

3 comments:

Sara Boyce said...

As an Instructional Technology student at Bloomsburg University, I had the opportunity to conduct an on-site needs assessment at a training confernece held by the organization for whom my team was creating training. We conducted surveys, interviews and small group discussions to collect the essential data needed for our project.

How many were you cooking for (target audience)?

Our audience consisted of 56 individuals who were volunteers for a non-profit organization.

How elaborate was your party?

At this training conference there were probably over 200 individuals in attendance, however, we had a chance to personally interview about 10-12 individuals who were part of our target audience.

Was your party a buffet style or sit down service (delivery method)?

This need assessment experience was both a buffet style and a sit down service. We had a chance to mingle and informally speak to many individuals but also had one-on-one formal interview sessions. We also conducted small group discussions. We also sent out surveys to our target audience members to collect additional data not collected at the conference.

What lessons did you learn?

One huge lesson that our team learned was that electronic surveys and survey software, such as Zoomerang and Wufu, are much better tools for gathering survey data than emailing out a Word document. For our needs assessment we typed up a 3 page survey in a Word document, sent it to our target audience expecting to get many complete surveys returned to us, and unfortunately, we received zero. We were very disappointed. For our evaluations at the end of the process, we used Zoomerang surveys to gather necessary data. This worked perfectly. We learned a very valuable lesson about surveys and how to collect data!

How did the Needs Assessment help you to plan a successful party?

Our Needs Assessment helped tremendously for this project. We discovered from our Needs Assessment that we need to develop our training course without using Flash elements. This was important to know. If we wouldn't have conducted the assessment, we probably would have used Flash to create the course, and most of our audience would not been able to view the training.

Was your facility too small for the party?

It would have been helpful to have more input from more individuals that were part of our target audience, especially regarding limits on technology. We ran into a few technical problems towards the end of development and implementation. Overall, the facility was adequate and the data collected during the needs assessment was extremely useful.

Was the party poorly planned?

The party couldn't have been planned any better. We had an amazing opportunity to speak with many of our target audience members who were all in one place on the same day. We worked with our client at a distance and our target audience was spread out over the entire state of Pennsylvania, so having the ability to conduct an on-site needs assessment was outstanding. Again, the only poorly planned part of the party were the surveys we sent out after the conference. We have learned from this mistake and next time we will definitely plan our surveys much better.

Katie Y. said...

As a student in the Instructional Technology department at Bloomsburg University, I conducted a needs analysis for a cyber charter school. We used a number of different assessment tools for the needs analysis, but I would like to focus on one experience, because it didn’t work very well. As a result, it taught me some important lessons about conducting a needs analysis.

How many were you cooking for (target audience)?

There were about 150 students in the school. They were enrolled in grades 9-12.

How elaborate was your party?

We attempted to gather information from as many of those 150 students as we could, but as you will see, our initial attempt did not work well at all.

Was your party a buffet style or sit down service (delivery method)?

The needs assessment was buffet style. Since the students were geographically spread out and at no one time could we gather them together for interviews, we had to rely on a quick survey to send out. Also, since the students were underage and school students, we would have needed to either have a teacher present or to receive clearances in order to directly speak with a student.

What lessons did you learn?

Like Sara said in her comment, our group quickly learned not to use a Word document survey as a way to conduct a needs analysis. We sent out surveys to students and didn’t get a single response! It was very disheartening. We then reconfigured our survey into a SurveyMonkey survey. We rewrote many questions from essay style into multiple choice or true/false—anything that the students could just click on instead of needing to write. Also, instead of sending out the survey in an email, we had the school post a link on their website. The combination of those two things caused us to get a much better response rate.

So, I guess one of the lessons I learned is that, even though you are conducting a needs analysis to learn about your audience, you already need to know a little about them in order to carry out your analysis in a way that will give you a good response. Make sure you approach your audience on terms that are acceptable to them. Also, take into account any factors that may limit your ability to carry out an analysis (such as needing clearances to interact with school students), since these factors will also influence how you can conduct small group sessions and other evaluations.

How did the Needs Assessment help you to plan a successful party?

We learned a tremendous amount of information from the needs assessment. Part of it related to the questions we asked: what information needed to be taught, what had worked well in the past, how the students preferred to learn information, etc. But part of what we learned could not be elicited directly from any questions and came about simply from the process of interacting with the students, even if it was indirectly through an online survey. We learned what types of questions students would respond to (online, multiple choice) and what types of questions to avoid (essay questions). We learned that students needed explicit directions on how to complete surveys, including how many pages there are in the survey (otherwise students might answer the first 4 questions and skip out without answering the rest). This was important information to incorporate into the training.

Was your facility too small for the party?

Our training turned out pretty well, I think. We got good feedback from our small group session, which allowed us to make some revisions, but they weren’t major revisions. The same was true of our formative evaluation. The needs assessment allowed to tailor the training into small, concrete chunks that seemed to work well for the students.

Was the party poorly planned?

Once we got over the initial hurdle of figuring out a way of connecting to our audience, the design process went much more smoothly. I think that without the needs analysis, we would have designed a tutorial that was much wordier and that would have gone over poorly with the students. We definitely would have designed a much wordier evaluation process that would have skewed the results.

Amanda Bradley said...

I, too, am an Instructional Technology student at Bloomsburg University. For my Advanced Insructional Design class, my team conducted a Needs Analysis for the Special Olympics of PA (SOPA). I had many of the same experiences that both Sara and Katie did.

How many were you cooking for (target audience)?

Our audience consisted of 53 people who volunteer their time for SOPA.

How elaborate was your party?

We attempted to gather information from all 53 of the learners, but only got responses from about 15 of them.

Was your party a buffet style or sit down service (delivery method)?

Our surveys were delivered buffet style. We sent the surveys out electronicly in a Word document attachment and then, after getting a poor response, decided to use SurveyMonkey.

What lessons did you learn?

Like the other two comments posted, we found that no one reponded to an e-mail attachment. People were more likely to follow the link to SurveyMonkey and complete it there. Though the results were a little disheartening, I think we got an even better picture of who are leanrners were. We, as Instructional Designers, really want to cater to our audience to have the training be effective.

How did the Needs Assessment help you to plan a successful party? Or how did the lack of a Needs Assessment affect your party planning and outcome?

Our Needs Assessment was very helpful. We learned that the learners were both busy and not particularly computer savy. We designed our training to be simple, thorough and easy to navigate.

Was your facility too small for the party?

We would have liked more responses from the volunteers. Many of our survey questions were based around the learners comfort level using a computer. This information is essential in building effective online training.

Was the party poorly planned?

The party was planned well, but we did run into a problem or two. We learned from these problems though, so they weren't in vain. We can assume since no one responded to the Word document attachment that using this method to gather information isn't as effective as using SurveyMonkey.