During the first portion of the semester, we focused on actually learning what usability testing is.  I confess I had no idea whatsoever what usability testing was before I began the class.  Through class lectures and reading Usability Testing and Research  by Carol Barnum, I learned that usability testing is a system evaluation of a production based on the observation of the interaction of everyday users with the product.  I learned that much prior planning was required in order to even begin the test.

This project was a huge project.  It would take the entire class the whole semester just to complete on time.  This would have been a complete communication fiasco if we had not split up the class into three groups:  the testing group, the writing and editing group, and the sponsor and participant management group.  These three groups were created in order to simplify the delegation of tasks and allow everyone in the class to specialize in a few aspect s of the project.  These groups helped the project very much.  Also, because we were usually collaborating with the same people everyday, we were able to get to know our group members very well, which increased our communication of information and made the class fun.  Furthermore, each group had a team leader, which led their group and coordinated with Dr. Barker and the other team leaders.

Fortunately, I was placed within the Testing Group of the IRB website usability test, the group I had applied for.  I wanted to be in this group because I wanted to actually conduct the tests in the usability lab and use the equipment and software that this process required.  Our first job as a team was to plan for the tests.  We had to come up with task objectives, which were designed to help us begin in the right direction by helping us see how we were going to evaluated the website in the future.  We created a task objectives sheet that covered the material in the website we wanted to focus on during our tests.  The first big job that we were apart of was the test plan.  This plan was a work in progress for a while because it was such a huge job.  Our team was responsible for creating the testing environment section, the users tasks, and the quantitative and qualitative data summary that would be used during the tests.

To learn more about our testing environment, we spent a day in the usability lag learning about the equipment and software.  We made sure we were able to use the cameras and work all of the recording equipment.  The quantitative and qualitative data we expected to be the actual observation of users interacting with the website and their responses about the website afterwards.  The user tasks section was much harder.  We had to use the user personas from the S&PM group to tailer a test for them that would evaluate the elements we wanted.

Then we created the documents that we would need for the tests, in the user tests.  When testing began we used our knowledge that we had learned from class lectures and our text book to operate the testing as smoothly as possible.  The planning really helped!

After the testing, the tedious process of editing the video clips began.  We had to find and edit the clips we wanted to use for our presentation to our clients.  Not only that, but we had to analyze our quantitative and qualitative data from the tests.

We gathered our entire data analysis and assembled it into the final report.  We presented our final report and it was amazing!

I have never been involved in such a big project as this, especially not in a professional setting.  This experience is a great introduction for the business world.

At the three of these tests, I coordinated the use of the documents to be used during the testing process and helped record data onto the observation form by watching the input TV screens and listening on the headphones. More importantly, I helped facilitate the tests by communicating with the users through the phones in order to answer questions and by prompting the users to move on to other questions if time expired.

The lasts test I served as the test facilitator. I greeted the user and administer the testing materials to him before the test while explaining what the test was attempting to evaluate on the website.

I arrived at the Usability Lab at 4:45 pm; the test was scheduled to begin at 5:00 pm.  Also there was Kyle, Chris, Matthew, and Mark from the sponsor and resource management group.  Dr. Barker could also be reached in the building.

As soon as I arrived, I had to use Dr. Barker’s computer to print the documents needed for the test, including: the faculty test, observation form, post test questionnaire, screening questionnaire, set up/break down checklist, facilitator checklist, and the facilitator script.

While that was going on, the others were preparing the Usability Lab.  The testing roles were

Mark, facilitator

Chris, time recorder

Kyle, supervisor

Matthew, audio recorder

Dustin, recorder

The test went quite well.  The user was easily able to perform the tasks, however, she was very accustomed to the website.  Chris recorded some good Pilot test results, and I noted where the questions needed to be improved.

The first day Dr. Still, the manager of the Usability Lab, showed us the entire lab. He explained the recording equipment, the audio equipment, the monitoring equipment, the cameras, the microphones, and the camtasia software on the computers.

Our second day in the lab we tested the equipment and recorded everything for the testing environment section of the test plan.

On October 11, 2007, the day of our first walk through, we tested our current test plan on Dan Stinnet. We found that the recording process went smoothly, but our test plan was inadequate. We needed a facilitator script, a facilitator checklist, an observation form, a set up/break down checklist, a pre-test questionnaire, a post-test questionnaire, and a screening test. We also discovered our current test questions needed to be revised. I volunteered to do the facilitator script and the facilitator checklist. Chris Flores volunteered to do the set up/break down checklist. Also the Bryony assigned members of her group to complete the screening questionnaire and the post test questionnaire.

On October 16 we went over the test plan as a whole class. Dr. Barker made helpful suggestions to improve our current test plan, including: changing the test questions to be more specific, making the testing environment section more general, making the task list chart. Kyle, the testing group manager, was unable to make this class, so I had to step in to take his place. I worked closely with Bryony, the sponsor and participant group manager, and Allison, the writing and editing manager, to organize the revision of the test plan.

This was an extremely hectic day. I had to work with two members of the editing and writing group to decide which questions we should use on the test plan. Also, we created the task chart for the questions we used. At the same time I had to delegate certain tasks to the other members of my group.

List of documents:

post test questionnaire

screening questionnaire

set up/break down checklist

facilitator checklist

facilitator script

method test design

Question and Concerns about Website

Email dialogue about sponsor questions

test objectives

Sponsor management email