SirsiDynix has partnered with INFOhio to develop SchoolRooms™, an online portal to resources selected by teacher-librarian teams which meet national and state academic content standards. Schoolrooms also features a federated search capability which simultaneously searches library catalogs, electronic databases, Web sites selected by the teacher-librarian teams, and the Web using a search engine.

In January and February 2006, a team of Kent State University researchers conducted usability testing in the Shaker Heights (OH) Schools using the nine rooms which have been developed for Earth Science, U.S. History, and Parents. Elementary, middle, and high school teachers in grades ranging from 2^nd to 12^th agreed to create an assignment that required students to use SchoolRooms and to have it completed under observation by both SirsiDynix staff and the Kent State researchers.

Overall, the Kent State team conducted 135 student observations. Of these, 97 students were video-recorded using Morae® usability software, while 38 students were eye-tracked using the Tobii® 1750 eye-tracking monitor. This monitor tracks the position of the subjects’ eyes on the screen using an infrared camera built into the monitor.  

These observations can be used to identify whether students were concentrating or confused, were reading or quickly browsing, or were engaged in or ignoring parts of the Web pages. “Hot-spot” images and screen capture video can be used to display the eye movements of the students over the Web pages to indicate where the student was looking while performing a given task.

An eye-tracking session shows eye fixation (blue dots) and eye movements (blue lines) for each Web page that is viewed. Each numbered circle shows the point that the student fixated on, and the larger the circle, the longer the time spent viewing that spot. The sample screen shows that the user started in the upper left-hand corner, quickly looked at other parts of the screen, and concentrated on the picture in the center.
 

After a group of comparable students (e.g., fifth graders) have viewed the same page, the individual session maps are turned into heat maps. Heat maps use colors to show group visual activity. Reds and other “warmer” colors show what most users looked at, while greens and other “cooler” colors indicate less viewing. Areas with no color were not viewed by any of the group. Eye-tracking studies have generally found a “golden triangle” where people tend to look (outlined in blue) and that most people do not scroll down “below the fold.”  

A usability report (Report on the Usability and Effectiveness of SirsiDynix SchoolRooms for K-12 Students, May 2006) has been provided to SirsiDynix, and they have already begun to modify and enhance certain aspects of the product to make it even more useful. 

However, it is already clear that the students liked SchoolRooms. In surveys, 89 percent said that they would use SchoolRooms again; 55 percent said they felt like they got more done using SchoolRooms; 86 percent indicated that the screens were easy to read, 67 percent reported that SchoolRooms looked “really good,” and 38 percent of the students even thought it was “fun” to use. Perhaps the best indicator of the success of SchoolRooms is that 46 percent said they preferred SchoolRooms over search engines like Google and Yahoo! and only 26 percent preferred those search engines.

Based on the success of the usability study, INFOhio and SirsiDynix have committed to undertaking a project this summer to complete the initial development of over 70 rooms for SchoolRooms by next fall. 

The researchers now will analyze the data collected to look broadly at the information-seeking process of students as they interact with electronic information sources. This will require substantial time and effort since each of the 38 eye-tracking sessions lasted 20-30 minutes and typically required more than 100MB to record the user camera and more than 2 GB for the actual eye tracking. The researchers also have approximately 45 hours of Morae® sessions to be coded and analyzed.

The relationship of cognitive load to performance, success rates, time on task, and errors will be studied. Cognitive load can easily be ascertained by changes in pupil size measured by the eye tracker. Increases in pupil size have been correlated with harder mental labor during tasks. Finally, the differences between grade levels, sex, academic background, and other variables will be studied. These can be measured by more closely comparing recorded performance on tasks and classroom assignments.

If the eyes are the mirror of the soul (old Yiddish proverb), they can also mirror levels of cognition and comprehension in the Age of the Internet. 

Dr. Greg Byerly, Dr. Jason Holmes, Dr. David Robins, Dr. Yin Zang, and Dr. Athena Salaba
Kent State University School of Library and Information Science