BlobFisk.com

Heatmap of the Channel4.com homepage (click to enlarge)

Many people have been asking me to share more of the results of our user testing on channel4.com. I posted my initial takeaways from the testing and also posted about the methodology we used of mixing eye tracking with qualitative user testing techniques.

Methodology

We captured the initial 30 seconds of eye movements (using a Tobii Eye Tracker) to get an insight into the initial fixation points on the page and what elements users looked at first. We then carried out traditional qualitative task based testing on that page, still tracking eye movements.

The initial 30 seconds of eye tracking information was aggregated into traditional heatmaps for the pages we tested. The note takers in the observation room noted the eye movements during the task based testing.

Results

In quantitative eye tracking testing the assumption is that fixation indicated attention. In general this is true, but not always. By blending eye tracking and qualitative testing we were able to determine that some areas receive fixation, but were not consciously noticed by users.

Gazeplot of the Channel4.com homepage (click to enlarge)

When asked to complete a task users had to seek the area of the page that had received fixation. This was incredibly useful for us, as it indicated that some areas needed to be reworked slightly visually to make them more noticeable.

The results of eye tracking studies are invaluable when used as a quantitative testing technique – however, since we were testing a live site carrying out quantitative studies would have been too late. We had carried out extensive testing during the design and development phase, so we were quite confident that our designs were ok! However, some of this testing was on static images, rather than on a working version. So, testing on the live site was necessary to hone the usability and some of the interactions of the site.

By mixing eye tracking with qualitative testing we were able to get some excellent results. The eye tracking data was a very powerful accompanyment to the task results, a further insight into how users interact with pages. The heatmaps and gazeplots also provide a very effective way of communicating those results. When I presented the initial results my slides contained only gazeplots, heatmaps and live trace videos.

Changes we’ve made

We’ve made some changes to the site on the back of the testing. The nice panel below the main image on the homepage has been modified to be more usable. We changed the language of the labels above the panels a little to make it clearer to users what the panels were. “Catch-Up Highlights” has now become “What’s Hot in Catch-Up”. There was some confusion as to what this meant, participants thought that may be an abridged version of a programme, showing only the best bits.

Previous Homepage media panel (click to enlarge)

Similarly “Tonight’s Highlights” is now “Tonight on Channel4″. This is to reinforce that the programme information being shown is for Channel 4 only. This is important as sometimes we advertise E4 or More4 programmes on the homepage, and some participants thought that they were on the porgramme page, rather than the landing page for Channel4 as a whole. To further strengthen this we have asked that the editors always put a “Welcome to Channel 4″ in the promo area (the text area in the lower left hand corner of the main image on the homepage) so that users are immediately told that they are on the Channel4 homepage, regardless of the image used. Sadly, this is regularly being ignored.

Heatmap of the TV Listings Page (click to enlarge)

Our TV listings page tested extremely well. Participants loved this page. It is very information rich, but also quite playful. Participants did not notice the zoom in and out buttons, and we debated changing the design to make these more prominent. In the end we decided to do nothing! Users were very happy with the level of detail we show by default, and I have always been a big fan of discoverable interfaces.

Playful interfaces

I like an element of playfulness in a UI – users can discover features that are not highlighted or advertised. This makes an interface more enjoyable to interact with and use. As long as the playfulness does not get in the way of the user getting at the information or completing whatever task the interface is designed to assist with, then making the interaction a little fun is a good thing!

Predictive Search Mechanism (click to enlarge)

Users really liked our new predictive search mechanism. I think that as well as being an incredible usability bump (less clicking, faster searching and faster browsing) adds another element of playfulness.

Users interacting with it for the first time like to play with it – using the arrow keys to change the show that is highlighted and adding more and less letters to see the change in the results being shown.

By making it easy to play with a feature, users will learn how to use if quicker and come to trust it more. Many users of distrustful of search mechanisms on websites. They have traditionally been very bad at giving users the data they need. By introducing our predictive search feature, users can see that the results that we provide are good and learn to trust our search.

Thanks to Jennifer for prompting me to write this post. Of all the user testing I’ve done with Channel4, this one was by far the most enjoyable. The mixture of the two techniques made it more enjoyable for the note takers and observers (more too look at) and the results we got by combining these testing techniques have been very powerful. Powerful in the analysis we did, but also very useful for presenting the results and influencing the changes that needed to be made.