Fall 2001 Volume 1 Issue 2
The Design of Complex Library Web Sites.
John Matylonek
Introduction
Library web design is changing. Library web sites are becoming increasingly complex. The plethora of links and services, confusion of print and electronic formats and the competition of general web resources all point to the need to focus the functionality of library web sites. This complex environment has arisen from various factors including:
· Competing goals of various library units in promotion, education and service and their relative prominence on the site.
· Novel ways of providing information and services such as electronic reserves, course specific library help pages and document delivery.
· Complex library research task flows due to the cross functionality and links between the online catalog, journal aggregator databases, electronic resources.
· The need to identify and present high quality free information resources on the web and distinguish those from library licensed material.
· The increasing expectations of users for interfaces to lead directly, without undue hunting, to the information or service they need.
As a result of these forces, many library web sites get congested with diluted objectives, competing task flows and duplication of services. It is no longer apparent how to do research with a tool that provides so many pathways for getting information. For instance, at Oregon State University, the Research Gateway page and Oasis Online Catalog page have many of the same links and overlapping functions. Which one of these should be used as the primary research tool? Is the reason they were created still valid? What design modifications should they undergo to make them better? How to measure the effectiveness and relationship of these pages and make design decisions is not obvious.
A systems design philosophy backed by usability measurements can answer some of these questions. It can, at least, determine the success of specific design objectives. This article will provide outline of a systems method of assessment and evaluation of your library web site. It will also provide specific usability techniques to measure the effect of continual redesign of complex library web sites.
A Systems Design Philosophy: Dealing with Complexity
One definition of a system is:
system: 1. Any organized assembly of resources and procedures united and regulated by interaction or interdependence to accomplish a set of specific functions
Systems are made up of individual components whose interacting elements allow an outcome not achievable by any one independent component. Library web sites have resources and prescribed procedures that will lead users to certain outcomes or functions (getting a couple articles for a brief library orientation assignment, systematic research from more general to very comprehensive resources, or filling out a Interlibrary loan form after searching local online catalogs). The challenge for library web designers is to see and define these interacting elements, define critical task paths and design interacting elements leading to a successful outcome. This challenge can be facilitated by following by developing an objective or task for each library web page on the site or element(s) on the web page and determining the relationship to other elements in the task flow. This means modeling what an expert searcher would do for a successful outcome and then creating design elements that emphasize these tasks. But, what outcomes should we choose?
system: 1. Any organized assembly of resources and procedures united and regulated by interaction or interdependence to accomplish a set of specific functions
Systems are made up of individual components whose interacting elements allow an outcome not achievable by any one independent component. Library web sites have resources and prescribed procedures that will lead users to certain outcomes or functions (getting a couple articles for a brief library orientation assignment, systematic research from more general to very comprehensive resources, or filling out a Interlibrary loan form after searching local online catalogs). The challenge for library web designers is to see and define these interacting elements, define critical task paths and design interacting elements leading to a successful outcome. This challenge can be facilitated by following by developing an objective or task for each library web page on the site or element(s) on the web page and determining the relationship to other elements in the task flow. This means modeling what an expert searcher would do for a successful outcome and then creating design elements that emphasize these tasks. But, what outcomes should we choose?
Guidelines for Prioritizing Tasks
Defining the priorities and outcomes of the Web site page is sometimes difficult. Library internal units often want equal representation in the design. Often a design will begin to clutter with competing services or information, distracting the attention away from real needs or diluting the main purpose of the page. One helpful guideline to follow here is that the relative prominence of a service category or task flow emphasis in the design should reflect the major objective or function of the page.
There are myriad of actions a library researcher must perform to successfully perform research or get information. A certain task flow emphasis could be based on priorities for instruction or promotion of certain collections, new services, on the major audience and their importance or its association with other tasks in the flow. The prominence of these tasks should follow a natural hierarchy of priority from the web site home page down to minor functions on the secondary pages. For instance, if using research tools is the major priority of the library web site, it should be emphasized on the home page. Deciding on what the most important function of your library web page design is essential because a library web site page cannot be the best at everything. In the end, the best design library web site designs have elements whose critical paths are prominent and have a clear one-to-one correlation with what they allow a user to do.
Measuring the Success of the Design Changes
A method of determining the success of critical paths is required once design changes are made. Success can only be measured by agreeing on objectives, establishment of specific criteria, measurement of those criteria, and then observation of the results. If a baseline for a particular performance measure is observed then the effect of a change in layout, logical flow, prominence, etc. should be reflected in a web performance factor. The following method can help library web site developers design your website in a systematic way. Following this method will allow your web design team to identify a purpose, track changes and document the evolution of your library website.
1. Establish Objectives: Reach an agreement between all parties involved about what you are trying to achieve (for example, to have users follow a critical path, or the promotion, placement and prominence of services on your web can all be reflected in web usage statistics).
2. Determine Criteria for Success: Define success specifically (for instance, the desired number of user sessions, increase in usage, decrease in usage, paths users take, time they spend, target population for your site).
3. Develop a Baseline: This step is essential because benchmarking is a comparative process. As library web sites are increasingly relied upon for basic services knowledge of the trend of usage in these services will be invaluable.
4. Select a Benchmark: Baselines provide an essential benchmark if you are comparing the web site's progress to itself. However, there may be more benchmarks that are more helpful. For instance, library home pages sometimes have a promotional function. Comparing web graphic designs with comparable institutions is appropriate if extensive prototyping is impractical.
5. Compare the results to objectives: Once the design change has been implemented examine the resulting differences in web statistics. Decide if the change succeeded based on Step 2.
6. Act on the results: If the change succeeded decide if further action is necessary. If the change failed, try something else. Record what succeeded in general design guidelines for the web site.
Useful web measurement methods and the information garnered
Usability studies measure how patrons actually use library web sites. General studies have resulted in many web style guidelines that can be incorporated into any web site design plan. The reader is strongly encouraged to use these guidelines in developing basic web site designs. However, complex library web sites are particular creatures and measuring library user behavior ought to allow library specific guidelines to develop. The list of methods outlined below are not a comprehensive list of usability study methods. However, the following outline of usability methods and web statistical procedures is especially efficient in the development of guidelines that will assist in dealing with complexity. Each section will tie the nature and results of the method to library web design procedure. Each result eventually leads to web site design questions, changes or decisions.
Refining the Purpose
Surveys
Refining the purpose, content and intended audience of the library web site is the major task of the first stage in the design. Confirm the audience to assess whether users you wish to serve are being served. A survey tests the overall purpose of the site by determining whether the information or services presented fulfill a real need. General surveys published and distributed in electronic and print format can determine users feelings about how an actual or potential library web should be designed. Major services can be identified as priority services. A general survey will also provide confirmation on what was done right and possible gaps in the service or design structure. In this way, needless backtracking can be avoided and initial prototypes will be more successful. (see Appendix A of Oregon State Universities' survey instrument)
Determining Critical Paths and Terminology
Cluster Analysis of Web Service Categories
Content usability is heightened by matching the terminology, hierarchy and layout so that hyperlinks “make sense," providing proper associations between hyperlinks and providing efficient routes with sufficient navigation cues. Cluster analyses of service categories can help identify critical paths and associations before actual design. This will increase the chances that the new web design elements serves the intended users and their needs.
Determining how to arrange categories, the terminology, hierarchy and layout so that hyperlinks “make sense” is one of the goals of cluster analysis. This design technique provides the associations between hyperlinks and possible navigation cues that users would use to enhance web navigation. Cluster analysis attempts to map the users mindset and classification of services to the creators of the web site. Card sorting, in which library users freely associate terminology on three by five cards performs the same function as cluster analysis. However, cluster analysis' statistical procedures make analysis much easier.
Cluster analysis asks users to categorize a list of possible service content of a web site. This content (See service categories in Appendix A) are then grouped by users and then statistically added to create the optimum organization. Sub-populations of the sample can be pulled from the study to determine differences in web terminology preferences among different user groups e.g. faculty, staff and students. (For a detailed description see Appendix A)
The ultimate goal of cluster analysis is to:
· Gather the demographics of the audience.
· Find which service describtions the user preferes to lump together
· Find which service describtions the user considers important, lacking, or irrelevant
· Reveal the prefered terminology of the user
The resulting graph, called a dendrogram, displays all those groups of services (here represented by a letter) that were naturally associated with one another during the survey. The seven groupings shown below may be considered for major headings on the web site. The vertical axis represents the decreasing frequency of association. Thus, the smaller the peak the more strongly it was associated by the target population.
Testing the Website
Evaluating Prototypes by Cognitive Walkthroughs
The library web can be put through its paces by submitting the design to an information retrieval challenge based on what the library web designers believe is important. Users are physically observed operating the web site in a cognitive walkthrough. A cognitive walk through is best performed like a focus group of one person. They are asked to find information derived from the priorities identified from the online surveys, content analyses and the educational agenda of the site. The subjects are encouraged to verbalize their search strategies and to explore the hyperlink options. The interviewer tries to be helpful while not giving the answer away. By encouraging free association and exploration, obstacles to navigation become conspicuous. These strategies are audio-recorded. The actual performance of the subject is recorded on a work sheet. These are analyzed later for any web design obstacles to navigation and retrieval. The most valuable information is the time, number of alternate paths and major search strategy used in finding the information. (See Appendix A for sample worksheets)
Determine the Success of Critical Paths and Services
Popular and Most Requested Pages
The number of user-sessions experienced by a library web site can show the most/least successful parts of the site. This can determine if the placement, prominence and promotion of services on the website are working, especially if current statistics are compared with a baseline. Whenever there is a change in placement, prominence or promotion then developers and maintainers of the site can determine effect.
Most Requested Pages |
|||||
|
|
Pages
|
Views
|
%age of Total Views
|
User Sessions
|
Average Time
|
|
1
|
Oregon State University Valley Library http://osu.orst.edu/dept/library
|
48,642
|
17.78%
|
31,670
|
00:00:42
|
|
2
|
Oregon State University Electronic Research Databases http://osu.orst.edu/dept/library/database.htm
|
16,931
|
6.19%
|
12,279
|
00:00:43
|
|
3
|
Oregon State University FirstSearch Access Page http://osu.orst.edu/dept/library/fsaccess.htm
|
13,095
|
4.78%
|
9,100
|
00:04:51
|
|
4
|
Oregon State University InfoTrac AccessPage http://osu.orst.edu/dept/library/iac.htm
|
9,804
|
3.58%
|
7,037
|
00:03:33
|
|
5
|
8,827
|
3.22%
|
7,431
|
00:04:53
|
|
|
|
Sub Total For the Page Views Above
|
145,600
|
53.24%
|
N/A
|
N/A
|
|
|
Total for Log File
|
273,450
|
100%
|
N/A
|
N/A
|
Adapted with Permission from NetIQ Corporation (all rights reserved)
Path Analysis
A path (or sometimes called a thread) shows us how groups of users navigated through a Web site by what links they choose to get an objective. Cluster analysis tries to determine these likely paths before the web site is created. Path analysis tests this in actual use.
Library users will choose many different paths to the same objective. Designs that do not specifically lead people through a critical path to an intended outcome cause users to depend on trail and error. This causes your website to be inefficient and ambiguous. If you have linked your library web from obvious gateway pages or have configured home page defaults on library workstations you may believe users are naturally disposed to using your site the way you intended. However, in libraries that have unrestricted web access, the plethora of hyperlink choices lead to many alternative paths, some of which may be considered more convenient than the library web site home pages.
Designing a library web site for users outside the library is even more demanding. These users cannot be helped by walking up to the reference desk and often will choose the most expedient and convenient network information sources - often not your library web site. Creating an environment where users are intuitively lead to the resources would be optimal. Competing goals for library web sites always hamper this goal.
Library web designers can study various web performance factors that lead to a picture of how your site is used.
1. Entrance - By studying what percentage of users enter a site on the homepage, compared to going directly to a page within a site, can help determine where to put crucial services and information.
2. Intermediate pages - If the path to crucial services requires specific paths then themost frequently used paths statistic can determine if the web site is leading users to the correct services.
3. Exit - What page tends to be the last page users are on before they leave the site? What may be found is that a large number of users leave a site on a specific page, and typically this is not on purpose. If one the purposes of the site is to lead users on a particular path to crucial services then this information can gauge how well the site is performing this goal.
4. Time - The amount of time users spend on particular pages may indicate how effectively they are using that section.
Changing the Website: Decisions, Actions, or Questions
From the survey results the major aims and functions of the library web site are validated. Significant omissions in services and information content is uncovered from the survey. At a more detailed level, the cluster study uncovers user preferences for web categories and associations. From these decisions layout, terminology and explanatory phrasing of the web site can be modified to reflect user preferences. The cluster analysis dendrogram graph shows that the computer services available to students (Peak 2 -B, K, N R, M, V, Y, X) are very loosely associated. In this case, these services could be made more apparent by explanatory phrasing or better definition and promotion.
Often, paths that lure users to dead-ends can be identified. Sometimes explanatory material meant to help can be shown to be a obstacle. Sometimes not enough explanation is offered. For instance, during our last study students simply did not recognize narrower subject headings linked under broader headings.
Preferences for certain placement of links can become apparent in the design. One user was biased to the links on the left side of the page. Another did not find the search engine window buried beyond the observable window. These and other phenomena will become apparent with more studies.
Evaluating an Ongoing Website
In the past, library gate counts, circulation statistics, etc. have provided a real gauge to library usage. Now, the library user community must include remote network users. The relative impact of remote access versus in-house service will become more important in library resource budgeting. Being able to justify resources toward design and of web-based services is important. Also, these users have special technical needs that often depend on the local Internet service provider and quirks of their local networks. Beyond the technical aspects of access, distance education initiatives have guidelines for service and content that may be assisted by knowledge of where and how users are accessing your library site. Once the web-site is made available to the public, ongoing aggregate statistics can indicate whether the site is achieving basic service goals. Web statistical software can aggregate web performance measures over time and are especially useful if baselines are observed at the onset of record-keeping. Some useful web performance measures include:
Domain and User Populations
Domain and User Populations
Identification of library web users and where and how they are accessing the library web no only identifies hidden populations but also justifies resources to support the service.
The following table shows the breakdown of campus ISP users of the library web site versus those users having private ISPs.
|
Most Active Organizations
|
|||||
|
|
Organizations
|
Hits
|
Percentage of Total Hits
|
User Sessions
|
|
|
1
|
Oregon State University
http://Orst.edu |
19,068
|
55.55%
|
4515
|
|
|
2
|
Rubis Network
http://rubis.net |
1,128
|
2.99%
|
45
|
|
|
3
|
America Online
http://aol.com |
852
|
2.25%
|
551
|
|
|
4
|
Peak.org
http://www.peak.org |
439
|
1.16%
|
116
|
|
|
|
Subtotal for Domains
|
21,487
|
26.95%
|
5227
|
|
|
|
Total for Log File
|
38,369
|
100%
|
9178
|
|
Adapted with Permission from NetIQ Corporation (all rights reserved)
Browser Analysis
The receiving end of web services is constantly evolving and the quality of the experience depends greatly on the user's situation. At the very least, a library web site ought compile with the American Disability Acts (ADA) standards for access. Lynx, a nongraphical browser, is still used by some people who access the Web with older computers (that cannot support the graphical browsers), and by users who are accessing the Web over slow modem connections. Others have lightning fast cable modems connected to the latest desktop computers. But even here, different browsers render html differently. The war over access methods is especially apparent in consumer electronic industry. Cell phones have rudimentary web capability and personal digital assistants will become more capable. Today's web is obviously a lot more graphical, interactive, and integrated with other information services then the web of three years ago. Decisions regarding the level of support for various access methods depends on knowing how users are accessing services, and what standards are commonplace.
Referrals from Search Engines and Metatags
Many referrals that a site receives are from search engines. Analyzing the web-site logs can tell us exactly what words users entered within the search engine request. Knowing this information can help us determine what terms should be used in a library web site's meta tags, which may help others find the site more easily from within a search engine. This quality of access is especially important from the local search engine serving the institution the library supports.
Keeping Track of Error 404 Messages and Bad Referrer Links
Whenever changes occur on a web-site, the bookmarks and links people have made are not changed. If we haven't created "re-directs" (pages or files that refer people to the proper web address) the changes this will show up as a "404 file not found" to the user. Log reports note the offending URL and referrer. Internally, maintenance of bad links can be performed. Externally, web-sites with obsolete or bad links can be informed and then modified so the user is connected to the proper address.
Conclusion
Libraries have always tried to remove obstacles to information access. A poorly designed website is certainly a barrier to the library user. A systematic method of designing, testing, and evaluating your website can insure quality and enhanced usability of the site. There is no need to guess or intuit what your library users are doing when visiting your website. By creating benchmarks for use, carefully experimenting with design changes, and then testing the results of those changes library web developers can make improve the experience of the user and provide better service.
References
Chisman, Janet K., Diller, Karen R., and Walbridge, Sharon L. "Usability Testing: a case study at Washington State University." College & Research Libraries 60 (November 1999): 552-69.
Dickstein, Ruth and Mills, Victoria. "Usability testing at the University of Arizona Library: how to let the users in on design." Information Technology and Libraries 19 (September 2000): 144-51.
McMullen, Susan. "Usability Testing in a Library Web Site Redesign Project." Reference Services Review 29 (1): 7-22
Michael D. Levi and Frederick G. Conrad "Usability Testing of World Wide Web Sites" Bureau of Labor Statistics Research Papers Available Online [August 2001] URL: http://stats.bls.gov/ore/htm_papers/st960150.htm
Reeves, Wayne W. Cognition and complexity : the cognitive science of managing complexity. Lanham, Md. : Scarecrow Press, 1996.
For More Information
Links to library website usability studies Available Online [August 2001] URL: http://www-hsc.usc.edu/~benjes/usability.html
U.S. Department of Justice Americans with Disabilities Act ADA HOME PAGE URL: http://www.usdoj.gov/crt/ada/adahom1.htm
Cluster Analysis in Web Design Available Online [August 2001] URL: http://osulibrary.orst.edu/staff/matylonj/cluster/sld001.htm
Webtrends User Manual Available Online [2001] URL: http://www.webtrends.com/products/log/default.htm
Karman, Robert N. AN INTRODUCTION TO SEARCH ENGINES URL: http://www.division42.org/MembersArea/Nws_Views/articles/IT/search_intro.html
Appendix A
For a detailed description see: http://osu.orst.edu/dept/library/libweb/survey.htm
|
The Valley Library is currently reviewing the design and organization of its Web pages. To help insure that our Web pages meet the needs of our users, we would appreciate a few minutes of your time to answer the following survey questions.
In completing this survey, you are participating in a research project to evaluate user responses to the OSU Libraries Web. The ultimate purpose of the project is to improve the usability of the library web. As an anonymous participant, your confidentiality will be maintained. If you have further questions about this project, please contact the Head of Reference & Instruction, (541)737-XXXX. For questions about your rights as a research subject, please contact the IRB Coordinator, OSU Research Office, (541)737-XXXX.
Question 1 - What is your status?
Question 2 - How would you characterize your level of familiarity with the World Wide Web:
Question 3 - In the past year, how often have you used the Library's Web page:
Question 4 - What kinds of services or tools do you use on the OSU Libraries Web Site? Please be as specific as possible. Question 5 - How successful are in you in finding the information that you want on the OSU Libraries Web Site?
Question 6 - How would you change the presentation of information on the OSU Libraries Web Site? What would you make better? Question 7 - What services or resources would you like to see on the OSU Libraries Web Site? Question 8 - Overall, how satisfied are you with the Library's Web Site
Question 9 - If you responded to the question above with "not satisfied", please comment on the source of dissatisfaction? |
Cluster Analysis
For a detailed description: http://osulibrary.orst.edu/staff/matylonj/cluster/sld001.htm
Methodology
An alphabetized list of service categories, representing all the content of the site, is presented to users. The user is asked to freely associate these categories in broader groupings. These groupings are then statistically added and graphed to show the natural groupings of a population of users.
Library and Computer Services Categories
A. Library Collections and what they contain
B. Problem resolution procedures for the network
C. Telephones and pagers
D. Newsletter of Information Services
E. Contact list of staff working for Information Services
F. Organization chart of IS; outline of responsibilities
G. Description of technology resource fees
H. Student computer lab resources
I. Resource for faculty developing multumedia for instruction
J. Databases to support university administration
K. Assistance for operating systems, software, mainframe, dail-up problems
L. Signage, printed materials, illustrations, logos, and animations's
M. Acquiring site-licensed software
N. Dorm computer connectivity
O. Adaptive technology for visual, speech, hearing, mobility or learning special needs
P. Departmental (not run by IS) Computing labs
Q. Network wiring for computer and phones
R. Central campus network description
S. Articles, books, newspapers, government information
T. Network support for departments
U. Computer classroom scheduling
V. Modem services (dialing-in by computer)
W. Event and documentary photography for special occasions
X. Workshops on instructional design using media, graphics, presentation software
Y. Documents for assistance on installing and configuring software
Z. Library hours
Cognitive walkthrough
A sample cognitive walkthrough workshhet follows:
|
Challenge Question: Find a journal article in your major
Topic: Education
Strategies and path used:
Attempt 1 - Subject Resource Guide...Education...Educational Abstracts
Attempt 2 - Electronic Reference Center...InforTrac Database...???
Attempt 3 - Electronic Reference Center...Database list
Comments by Subject:
The subject guide has too many links to get to the database
Obstacles observed:
She didn't know what oasis is.
|
John Matylonek is the Engineering Librarian at the Oregon State University Valley Library, Corvallis Oregon.
His other research interests includes developing engineering library instructional websites, identifying standards for the accreditation of engineering libraries, and developing information science techniques and services to expand the roles of librarians and libraries. When not performing research, providing instruction or providing reference service, he enjoys flying hang gliders, airplanes and snowboards
His other research interests includes developing engineering library instructional websites, identifying standards for the accreditation of engineering libraries, and developing information science techniques and services to expand the roles of librarians and libraries. When not performing research, providing instruction or providing reference service, he enjoys flying hang gliders, airplanes and snowboards