The purpose of this bibliography is to provide an introductory review of selected resources related to website evaluation. It is not a comprehensive list, but rather, offers a general sense of the range of existing resources that might be considered in the evaluation of websites, as well as the use of websites. Of special interest are the various tools and techniques related to website evaluation (including performance measures) that can be accessed via these sources. This bibliography suggests there is much work that has been done in the area of website evaluation and evaluators should draw upon this knowledge before developing their plan to evaluate websites.

Alexander, J. and Tate, M .A. 1999. Widener University Wolfgram Memorial Library:

Evaluating Web Resources. Available at: http://www2.widener.edu/Wolfgram-Memorial- Library/webeval.htm . [Accessed 18 May 1999].

Includes resources to assist in teaching how to evaluate the informational content of Web sites.

Aloia, G.F. 1998. Journey into Technological Integration: Key Steps and Strategies: Evaluation Web Sites. Available at: < http://cube.ice.net/~edutech/floridaweb/evalscale.htm > [Accessed 17 May 1999].

The author, Professor of Special Education at Illinois State University, targets instructors interested in Web site evaluation. This source presents a web site evaluation scale which relates the components of a site (information, creators, aesthetics, technology, and maintenance) to measures and descriptions of the measures. This web site was established to be a companion piece to the material presented at a Florida Conference in April of 1998.

Auer, N. 1999. Bibliography on Evaluation Internet Resources. Available at < http://refserver.lib.vt.edu/libinst/critTHINK.HTM >. [Accessed 17 May 1999].

Addresses problems and issues related to teaching and using critical thinking skills to evaluate Internet resources. Includes links to other electronic resources dealing with these issues, as well as a bibliography of print resources and useful LISTSERVs.

Csir, F. J. 1996. Evaluation and Criteria of the World Wide Web: Reference Web Sites. Available via ERIC ED 413 919, microfiche.

This master’s thesis applies and evaluation method for World Wide Web sites that provide access to online reference materials at academic and public libraries. The evaluation of web sites was performed with a questionnaire form focusing on: web site currency, accuracy and relevancy; web site organization/structure; web site presentation; URL maintenance; and web site features.

Livengood, S. P. 1997. An Evaluation Instrument for Internet Web Sites. Available via ERIC ED 413 899, microfiche.

This master’s thesis describes the creation of a comprehensive evaluation tool for reference librarians in adult service divisions to use in selecting World Wide Web sites as reference sources. A series of evaluation forms organize and display critical criteria taken from the current literature using the methodologies of content analysis and feature analysis.

McClure, C. R., and Lopata, C. 1996. Assessing the Academic Networked Environment: Strategies and Options. Available at: < http://istweb.syr.edu/~mcclure/network/toc.html>. [Accessed 7 April 1999].

This manual is an initial attempt to offer strategies and options for academic institutions to use in collecting information to assess their academic networked environment. Among others, the manual has the following objectives: describe a range of techniques that assess the academic networked environment; provide procedures for collecting and analyzing the data needed to produce an assessment of the academic networked environment; and identify and discuss data collection issues and problems that may be encountered when conducting such assessments. Many of the specific performance measures discussed in this manual are appropriate to, and easily transferred to, the federal environment.

Rubin, J., McClure, C. R., and Bertot, J. C. 1998. Log File Analysis Of DelAWARE: Selected Statistics And Discussion Items. Syracuse, NY: Syracuse University, School of Information Studies.

This report analyzed DelAWARE, the Digital Library of the First State which provides online information and services to Delaware's public libraries, via analysis of its log server files.

Federal Government

Bertot, J. C., McClure, C. R., Moen, W. E., and Rubin, J. 1997. Web usage statistics: Measurement issues and analytical techniques. Government Information Quarterly 14 (4): 373-395.

As government use of the web rises, so too does the need for assessing the extent and nature of public use of agency web sites. One means of web use evaluation is through the analysis of web server-generated log files. This article presents various log file analysis techniques and issues related to the interpretation of log file data.

Center for Democracy & Technology. 1999. Policy Vs. Practice: A Progress Report on Federal Government Privacy notice on the World Wide Web. Available at: < http://www.cdt.org/privacy/fedprivacystatus.shtml>. [Accessed 17 May 1999].

This report illustrates that although privacy is seen as essential in the WWW reaching its potential and there is increasing concern regarding government surveillance of individuals, many federal agencies have failed to clearly report their privacy policies on agency web sites.

Eschenfelder, K. R., Beachboard, J. C., McClure, C. R., and Wyman, S. K. 1997. Assessing U.S. federal government Websites. Government Information Quarterly 14 (2): 173-89.

This article provides an overview of information policy issues which affect federal web sites and introduces assessment techniques which federal web site designers and administrators could use to evaluate the design and management of their web-based information sources.

Federal Webmasters Forum. Top Privacy Principles for Federal Web Sites. General Services Administration, Office of Governmentwide Policy. Available at: < http://www.itpolicy.gsa.gov/mke/fedwebm/privacy.htm >. [Accessed 2 April 1999].

This purpose of this online GSA memorandum is to highlight the top privacy issues to consider for developing and maintaining Federal web sites. Privacy concerns impact the acquisition, use, and disclosure of personal information.

General Services Administration. 1997. Performance-Based Management: Eight Steps to Develop and Use Information Technology Performance Measures Effectively. Available at: <http://www.itpolicy.gsa.gov/mkm/pathways/8-steps.htm>. [Accessed 18 May 1999].

Offered by the General Services Administration’s Office of Governmentwide Policy, this guide describes the major tasks to follow when measuring the contribution of IT projects to an organization’s goals and objectives.

General Services Administration. 1997. Performance Pathways. Available at: <http://www.itpolicy.gsa.gov/mkm/pathways/ pathways.htm>. [Accessed 18 May 1999].

This Web site offers links to information related to performance measures. Includes links to information regarding strategic plans and sample performance measures.

Harreld, H. 1999, 11 January. A tangled web. Federal Computer Week 13 (1): 1, 32.

Article discusses federal agencies’ failure to develop a stronger job role for federal Webmasters and the fact that this failure may threaten the development of a digital government.

Hert, C. A. 1998. Facilitating Statistical Information Seeking on Websites: Intermediaries, Organizational Tools, and Other Approaches. Final Report to the Bureau of Labor Statistics. Available at: http://istweb.syr.edu/~hert/BLSphase2.html. [Accessed 7 April 1999.]

This study extends previous work (Hert & Marchionini, 1997) on the FedStats web site < http://www.fedstats.gov> which identified both successes of the web site as well as several areas for further exploration and enhancement. The project reported here builds on that work and aims to assist the agencies participating in FedStats in their efforts at continuing enhancement of the site.

Hert, C. A., Eschenfelder, K. R., McClure, C. R., et. al. 1999, January. Evaluation of Selected Websites at the U.S. Department of Education: Increasing Access to Web-Based Resources. Washington, D.C.: U.S. Department of Education. Available at: < http://iis.syr.edu/webeval/>. [Accessed 2 April 1999].

This study was a comprehensive evaluation of the U.S. Department of Education’s website. The evaluation was conducted using the following analyses: management assessment, policy analysis, log and transaction analysis, and usability testing.

Hert, C. A., and Marchionini, G. 1997. Seeking Statistical Information in Federal Websites: Users, Tasks, Strategies, and Design Recommendations. Final Report to the Bureau of Labor Statistics. Available at: <http://ils.unc.edu/~march/blsreport/mainbls.html>. [Accessed 7 April 1999].

This report presents the results from a nine-month study of three federal government statistical websites. The study was undertaken from October 1996 to June 1997 under the auspices of the Bureau of Labor Statistics and examined the BLS web site, CPS web site co-sponsored by BLS and the Bureau of the Census, and the FedStats web site sponsored by the Interagency Council on Statistical Policy. The main objectives of the study were to determine who uses these services, what types of tasks they bring to the sites, what strategies they use for finding statistical information, and to make recommendations for design improvements.

McClure, C. R. 1999, March. Performance Measures for Federal Websites. Internet Connection. Available at: <http://istweb.syr.edu/~mcclure/InternetCon4_99.html>. [Accessed 2 April 1999].

This article advocates the creation of performance measurements designed to measure the overall performance of web sites in successfully conveying government information to the public.

McClure, C. R., Wyman, S. K., and Beachboard, J. C. 1997. Quality Criteria for Evaluating Information Resources and Services Available from Federal Web Sites. Available at: <http://www.oclc.org/oclc/research/publications/review97/mcclure/mcclure_frameset.htm>. [Accessed 2 April 1999].

This study was conducted to explore the degree to which federal agencies’ World Wide Web sites meet the needs of their constituencies. Among other goals, the study was designed: to establish evaluative criteria for reviewing webpages; and to evaluate a sample of federal webpages.

Moen, W. E., and McClure, C. R. 1997. An Evaluation of the Federal Government’s Implementation of the Government Information Locator Service (GILS): Final Report. Available at: <http://www.unt.edu/slis/research/gilseval/titpag.htm>. [Accessed 7 April 1999].

This document reports the results of the evaluation study commissioned in response to the GILS Board’s request for an assessment of GILS. The goal of the study was to understand how:

• GILS serves various user groups
• GILS improves public access to government information
• Agencies are progressing with their implementations
• GILS works as a tool for information resources management.

The principal investigators used a variety of data collection and analysis techniques to assess the current status, use, and user satisfaction with the U.S. Federal implementation of GILS.

This document establishes the procedures that the U.S. Department of Education (ED) will use to make information available on the World Wide Web (WWW). This document provides guidance to ensure high quality and consistent content, organization, and presentation of information on ED WWW servers, to help users find pertinent information to answer their education questions.

U. S. Department of Justice. 1999. Section 508 Home Page. Available at: <http://www.usdoj.gov/crt/508/508home.html>. [Accessed 13 April 1999].

Section 508 establishes requirements for any electronic and information technology developed, maintained, procured, or used by the Federal government. As the initial step for implementing Section 508, all Federal agencies and departments must report to the Attorney General by June 15, 1999, the extent to which their electronic and information technology is accessible to and usable by people with disabilities. Links to information to assist in the self-evaluation process to determine this accessibility is available on this web site.

Verton, D. 1998, 21 December. Critics: DoD web policies too strict. Federal Computer Week.

While the Department of Defense says its new policies (U.S. Department of Defense, 1998) will not impede the free flow of information, others claim the guidance risks shuttering sites.

Standards and Guidelines

Defense Technical Information Center. 1996. DTIC WWW Server Standards and Guidelines. Available at: <http://www.dtic.mil/hovlane/standards.html>. [Accessed 14 April 1999].

This document lists the specific standards and general guidelines by which DTIC will make information available on the World Wide Web (WWW). It is the goal of this document to provide enough standardization in DTIC WWW projects to ensure a high-quality presentation and consistency for users. It is also the goal of this document to help those implementing WWW services at DTIC to identify the issues which should be addressed as a project is developed and made available to users.

Directorate for Computer and Information Science and Engineering. 1998. Policy for Internet Information Dissemination Using the World Wide Web. Available at: <http://www.cise.nsf.gov/general/CISEWebPolicy.html>. [Accessed 14 April 1999].

Provides guidelines for web site design for CISE web pages.

ED’s Oasis. Evaluation Center. Available at: <http://oasis.syr.edu/Evaluation.html>. [Accessed 2 April 1999].

Provides guidelines for evaluation of educational web sites. Affiliated with ERIC.

National Biological Service. 1997. Style Guidelines for NBS Home Pages. Available at: <http://www.its.nbs.gov/irm/HPStyleGuide.html >. [Accessed 14 April 1999].

The goal of this paper is to define a set of guidelines which will produce a consistent look and feel throughout all National Biological Service (NBS) World Wide Web Home Pages. The intention of this guide is that if someone, NBS or otherwise, were to call up any NBS Home Page, that person would know immediately that they had just queried an official NBS site. We feel that the best way to accomplish this is through a minimum set of HyperText Markup Language (HTML) tags.

World Wide Web Federal Consortium. 1996, November. World Wide Web Home Page: Guidelines and Best Practices. Available at: <http://www.dtic.mil/staff/cthomps/guidelines/ >. [Accessed 2 April 1999].

This site contains guidelines for developing and maintaining web sites which the World Wide Web Federal Consortium offers as suggestions to help the Federal community accomplish agency missions.

Tools and Methods

Byrne, T. 1997. Web Site Evaluation Questionnaire. Available at: <http://www.soc.staffs.ac.uk/~cmtrp/questns.htm>. [Accessed 2 April 1999].

Tony Byrne, a student at Staffordshire University School of Computing, developed this web site as part of his dissertation. The site offers an example of a practical, user-oriented web site evaluation form.

Colorado State University Library. 1997. Writing Center: Overview: Content Analysis.

Available at: <http://www.colostate.edu/Depts/WritingCenter/references/research/content/page2.htm>. [Accessed 18 May 1999].

A detailed introduction to content analysis. Includes types and methods of analysis, as well as an annotated bibliography.

Erskine, L. E., Carter-Tod, D. R. N., and Burton, J. K. 1997. Dialogical techniques for the design of web sites. International Journal of Human-Computer Studies 47 (1): 169-195.

The authors present an empirical study of their use of scenario-based design in the evaluation and redesign of a web site.

Georgia Tech Research Corporation. 1997. GVU’s WWW User Surveys: GVU’s 10^th WWW

User Survey. Available at: <http://www.gvu.gatech.edu/user_surveys/survey-1998-10/ >. [Accessed 18 May 1999].

This page is sponsored by the Graphic, Visualization, and Usability (GVU) Center, an interdisciplinary research lab affiliated with Georgia Tech’s College of Computing. It presents the surveys and the results from a user survey given online from October 10, 1998 through December 15, 1998 on topics such as online privacy and security, software filters and content rating, and finding product information. Includes the methodology of the survey, including its limitations. Offers advice on how to conduct analysis.

Intelliquest. Quantitatively Assessing Your Web Site: IntelliQuest Web Evaluation Services. <http://www.intelliquest.com/tools/wp9.asp>. [Accessed 12 May 1999].

IntelliQuest is a worldwide leader in providing technology-based companies with timely, statistically valid information critical in supporting their marketing decision. To meet the need of objective and quantitative assessment of web sites, IntelliQuest has developed another industry-leading solution: IntelliQuest Web Evaluation Services. Our quantitative assessment is a unique resource for corporations, publishers and agencies to assess the effectiveness of their web sites.

Kirkwood, H. P., Jr. 1998. Beyond evaluation: A model for cooperative evaluation of Internet resources. Online 22 (4): 66-68, 70-72.

This article updates the evaluation environment of the web and offers a simple model for evaluating web sites.

Lanius, C. 1997. What Makes a Good Webpage? Available at: <http://www.crpc.rice.edu/CRPC/Women/GirlTECH/Materials/rubric.html>. [Accessed 2 April 1999].

Provides a simple rubric for assessing the quality of a website.

Lavoie, B. 1999. Web Characterization Metrics. Available at: <http://www.oclc.org/oclc/research/projects/webstats/currmetrics.htm>. [Accessed 11 May 1999].

Details a number of metrics to be used in measuring: the web, web users, web clients, web servers, web sites, web pages, and web collections.

Massey, B.L. 1998. Content Analysis. Available at:
<http://www.ntu.edu.sg/home/tblmassey/content_analysis/ca.htm>. [Accessed 18 May 1999].

A list of links regarding content analysis, including links to overviews, examples, LISTSERVs, and software to aid analysis.

National Institute of Standards and Technology. 1998. WebSAT. Available at: <http://zing.ncsl.nist.gov/~webmet/sat/websat-process.html>. [Accessed 14 April 1999].

WebSAT uses a subset of usability guidelines and checks the html of a web page to determine "how well" the guidelines have been followed. Many guidelines are subjective and it is not possible to automatically judge conformance. This site also has a supplemental list of guidelines that should be reviewed before and during the design of a web site. These guidelines are grouped into the six categories.

Noonan, D. 1998. Making Sense of Web Usage Statistics. Available at: <http://www.piperinfo.com/pl01/usage.html>. [Accessed 7 April 1999].

O’Neill, E. T., McClain, P. D., and Lavoie, B. F. 1997. A Methodology for Sampling the World Wide Web. Available at: <http://www.oclc.org/oclc/research/publications/review97/oneill/o'neillar980213.htm>. [Accessed 12 May 1999].

The research reported in this paper documents a methodology for collecting a representative sample of the Web’s content through the use of random IP addresses. The ability to create a random sample of the content of the Web is a vital first step in any effort aimed at producing estimates of the size, scope, and characteristics of Web-accessible information. Web statistics based on the methodology described in this report should be relatively free from bias and other forms of sampling error. The methodology is also scalable in order to account for future expansions of the valid address space.

Perlman, G. 1998. Web-Based User Interface Evaluation with Questionnaires. Available at <http://acm.org/~perlman/question.html >. [Accessed 18 May 1999].

This online manual allows users to create their own questionnaires to evaluate Web sites. Provides links to sample questionnaires, as well as information on question.cgi, a system developed to help users create online questionnaires.

Rubin, J. 1998. A Strategic Look at Log Analysis. Available at: <http://headcase.syr.edu/NEW/Research/>. [Accessed 7 April 1999].

Provides background about using log analysis as an evaluation technique.

Trochim, W. M. K. 1996. Evaluating Websites. Atlanta, GA: Paper presented at the annual meeting of the American Evaluation Association. Available at: <http://trochim.human.cornell.edu/webeval/webintro/webintro.htm>. [Accessed 2 April 1999].

Proposes a general evaluation model which is based on an idealized development cycle for websites. The model suggests that website development is an ongoing endeavor and one that progresses through several definable stages.

Virtual Surveys Limited. 1999. Web Site Evaluation. Available at: <http://www.virtualsurveys.com/papers/webeval.html>. [Accessed 5 April 1999].

This site describes the stages of web site testing/evaluation as well as the migration from qualitative evaluation in the early stages to quantitative evaluation after implementation. Examples of both quantitative and qualitative measures are included.

W3C Web Accessibility Initiative. 1999. References on Web Accessibility: Tools and Utilities. Available at: < http://www.w3.org/WAI/References/#TOOLS >. [Accessed 18 May 1999].

Usability

Abels, E. G., White, M. D., and Hahn, K. 1998. A user-based design process for web sites. Internet Research: Electronic Networking Applications and Policy 8 (1): 39-48.

This paper reports on Phase II of a project to identify and implement user-based design criteria in World Wide Web sites. The authors developed a user-based design process which gathers user input at three different times in the process.

Abels, E. G., White, M.D., and Hahn, K. 1997. Identifying user-based criteria for web pages. Internet Research 7 (4): 252-262.

Reports on the first phase of a project to identify and implement user-based design criteria in World Wide Web pages. Focus group interviews and questionnaires were used to identify the criteria that influence the use of the Web and to analyze them within the context of the users’ information-gathering behavior.

Helinski, P. 1997, June. "Building a Quality Web Site: Part 2" Web Techniques Magazine.

This article demonstrates how to test for usability, such as using prototypes, and to avoid common pitfalls through addressing the aspects of a good interface design. Some aspects of a good interface include being easy to learn, containing few errors, and being satisfying and pleasant.

Hom, J. 1996. The Usability Methods Toolbox. Available at: <http://www.best.com/~jthom/usability/>. [Accessed 12 May 1999].

A very detailed collection of links to methods and techniques used in usability evaluation.

Human Factors International. Available at: <http://www.humanfactors.com/home/>. [Accessed 12 May 1999].

The goal of this commercial firm is to help make software usable. HFI has been in business since 1981 refining and field-testing a combination of usability consulting, training, and products.

Instone, K. 1997. Usability heuristics for the web. Webreview.com. Available at: <http://webreview.com/wr/pub/97/10/10/usability/sidebar.html>. [Accessed 12 May 1999].

Describes Jakob Nielsen’s ten usability heuristics for designing user-friendly web sites.

Instone, K. 1997. Site Usability Evaluation. Webreview.com. Available at: <http://webreview.com/wr/pub/97/10/10/usability/index.html>. [Accessed 12 May 1999].

Discusses heuristic evaluation of web site usability.

Available at: <http://www.cio.com/archive/webbusiness/040199_use_content.html>. [Accessed 19 May 1999].

This article discusses how web site design can both help and hinder a company. The author offers testing users as a way to truly find out if the web site is benefiting the company. Several fixes for bad web site design are given, ranging in price to fit any company’s budget.

Laskowski, S. J., and Downey, L. L. 199?. Evaluation in the Trenches: Towards Rapid Evaluation. Available at: <http://www.nist.gov/itl/div894/vvrg/position_paper.html>. [Accessed 14 April 1999].

In this position paper, the authors outline four case studies of their attempts at encouraging and incorporating usability in Web interfaces. Next, they describe the web interface and organizational characteristics that shaped the evaluation methodology presented in this paper. During the evaluations, the authors observed that, while typical evaluation approaches are useful, they are not sufficiently robust when applied to web interfaces. Based on these experiences, they have come to the conclusion that often one has no choice but to rely on techniques which support rapid, ongoing evaluation.

McCann, J. 1998, December. "Is Your Site User-Friendly?". Targeting Trends: Industry

Focus. Available at: <http://www.polysort.com/humanfactors/focus1-2.htm>. [Accessed 18 May 1999].

The importance of navigation, text, and visual aids in creating a user-friendly web site is discussed in this article. The author stresses that of these three aspects, navigation is the most important in aiding users.

Nielsen, J. Heuristic Evaluation. Available at: <http://www.useit.com/papers/heuristic/>. [Accessed 12 May 1999].

Collection of links to Nielsen’s online writing about heuristic evaluation.

Nielsen, J. 1996. Top Ten Mistakes in Web Design. Available at: <http://www.useit.com/alertbox/9605.html>. [Accessed 13 May 1999].

Jakob Nielsen, an expert on usability, describes ten mistakes in web design, such as using frames or having complex URLs.

Nielsen, J. 1999. "Top Ten Mistakes" Revisited Three Years Later. <http://www.useit.com/alertbox/990502.html>. [Accessed 13 May 1999].

Nielsen reexamines his 1996 top ten usability mistakes and scores them in light of current user attitudes. He concludes that all ten of the 1996 mistakes are still mistakes in 1999.

Nielsen, J. 1999. useit.com: Jakob Nielsen’s Website. Available at: <http://www.useit.com/>. [Accessed 13 May 1999].

Nielsen’s home page has many useful links to his work on usability, particularly invaluable is the link to heuristic evaluation.

"Our Global Community" Fourth conference on Human factor and the Web. 1998, June 5. Available at: < http://www.research.att.com/conf/hfweb/proceedings/proceedings.en.html> [Accessed 19 May 1999].

A conference sponsored by AT&T Labs which provides a forum for sharing information on producing web sites that are more useful and usable.

Scholtz, J., and Downey, L. 1998 (?). Methods for Identifying Usability Problems with Web Sites. Available at: <http://www.itl.nist.gov/div894/vvrg/jpaper/final.html>. [Accessed 14 April 1999].

We believe that rapid, remote, and automated evaluation techniques are key to ensuring usable Web sites. In this paper, we describe three studies we carried out to explore the feasibility of using modified usability testing methods or non-traditional methods of obtaining information about usability to satisfy our criteria of rapid, remote, and automated evaluation. Based on lessons learned in these case studies, we are developing tools for rapid, remote, and automated usability evaluations.

Scholtz, J., and Laskowski, S. 1998 (?). Developing Usability Tools and Techniques for Designing and Testing Web Sites. Available at: <http://www.itl.nist.gov/div894/vvrg/jpaper/hf_and_web.html>. [Accessed 14 April 1999].

Traditional usability methods are expensive, time consuming, and many require professional usability engineers. The environment in which web sites are designed cannot easily support these constraints. The approach of the authors assumes that evaluation of web sites must be rapid, remote, and as automated as possible. In this paper, we'll discuss three software tools and two techniques that we have developed to facilitate quick evaluation. These tools and techniques can be found at http://zing.ncsl.nist.gov/~webmet/.

Shneiderman, B. 1998. Designing Information-Abundant Web Sites: Issues And Recommendations. Available at: <http://ijhcs.open.ac.uk/shneiderman/shneiderman-nf.html>. [Accessed 5 April 1999].

Improved web site design may increase users' successful experiences and positive attitudes. This review of design issues identifies genres of websites, goals of designers, communities of users, and a spectrum of tasks. An Objects/Actions Interface Model is offered as a way to think about designing and evaluating websites. Finally, search and navigation improvements are described to bring consistency, comprehensibility, and user control.

Spool, J., DeAngelo, T., Scanlon, T., Schroeder, W., and Snyder, C. 1998. Web Site Usability: A Designer’s Guide. San Francisco: Morgan Kaufmann Publishers.

Based on data collected from research done at User Interface Engineering, this book describes how well or poorly some information-rich web sites performed when real users accessed them to find specific answers. Authors offer some guidance for evaluating and improving the usability of other web sites.

USINACTS. 1998. USINACTS: Tutorial. Available at: <http://atwww.hhi.de/USINACTS/tutorial/index.html>. [Accessed 18 May 1999].

An updated review of the tools and methods available for usability evaluations for telecommunication systems and services. Offers definition of usability, introduces statistical methods to analyze data, and aids in deciding which methods to use. Also discusses the human-centered design process.

U.S. West Information Technologies. No date available. Human factors Engineering: Usability Evaluation Methods. Available at: < http://www.cs.umd.edu/~zzj/UsabilityHome.html> [Accessed 17 May 1999].

Presents the various methods of testing, inspection, and inquiry to evaluate usability. Discusses characteristics of each method, as well as an overview, a description of procedure, and references when applicable.

W3C Web Accessibility Initiative. 1999. Web Content Accessibility Guidelines 1.0. Available at: <http://www.w3.org/TR/1999/WAI-WEBCONTENT-19990505/>. [Accessed 12 May 1999.]

This document is a list of guidelines that web content developers should follow in order to make their pages more accessible to people with disabilities. Following these guidelines will also make pages more useful to people using a broad range of devices (desktop browsers, voice browsers, mobile phones, automobile-based PC's, etc.) and to search engines. Tools that create web content (HTML editors, document conversion tools, tools that generate web content from databases) should generate content that is consistent with these guidelines.

W3C Web Accessibility Initiative. 1999. Techniques for web content accessibility; Guidelines 1.0. Available at: <http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/>. [Accessed 12 May 1999].

This document is a list of checkpoints arising from Web Content Accessibility Guidelines 1.0 that designers should address when creating web sites. It contains a number of examples offered as solutions to issues.

Webreview.com. 1998. Usability: Jared Spool Presents Hard Evidence. Available at: <http://webreview.com/wr/pub/web98east/23/spoolintro.html>. [Accessed 12 May 1999].

Presents links to three articles about web usability and usability testing by Jared Spool.

General Information

Alves, S. 1997, July. InfoGuide: World Wide Web Site Evaluation Guide for Information Professionals. Available at: <http://www.bf.rmit.edu.au:81/Dimals/rguides/website.htm>. [Accessed 5 April 1999].

This is an InfoGuide prepared by a student of RMIT University in Australia. This guide contains many resources to assist Information Professionals in their role of evaluating Internet resources, specifically relating to evaluating information on websites.

Bailey, S. 1997. Web site evaluation: Keeping your web site afloat. Webreview.com. Available at: <http://webreview.com/wr/pub/97/01/03/arch/index.html>. [Accessed 12 May 1999].

Offers justifications for why a web site author should evaluate their site.

Beck, S. 1997. "Evaluation Criteria." The Good, The Bad & The Ugly: or, Why It’s a Good Idea to Evaluate Web Sources. Available at: <http://lib.nmsu.edu/staff/susabeck/evalcrit.html>. [Accessed 17 May 1999].

The author, Instruction Coordinator at New Mexico State University Library, includes evaluation criteria for web sites (federal and non-federal), as well as rationale for using each criterion. Also offers links to example web sites.

Bevan, N. 1997. Usability issues in web site design. Advances in Human Factors/Ergonomics. Amsterdam; New York: Elsevier.

Describes various issues to consider when designing web sites, including planning the site, site structure and content, navigation within the site, page design, and site management and maintenance. Includes description of several evaluation methods.

This site offers criteria to evaluate web sites both in content and visual presentation. Also, provides links to several other university web sites on internet resource evaluation.

Bishop, R. 1997. Web Site Evaluation. Available at: <http://www.slis.ualberta.ca/598/robin/pap598.htm>. [Accessed 5 April 1999].

This paper of an MLIS student at the University of Alberta briefly explores the evaluation criteria available for print resources and borrows from these criteria to fit the context of web sites.

Gellman, R. 1999, April. "How to Tell If Your Organization Has a Privacy Problem and What to Do About It". The Internet Connection: Your Guide to Government Resources 5 (4): 1-4.

This article cautions organizations to be aware of what personal information they are collecting from employees, customers, or users of their web sites. Potential problems and solutions are discussed, including privacy policies.

Gordon-Murnane, L. 1999. Evaluating the net evaluators. Searcher 7 (2): 57-66.

This article looks at the organizations which act as watchdogs over all the sites on the Internet and underlines the need for reliable web evaluation services that can be trusted to provide careful reviews based on evaluation criteria, not site popularity. Describes 12 evaluators and their evaluation criteria.

Grassian, Esther. 1997. UCLA College Library Instruction: Thinking Critically about World Wide Web Resources. Available at: <http://www.library.ucla.edu/libraries/college/instruct/web/critical.htm>. [Accessed 18 May 1999].

Suggests points, in question format, to consider when viewing electronic resources. Includes considerations of content, source, date, and structure.

Harris, Robert. 1997. Evaluating Internet research sources. [Online]. Available WWW: http://www.sccu.edu/faculty/R_Harris/evalu8it.htm. [Accessed 19 May 1999].

The author provides a detailed discussion of screening information in this age of information overload. He offers the CARS checklist that can be easily used to check the reliability of information as well as the CAFÉ advice to aid users in selecting information.

Oliver, K. M., Wilkinson, G. L., and Bennett, L. T. 1997. Evaluating the Quality of Internet Information Sources. Available via ERIC ED 412 927, microfiche.

This paper reports on quality indicators which have been identified and classified within eleven criterion categories, including: site access and usability; resource identification and documentation; author identification; authority of author; information structure and design; relevance and scope of content; validity of content; accuracy and balance of content; navigation within document; quality of links; and aesthetic and affective aspects.

Pealer, L. N., and Dorman, S. M. 1997. Evaluating health-related web sites. Journal of School Health 67 (6): 232-235.

This article descries elements to consider in constructing and evaluating web pages (content, authorship, purpose, and page aesthetics).

Schrock, K. 1998, August. Evaluation of World Wide Web sites: An annotated bibliography. ERIC Digest. EDO-IR-98-02.

List of annotated sources relating to the critical evaluation of information on web sites.

Small, R. V. 1997. Assessing the Motivational Quality of World Wide Websites. Available via ERIC ED 407 930, microfiche.

This paper details the Website Motivational Analysis Checklist, a set of motivational criteria by which websites can be assessed. Criteria include the following: engaging; meaningful; organized; enjoyable.

Smith, A. 1999. Evaluation of Information Sources. Available at: <http://www.vuw.ac.nz/~agsmith/evaln/evaln.htm>. [Accessed 14 April 1999].

This web site contains a list of links to additional sources which discuss evaluation of information sources on the Internet.

Sowards, S. W. 1997. "Save the time of the surfer": Evaluating web sites for users. Library Hi Tech 15 (3/4): 155-158.

Details criteria for evaluating purposes, identification, authority, layout and design, links, and content of web sites.

Tillman, H. N. 1999. Evaluating Quality on the Net. Available at: <http://www.tiac.net/users/hope/findqual.html>. [Accessed 18 May 1999].

The author, Director of Libraries at Babson College in Massachusetts, discusses evaluation criteria, both web-specific and more general, that can be used to assess a web site’s quality.

Toub, S. 1999. "Bringing Up Baby". Webreview.com. Available at: <http://www.webreview.com/wr/pub/97/09/12/arch/index.html>. [Accessed 18 May 1999].

This article discusses the issues involved with adding a search engine to a web site. The author stresses that, like raising a child, a web manager must constantly care for the search engine in order for it to benefit users.

Tweddle, S., Avis, P., Wright, J., and Waller, T. 1998. Towards criteria for evaluating web sites. British Journal of Educational Technology 29 (3): 267-270.

The authors present criteria for web site evaluation. The criteria were originally developed to enable evaluation of cancer web sites for the general public.

Wilkinson, G. L., and others. 1997. Evaluation criteria and indicators of quality for Internet resources. Educational Technology 37 (3): 52-58.

Similar to the information included in Oliver, Wilkinson, and Bennett (1997), details University of Georgia’s information quality criteria.

Wittman, S. 1999. Evaluating Web Sites. Available at: <http://servercc.oakton.edu/~wittman/find/eval.htm>. [Accessed 14 April 1999].

Provides a set of criteria for evaluating web sites as well as a list of links to other web sites which discuss evaluation.