Appendix A

Analysis and Development of Model Quality Guidelines for
Electronic Records Management on State and Federal Websites

APPENDIX A

SITE VISIT PROBES

EXHIBIT 1

TOPIC GUIDE FOR FOCUS GROUP

COMMONWEALTH OF VIRGINIA

EXHIBIT 2

TOPICS FOR DISCUSSION

DEPARTMENT OF TREASURY

EXHIBIT 3

SITE VISIT PROBES

NAGARA

EXHIBIT 1

June 24, 1997
NHPRC Project

TOPIC GUIDE FOR FOCUS GROUP

COMMONWEALTH OF VIRGINIA

Internet. Please describe the situation with regard to Internet access for state government agencies and employees.
Websites. Please describe what home pages or websites the Commonwealth has.

- How are they organized?

- Does any overall policy govern agency websites

- Do individual agencies have policies?
Records management. Please describe the nature and organization of the Commonwealth's records management program.
Electronic records management. Please describe the state of electronic records management in the Commonwealth. Do you do any electronic records management?
Website Records Management. Have you given any thought, or are there any initiatives, to the records management aspects of websites? What are your ideas about this?

- Original materials? Now? In the future?

- Accountability? Have there been any embarassments? Near misses? Court cases?

- What do you advise people when they ask?

EXHIBIT 2

Charles R. McClure

J. Timothy Sprehe

Department of Treasury Site Visit

September 18, 1997

TOPICS FOR DISCUSSION

The following probes have been developed from previous data collection activities related to the project. Also attached is a participant survey to be filled out by those interviewed during the site visits, and participants at any of the focus groups. Topics are divided into a number of specific areas. Although it is unlikely that the researchers will be able to address all the various points listed below, they suggest a flavor of the topics to be explored.

Topic 1: Developing Electronic Records Management Guidelines and Policies

What led your agency to begin establishing electronic records management guidelines and/or policies?
What factors determined those guidelines and policies?
Did you model your guidelines and policies off existing ones (federal)?
Do you have guidelines or policies in place or in draft?

Topic 2: Responsibility

Who is ultimately responsible for the development of electronic records management guidelines and policies?
Who did you decide would be responsible for developing electronic records management guidelines and policies?
What areas of responsibility and authority do you feel this individual should have?

Topic 3: Creation of Electronic Records

What criteria did you use, if any, in determining what constitutes an electronic record?
Are there certain records which you decided should not be made available electronically?
Under what circumstances do the postings on your website(s) qualify as official records?
Did you develop any criteria for determining which records are considered public records and therefore need to be managed appropriately?

Topic 4: Disclaimers

Do you use any type of disclaimers on your electronic records? Examples?
How did you determine what type of disclaimers to use?
Are there certain documents that you use disclaimers with and others that you do not?
How did you determine which documents needed disclaimers?

Topic 5: Privacy

How do you address the issue of privacy in your records management policies or on your websites?
Are you concerned about privacy issues in relation to electronic records management?

Topic 6: Security

How do you deal with security issues pertaining to electronic records on your websites?
Are you concerned about the security of your electronic records?
Do you have any policies which deal specifically with breach of security in relation to electronic records and those, more specifically, on your websites?

Topic 7: User Evaluation

Do you know who your users of your website are?
Do you have any mechanisms in place to identify who these users are?
Do you currently maintain any web log files?
If so, do you use these log files to determine who you users are and how you might better serve them?
What do you consider to be the most important aspect about the users of your website?
Do you include an e-mail link on your documents for user feedback?

Topic 8: Authenticity

How do you ensure that users of your websites will find the information authentic?
Are there any steps you undergo to insure that the records you post on your website are authentic and will remain as such?
Do you include a statement of responsibility on your documents such as "maintained by" and/or "date last updated?"

Topic 9: Training and Education

Are you involved in any training of agency/department employees regarding electronic records maintenance and use?
What is your agency doing to promote the importance of electronic records management of websites to its employees?
Do you conduct or attend any workshops or trainings regarding electronic records management in general and of websites in particular?
What are the issues pertaining to electronic records management which you feel require the most training and sensitivity?

Topic 10: Maintenance and Use

What steps are you taking to ensure that website records as evidence of agency actions and transactions are not being altered during the course of use and preservation?
Are your guidelines for managing electronic records on a website flexible enough to be useful in all types of website environments?
How do you go about keeping pace with technological advances and the evolving Internet?
How do you determine when you should update the records on your websites?

Topic 11: Records Disposition

Do you have in place procedures for removing website records that are no longer active?
How do you maintain historical records of what has been previously posted on your websites?
What steps are you taking to incorporate disposition of electronic website records as early in the life cycle as possible?
Do you have any guidelines which deal with the scheduling of electronic records on the website(s)?
Are there any changes in a website document which may constitute an update in its scheduling?
How do you go about describing the content or composition of your website records for scheduling and disposition purposes?

Are you taking any steps to incorporate appraisal and retention functionalities into the design of website information systems supporting records creation and use applications?

Topic 12: Preservation

Are you taking any steps to ensure preservation of your electronic records on your website(s)?
In what format are you preserving your such records?
Are you concerned with migration?
Do you feel it is necessary for federal agencies to have weekly or daily snapshots of their websites?
How do you maintain the readability of electronic website records over time as software and operating systems become obsolescent?

Topic 13: Liability

How do you go about preserving records on your website for faithful reproduction in court if called upon to do so in the future?
What attributes of website records are essential to preserve for liability purposes?
Do you have any policies in place which deal with the retention and disposition of records to protect against liability issues?

Topic 14: Feasibility/Costs

Who is in charge of managing the costs of electronic records management on your agency's websites? What cost are involved in creating and storing these electronic records?
Was cost a factor in determining which records to place on the website?
Are costs a factor in the creation and storage of electronic records on your website?

Topic 15: Accessibility

What steps are you taking to ensure that electronic website records no longer in active use can still be accessed?
Do any of your websites function as an formal information repository for inactive records?

Topic 16: Accountability - The Ability to Reconstruct the Past

Do any activities occur on your website(s) that constitute official Departmental or agency business?
Are there any procedures in place to protect against the likelihood of having to reconstruct with precision what was displayed on your website(s) at a given moment in time?
Do you maintain a historical log of what has been posted on your websites in the past? When added, and when deleted or removed?

Topic 17: Policy Issues

Who are the key individuals in the development of policies pertaining to electronic records related to your departmental/agency website?
To what degree is Federal legislation or Executive regulations affecting electronic records management of websites?
Are you taking any actions to ensure compliance with particular government policies relating to electronic records management?
What policies have helped or hindered the development of your electronic records management guidelines -- and especially those affecting your website?
What policies would you like to see in place regarding electronic records management of websites?

Topic 18: Lessons Learned and Future Directions

What do you feel are critical factors leading to the success of electronic records management on Federal websites?
What do you feel are the key issues relating to electronic records management of Federal websites?
What are the most critical problems you have encountered in trying to develop and maintain electronic records management of websites?
What are the next steps for you/your agency/department in regards to electronic records management of websites?

EXHIBIT 3

July 8, 1997

SITE VISIT PROBES

NAGARA Focus Group

The investigators developed the probes listed below as an aid while conducting a focus group at the July 1997 meeting of NAGARA in Sacramento, CA. They derived the probes from findings of the Phase I report and from their experience with three state site visits: Connecticut, New York, and Virginia.

Topic 1: Developing Electronic Records Management Guidelines and Policies

Have your states developed electronic records management (ERM) guidelines or policies?
What led you to begin establishing ERM guidelines/policies?
Did you model your guidelines/policies off existing ones from another state or federal agency?
What did you consider the most important issues in creating such policies?
What were you trying to achieve by developing ERM policies?

Topic 2: Internet and Email

Do your states have any policies governing use of the Internet? What topics do such policies cover?
Do your states have any policies governing the use of electronic mail? What topics do such policies cover?
Are there any established linkages in your states between Internet and email, on the one hand, and recordkeeping systems, on the other?

Topic 3: State Website (Home Page) Management

How are state agency websites organized in your state?
Are state agency websites coordinated or controlled in any centralized manner?
What kinds of backgrounds do webmasters come from?
Is there any regular communication or coordination among webmasters?
Are there established relationships and communications between webmasters and records managers/archivists?
Are there any linkages in your states between websites and recordkeeping systems?

Topic 4: Original Records on State Websites

Do you think your state agencies have any original materials on their that could be considered official records?
Do you have any guidelines or policies about whether state agencies may post original materials to websites?
What records management steps do you expect agencies to take if they have original record materials on their websites?

Topic 5: Accountability

Do your state agencies keep any kind of historical log as to what has been posted on their websites?
Have any postings on websites caused public embarassment or controversies that have caused top state officials to be concerned about their state agencies' websites? What has been the outcome of these incidents?