Standards for Records Preservation

TABLE OF CONTENTS

RATIONALE
APPRAISAL AND RETENTION SCHEDULES
PRESERVATION AND STORAGE
INVENTORY AND ACCESS

Rationale

All electronic media (magnetic tape, floppy discs, compact disc, etc.) are highly perishable. GIS files stored on electronic media do not last as long as traditional records maintained on paper, linen, mylar or microfilm. At a time when public records are increasingly created, maintained, and made accessible only in electronic form, it is vital that standards and procedures be established to identify and preserve these records to make them accessible to government and to the public over time. MSGIC members, and other organizations creating, using, or maintaining GIS files, can benefit both from standards for preservation applicable to all electronic records and from the specific recommendations included here for GIS files.

Agencies involved in GIS operations should 1) appraise, 2) inventory, and 3) preserve their electronic records to insure both their short- and long-term accessibility. GIS files are essential to the functions of many agencies, are expensive to produce, and will have significant uses in the near and far future. Retrospective and prospective studies are only possible when essential data from the past are still available. Records of such importance must be identified, catalogued, and preserved.

Accessibility of GIS records over time has legal, practical, and historical implications. The availability of records helps safeguard the government's legal and fiscal accountability, as in the case of disputes over wetland designations or historical easements. Future projects can rely on the data gathered by current programs, saving on the expense of creating datasets, and documenting change over time, such as shoreline erosion. Files from GIS projects help document the administrative history of government by illustrating an agency's goals and accomplishments, allowing civil servants to benefit from the efforts of their predecessors (or even their own earlier work) and to avoid the pitfalls of the past.

Control of the life cycle of electronic records is an essential feature of GIS management. Plans must be made and carried out for the creation and properly timed deletion, or the permenant preservation, of electronic records. The necessary appraisal, cataloguing, and preservation techniques are not difficult. These procedures are often already in place and utilize information already being documented in metadata forms, administrative files, and record retention schedules.

Appraisal and Scheduling consists of identifying records and determining how long they should be retained.

Inventory consists of accurately and completely describing the component parts of the files when they are created to insure that they can later be found and retrieved easily.

Preservation includes permenant storage, migration to new media, back-up procedures, and off-site security copies.

Appraisal and Retention Scheduling

Appraisal means identifying records and determining how long they should be maintained. The agency's Retention and Disposal Schedule is an essential tool in deciding which records should be deleted and which are of permenant value. The process of scheduling can also generate useful finding aids, such as inventories and catalogs, both on paper and using database management techniques. Data documentation, or metadata, is particularly important, because inventories of all records and and descriptions of permenant records depend on the same information as do metadata files. Because each GIS record should be documented through metadata procedures (See MSGIC'S Metadata Standards, 1995), inventories and scheduling rely on information already recorded, not newly reported.

Retention schedules are required by the Code of Maryland Regulations (Title 14 INDEPENDENT AGENCIES, Subtitle 18 STATE ARCHIVES, Chapter 02 Records Retention and Disposition Schedules; Authority: State Government Article, §§9-1007 and 10-632, Annotated Code of Maryland). According to this regulation, retention and disposal schedules are official documents that list and describe all records of an agency or office. The schedule authorizes both preservation and destruction of record materials in any form, format or media which are created or received by an agency or office in connection with the transaction of public business. GIS datasets, theme layers, and digital map or image files are all government records, whether created or acquired by an agency or office.

By law, each agency is responsible for devising its own retention schedules and taking its own inventories. The Records Management Division of the Department of General Services and the State Archives assist agencies in developing these documents and making correct preservation and disposal decisions. The State Archivist must approve all retention schedules.

An agency's records inventory and retention schedule are kept on standard DGS forms [call: (410) 799-1930]

• Inventory - DGS 550-4
• Electronic Records Inventory - DGS 550-6
• Retention Schedule - DGS 550-1

Although many records are routinely preserved in paper form, digital files that go into producing a final product are often discarded. The components of a project can be as valuable as the completed product, especially since digital files can be used in other ways than originaly planned. Datatsets can be useful for reasons other than the one for which the data was first collected, just as Maryland Geological Survey is now using navigational charting to determine shoreline erosion. The digital files that the State Highway Administration uses to produce road maps should be preserved as well as the paper maps. The use of those computer files could make future historical research or GIS operations possible. Thus, components should also be scheduled.

Each agency is responsible for appraising records to determine how long a record should be kept. GIS operators and project managers are in the best position to assist in that appraisal process for the records they use or produce. An entry for government records inventories should be completed by the GIS operator at the time the files are created or received. A Retention Schedule should be planned at that time. The forms can be completed at that time, but a more efficient way to handle the information is to create a database catalog.

In developing Retention and Disposal Schedules, agencies must consider the historical, administrative, legal, fiscal and other archival value of the record. Not least of these considerations of even temporary records is the value to the GIS community, including the original user.

Although writing comprehensive and all-inclusive rules for scheduling GIS records is not possible, certain guidelines are helpful and appropriate.

1. Inventory and schedule files by theme layer whenever possible. The data may be important for applications other than those for which it was originally developed.

   Example: Modern land use/land cover data and satellite images can assist archeologists in locating historical sites.

2. Inventory and schedule project files when the use and interaction of the themes produce a unique and discrete document.

   Example: The Office of Planning's MdPropertyView is a permenant record.

   Example: The Archives' use of image files of eighteenth century patent surveys superimposed on modern tax maps helped decide a particular land patent application case. The project file is of permenant value on legal and historical grounds.

3. If a theme layer is used repeatedly in your agency, especially for different applications, it should be permanent.

   Example: all base maps (see MSGIC theme layer and base map development reports)

4. The agency of origin has the ultimate responsibility for scheduling its records.
5. An agency that acquires data should schedule those copies based on the retention schedule of the agency of origin. Remember that scheduling identifies disposable records as well as those of permanent value.

   Example: If an agency acquires a valuable dataset from the Federal Government it should be scheduled as a permenant state record.

   Example: If an agency acquires data from another which has scheduled the record as permanent, the acquiring agency may indicate disposal for their copy after an allotted period of time.

   Example: If an agency PAID for data, that data should be deemed permenant. (Fiscal value is a valid basis for determining the permenancy of records.)

6. When in doubt, GIS records are likely to be permenant.
7. The Maryland State Archives will assist in the appraisal and scheduling process beyond its legal mandate to authorize the forms. Kevin Swanson, the Director of Appraisal and Preservation, and other Archives staff will be available for consultation [(410) 974-3914].
8. For further instructions on scheduling see COMAR Title 14, Subtitle 18, Chapter 2 or contact Records Management or the State Archives. For an introduction to records retention, see the regulations site at the State Archives' website (http://www.mdarchives.state.md.us).

Preservation and Storage

Ultimately, preservation of fragile electronic data depends on GIS operators developing both short- and long-term storage techniques. The sheer size of GIS files makes storage, communication, and transportion challenging. However difficult, adequate resources must be allocated to safe-keeping of these important and often expensive GIS records. The cost of recreating these files, or the even greater costs of not having essential data in the future, is too high to be ignored. In the end, long-term preservation of these records will depend upon managers practicing system discipline, the adoption of open file formats, the use of suitable storage media, and regular transfer of permanent records to the State Archives.

System Discipline

Routine computer procedures in any discipline should provide for daily file back-up. GIS files present a greater challenge because of their size. Intranet and network connections make saving copies of such large files practicable. Different electronic media also provide answers to short-term security storage issues. No single solution is applicable to every GIS operation.

Many work-stations are equipped with tape, or removable hard drives for back-up. Others simply keep files both on local workstations and on network servers, especially when files need to be shared. Setting a back-up routine and having back-up programs in place makes the creation of security copies second nature. Timing can be critical. Over-night transfers are sometimes necessary, given the time it may take to make such a copy. Some operators periodically make copies on CD-ROM. Whatever separate, portable medium is selected, the cassette, CD, dtata cartridge, or tape reel must be kept in a protective case and in a proper atmospheric environment: avoid extremes of temperature, humidity, and exposure to sunlight, radiation, or magnetic fields.

The most desirable storage location for such transportable media is off-site. To avoid catastrophic losses, such media should be sent to another building, preferably one near enough to allow frequent deposit and recovery. If an agency has only one facility, MSGIC members with adequate storage facilities can offer reciprocal deposit arrangements. Universities have the advantage of several buildings and departments, as well as institutional archives. The Division of Records Management in Jessup has responsibility for providing space for the off-site storage of files, especially since CDs, discs, and cassettes require so little room. The Annapolis Data Center may also provide back-up and off-site storage services. Permanent records should be deposited with the Maryland State Archives (see below) will always be accepted and given priority over temporary records.

Storage Media and Format

Safe short- or long-term storage of electronic records depends on proper procedures and the appropriate use of available technology. The Code of Maryland Regulations contains instructions on both procedures and media (Article 14 Subarticle 18 Chapter 4).

Several procedural considerations must be met. All authorized users of the system must be able to retrieve the desired documents. An appropriate level of security must be provided. Whenever possible, use the standard interchange format for permanant records. Temporary files are often best left in their native format. The selection of any format should provide for easy transfer and migration of the records. Whatever format or media is chosen, GIS operators should periodically examine and test files they have stored to be certain of their safe-keeping and accessiblity.

Although there is a range of electronic storage media available, CD-ROM has proven to be the most reliable, durable, and secure at the current time. If an agency or office head decides to transfer records on CD-ROM, which include database or text files, the standards recommended by "Read Only Memory (CD-ROM) Medium to Transfer Records to the National Archives (NARA Bulletin No. 94-4, 1994 July 20), "Information Processing Volume and File Structure of CD-ROM for Information Interchange (ISO 9660, 1988 September 1)," and the American Standard Code for Information Interchange (ASCII) format as required by the "Code for Information Interchange, Its Representations, Subsets and Extensions (FIPS Pub 1-2, 1984 November 14)" have been mandated by Maryland code.

For short-term back-up or security copies, network servers allocated for the purpose are useful to GIS operators whose files take a long time to be copied to disks. Some materials are especially well suited to the use of removable drives for easy portability and retrieval. Such devises, however, should not be considered permanent storage options, nor should they be exposed to potentially damaging environments.

All permanent records, and all those likely to be exchanged with other agencies or oganizations should be maintained in as universal a format as possible. According to COMAR:

"Word processing or text creation system shall be able to import and export files in American Standard Code for Information Interchange (ASCII) format as required by the "Code for Information Interchange, Its Representations, Subsets, and Extensions (FIPS Pub 1-2, 1984 November 14)" which is incorporated by reference."

"Cartographic or graphic file systems shall be able to import and export files in Group III or Group IV compressed facsimile transmission format as specified in "Terminal Equipment and, Protocols for Telematic Services (The International Telegraph and Telephone Consultative Committee, Blue Book Volume VII - Fascicle VII.3, 14-25 November 1988)." The second COMAR section above refers to the use of Group III or IV compression TIFF images.

Transfer of Permanent Records to the Archives

As with all records of permaneant value, such GIS records must be deposited with the State Archives. For this reason, and in the interest of protecting agency's work, the State Archives will assist in record appraisal and the establishment of preservation procedures. The State Archives fully supports MSGIC and agency GIS operations and recognizes the considerable investment GIS functions represent to state agencies. The Archives also cooperates with state agencies in their efforts to market their records and recognizes the retail value of even permenant public records. The Archives will work with state agencies to safeguard their income stream resulting from material also on deposit at the Hall of Records.

The standards governing the preservation of permanent electronic records are dictated by COMAR (Article 18, Subarticle 14, Chapter 4). The highlights of these standards include the use of universal media and formats. While ASCII has created a universal format for text files, GIS still does not enjoy such a standard. Inclusion of metadata, compression programs, and all requisite software to utilize the files is thus paramount. At present, the most desirable storage media is CD-ROM (see above), but CDs are not required for deposits. The regulations charge the depositing agency with the responsibility of providing any additional technical information, software, or platforms needed to read, copy, or process the records. These measures protect the user as well as the information, insuring that files will be available for reproduction and return to the agency of origin if needed. In the cases of the loss of the agency's own copies or the need for retrospective data, these procedures can mean the difference between being able to use such data and the files being inaccessible to future users.

Microfilm is one of the most durable and reliable records preservation media. ASCII text files, text printouts and raster or vector map image can be preserved by microfilming the text or images. Digital exchange between microfilm and computers makes this a viable option. However, microfilm preservation of actual ASCII or hexadecimal coding or other program data is not feasible at this time, although the potential of film-to-computer interaction exists in the near future.

Inventory and Access

Accurate and detailed inventories of GIS files, themes, and projects serve many functions. Inventories help identify and evaluate datasets and permit retrieval. Catalogues derived from the inventories can assist in distribution or marketing. Such lists form the basis for retention scheduling. Inventories can be paper documents or electronic files, local and private copies, or public and internet services. Inventories can be made to serve the agency, the government, and any constituency, clients, or customers.

Inventories can usually be created from existing metadata. The standardization practiced in metadata forms should be emulated in any inventory. Uniformity promotes easier access by allowing the use of common search terms, provides comparability, and insures that updates and editions of the same records are identified as such.

The Internet holds the promise of making inventories available to a wider audience. Intranet technology means the same service can be provided within an agency without such information being made more public. In either case, descriptions must be complete and accurate enough to guide the user to identify useful datasets and to disregard less useful products.

While standard practice is to catalogue layers or themes, project inventories can be just as important. The character and intentions of a project are important ingredients of an inventory description, as are the component parts of any project. Although these elements are obvious at the time of the project's creation, such information may not be available to others in the future without adequate inventory descriptions.

In addition to standard metadata entries, a catalogue or inventory should use key words and phrases that a researcher might use to search for the files. Inventories should also include the location of files on network drives as well as the location of back-up or off-site copies.

The University of Maryland, Baltimore County is compiling an on-line GIS resource catalogue. It provides a model for inventories and offers the opportunity to broadcast an agency's holdings. MSGIC members are encouraged to support this cataloguing effort and to work with University staff to develop a system the will adequately serve MSGIC members. The website address of the GEM! site is http://www.umbc.edu/bwrdc

Please contact Kristine Kulhman at kuhlman@umbc.edu or (410) 455-3847; fax (410) 455-1056 for more information.

The federal government is also developing several on-line catalogues, particularly the National Geospatial Data Clearinghouse under the auspices of Federal Geographic Data Committee - FGDC -( http://fgdclearhs.er.usgs.gov/). MSGIC members are encouraged to submit dataset information to such programs. Remember, catalogues are derived from information already entered into necessary metadata files and can even be used as metadata recordation tools.

In addition to the invaluable metadata technical description, certain other information is also necessary to provide access to files over time. Noting the location of files, in terms of network or workstation drives, or building/room/shelf locations for CDs is vital. The inventory must be kept up-to-date when files or CDs are moved. What may appear as a usual and obvious or consistant location today may change over time. Such changes need to be recorded.

When preparing permanent electronic records for deposit at the Maryland State Archives, certain procedures steps must be followed. These provide useful guidelines for internal physical inventories as well. The guidleines for deposit include the following instructions, modified here for MSGIC purposes:

All records transfered to the Archives must be described in a database format. The Archives has a records description program for use with Microsoft Access which can be furnished for agency use. The Archives will accept submissions in formats which are compatible with Access. Field specifications are available on request. Descriptions of records should be at the unit [theme layer] level, (one database record entry for each box or volume to be transferred.) The description should be sufficient to facilitate retrieval of the materials: series title as designated by the records retention schedule box or volume number [disks], date of contents, and file numbers or general description of contents. In some instances, agencies may have existing electronic resources that describe records which may provide sufficient control for the transfer of records. All descriptions of materials shall be furnished before records are transferred. The data received will be uploaded into A Guide to Government Records on the Archives' Internet site for agency and public reference.

Suggested fields for GIS deposits include:

• Agency of Origin
• Series name (i.e. name of project or component files)
• Date (of creation)
• Description (text/memo) - metadata text file
• Notes (text/memo) - restrictions, contact persons, cross-references (including to components), etc.

Each "series" is a distinct and separately defined project or theme; for example:

• DNR (Wetlands - RVC)
• MOP (MdPropertyView)
• DHCD-MHT (Historical Easements)

Series descriptions should also reflect data type (raster, vector) and data format (RVC, etc).

Inventories are records in and of themselves and should be permenant, whether on paper or as electronic files. Such inventories help track file development, document agency activities, changes in scheduleing, and other institutional history, as well as permitting record or file retrieval.