How do you know what you think you know?

When preparing this presentation I initially found it somewhat difficult to decide what its focus would be. Archival and records management theory and practice, knowledge management, information technology and diplomacy at first glance seem to be a pretty disparate group of topics for a 30 minute presentation. It can, however, be done. I chose to focus on four records-related areas where these issues come together and provide the greatest challenges to archivists, diplomats, historians and technology providers. These areas are: (1) validation, (2) trustworthiness, (3) context and (4) longevity.

Having been intimately concerned with diplomatic records as a creator of them, a consumer of them and a custodian of them has given me a well-rounded view of their importance, their functions and their limitations. As a result of my background with the State Archives of Kentucky, the U.S. Department of State, the U.S. National Archives and my current position as head of the records section of the Inter-American Development Bank I have become more and more convinced that an organization which has a well conceived and fully functioning records management program will have a solid archives and that that archives can be an essential contributor to the information and knowledge needs of the organization. On the other hand, an organization which lacks a solid foundation regarding both its current and its non-current records will be building its information structures on sand and will pay a heavy price for such a state of affairs. So, with my biases exposed, let us turn to some definitions of archives and look at their implications for diplomacy, especially in light of the new technologies.

DEFINITIONS OF ARCHIVES

The archival Magna Carta is Sir Hilary Jenkinson’s A Manual of Archive Administration, published in 1922. This manual is the departure point for discussions of archival theory and practice, at least in the English speaking world. Sir Hilary, in a later essay, defined archives as “the Documents accumulated by a natural process in the course of the Conduct of Affairs of any kind, Public or Private, at any date; and preserved thereafter for Reference, in their own Custody, by the persons responsible for the Affairs in question or their successors.” In an article in the Spring, 1994 issue of the American Archivist, Luciana Duranti quotes this definition and further cites Jenkinson’s Manual in which she notes that “because they [archival documents] are created as a means for, and a by-product of, action, not ‘in the interest or for the information of Posterity,’ and because they are ‘free from the suspicion of prejudice in regard to the interests in which we now use them,’ archival documents are impartial and ‘cannot tell…anything but the truth.’” Now this is a startling concept—that archives are inherently trustworthy and useful precisely because they were generated as a by-product of recording the daily business transactions of an organization (or individual) and without regard to how they might be used by other people for other reasons in other times.

Laundry lists

A couple of examples might be illustrative. Consider, for example, the laundry lists of a medieval monastery and why they might be a very useful research tool. Some monk, or succession of monks, created, over a period of time, lists of dirty linen and what happened to it. These records express no interest in anything else going on in their organization, much less in their society as a whole. Whatever went into the laundry lists was validated as having to do with that topic by the mere fact of inclusion. Those responsible for the laundry records did not intermix records dealing with other matters; if it didn’t have to do with dirty linen, they didn’t accept them. Once accepted as germane, the records were arranged in ways most useful to the laundry department—by name of monk, by type of material, by date, whatever was organically most useful to them. The fourth point to note is, of course, that they survived the vagaries of the centuries.

Exactly because of this specificity in why they were gathered together (validation); the fact of their being accepted by the organization as reliable (trustworthy); their internal groupings (context and arrangement) and their survival (longevity) it is possible to use such records with confidence in, for example: an analysis of administrative costs of religious institutions; the names and status of individual monks based on the numbers of articles to be washed or numbers of changes of clothing; studies of medieval textile trade patterns, perhaps even determining the names of individual weavers or cloth merchants; the internal hierarchy of the monastery; liturgical customs based on the use of various vestments; the dating of visits by passing royalty; seasonal changes in clothing; etc., etc. The very fact that the creators of the laundry lists were supremely indifferent to providing information on administrative costs, names and status of monks, the textile trade, weavers, hierarchies, liturgical issues, royal visits, seasonal changes, etc., etc.—makes the information which they provide about these areas so very valuable. They unconsciously provide the peripheral vision of history, without which history would suffer from tunnel vision.

As an aside, the examples I am using here illustrate that diplomatic (and indeed all) archives share certain common traits which offer particular challenges to the consumers of today’s information technology. These challenges include the development of computer functionality for the creation and maintenance of true records. To do this, records produced electronically must: (1) be able to be validated as being relevant to the business transaction at hand, (2) provide an environment which will allow records created in the normal course of business to maintain their inherent trustworthiness, (3) provide some architecture for maintaining a meaningful relationship among records and, perhaps the most difficult, (4) ensure survival over time. Surely it is not too much to expect the latest technology to at least provide the functionalities available to medieval monks.

Concentration camp records

This continuity of traits could be illustrated in any number of other examples. The records of Hitler’s Germany, for instance, captured by the U.S. Army after the fall of Berlin in 1945 were useful at the War Crimes trials in Nuremberg precisely because they were created without consideration of how they might be used outside the context of their creation. They did not have subject files arranged under the title “Atrocities” or “the Holocaust” but rather they were organic records of routine transactions relating to, for example, the administration of concentration camps. These routine transactional records might include orders of the day, receipts for supplies (such as poison gas), bills of lading for the shipment of personal effects (such as eye-glasses and gold teeth), personnel records which listed everyone from the camp commanders to the guards (including periods of service and position descriptions), routine periodic reports from the camp medical unit (which might include the results of experiments on human beings), mortality registers, incident reports of uprisings and how they were quelled; in short, all the usual, mundane records likely to be produced in the daily course of business in a well-ordered military installation. Only by reviewing such routine records does the full impact of what went on in these camps hit home. And these routine records were accepted without question by the War Crimes Tribunal because they were inherently trustworthy. As unalike as these records are from the monastery laundry lists, they share the common threads mentioned above—they are trustworthy because they had no interest in documenting anything other than the routine transaction at hand, the act of inclusion in the files served to validated the records, they were maintained in a meaningful order and they were preserved over time.

State Department records

Another body of records to consider might be those of the U.S. Department of State. Regular reports are submitted to Washington from all diplomatic and consular posts, and have been since the 1790s. Studies of U.S. foreign policy can be enhanced by going beyond the selected documents published in the “Foreign Relations of the United States” (the FRUS) and looking at these raw reports. Indeed, since the documents printed in the FRUS are selected after the fact, they are inherently less trustworthy than the original reports which must be, by definition, trustworthy. These reports were filed by name of diplomatic post and chronologically thereunder. The pre-1903 reports have been microfilmed and are available for use and purchase at the U.S. National Archives. An early 19th century consul might never have mentioned the words “foreign policy” in a report, but his comments on the treatment of U.S. ships by the local harbour master, the relative status of the U.S. ex-patriot community, the progress of civil and criminal cases through the local courts, the treatment of U.S. prisoners, the status of negotiations over export licenses, local gossip, rumours of coups, complaints about the unhealthy climate, currency fluctuations, local customs, language issues, legal issues, etc., provide a rich soup of information which was used then in the conduct of U.S. foreign policy and which can be mined for a variety of purposes today. This body of records is still receiving accretions to this day.

Let’s follow the path of how more recent diplomatic information becomes a part of these files by looking at the first foray of a junior foreign service officer into diplomatic reporting in the mid-1980s. To protect the innocent, let’s call him, oh, Tom, for the sake of this discussion. Shortly after his arrival at post, where he had begun his first tour as third secretary working in the consular section on the visa line, Tom attended a large reception at the ambassador’s residence. The food and liquor were excellent, the music and the lights were soft, all the movers and shakers were there. In other words, all the elements were in place for a hard night of diplomatic representational work. Now I’m not being sarcastic here—this is the setting in which much of the most important work of diplomacy is done. Here is where information (and misinformation) is exchanged, where friend and foe are sized up, where friendships are made and rivalries contested. What is of interest to the archivist, as well as to the historian and the diplomat, is how such environments produce meaningful records.

Well, back to Tom. During the course of the evening, he acquired some startling information. It could have been about currency fluctuations, or export restrictions, or the flow of illegal aliens, or the love-life of a prominent local figure or any one of a thousand topics. For the sake of this discussion, let us say that it related to a coup d’etat planned for the following month. As soon as he could find his consul general, Tom told her of his hot item and suggested that they notify Washington immediately. She, much wiser than Tom in the ways of diplomacy and how information was treated, listened and suggested that they follow the usual channels. (She certainly knew that awakening the Secretary of State at one in the morning to discuss coup rumours with a junior officer would not be a career-enhancing move.) Disappointed but undaunted, Tom began to navigate those channels which would lead his information into the safe waters of the diplomatic archives.

At Tom’s earliest opportunity he wrote a standard “memcon”, or memorandum of conversation, going on for pages about what an important piece of information he had and how it would change history. He took the time and trouble to set the scene, providing copious background information about the party, the ambiance, his reactions to it, how he thought the information he was sending should be used, whose side we should take in the imminent civil war, what he thought of all parties concerned, etc., etc. Little did he realize the process through which his multi-page opus would go before being transformed from information to record.

The first step was to transcribe it from hand-written notes into a customized word processing package and print it out onto a standard form, compatible with the communications equipment in use at the time which would convert it to a cable and transmit it to Washington. (Now this is an important part of the validation process—the physical format must be correct or the record will be summarily rejected.) Excitedly he sat down at the terminal and began to work. Certain information fields were required, such as drafter. That would be proud young Tom. Then the system asked for clearances. Young Tom put down his boss, the consul general. He put in certain codes which were attached to the cable-to-be, such as CO for consular matters, reflecting his assigned position. Proudly young Tom took the great document to his boss, sure it would be flashed to Washington at once, re-writing diplomatic history. Alas, illusions are grand but not always long-lived. The consul general took a heavy blue pen to the draft, pruning out much deathless prose, not even trying to be gentle as she pointed out that no one cared that Tom’s favourite scotch had been served at the party, whether or not the minister of justice’s wife had on the same dress as someone else, how the newly redecorated residence looked, and certainly no one wanted Tom’s opinions about the past, present or future. The purpose of the memcon was to report facts. Analysis was outside its scope. She also pointed out that even the subject code was incorrect. It had nothing to do with consular affairs, even though Tom might have been a consular officer. The proper subject was PO for Political issues. She increased the number of clearances to include the Deputy Chief of Mission and several others who had been at the reception. Unconsciously, as an organic part of her function, she was deepening the validation process by making sure that this embryonic document conformed to the standards of the records system of which it would form a part, both in terms of format and content. Once she had finished pruning, it went through the clearance process. The DCM made a few additions to the message, such as the fact that the high-ranking officer who was the source of this report was involved in a simmering dispute with another high-ranking officer whose career he had often tried to damage. Another clearer mentioned that the source had been drinking heavily all evening and that he had been overheard saying that he was going to “get el Colonel” that night. Other clearers added other details. When all the clearances were signed off on, the much shorter cable was taken to communications where it was sent. Once received, it was copied, distributed and filed with two centuries of similar reports.

Diplomatic history was not, alas, re-written. No coup occurred. Luckily the “fact” had been vetted, placed in its context, recorded as a rumour and then properly filed away with all its brother and sister reports going back two hundred years. Young Tom could, however, take comfort in the knowledge that he had contributed to both diplomacy and history, albeit in a very small way, since his information had been converted into something worthy to be part of the archives of the U.S. Department of State. It had been (1) validated, making it (2) inherently trustworthy, (3) it had joined with many other accretions to the reporting files, putting it in its proper context and (4) it would be kept for further reference.

Other examples could include aerial photographs of Europe and Japan, Stasi files, secret correspondence of Louis XV, baggage lists from U.S. immigration, visa files, Czar’s secret police, and so on and so on, but I think that I have made my point about what gives a record archival value. I would now like to turn again to the issue which I raised earlier about new information technologies and the challenges and opportunities they offer to the “traditional” archives and records keeping systems.

CHANGES AND CHALLENGES OCCASIONED BY TECHNOLOGY

Because of the very success of archives in storing knowledge, new types of researchers are clamouring at their doors, demanding information in new formats and with new expectations of what can and should be done with the information “locked away in dusty old boxes”. These new demands hold great promise and create perils for archives and their users.

One of the results of this change of users and uses is the blurring of the distinctions between information and records. All records are information but certainly not all information is a record. Those who ignore this distinction do so at their peril. Let me give you a couple of instances to illustrate this distinction and how technology has been involved.

TWA Flight 800

This blurring of lines was brought home to me when Pierre Salinger, the former aide to John F. Kennedy, claimed to have proof that the U.S. Air Force shot down TWA flight 800 over Long Island a couple of years ago. I remember him standing before the TV cameras in Paris, waving a piece of paper and saying, “Here is proof that the Air Force did it.” As near as I can figure it out, the following happened. Mr. Salinger knew someone who worked in a French intelligence service. This person had obtained a copy of something which reportedly had been acquired by someone with access to the CIA’s computer system. What they found, and posted on the Internet, was a statement that the Air Force had been responsible. The mistake that Mr. Salinger made was to transfer the assumed trustworthiness of a properly “archived” document to a piece of free-floating information. The information which was obtained (or leaked or planted) was deracinated, it had no context. Where was it from? Was it indeed from the CIA? If so, how had it been identified? Was it from their “rumours” file? Or from their “usually reliable sources” file? Was it in their cables from the field or was it a photocopy of something from a tabloid which they had as part of their reference files of nonvalidated information? Had it been forged and put out as part of an attempt to exculpate the airlines or the manufacturers of the plane or by some conspiracy-obsessed individual or by someone who wanted to make a movie? Where is it now? It fails every test of “archivability”, to coin an inelegant word. It was nonvalidated, untrustworthy, without context and impermanent. And yet, because it came from a computer a person used to handling important information accepted it at face value. Where is technology taking us?

In the interests of time, I will pass over the issue of subject files and reference files and how they relate to archives. This could be the topic of a paper in itself.

To avoid merely complaining about misapplied technology and get specific I offer the following as minimum requirements which must be in place before records in electronic format could be acceptable to archivists, historians, diplomats or anyone else who depends of records on a daily basis:

• A sine qua non is meaningful metadata. The capability exists to ensure that all relevant data about data being created and preserved electronically is present and accounted for. I would want to know that any system purporting to be an electronic records system is robust enough to carry the heavy burden of validating the information.
• If electronic information is being represented as an archives, the standards which make it inherently trustworthy should be clearly defined. This can be done, but is expensive and complicated. Already there are records-keeping standards being developed in Australia and by the U.S. Department of Defence which are widely available. All of us must be clear that the computer term “archives” means to make a back-up copy. It has nothing to do with the functions of an archive. To mention a pet peeve of mine, “archives” is a noun, never a verb.
• Provide context. A simple, direct-access, relational database is wonderful for quick answers, but as anyone who has done even a simple search on the Internet knows, the results of that search may provide you with every single one of the thirty references you may want for Marie, Princess of Battenberg of the House of Hesse, but if those references are buried within 250,000 hits covering Battenberg lace, the Hesse Oil Company, the Princess cruise line, every real estate company selling houses and Herman Hesse, what have you got? In the field of archival cataloguing some of the most fiendishly difficult challenges involve the linkages between related files, series, sub-series, documents, etc. It can be done, but is also expensive and complicated.
• Provide longevity. (This may be the biggest challenge of all.) The current life-expectancy of five years for a computer application is not enough to guarantee that records will survive for as long as they are needed. In a related area, I also expect clear distinctions between the use of computers for access and for preservation and I expect both issues to be addressed. Confusion between them may present the greatest threat to the continued value of archives. They are not the same. Indeed they are sometimes diametrically opposed. Increasing access may put records at risk. Preservation issues may limit access. Those who say “Let’s use the power of the computers and scanners. Just put everything on the computer where we can search it and don’t worry about that old-fashioned paper” make ice-cold chills run up my spine. It’s not the paper I continue to long for, it’s the overt context I want to see. Another related spine-chilling comment is, “I’ve got what I need from this database. Just dump it.” We do not have to be Luddites to have concerns about any organization’s commitment to keep data fresh, to migrate it every five years or so, to keep all the metadata intact and to keep all the archival relationships clear, especially when the original creator or user has finished with it. Archivists and historians may know that there are lots of secondary uses of information, many of which are more important (to us, anyway) than the original use was. We must make sure that the IT department understands this use. I wonder how many electronic records will not make it across the Y2K divide because someone will decide that what’s on them is “just old stuff” or “we’re through with that file” and throw out the diskettes rather than spend valuable time and money ensuring that the information is preserved?

CONCLUSION

So where are we? Right where we started as far as archival principles go but light-years ahead as far as the tools available are concerned. Archives (and archivists) have survived paradigm shifts before; we have gone from clay tablets to papyrus to moveable type to punch cards and, yes, even diskettes. We’ll survive again. It’s the attrition rate of information loss as we cross that technological boundary which I would hope to reduce to a minimum. So let us embrace the power of the computer to provide us with access speeds we never imagined even a few years ago; let us take advantage of data-mining technologies, of messaging possibilities, of the Internet and the Intranet and the Web, let us do all we can do using the technologies on offer, but let us also demand the validating and relational functions which give archives trustworthiness for as long as they need to be kept. It would be ironic if the 21st century, instead of leading to a technological utopia were to lead us backward to an informational Dark Ages where institutions loose the ability to create and maintain trustworthy records which can stand as guardians of individual rights and as sources of information which can be used to unite us. Working together we can ensure that computers will bring us great benefits. Failure to articulate our mutual needs as information consumers and custodians might well mean that records-keeping concerns are not addressed when designing information systems and we will all inadvertently contribute to an epidemic of organizational Alzheimer’s.