* The Secret Sentry discloses that the U.S. invasion of Iraq in 2003 was far from the first time when U.S. government officials, including senior military commanders and the White House, “cherry picked” intelligence information to fit preconceived notions or policies and ignored intelligence which ran contrary to their expectations. The Secret Sentry and the documents posted today show that widespread manipulation of intelligence also occurred during the Korean and Vietnam Wars for example, when Washington ignored intelligence on Chinese intervention in Korea, resulting in catastrophic consequences. [See Document 9 and Document 21]
* The Secret Sentry also details how since the end of World War II, constant changes in computer, telecommunications, and communications security technologies have been the most important determinants of NSA’s ability to produce intelligence. NSA has oftentimes found itself behind the curve in terms of its ability or willingness to adapt to technological changes, with delays and bureaucratic inertia causing immense harm to the agency’s ability to perform its mission. [See Document 16, for example]. As a result, during the past four decades NSA has dramatically increased the amount of the raw material that it collects, even while it has produced less and less intelligence information. According to Matthew Aid’s informed sources, during the Reagan administration in the 1980s, NSA processed, analyzed, and reported approximately 20 percent of the communications traffic it intercepted. Today, that number has dropped to less than 1 percent. For example, during the 2003 invasion of Iraq, NSA was unable to process 60 percent of the Iraqi messages it intercepted, while the U.S. military SIGINT units participating in the invasion processed less than 2 percent of the Iraqi military communications traffic that they intercepted.
Today’s posting of 24 documents consists of a selection of reports and memoranda prepared by NSA officials concerning the role played by Signals Intelligence (SIGINT) in selected military conflicts and crises, a number of classified internal histories written by NSA historians on key events in the agency’s past, and a selection of declassified articles from NSA internal journals.
Archive Visiting Fellow Matthew M. Aid obtained the documents while conducting research for his new book, The Secret Sentry: The Untold History of the National Security Agency (Bloomsbury, 2009). For the National Security Archive, Aid has edited a comprehensive set of declassified documents on the history of the NSA and its predecessor organizations from 1945 to the present, which ProQuest will publish later this year.
The National Security Agency
Edited by Matthew M. Aid
From its headquarters at Fort George G. Meade, Maryland, the National Security Agency (NSA) today controls a massive 60,000+ person intelligence organization with an annual budget estimated at over $10 billion, making it the largest spy agency in America and probably the world. NSA’s power and influence within the U.S. intelligence community has grown without interruption since it was created in November 1952, thanks in large part to the agency’s ability to consistently produce copious quantities of reliable intelligence information on a vast array of subjects from around the world (Document 5). And despite the agency’s much publicized recent troubles, SIGINT continues to be the single most important intelligence source available to the President of the United States. As of September 2001, NSA was producing 60 percent of all the material that was to be found in President George W. Bush’s daily intelligence summary, the President’s Daily Brief (Document 4).
But at the time of its birth in November 1952, declassified documents show that NSA was struggling just to survive in a period that a CIA official would later describe as “the Dark Ages of American cryptology." (Note 1) The nascent 7,600-person agency was striving against significant odds to establish itself on a firm footing, build up its intelligence gathering networks aimed at the Soviet Union and the People’s Republic of China, and trying to provide intelligence information for the U.S. military forces fighting in Korea (Document 2, Document 13).
Nowhere were NSA’s problems more evident than in the Korean War, which eventually degenerated into a bloody stalemate pitting U.S. and United Nations forces against the significantly larger Chinese and North Korean militaries. (Note 2) Despite some significant early successes against North Korean communications during the first year of the war (Document 7, Document 23), by the fall of 1952 NSA’s ability to provide meaningful intelligence declined markedly because North Korean and Chinese forces had systematically improved their communications security practices and procedures, reducing NSA’s SIGINT effort to whatever could be picked up from monitoring the enemy’s low-level walkie-talkie communications.
The Korean War also revealed one of the inherent problems afflicting all intelligence agencies--what to do when policymakers and military commanders refuse to heed the intelligence you provide because it conflicts with their own preconceived notions. For example, in the summer and fall of 1950, General Douglas MacArthur stubbornly discounted the SIGINT being sent to him showing an alarming increase in the size of Chinese forces based in Manchuria opposite Korea. By October 1950, the information derived from SIGINT and other intelligence sources overwhelmingly indicated that the Chinese intended to intervene militarily in the Korean War. General MacArthur and Washington ignored all of the indicators, with disastrous consequences (Document 21). In the end, NSA’s experience in the Korea War was not a happy one. A former NSA historian would later write that in Korea “There were successes and there were failures, but the failures tended to overshadow the successes” (Document 22).
NSA’s survival during the dark days of the 1950s was largely due to the efforts of two men. The first was Lt. General Ralph J. Canine, who served as director of NSA and its antecedent, AFSA, from 1951 to 1956. The combative Canine is today credited with lifting NSA out of the chaos that it found itself in by November 1952, and, in the span of four short years, making NSA a top-flight SIGINT organization and a force to be reckoned with inside the U.S. intelligence community (Document 8, Document 18). The second person responsible for NSA’s success during the 1950s was President Dwight D. Eisenhower, who proved to be a critically important supporter of the agency. NSA was repeatedly investigated during the 1950s by a series of panels, all of whom issued highly critical reports about the agency’s health and future prospects (Document 11). But Eisenhower, believing implicitly in the vital importance of the agency and its intelligence product, beat back all attempts by these committees to scale back or alter NSA’s efforts. In retrospect, it is doubtful that NSA would have survived the decade of the 1950s but for Eisenhower’s behind-the-scenes support and encouragement (Document 12).
The agency’s top target throughout the Cold War was the Soviet Union, which consistently ate up more than half of NSA’s collection and analytic resources between 1945 and 1989 (Document 6). The public is generally familiar with the significant successes by NSA’s cryptanalysts in breaking the codes used by Soviet intelligence operatives during World War II, an effort that has become known to history as the Venona decrypts (Document 1). But once the Moscow realized that the U.S. and Britain were breaking its codes, in the fall of 1948 the Soviets changed all their ciphers. It took three decades before Washington regained a degree of access to high-level Soviet communications. Nevertheless, the agency’s SIGINT efforts proved to be critically important in helping the U.S. intelligence community monitor the rising threat posed by the Soviet Intercontinental Ballistic Missile (ICBM) program (Document 17), which was the highest priority collection target during the late 1950s and early 1960s. Another high priority operation during the 1960s was NSA’s collection efforts aimed at the Soviet’s manned and unmanned space program (Document 21). NSA was also involved, albeit indirectly, in the CIA’s famous Berlin Tunnel Operation, which was not discovered by the Soviets until 1956 (Document 14).
NSA’s performance during the Vietnam War (1961-1975) was marked by a number of notable successes on the battlefield (Document 15, Document 19), which were credited with being instrumental in helping U.S. forces in Southeast Asia win a series of battles against the North Vietnamese and Viet Cong forces. (Note 3) But NSA was also involved in a number of costly intelligence failures in Vietnam, such as the agency’s mishandling of intelligence during the Tonkin Gulf Crisis of August 1964 (Document 9), which led to America’s full-fledged entry into the Vietnam War.
By the early 1970s, with U.S. involvement in the Vietnam War winding down to its tragic climax, NSA found that it had to dramatically revamp itself in order to handle a series of new technological advances in the computer and telecommunications fields, which were threatening to severely impair the agency’s ability to perform its mission (Document 16). NSA also had to restructure its SIGINT collection efforts to encompass a host of new transnational targets, such as economic intelligence and monitoring the rising threat posed by international terrorism (Document 10).
Throughout its history, a series of catastrophic events have damaged NSA’s codebreaking efforts. One of the worst of these events occurred in January 1968, when North Korean naval forces seized the U.S. Navy spy ship USS Pueblo while it was engaged in an intelligence collection mission off the east coast of North Korea. Within weeks of the loss of the ship, many of NSA’s top intelligence sources in North Korea and elsewhere in the Far East suddenly went off the air (Document 3). After the seizure of the Pueblo, the North Korean government published an expose on the ship’s mission, with a few pages reproducing some of the captured documents (Document 24).
Click here for documents
Notes
1. This period in NSA’s history is well covered in Thomas R. Johnson, American Cryptology during the Cold War, 1945-1989: Book I: The Struggle for Centralization, 1945-1960 (National Security Agency: Center for Cryptologic History, 1995), Top Secret Umbra, http://www.gwu.edu/~nsarchiv/NSAEBB/NSAEBB260/index.htm
2. A good general overview of the role played by SIGINT in the Korean War can be found in David A. Hatch and Robert Louis Benson, The Korean War: The SIGINT Background, U.S. Cryptologic History, Series V, Vol. 3 (Ft. George G. Meade: Center for Cryptologic History, 2000), Unclassified, http://www.nsa.gov/public_info/declass/korean_war/sigint_bg.shtml. For an example of a major SIGINT success during the Korean War, see Jill E. Frahm, So Power Can Be Brought Into Play: SIGINT and the Pusan Perimeter, U.S. Cryptologic History, Series V, Vol. 4 (Ft. George G. Meade: Center for Cryptologic History, 2000), Unclassified, http://www.nsa.gov/about/cryptologic_heritage/center_crypt_history/publications/sigint_and_pusan_perimeter.shtml.
3. The most detailed single-volume account of NSA’s successes and failures during the Vietnam War can be found in Robert J. Hanyok, Spartans In Darkness: American SIGINT and the Indochina War, 1945-1975 U.S. Cryptologic History, Series VI, Vol. 7 (Ft. George G. Meade: Center for Cryptologic History, 2002), Top Secret/COMINT/X1, http://www.nsa.gov/public_info/_files/cryptologic_histories/spartans_in_darkness.pdf.
