The most prickly area of alliance relations was the business of cypher security, which protected the secrecy of diplomatic communications. Foolishly perhaps, at a meeting in London in May 1947, the British launched an audacious bid to persuade the Americans to share the innermost secrets on matters of their code-making. The discussion revolved around the replacement of the Combined Cypher Machine, which had been developed for inter-allied communications during the war, but was now thought obsolete and vulnerable. The British were also keen to replace their own national machine, the Typex, but were desperately short of money as a result of post-war austerity, and argued that for reasons of economy any new cypher machines should be capable of inter-allied use, and proposed joint research and development with the Americans.[38] The Americans were startled: it had been a cardinal principle never to share the secrets of their unique and highly prized Sigaba machine. Hoping to overcome this psychological barrier, the British revealed that they were in fact already knowledgeable about Sigaba. They not only described its inner workings ‘quite accurately’, but confessed that they had ‘incorporated its principles in a radioteletype machine for their own use’. Hoping that they had pushed the Sigaba obstacle aside, the British then made their pitch. They claimed that they had developed an approach to cypher machines that was ‘new and revolutionary’, and ‘superior to the Sigaba principle’. They were happy to share this with the Americans, and perhaps make use of it in joint machines that might be developed for both national and allied use.

Far from being reassured, the Americans were horrified. Discussion had to be ‘temporarily discontinued’ while they withdrew to confer amongst themselves. The US Army could see no objection to releasing the Sigaba principle for use in a combined allied machine, since the British had clearly unravelled it. However, the US Navy offered ‘serious objections’, and used their veto. Thus the British were told that Sigaba had to be completely eliminated from the discussions. At this point they revealed their ‘new and revolutionary idea’ for future cypher machines, only to find that their American colleagues sneered at it and dismissed it as ‘impractical’ on engineering grounds. The two sides parted without agreement.[39]

While GCHQ was overawed by the scale of American sigint resources, matters looked quite different from Washington. With the Second World War now over, and an economising Republican Congress controlling the federal purse-strings, resources for American comint interception activities were remarkably tight. This contributed to American under-preparedness prior to the Korean War. It also enforced a division of labour between GCHQ and the Americans, and prevented American sigint from expanding its activities in Europe in the way it had hoped. In 1949, US Army Security Agency interception units in Europe were still passing much of their product to GCHQ for analysis, rather than back to Washington. Moreover, GCHQ retained primary responsibility for areas such as Eastern Europe, the Near East and Africa.[40] Because of this division of labour, the late 1940s saw the gradual development of American and British spheres of influence. In Scandinavia, for example, relations with Norway were an American responsibility, while those with the Swedes belonged to GCHQ, although this demarcation was not always strictly adhered to.[41] GCHQ enjoyed the additional benefits of the panoply of bases provided by Britain’s imperial and post-imperial presence. Although the Empire was shrinking, the very process of retreat and the euphoria of independence often rendered the new successor states willing to grant limited base facilities to the departing British. These ‘communications relay facilities’ may have seemed innocuous, but in fact many countries were unwitting hosts to important GCHQ collection sites.[42]

The outbreak of the Korean War early on the morning of Sunday, 25 June 1950 took Britain and the United States by complete surprise. Although they had comint units in locations such as Hong Kong and Japan, their main focus was Russian traffic, and their sigint capabilities against North Korea were non-existent. The NSA official history notes that there was ‘no person or group of persons working on the North Korean problem’, and even had they done so, they had ‘no Korean linguists, no Korean dictionaries and no typewriters’. Although the CIA had picked up what might be called ‘rumours of war’ from human agents, there was no high-profile attack warning delivered to policy-makers. During the first few weeks of the war, the Americans and their South Korean allies suffered serious reverses and were almost overrun. Sigint helped the Americans to beat back the attacks on their rapidly shrinking perimeter by providing excellent tactical intelligence, but they blundered again by missing the entry of the Chinese into the war in October 1950.[43]

The Korean War resulted in a headlong expansion of American sigint. More than two thousand additional staff were recruited, and more than $5 million of additional spending on comint and comsec was authorised within weeks of the war commencing.[44] The outbreak of the war also meant crash expansion in Asia. The Americans informed London of their ‘urgent need’ for a US Air Force sigint unit to be deployed to Hong Kong, and other sites were quickly developed on Taiwan in an attempt to remedy the yawning intelligence gap in East Asia.[45]

Korea had another important impact. With new employees flooding into the training wing at Nebraska Avenue in central Washington DC, the Americans soon had a vast backlog of people requiring security clearance before they could begin work. By the end of 1950, more than a third of sigint employees were ‘uncleared’. It was then discovered that since 1948 the CIA had been using the polygraph, or lie detector test, initially only to screen people who had access to sigint, although its use was soon extended to all CIA employees. By May 1951 it had been adopted for all American code-breakers, and polygraph examiners were testing ‘from seven in the morning till eleven at night’ to clear the backlog. Polygraphs soon became an embedded part of American sigint culture, but were not introduced at Britain’s GCHQ.[46]

The Korean War was of enormous importance for GCHQ because it fundamentally reshaped the American sigint community. There had been several failed attempts to create a single unified American sigint organisation along the lines of GCHQ.[47] The war broke the logjam. In 1952, President Truman suddenly insisted on the creation of a strong central body called the National Security Agency, or ‘NSA’, under General Ralph Canine. The armed services fought a desperate rearguard action: in August 1952, General Samford of US Air Force intelligence denounced Truman’s desire for ‘strong central control’ as nothing short of a ‘major error’. However, Truman’s mind was made up, and in November he signed the order for the reshaping of American comint.[48] NSA was given unambiguous control over comint in a historic document called NSCID-9.[49] This brought about a reduction in, but not the elimination of, what the leading historian of NSA has called ‘the fractious and seemingly never-ending internecine warfare’ between the American service comint organisations. The British were immensely relieved. In the background, figures like Edward Travis had been quietly urging inter-service unity on their American collaborators since the summer of 1945.[50]

The creation of NSA also had physical consequences. Up until this point there had been a plan to relocate the headquarters of American sigint to Fort Knox, near Louisville in Kentucky. However, it was now realised that the need for high-grade communications circuits and for civilian workers made this impossible. The policy-makers, who were the consumers of their intelligence ‘product’ in Washington, also protested about the move, rightly anticipating that it would mean a worse sigint service. They insisted that the new headquarters be within a twenty-five-mile radius of the Washington area.[51] On 3 November 1952, Fort Meade on the northern edge of Washington’s Beltway was designated the likely new headquarters for NSA under a secret programme entitled ‘Project K’. Over the next five years this location would become the headquarters of the world’s largest, most expensive and most secretive intelligence agency.[52]

Britain’s GCHQ had made precisely the opposite decision. In 1952 it moved away from its suburban site at Eastcote on the perimeter of London to a comparatively distant location at Cheltenham in Gloucestershire. Although GCHQ had only finished the move to Eastcote in June 1946, by April 1947 it was already looking for a new home. This was partly because of the physical limitations of the Eastcote site, and also because Travis realised that in any future war there would not be time to relocate to a safe place like Bletchley, away from Soviet bombing. Several possibilities were considered, and in October 1947 GCHQ scouts had found promising twin sites at Oakley Farm and Benhall Farm, near Cheltenham, which were occupied by the Ministry of Pensions. These single-storey temporary office complexes had been built in 1940, initially for the possible evacuation of government from London during the Blitz. After 1942 they were used for the logistical organisation of the US Army in Europe. The wartime presence of the Americans was the key, since it had left a helpful legacy of improved trunk cable communications.[53]

An alternative explanation for the choice of Cheltenham is offered by Professor R.V. Jones, one of the famous architects of the ‘Wizard War’ which deployed British science against Nazi Germany. Jones served as scientific adviser to GCHQ in the early 1950s, and recalls that the scout who initially found the Cheltenham site was Claude Daubney, one of the senior GCHQ staff who liaised with the Y Units of the armed services, and who spent much of his time in Whitehall. Daubney was a typical RAF officer of the thirties, handsome and ‘heavily moustached’. His main relaxation was betting on horseraces, and he argued endlessly with Jones about the theoretical possibility of beating the bookmakers. He ‘deliberately chose Cheltenham, as he told me, so that he could combine visits from his London office to GCHQ with attendance at Cheltenham races’.[54] Others support the idea that GCHQ was attracted to Cheltenham by its proximity to the racecourse.[55] Whatever the truth of the matter, the course would serve as an occasional helicopter landing pad for future visits by British Foreign Secretaries.[56] The move from Eastcote took place between 1952 and 1954, but the rapid growth of GCHQ during the Korean War meant space was tight even at the new location. This contributed to the decision to constitute comsec as a separate organisation called the London Communications Security Agency. Its staff either moved to central London offices in Palmer Street, or stayed at Eastcote.[57]

The creation of new headquarters for GCHQ and NSA coincided with further additions to the transatlantic sigint agreements. In March 1953, Clive Loehnis from GCHQ led a team that spent fifteen days in Washington discussing closer agreements between the two sigint leviathans. This included a new team of American liaison officers based at Cheltenham called ‘USGRO’, who would sift through GCHQ’s intelligence product looking for material of interest to a variety of customers in Washington, including the CIA, the FBI and the armed services. The ‘USGRO’ team lived with GCHQ’s Z Division, who looked after British sigint customers in Whitehall in a similar way. From 1953, the term ‘UKUSA’ agreement entered common usage.[58]

In its move to Cheltenham, GCHQ created a unique intelligence town. It brought with it Bletchley Park’s formidable reputation for secrecy. Within a decade almost everyone in Cheltenham had a family member or friends who worked at GCHQ, but nobody talked about what they did. The town welcomed the ‘Foreign Office types’ with open arms, and GCHQ was soon contributing an enormous amount to its intellectual and artistic life, quite apart from being its biggest employer. The GCHQ staff were also sporty, providing most of the players in the Foreign Office football team that won the Civil Service Football Cup in 1952. This could present some peculiar problems. When local reporters covered matches in Cheltenham, they were told they could name the goal-scorers of the visitors, but not of the local team. Reporting these games tested their copywriting skills to the very limit.[59]

The previous ten years had been a formative decade for GCHQ. By 1944, the code-breakers of Bletchley Park had made themselves Britain’s premier intelligence service. Out of the chaotic brilliance housed in a few wooden huts in the Buckinghamshire countryside came one of Britain’s most forward-looking and innovative organisations. Another crucial legacy of the late 1940s was the agreements with the United States and the Commonwealth that laid the foundations of UKUSA, a worldwide sigint alliance, agreements that are still in force today. Anxiety about Moscow had been a driving force behind these agreements, and even before the Second World War was properly over, the Western allies had been paying increased attention to Soviet cyphers. Despite the triumphs of Venona, and the uncovering of key KGB agents like Klaus Fuchs and Donald Maclean, Moscow’s higher-level communications remained mostly unbreakable after 1948. Accordingly, GCHQ and its partners were already searching for new kinds of intelligence-gathering to use against the Soviet Union, opening up a whole new vista in the electronic war.[60]

THE 1950s

6
‘Elint’ and the Soviet Nuclear Target

Our intelligence about Soviet development of atomic weapons is very scanty.

Joint Intelligence Committee, 29 October 1947[1]

In late August 1949, Lavrentii Beria, chief of the KGB, arrived at a small settlement on the steppes of Kazakhstan, not far from the city of Semipalatinsk. Here, Soviet scientists were hard at work in a set of temporary laboratories, intently focused on what they obliquely called ‘The Article’. They were referring to the Soviet Union’s first atomic bomb, now situated precariously on top of a scaffolding tower fifty miles away. Houses, locomotives, buses and even tanks, together with some unfortunate farm animals, had been placed close to the weapon to gauge the effects of an explosion. The Soviet Union’s chief nuclear scientist, Igor Kurchatov, gave the command to detonate, and a small but incredibly bright light appeared at the top of the tower. Suddenly it became a white fireball. A blast wave swept out, clearing everything in its path, as the explosion itself rapidly turned into a chaotic mix of orange, red and black. A dark mushroom cloud, five miles high, formed over the test site. Back in the laboratories, the scientists were jubilant and kissed each other on the foreheads. A month later, nineteen key figures from the nuclear programme, including a German scientist, were made Heroes of Socialist Labour. Beria is reported to have used the same list of names that identified those who would have been shot immediately had the test failed.[2]

In the event, it was Western intelligence that had failed. Soviet progress towards a nuclear weapon had been a top intelligence target. The predictions of Britain’s Joint Intelligence Committee, the highest authority for analysis, had actually become steadily less accurate. By 1949, when the test took place, Britain’s top intelligence analysts were arguing that the probable date of the first Soviet test would be mid-1953. They were adrift by no less than four years. The CIA was no more accurate, and had similarly told President Truman that mid-1953 was the most likely date. Shocked by the surprise atomic test, Britain and America now redoubled their intelligence effort in the field of Soviet strategic weapons.[3] Throughout the Cold War, the key target for GCHQ would remain Soviet nuclear weaponry. This included not only the atomic bomb programme, but work on ballistic missiles, bombers and other means of delivery. The Chiefs of Staff were worried by Britain’s relative vulnerability to nuclear attack, and wanted intelligence forecasts on this crucial issue. In their list of ‘sigint targets’ for 1948, the JIC exhorted Britain’s code-breakers to focus their efforts on this area, together with parallel strategic threats such as chemical and biological weapons.[4] Other Soviet activities, including KGB espionage and diplomatic initiatives, only constituted GCHQ’s second and third priorities.

But ever since the massive revision of Soviet cypher procedure on ‘Black Friday’ in 1948, GCHQ had been having a hard time with its main target. The one-time pads employed for the highest-grade Soviet messages were now being correctly used, and so could not be broken, and machine cypher procedure on systems like Taper, effectively a Soviet military version of Enigma, had also been tightened up. Moreover, Moscow and its satellites enjoyed common borders and so often used landlines instead of wireless transmissions, which could not be easily intercepted. All this eventually prompted the British to follow the Soviets down the path of more extensive physical bugging of diplomatic premises in the mid-1950s.[5]

GCHQ was nevertheless providing Whitehall with large quantities of useful material on lower-priority issues. It continued its long tradition of attacking the communications of smaller states like France, Turkey and Egypt. The JIC had also asked it to look at subjects such as Arab nationalism and the relations of Arab states with Britain and the USA, and the attitude of France, Italy and the Arab states to the future of North Africa, especially Libya. Because of the ongoing insurgency in Palestine, GCHQ was also urged to focus on the Zionist movement, including its various intelligence services. All of these proved more accessible than Soviet traffic.[6] The diplomatic traffic of smaller states also provided an excellent window on the Soviets. Conversations between Soviet diplomats and the officials of these countries were often captured in telegrams sent from Moscow that could be read with ease. In 1946 Alan Stripp, a British code-breaker who had spent the war working on Japanese codes, found himself redeployed to the Iranian border. Throughout the Azerbaijan crisis of that year, when the Soviet Union appeared to be behind a potential breakaway state in northern Iran, he worked on Iranian and Afghan communications.[7] This sigint revealed the scale of Soviet activities and ambitions in the region, helping to trigger robust counter-pressure by President Truman.[8]

How the main target lists for sigint were drawn up was of central importance. In theory, they were created by the JIC. However, much of the preliminary work was undertaken by more shadowy committees working under the London Signals Intelligence Board, Britain’s supreme sigint authority. These enjoyed strong military input. When diplomats complained that political and colonial subjects were not getting enough attention, they did not get an enthusiastic response. Many felt that undue weight was being given to defence priorities, and to nuclear warfare in particular. The overwhelming emphasis given to defence had profound implications for the shape of British sigint. It accelerated the development of a revolutionary new kind of sigint that focused on equipment and military formations.[9]

During the Second World War, Bletchley Park’s primary emphasis had been the interception of communications signals for intelligence purposes. However, as the war progressed, there was growing interest in another kind of sigint that was derived from intercepting electronic signals, such as radar, which had been developed during the Second World War. A related field of interest was the growing use of radio waves to create missile guidance systems. Examining these enemy radio signals was known as electronic intelligence, or ‘elint’. Elint revealed a great deal about enemy weapons, and was also essential for conducting ‘radio warfare’, which involved jamming enemy signals and radar. The first example of this had been the successful efforts of Professor R.V. Jones to divert the beams used to guide the German bombers attacking London. These techniques were refined during the war against Japan. An elaborate elint unit was set up within Mountbatten’s South-East Asia Command under the improbable cover name of the ‘Noise Investigation Bureau’. In the summer of 1945, elint-equipped aircraft called ‘ferrets’ patrolled the night skies over Rangoon listening to Japanese radar.[10] Elint and radio countermeasures, conducted jointly by GCHQ and special units of the RAF, were a massive growth area after 1945, partly because they were so closely linked to strategic weapons. One great advantage of elint was that it rarely required the reading of complex enemy codes.[11] Increasingly, the business of signals intelligence would consist of two branches, the familiar one of communications intelligence or ‘comint’ and the new one of elint.

Britain had excelled in the use of both comint and elint during the Battle of Britain, and later in bombing raids over Germany. One of the architects of this system was Arthur ‘Bill’ Bonsall, who would become Director of GCHQ in 1973.[12] This success had made a deep impression on American intelligence officers in Europe, who felt admiration and not a little envy.[13] In early February 1945, the US Army Air Force held a conference of all senior air intelligence officers (A-2s) across Europe, at which ‘every A-2 expressed his disappointment at our utter dependence on the R.A.F.’ in sigint matters. The US Ninth Air Force had deployed some very effective converted Flying Fortresses as airborne listening stations, but the British had controlled the flow of strategic sigint. The lesson was clear. Colonel Robert D. Hughes, Director of Intelligence for the Ninth Air Force, told Washington that he wanted his own air sigint units with control over sigint policy and sigint research: ‘We feel that you should demand, and organize under your control, for peace as well as war, an organization similar to that of the R.A.F. … Unlike other highly technical forms of intelligence, in which our American Air Forces have shared, we have continued to depend entirely on the R.A.F. for this level of work in “Y”.’[14]

Elint formed one of the closest parts of the Anglo–American sigint relationship during the immediate post-war period because it focused on the Soviet military target. Exchange on elint was not initially linked to the Allied sigint agreements reached at the end of the war, but in 1948 it was being brought within the growing body of Western intelligence pacts that formed UKUSA. GCHQ approached Washington with a proposal to ‘extend the present British–US Comint collaboration to include countermeasures, intercept activities and intelligence’ in the field of elint. This meant coordinated patterns of ‘ferret’ flights – effectively a division of labour – with the resulting intelligence being swapped ‘via Comint channels’.[15] By the 1950s, GCHQ had achieved control over elint in Britain, and so was managing relations with all the various American outfits in this field. This had meant redrawing GCHQ’s charter to include not only comint but also elint, something which had not pleased everyone. R.V. Jones, who was Director of Scientific Intelligence at the Ministry of Defence in the early 1950s, strongly resented losing this part of his empire; the benefits of having all activities superintended by GCHQ were nevertheless overwhelming.[16]

Anglo–American sharing was important, because elint was an expensive business. Many of the target Soviet signals were short-range and could only be collected from ‘ferrets’, which were effectively flying intelligence stations. Initially, the RAF was ahead in this new field. By 1947 a fleet of specially equipped Lancaster and Lincoln aircraft patrolled the East German border, monitoring Soviet air activity. This was complemented by ground stations at locations such as RAF Gatow in Berlin listening to basic low-level Soviet voice traffic. British ‘ferrets’ made adventurous forays over the Baltic in June 1948 and the Black Sea in September 1948. Remarkably, they were soon crossing Iran to reach the Caspian Sea, thus flying perilous missions close to the very heart of the Soviet Union.[17] On the ground, a British undercover team was also operating in northern Iran, monitoring Soviet radar in the Caucasus as well as Soviet missile tests at Kasputin Yar on the edge of the Caspian. The team conducting this work were posing as archaeologists, a favourite British cover for all sorts of intelligence work. Once a week they drove from the Iranian border with the Soviet Union to the British Embassy in Tehran to deliver their precious tapes.[18]

Early Western elint efforts in the air were spurred on by the knowledge that the Soviets had launched their own secret ‘ferret’ programme. In April 1948 an American radar station in Germany reported that it was being probed by ‘ferret’ aircraft, and in November a Soviet plane circled a US radar station at Hokkaido in Japan collecting signals for an hour, and then escaped without interception due to bad weather. Defectors also brought tantalising snippets. In May 1948 Baclav Cukr, General Secretary of the Czech Air Force Association, escaped to the West bringing knowledge of a group of Dakota-like planes at Zote airfield outside Prague. These mysterious aircraft were kept under constant guard in special hangars, and had ‘several special antennae on the outside’, a sure sign that they were elint collectors.[19]

During the war the vested interests of many different RAF commands had made it difficult to create a single coherent elint organisation.[20] Post-war rationalisation allowed the fusion of these elements. RAF Watton in Norfolk was selected as the new home of elint, and collected the remnants of many wartime units into 100 Countermeasures Group.[21] The result was a weird menagerie of aircraft which were one-off flying laboratories adapted for various special tasks. The mainstays were twenty ageing Handley Page Halifax bombers. There were also B-17 Flying Fortresses, Lancasters, Mosquitoes and Avro Ansons, together with an Airspeed Oxford and a Percival Proctor. The unit at Watton soon received some new Avro Lincolns, effectively updated Lancasters. All were stuffed with unique items of electronic listening equipment and primitive wire recorders for collecting voice traffic.

Christened the Central Signals Establishment, or ‘CSE’, Watton boasted a Signals Research Squadron, a dedicated sigint unit known as Monitoring Squadron and a Radio Countermeasures Squadron. The Avro Lincolns were the best aircraft available, and they were given over to radio countermeasures and radar jamming, since they would have to work closely with RAF bomber formations in any future war with the Soviet Union. The Lincolns would soon be fitted with the revolutionary new carcinotron, or ‘backward wave oscillator’, in effect an electronic gun that produced powerful microwaves of the same frequencies used by radar, giving them enormous onboard jamming power. By contrast, some of the other airframes used for listening were antique. However, it was a venerable Halifax from Monitoring Squadron that was despatched on sigint collection duty over the Soviet Zone of Germany during the early stages of the Berlin Blockade in 1948. This was the first sign of a possible ‘hot war’ between East and West, and GCHQ decided it was time to share the lessons of sigint more widely. For the first time lectures on radio countermeasures and tactical sigint were given to officers passing through the RAF Staff College at Bracknell. A handbook on tactical sigint was prepared for all staff officers, albeit no mention was made of the mysterious ‘Ultra’.[22]

In 1951 the monitoring aircraft were rechristened 192 Squadron, and worked ever more closely with GCHQ. Meanwhile the radio countermeasures and jamming unit was rebranded as 199 Squadron. The RAF received four Boeing RB-29 ‘Washington’ aircraft, which were really American B-29 Superfortresses modified for listening. Their vast internal space allowed additional sigint equipment to be fitted by the sigint ground engineers at RAF Watton, who were known as the Special Radio Installation Flight, or ‘SRIF’. In 1953 two English Electric Canberras were acquired and refitted for secret sigint operations by SRIF. Their standard duty was flights along the borders of the Warsaw Pact, alternating with longer visits to the Baltic and the Mediterranean. Training of special operators was undertaken on slow but reliable Vickers Varsity aircraft.[23]

Beneath the sea, an even more sensitive sigint programme was under way. Much of what London and Washington knew about the Soviet Navy had been derived from captured German intelligence material harvested from Berlin in 1945, or from what the British had gleaned directly from their surprisingly good relations with the Soviet Navy during the war. However, this information was now outdated. The US Navy decided to send two submarines into the Bering Sea to test the possibility of undertaking listening operations off the major Arctic ports used by the Soviets. These successful pilot operations were limited to an investigation of the area using sonar. A much more ambitious mission was then attempted. This was a proper sigint collection operation, designed to scoop the signals that emanated from regular Soviet missile tests in the Barents Sea.[24]