The Black Door: Spies, Secret Intelligence and British Prime Ministers

Jack Straw, one of the most prominent cabinet ministers of recent years, recalls his initial security vetting when he first joined the government. ‘A man in a mac, with a skin disease which meant he could not shake my hand … came to interview me … for three hours.’ Later the same man came back for another three hours, and in this second interview he suddenly leaned across the desk, looked his subject in the eye and asked, ‘Mr Straw, do you like men?’ This reflected the fact that historically, several people who had spied for the Soviets had been trapped by sexual blackmail. In 1974, when Straw was an adviser at the Department of Health and Social Security, his file was already two inches thick, and from their questions it was clear that MI5 had been collecting material on his family members since he was fifteen. Straw reflected on the scale of the surveillance operation that this implied. But, he recalls, this ‘neither surprised nor shocked me’. He saw it as part of everyday life on the strange planet that was Cold War Britain, where the KGB had to be kept at bay. Years later, when Straw became home secretary, MI5 was nervous that he might wish to see his own file. But he did not request this privilege, taking the view that he had no more right to see this secret material than any other citizen – and he gave the same response to Peter Mandelson, who was characteristically eager to peruse his own MI5 dossier.17

In the 1970s, Downing Street also lived in the shadow of Watergate, and was engulfed by growing paranoia about grand political conspiracy. Plots and bugging seemed to be almost normal. In March 1976, during the last days of the Wilson government, Tony Benn, then secretary of state for energy, attended a reception at the American embassy at which he chatted amiably to Cord Meyer, head of the CIA station in London. They reviewed the continuing fallout from Watergate, and Benn offered the opinion that Nixon was in fact quite charming, and that the media had been ‘unfair to him’. Meyer countered that Nixon was a ‘terrible man’, and had done a lot of damage to his service. But Benn took the wider view that given the catalogue of human error that went with the political experience, ‘bugging your opponent wasn’t so bad’.18 A year later, James Callaghan and the cabinet secretary were debating whether Benn himself could be trusted to see intercepts and ‘sigint’ material from GCHQ.19

Labour prime ministers also worried about plots. Ramsay MacDonald famously feared that MI5 was working against him. During the 1930s, Neville Chamberlain employed a former MI5 officer, Joseph Ball, to spy on both the Labour Party and his own rivals within the Conservative Party, using human agents and telephone taps. The Harold Wilson government rightly feared that a number of secret elements – both domestic and foreign – were seeking to destabilise his regime. For reasons yet unknown, Harold Macmillan had insisted that Downing Street be wired for sound to allow recording of conversations, much in the same style as the Kennedy White House. The extent to which this was used or abused during his time in office and subsequent administrations remains a mystery. The sensitivity of this subject was such that the Cabinet Office insisted that all references to it be cut out of what was otherwise a remarkably candid authorised history of MI5 published in 2009.

Harold Wilson was not the only senior figure who feared bugs. In 1973, in the final days of Edward Heath’s Conservative government, William Armstrong, the head of the home civil service, turned up in the Cabinet Office and demanded to speak to the cabinet secretary somewhere that was ‘not bugged’. Ushered into a suitably secure room, he took off his clothes, lay on the floor chain-smoking and talked ‘very wildly’ about the whole system collapsing and ‘the world coming to an end’. The next day he summoned a meeting of all Whitehall permanent secretaries and told them to prepare for ‘Armageddon’. ‘He was babbling incoherently’, and was ‘taken off to hospital for treatment’. He later spent a month recovering at Lord Rothschild’s private villa in Barbados. Although Wilson said almost nothing about the security services in his first volume of memoirs, penning a bizarre chapter of just two and a half pages on the subject, he later said much more to the press. Cabinet Office officials were by turns amused, embarrassed and then dismissive when these stories first appeared. But by 1979, evidence of interference by South African intelligence in London was mounting, and officials gradually came to accept that there was real substance to Wilson’s fears.20

Wilson’s public ramblings on intelligence required a new approach by Downing Street. Since the early 1960s, prime ministers had been forced to confront a new era of exposure. Now, a decade later, they had begun to manage their own gentle counter-offensive. Encouraged by the cabinet secretary, Burke Trend, with whom Wilson had enjoyed a good relationship, the intelligence agencies initiated a deliberate policy of emerging from the shadows. The Cabinet Office presided over the writing of the official history of wartime intelligence, and approved the release of the papers of Bletchley Park. This project was expressly about intelligence at the top, and traced the interaction between secrets and high strategy. Throughout the 1970s, British prime ministers were feeling their way towards a more public profile for the security agencies and their own engagement with them, pondering the possibility of public avowal of their existence and activities.

Margaret Thatcher hated this. Even as opposition leader in the late 1970s, she had repeatedly attempted to veto any public revelations about intelligence, however carefully controlled. Once she entered Downing Street, she was immediately confronted by the media frenzy surrounding the revelation that Anthony Blunt, Surveyor of the Queen’s Pictures, was the ‘Fourth Man’. Tasked with explaining the decision not to prosecute him, she went against her instincts and, rather than saying as little as possible, made a detailed statement to the House of Commons. She soon regretted it. Each morsel of detail was picked over, and seemed to draw out further press revelations. Now confirmed in her personal commitment to absolute secrecy, Thatcher’s later years were partially defined by the absurd battle with MI5’s supreme mole-hunter Peter Wright, whose Spycatcher memoir she sought to suppress. Her cabinet secretary endured public humiliation in the Australian courts in a futile attempt to hide what was already in the public domain.

By contrast, John Major found himself surrounded by modernisers. Not only were the intelligence chiefs of the 1990s happy to adopt an avowed legal identity, they were also keen to emerge from the shadows. In 1994 the new director-general of MI5, Stella Rimington, gave the BBC Dimbleby Lecture on ‘Security and Democracy’ – despite the intense unease of Home Office officials. MI6 opened an extraordinary new building at Vauxhall Cross, on the south bank of the Thames, that looked like something out of science fiction. Not to be outdone, GCHQ then commissioned its own new headquarters that looked like a flying saucer, and became the first British agency to have a presence on the internet. Yet the press still seemed determined to turn success into scandal. In 1999, when MI6 revealed the treasures of the Mitrokhin archive, compiled by a KGB archivist who had given his secrets to Britain, surely one of the most magnificent intelligence successes of the late Cold War, the press instead focused on the failure to prosecute some of the espionage ‘small fry’ this had revealed. Government spin doctors puzzled over how MI6 had managed to turn a golden story to ashes, and Duncan Wilson of the Cabinet Office agreed that the secret services now needed assistance from Downing Street in managing their public image.

Tony Blair exuded confidence in all areas of public affairs, including secret service. When asked about MI6 operations in Moscow he quipped to a waiting band of journalists, ‘We never comment on intelligence matters … except when we want to, obviously.’21 Even before 9/11, his public relations team were fascinated by the media operations of MI6 and the interface between openness and secrecy. After the attacks on the World Trade Center in New York, secret matters were on the front pages of the newspapers each week during a decade-long ‘War on Terror’. This raised important questions. To what extent should prime ministers place intelligence in the public domain to explain policy? And how should this be done? In two notorious Downing Street dossiers, Tony Blair and his press secretary Alastair Campbell did this badly, and paid a considerable price in terms of trust and public confidence. In the background, senior MI6 officers, some of them modernisers like Richard Dearlove and some old-school types like John Scarlett, argued over issues of public image and the proximity of their own service to Number 10. None of them had any idea how disastrously this episode would explode in their faces.

Yet David Cameron retraced Blair’s footsteps. In early 2013, faced with the issue of the use of chemical weapons against its own people by the government of Syria, Cameron released JIC material to the press, going even further than Blair in the public use of intelligence. He realised that prime ministers must balance intelligence’s inherent need for secrecy with their public duties leading the government. If this was an effort to reassert control and confidence, it surely failed. After Blair’s Iraq fiasco, the public were deeply sceptical about intelligence as a justification for pre-emption, and Parliament voted against military action in Syria. Moreover, Downing Street was about to discover that it did not have control over what became public in this realm. Even as it agonised over the release of a two-page letter from Jon Day, the chair of the JIC, to Cameron, an unknown figure called Edward Snowden was preparing to release a deluge of secret material that amounted to thousands of documents. Much of this was British, and each remarkable revelation was more eye-wateringly secret than the last.22

Cameron said almost nothing about Snowden. This was because much of the material shone a harsh light on the personal interaction between premiers and espionage. In a world in which diplomacy has become increasingly personal, characterised by G7 summits and mobile-phone calls, spies and statesmen rub shoulders with increasing frequency. For wealthy and powerful states, Snowden’s revelations were a story of hacked emails and rude words. GCHQ was specifically accused of hacking into the Belgian telecoms firm Belgacom, which includes EU institutions as clients. Cameron refused to answer questions about whether he had been able to reassure allies that British intelligence had not been involved in any bugging.23 In fact, this was hardly news. Generations of British cabinet ministers have been amazed by the scale of Britain’s surveillance of its European partners.24 But what Snowden, and previously Chelsea Manning, also revealed was the jostling of less powerful states, such as Chile and Mexico, which suffered not only stolen briefcases, but also bullying and bribery by the West. This rather raw business of personal espionage has now been exposed, and to the surprise of many, we have realised that James Bond often takes his orders in these delicate matters direct from the prime minister in Downing Street, rather than from ‘M’ in MI6 headquarters.

In the summer of 2007, the British embassy in Baghdad received five bloody index fingers through the post. They were claimed to be the body parts of five British citizens who had recently been kidnapped by a pro-Iranian offshoot of Hezbollah in Baghdad. On 29 May that year, Shia militants disguised as policemen had conducted a violent raid on the Iraqi Ministry of Finance. Their main target was Peter Moore, a British consultant who had been installing software designed to prevent fraud surrounding the billions of dollars of aid flowing into the new government’s accounts. Downing Street was determined to secure Moore’s release. SAS and MI6 representatives attended four separate cabinet-level security conferences to plan his rescue with ministers, and as a result special forces conducted more than two dozen house assaults seeking persons involved in the kidnap. Gordon Brown came within minutes of ordering a raid on the kidnappers, but the intelligence on their location was judged to be too thin. Eventually, in December 2009, after 946 days of captivity, Moore was swapped for Qais Khazali, a Shia militant leader, and his brother, who had been seized by the SAS on the streets of Basra at around the same time he was kidnapped. The negotiations were carried out by an MI6 officer operating in Baghdad. The other four Britons, who had served as his security guards, had all been executed the previous year.25

The dirtiest diplomacy is talking to terrorists, kidnappers and insurgents. This has to be deniable, since even if governments are willing to admit their part, terrorists frequently are not, and all parties ultimately fear being perceived as ‘weak’. This can place prime ministers in a difficult position, potentially forcing them to mislead the House of Commons. Harold Wilson kept his MI6 back channel with the IRA incredibly secret – many in his own cabinet did not know what was going on. Similarly, Margaret Thatcher publicly insisted that Britain did not talk to terrorists, and yet, as under her predecessors Wilson, Heath and Callaghan, various back channels remained open. In fact, Thatcher was personally involved in some exchanges. She made handwritten comments on one statement sent to the IRA as part of negotiations over the hunger strikers in 1981, and personally amended and approved Britain’s negotiating position.26 Her successor, John Major, faced a similar dilemma. Despite stating in the House of Commons that he did not negotiate with terrorists, he had approved precisely that, and his emissaries were secret service officers whom the IRA regarded as ‘untouchable’. But when the secret talks leaked, Major and the IRA descended into a welter of accusation and counter-accusation, with each side publicising its version of the hitherto secret communiqués.

Bizarrely, even as Thatcher and Major talked to terrorists, they were also their top targets. Both came within inches of being eliminated by the IRA, and one of the darker stories of intelligence, security and 10 Downing Street is the ever greater level of protection required to prevent assassination. Margaret Thatcher, characteristically, had just finished rehearsing a speech with her private secretary, Robin Butler, at ten to three in the morning when a bomb destroyed a large part of the Grand Hotel in Brighton during the 1984 Conservative Party Conference. One minister recalled the security shambles in the immediate aftermath, as injured politicians made their way out of the rubble with barely a policeman in sight. Two IRA gunmen could have ‘got the whole Government as they blearily emerged’, and made their getaway unimpeded. The IRA also purposely eliminated two of Thatcher’s closest advisers, with the assassinations of Airey Neave in 1979 and Ian Gow in 1990, bracketing her arrival and her departure from office.27

As a result, Downing Street was turned into a fortress. Large reinforced black gates prevented public access. Harold Wilson had resisted elaborate gates when they were suggested in 1974, but now they arrived, together with armed policemen.28 Several levels below Downing Street, technicians were putting the final touches to ‘Project Pindar’, a command bunker deep beneath the Ministry of Defence on Whitehall to which selected denizens of Downing Street could retreat in times of peril, at a cost of £126.3 million.29

By 2005, threats against the life of the prime minister had become an almost daily occurrence. The intelligence warnings of assassination attempts often landed on the desk of David Blunkett, the home secretary, who observed with delightful sangfroid that these plots did not worry him unduly – unless they related to an event at which he was likely to be sitting next to Tony Blair. Towards the end of Blair’s administration, simply moving outside the Whitehall government complex became a problem. Alastair Campbell gazed at the vast prime ministerial convoy of armoured vehicles proofed against nerve gas, the motorcycle outriders stopping the traffic amid a cacophony of wailing sirens, and wondered how many votes were lost every time this baroque parade of vehicles took the prime minister to the airport.

The personal threat to the prime minister is at its worst overseas. In October 2001, Blair set off on a trip to Russia, India, Pakistan, Saudi Arabia and finally Egypt. The BBC ignored security guidance and announced the prime minister’s destinations to the world, thereby increasing security concerns. As a result, Blair’s entourage had to be ‘surrounded by outriders with machine guns and tanks carrying anti-aircraft missiles’.30 On their return to the UK, Blair’s team told the BBC director of news that Number 10 was ‘exasperated by their continued reporting of his movements’.31 Blair remains a high-profile target. In 2014, MI5 planted a listening device in the car of one terrorism suspect and listened in on his conversations for two weeks after searching the vehicle and finding Blair’s home address on a scrap of paper folded inside a Versace glasses case.32 MI5 had previously established covert surveillance of Ishaq Kanmi, the self-proclaimed leader of al-Qaeda in Britain. His stated aims included ‘the elimination of political leaders and capitalists Blair and Brown’.33 In the twenty-first century, British prime ministers value their intelligence and security services partly because they keep them alive.

No less worrying are the threats to visiting heads of state. Each premier on a state visit to Downing Street, whether from Russia, Israel, Sri Lanka or Saudi Arabia, has brought in his wake an exotic trail of would-be assassins. The grim prelude to every visit is the backstairs diplomacy of death. As the leaders make their way around London, they are shadowed by ambulances carrying copious supplies of their blood group. The most vexed discussions concern the intelligence precautions involved in each visit, with every drain and culvert along the routes taken by visiting dignitaries being searched for explosives. Defensive weaponry is an issue, with each American president asking to bring with him an increasingly formidable array of automatic weapons and ground-to-air missiles. George W. Bush was denied permission to bring an SUV containing an M134D mini-gun in an armoured pop-up turret. MI5 routinely regard the US president’s secret service as a more dangerous threat to the British public than potential assassins.34

Death has hovered over Downing Street for more than a century. While attacks and plots have become steadily more serious in recent years, enterprising amateurs have always abounded, and earlier prime ministers were more accessible to the public. On 28 August 1939, panic swept through a crowd of protesters immediately outside the door of 10 Downing Street when a London clerk called Laurence Hislam opened a small suitcase and scattered its contents, which resembled the deadly devices beloved of bearded bomb-throwing anarchists. Mayhem ensued, until the crowd realised that the ‘bombs’ were actually large black rubber balls inscribed with the message ‘Peace Conference Now’. Hislam was in fact a pacifist protesting against the mounting international crisis and the spectre of another major war in Europe. The police led him away, and despite his defence that he acted ‘in the cause of peace’, the court sentenced him to six months’ hard labour.35

No prime minister illustrates this interface between premiers and personal hazard more than Churchill. Britain’s wartime leader accepted the advice of the Special Operations Executive that assassinating Hitler would be counter-productive, since his strategy was increasingly incompetent and damaging to Germany. However, in 1943 Churchill did approve an assassination attempt against Mussolini, with female agents being dropped into Italy, together with aggressive kill missions against Rommel and other senior German generals. Meanwhile, lesser fascists were earmarked for mere bribery, with an initial £100,000 personally approved by Churchill and then passed over in a large bag to members of Franco’s circle on a Spanish golf course as an inducement to keep Spain out of the Second World War.36 Eden was appalled by the operation, but when more money was required for Franco, Churchill eagerly minuted in red ink: ‘Yes Indeed. WSC.’37

Britain’s wartime leader was oddly relaxed about his own personal safety, despite more than a dozen serious attempts on his life. Hitler certainly ordered his multiple secret services to target Churchill, the oddest such operation being a plot to attack Allied leaders at the Tehran Conference in 1943 using a team of Nazi agents transported by camel.38 Much of Churchill’s remarkable ability to survive the attempts on his life must be attributed to good fortune and his ever-vigilant bodyguard, Detective Inspector Walter Thompson. Occasionally Churchill insisted on carrying his own revolver, but for the most part brazen and carefree with only a single bodyguard, he travelled over 200,000 miles across Europe, America and the Middle East in the course of the war. No British prime minister will ever do that again, so in many ways Churchill stands at the turning point of security and intelligence in Downing Street, representing both the first of a new wave of premiers who exploited intelligence properly, but also the last of a dying breed.

Twenty prime ministers have led Britain since 1909. From Herbert Asquith to Theresa May and from Ramsay MacDonald to Gordon Brown, these premiers have held diverging political views, possessed wildly different leadership styles, and have confronted the full spectrum of problems, threats and crises. This book examines each prime minister in turn. It traces their personal approaches to intelligence and uses each premiership as a vehicle to explore the most pressing national security issues of the day. It reveals that despite the vagaries of personality and politics, intelligence has become increasingly central to all prime ministers. This was not always the case. Back in the first decade of the twentieth century, the secret world was alien to Asquith. Churchill and Attlee addressed this problem together, and forged a quiet revolution. Now, as a result, the prime minister presides personally over an organised intelligence community operating at the heart of Whitehall. This book traces that secret service journey from the periphery to the centre of power.