Leveraging Data Science to Investigate Intelligence Failures

Abstract

This article challenges the conventional assumption underpinning the ‘First Law of Intelligence Failure’ – that warning signs are always available, but ignored, prior to intelligence breakdowns. Employing advanced natural language processing and machine learning techniques, the authors analyse declassified US State Department cables from the 1970s, focusing on two case studies often deemed intelligence failures: the Soviet invasion of Afghanistan and the Iranian Revolution. Using semantic outlier and change-point detection algorithms, they test whether meaningful signals (‘signal in the noise’) or emergent patterns (‘connecting the dots’) were more prevalent prior to failure than in earlier, ‘successful’ periods. The study finds this is not consistently the case, suggesting that indicators are not uniformly available or discernible before failures occur. By demonstrating the limitations of this study, the article concludes that the binary framing of intelligence as either success or failure is analytically flawed and potentially misleading. It offers a proof-of-concept for applying data science to intelligence analysis and advocates for a more nuanced understanding based on baselines and deviations, rather than retrospective judgements shaped by hindsight.