A very interesting look at classified government information and how it is (or isn't) entering into the public domain. ( )

The premise of this book is promising, alluring, and troubling at the same time--using the tools of data analysis to get at government secrets. To be honest and up front, I was part of what the author calls "the dark state" for 23 years, working in declassification at the National Archives. So my review reflects that experience.

"The Declassification Engine" was published by Pantheon Press in 2023. Filling a total of 540 pages in my Kindle edition, the book begins with a preface ("Should This Book Be Legal?") and an introduction (the author calls it a "reintroduction". Connelly then lays out his case in 10 numbered chapters, a conclusion, followed by acknowledgements, endnotes, a list and links to various archives and databases, and an index.

The author begins the book with a "reintroduction" that serves as a history of how the United States handled its secrets prior to World War II. It is here that Connolly produces the first of his data analysis and injects the topic of social justice into the discussion. Chapter 1, Pearl Harbor: the Original Secret, follows a familiar and well-trod path of conspiracy that brought an unwilling United States into World War II. Chapter 2, The Bomb: Born Secret, dives into the development of the secrecy culture that enveloped the Manhattan Project and the drive to develop a deliverable nuclear weapon. Chapter 3, Code-Making and Code-Breaking: The Secret of Secrets, specifically targets the National Security Agency, an agency born in secrecy that continues its prominence in the intelligence community to this day. Chapter 4, The Military-Industrial Complex: The Dirty Secret of Civil-Military Relations, speaks to the handling of secrets at the Departments of Defense and State. Chapter 5, Surveillance: Other People's Secrets, addresses how the intelligence agencies uncover the secrets of individuals, organizations, and other countries' governments.

Chapter 6, Weird Science: Secrets That Are Stranger Than Fiction, discusses classified projects across the range of government agencies over the post-WW2 years. Chapter 7, Following the Money: Trade Secrets, follows the impact of American business on government secrecy and foreign relations. Chapter 8, Spin: The Flip Side of Secrecy, follows how the U.S. government manipulates its secrets and their declassification for a variety of purposes. Chapter 9, There is No There There: The Best-Kept Secret, is about the over-classification issue, a sore point for the declassification community for many years. Finally, in Chapter 10, Deleting the Archive: The Ultimate Secret, the author details the dangers of ungoverned records management, where agencies feel free to destroy permanently valuable records, either deliberately or due to negligence and ignorance.

Connelly give the reader much to unpack here, especially this reader. There are times when he is spot-on in his analysis and other times when he is flat-out wrong. A fundamental problem with his data analysis is that it depends upon a relatively small data set--State Department cable files. He does some analysis based upon redactions (information the agency removed from the declassified record) which can be faulty because redactions differ depending upon who did the review and when the review was completed.

If Connelly had just stuck with his "engine" and his analysis of its results I think this book would have been fine. However, he expanded the boundaries of what this book should have been about. Chapter 1 regurgitates allegations that have been raised for decades without direct proof and fails to provide context for the decisions made in the period before the U.S. entered the war. I was hoping that Chapter 2 would delve into the problems of the Atomic Energy Act of 1954, as amended, as well has the concepts of Restricted Data and Formerly Restricted Data--concepts that have a great deal to do with the backlog of needed declassification work, but Connolly does a shallow dive on the subject. He doesn't even mention the differences between information protected by Congress via legislation like the Atomic Energy Act and Executive Orders promulgated by the President. In Chapter 3, Connelly brings up the secrets of the National Security Agency; however, he centers the chapter mostly on purported and unproven vulnerabilities of the agency in the building of its 1950's-era headquarters at Fort Meade. In Chapter 4, the author misses completely on the fact that records containing possible Formerly Restricted Data require joint review by the Department of Defense as well as the Department of Energy, the only equity with this type of requirement.

This kind of faulty analysis continues throughout the book. The strong points of the book are the final two chapters, which point to the inherent weaknesses in the authority of the National Archives in legislation and the United States Code, weaknesses that have been spotlighted in recent months with the focus on the Federal Records Act and the Presidential Records Act. The disappearance of permanently valuable Federal and Presidential records is a serious threat to democracy. Sadly, the author does not suggest any solutions. I will do so. NARA's statutory authority requires strengthening with amendments both of the Federal Records Act and the Presidential Records Act. There must be a much closer relationship between NARA and the Department of Justice so that if either the FRA or PRA require enforcement action, DOJ provides better qualified staff to make NARA's case. NARA itself needs to be resourced far better than it has been for the better part of the last decade. There must be changes at the Office of Personnel Management to permit NARA to recruit better qualified technicians, specialists, and archivists to meet the challenges of this Digital Era. And, finally, NARA needs to be a part of an interagency working group involving all members of the declassification community tasked and funded to build a single standardized declassification system capable of ingesting digital records, using AI to select candidate records that require human review. Without these changes, NARA will continue to fail in meeting its declassification mission, and our nation's history and heritage will suffer as a result.

I had hoped this book would herald a call for action. Unfortunately, its faulty analysis coupled by its distraction by conspiracy theories and inappropriate search for social justice detracts from this worthy subject. ( )

This book gives a historical overview of the United States "dark state," the potential dangers such secrecy poses, and an introduction to the author's data science project History Lab (the title's "Declassification Engine"), which it seems started 8 or so years ago and uses data mining, AI and machine learning to sift through various classifications secrecy levels in order to ultimately help with all sorts of problems like freedom to research and communicate, and so on. It's easy to read and is a superficial treatment on the history stuff, which makes sense to keep the book from getting too unwieldy. However, I would have liked the book to offer more info technology aspects, such as use of blockchain, which is not mentioned at all. Solid book. ( )

The Declassification Engine by Matthew Connelly is an engaging and surprisingly (for me, since it is about data) very interesting read. Both as a history and a statement about our current state of government, this book delivers.

I think what made this a particularly appealing book is the way data, and the analysis of data, is presented in both an understandable and relatable manner. From discussing the history surrounding various moments in the past to the author's own journey trying to get the project off the ground, it was easy to see why this is an important and valuable research project. Not to mention anyone who has ever had to deal with a bureaucracy that has multiple personalities will relate to his dealings with foundations and the government.

I read this while also reading a book about the history of the deep state and what became apparent is the various ways one defines the term. Here, when Connelly discusses the deep state, he is specifically talking about the use of state secrets to mask accountability, as well as the associated disorganization of any supposed guidelines, whether for classifying or declassifying. The other book is more about basic dishonesty and greed and how keeping things hidden helps that. There is overlap but the other book is using a broad and loose definition of deep state.

Even knowing beforehand that the United States used to be more transparent, I was surprised at some of the ways. That said, we have never been as transparent as I think we should be to minimize abuse of the powers we give our government. And since WWII we have become dysfunctional as people and organizations have used the guise of national security to hide far too much that the citizens should have access to.

Highly recommended for anyone who wants a glimpse at what and how things are classified, as well as steps that might help increase accountability, both in the moment and in the future through historical research.

Reviewed from a copy made available by the publisher via NetGalley. ( )