The need to accelerate deployment of Artificial Intelligence (AI) by executive branch agencies also requires the development and retention of human expertise, according to Dr. Deborah Frincke, Director of Research at the National Security Agency (NSA). Speaking at the industry-sponsored media event “Accelerated AI: Shaping the Future of Government with Artificial Intelligence” (11/13/2018), in Washington, D.C., Dr. Frincke argued that vast and growing sets of “noisy data” (volumes of structured and unstructured data), make AI indispensable to human analysts. Dr. Frincke stressed that as the government digitizes and upgrades outmoded systems to deploy AI, demand grows for more analysts to “say yea or nay on the AI output – the mechanical answer needs human judgment to get to the ground truth.”
Dr. Frincke explained that Federal programs should tailor AI technologies to specific processes where they can be most productive, rather than implementing AI for its own sake. Where deployed most effectively in the Intelligence Community (IC), the infrastructure underpinning AI innovations focuses on getting the right people working in teams to advance research by using technology to better collect and evaluate data.
As a first principle, guidance from the Director of National Intelligence (DNI) Dan Coates, and his Principal Deputy Director (PDDNI) Susan Gordon, further emphasizes that the infrastructure improvements supporting AI need to facilitate the ability to share information within the IC, across government, and with all American citizens to the greatest extent possible. Dr. Frincke noted that the need for policies to effectively manage AI, and the issues of privacy and search biases in algorithms that concern “Ethical AI,” will shape how human analysts handle AI output.
At the same event, the Bureau of Labor Statistics (BLS) Senior Economist Alex Measure presented an AI use case that illustrates similar principles for implementing AI tools at a Federal agency outside the IC. The BLS receives a large amount of structured and unstructured data, particularly in the form of written descriptions submitted for injury claims. Mr. Measure led teams in the development of algorithms that were then used to read descriptive narratives to classify these injury claims.
The technical solutions developed at the BLS employ “training data,” a type of AI based on supervised Machine Learning (ML) to find patterns in large amounts of inputs received. In just the last ten years, this method has resulted in significant advances in AI such as facial recognition. The technology works by tagging and orchestrating large amounts of “noisy data” into a system.
In the past, the use of training data to produce AI outcomes required supercomputers, but computing power has advanced so far that teams at the BLS are able to train a system on a million data points by using no more than government-issued laptops. Despite the enormous volumes of data, computing power was not the most difficult piece of the project.
Mr. Measure found that the biggest challenge at the BLS proved to be communicating the need to integrate the AI solutions into the organizational system. This required cultivating the support of upper management, and benchmarking human outputs to compare with the automated solutions. Human experts re-coded injury narratives and used standard classification methods, constantly measuring biases to adjust quantitative measures. He stated that correctly assigning codes also requires the continuous monitoring and re-evaluation of automated outputs.
As discussed by NSA Research Director Dr. Frincke, the DNI’s emphasis on managing AI to facilitate information sharing, and augment the work of human analysts in the IC, reflects the imperative for modernizing the national security classification system long supported by the PIDB. Just as the DNI recognizes the need for new policies and practices to implement emerging technologies in the IC, the automation of data classification at the BLS demonstrates the power of AI and ML to achieve efficient outcomes.
The PIDB promotes modernization to upgrade the efficiencies of similar processes in the classification and declassification of national security information at the enterprise level across the Federal government. The success of IC teams in implementing AI projects, and the currently dispersed nature of innovations such as the use case presented by the BLS, demonstrate that the executive branch agencies would do well to follow the example of the Office of the Director of National Intelligence in coordinating the implementation of emerging technologies across administrative silos.