Most agency AI inventories are ‘not fully comprehensive and accurate,’ GAO reports
A detailed government watchdog review of federal agencies’ artificial intelligence use case inventories found that most of the disclosures weren’t “fully comprehensive and accurate,” echoing similar findings in Stanford research and subsequent FedScoop reporting.
The Government Accountability Office report released Tuesday focuses on how 23 agencies — the civilian agencies under the Chief Financial Officers Act — have progressed with a requirement that they must annually inventory their current and planned deployments of AI and disclose non-sensitive, non-classified uses publicly. That obligation stems from a Trump-era executive order (EO 13960) focused on AI.
The report found that those agencies’ inventories did not always follow requirements and sometimes contained “data gaps and inaccuracies.” Of the 20 agencies that submitted inventories to the Office of Management and Budget, just five “provided comprehensive information” for each use case, GAO found.
“Without accurate inventories, the government’s management of its use of AI will be hindered by incomplete and inaccurate data,” the report stated.
The report also outlined the extent to which CFO Act agencies are using artificial intelligence. While more than 700 use cases were publicly disclosed in a consolidated spreadsheet uploaded to the AI.gov website in October, the GAO reports that agencies publicly disclosed about 888 use cases and there are about 1,200 reported by those agencies in total.
NASA had the most uses of any agency, followed by the Department of Commerce and the Department of Energy, GAO found, which differs from the totals in the existing consolidated inventory. NASA, for example, which has 375 total public uses of AI according to GAO, has just 33 in the consolidated list online.
About half of government AI use cases are focused on science, while the rest span areas including internal management, agriculture, transportation, public services and law enforcement, the GAO said.
A major Stanford report, along with a series of subsequent investigative pieces produced by FedScoop, had previously highlighted challenges, including inaccuracies and discrepancies, within these inventories. The inventories were also discussed in a House Oversight subcommittee hearing earlier this fall.
The disclosures could become more important. In a November interview with FedScoop following President Joe Biden’s recent AI executive order, Conrad Stosz, director of AI in the White House Office of the Federal Chief Information Officer, said the disclosures are expected to be more central to government and public insight into AI uses going forward.
In addition to not following requirements, the GAO also found that several inventories included duplicative use cases, and two included references to technology that were not, in fact, AI. Nine agencies also included references to AI used within research and development, which were supposed to be excluded.
Reasons for the issues, agencies told the watchdog, included staff errors and different interpretations of the CIO Council’s guidance for the inventories.
Department of Veterans Affairs officials, for example, told GAO that they used different responses for life cycle stage than the ones in the template provided by the CIO Council “because the selections did not accurately reflect the life cycle stage for their AI use cases and there was no option for ‘other,’” according to the report.
FedScoop similarly reported that DOE said it was able to produce a more accurate and larger use case inventory in the second year of reporting, following “enhanced” guidance.
The GAO report also found that agencies have taken only initial steps to implement existing AI requirements in executive orders and statutes. OMB and the Office of Personnel Management, for example, didn’t “fully implement” requirements for AI acquisition and workforce assessments, respectively.
Ultimately, the GAO made 35 recommendations to 19 agencies. That included recommendations that 15 agencies update their inventory to include information required, and that OMB, OPM and the Office of Science and Technology Policy implement the outstanding AI requirements.
Of the 19 agencies, GAO said 10 agreed with their recommendations, three partially agreed, four didn’t agree or disagree and one ultimately didn’t agree with a recommendation.
