GAO says AI oversight framework will help in continuously monitoring agencies
The Government Accountability Office’s forthcoming artificial intelligence oversight framework will help auditors work with inspectors general to continuously monitor executive agencies’ progress with the technology, says the agency’s first-ever chief data scientist.
Right now GAO is developing a “wireframe” of what oversight might look like in the areas of explainability and transparency, bias and fairness, integrity and resilience, and data quality and lineage, said Taka Ariga, who also directs his agency’s Innovation Lab.
Federal officials floated the possibility of the AI framework functioning like a scorecard for agencies in September, and Ariga said “practical” principles are needed beyond simply “do no harm.”
“The proliferation of data has really accelerated machine learning deployment,” Ariga said, during a Data Coalition event Tuesday. “Coming with that are a lot of disparate impacts, societal impacts that at this point are very high level from a governance perspective.”
GAO stands to benefit as well as it employs advanced analytics on other agencies’ open source, subscription, operational, and classified data in the course of its audits, he added.
The agency is conducting more than 100 pandemic-related audits and evaluations on top of its usual oversight and has aspirations of prototyping AI, cloud services and blockchain to help securely process all the incoming data, Ariga said.
At the same time the Innovation Lab is fostering holistic data literacy among GAO auditors.
“Complicated analytical exercises shouldn’t always be outsourced to data scientists, statisticians and other specialized skillsets,” Ariga said. “We want all GAO personnel to have generalized awareness of what ‘right’ looks like and what ‘wrong’ looks like.”
GAO is implementing a data catalog so auditors can see what information is available and how they can use it, which also requires policies for who can access what.
And pipelines and containers are being developed for algorithms so they can not only scale but have their code “tweaked” as needed based on changes in requirements and circumstances, rather than having to be refactored or entirely redeveloped, Ariga said.
The Innovation Lab is taking care of data transfers, quality and visualization so GAO analysts can focus on researchable questions using the information.
“I consider all of those utility functions just like electricity, just like water,” Ariga said. “When you turn them on, they should just work.”
