Pentagon IG office shares new details about its unfolding evaluation of CDAO’s effectiveness

The Defense Department’s top watchdog is conducting a comprehensive assessment to ultimately determine the impacts and “effectiveness” of the Chief Digital and Artificial Intelligence Office’s procurement, strategy and policy development, and adoption of AI since its inception, according to a recent memorandum.  

“This is a self-initiated evaluation that the DOD [Office of Inspector General] identified as part of our ongoing series of oversight projects on DOD’s implementation of artificial intelligence,” Pentagon OIG spokeswoman Mollie Halpern told DefenseScoop this week. 

Employees from four legacy organizations — the Joint Artificial Intelligence Center (JAIC), Defense Digital Service (DDS), Office of the Chief Data Officer, and the Advana program — were repositioned to collectively form the new CDAO in late 2021. In a memo announcing its launch, Deputy Defense Secretary Kathleen Hicks wrote that the move was needed to synchronize, integrate and strengthen data, AI and associated digital solutions across all department and military components.

The office’s first-ever chief, Craig Martell, left his post as Lyft’s head of machine learning in 2022 to steer CDAO, which reached full operating capability that year.

Some of its notable early outputs since then include the online “Tradewind Solution Marketplace,” the production of the Pentagon’s latest Data, Analytics, and AI Adoption Strategy, and the military’s highly anticipated minimum viable capability for Combined Joint All-Domain Command and Control (CJADC2).

That progress, however, has not come without some controversy.   

For instance, questions continue to swirl about the office’s role as a new portfolio owner for what was formerly known as Project Maven — now Maven — the evolving, computer vision work within the Defense Department. A recent Government Accountability Office report also found that the CDAO must complete “the additional steps necessary to fully define and identify DOD’s AI workforce.”

Separately, in May 2023, DefenseScoop obtained slides that suggested an internal CDAO review had revealed that certain personnel were deeply dissatisfied with how senior leaders were running the office. 

“We are trying very hard to make pretty large mind-shift changes. As we make these mind-shift changes, it will create some discomfort,” Martell told DefenseScoop when asked about that during a media roundtable at the Advantage summit last month. 

In their announcement about the unfolding evaluation, Defense Department OIG officials wrote that the overarching objective “is to assess the effectiveness of the [CDAO’s] development of [AI] strategy and policy for the DOD, and the CDAO’s acquisition and development of AI products and services.”

They also confirmed that those aims may be revised based on learnings as the evaluation is conducted.

“To protect the integrity of the evaluation process, we do not preemptively release any findings before the evaluation is complete. We have no timeline for results as the length of the investigation depends on many variables,” Halpern said. 

In response to questions from DefenseScoop on Wednesday, a CDAO spokesperson noted that the office routinely works with the Pentagon’s OIG on evaluations of their core processes and capabilities. 

“As a new organization, only about two years in existence, we are particularly interested in how we can continually improve and make our efforts more effective and efficient. We welcome this evaluation and look forward to their recommendations,” the spokesperson said.