Evaluating Adaptive AI for Contracting Officer Readiness: Design and Pedagogical Proposal for the Warrant Board RAG Chatbot

Abstract

The United States Air Force (USAF) requires a sustained and expanding pool of warranted Contracting Officers (COs) to meet growing operational and fiscal demands across its global enterprise. The authority to obligate funds and bind the government contractually—granted through the issuance of a warrant—requires successful completion of a multi-stage evaluation process culminating in a scenario-based oral board. This final interview assesses a candidate’s ability to interpret and apply acquisition policy under complex and ambiguous conditions. This paper proposes the design of an adaptive artificial intelligence (AI) training system—the Warrant Board Retrieval- Augmented Generation (RAG) Chatbot—to serve as a simulated board-preparation environment. This chatbot inverts the common ’user question, AI answer’ model, and instead has the chatbot ask the learner a series of critical thinking scenariobased questions. The prototype design adopts a model-agnostic LLM gateway capable of operation through either commercial APIs (e.g., OpenAI) or secure, government-hosted environments such as GenAI.mil, ensuring accessibility within unclassified Air Force networks. This research contributes to the emerging field of AI-assisted professional education by developing a transparent, auditable, and pedagogically grounded framework for formative learning in acquisition training.