Evaluating Expert-Authored Rules for Military Reasoning

Authors: Mike Pool, Ken Murray, Julie Fitzgerald, Mala Mehrotra, Robert Schrag, Jim Blythe, Jihie Kim, Hans Chalupsky, Pierluigi Miraglia, Thomas Russ, Dave Schneider.


Eliciting complex logical rules directly from logic-naïve subject matter experts (SMEs) is a challenging knowledge capture task. We describe a large-scale experiment to evaluate tools designed to produce SME-authored rule bases. We assess the quality of the rule bases with respect to the: 1) performance on the addressed functional task (military course of action (COA) critiquing); and 2) intrinsic knowledge representation quality. In the course of this assessment, we note both strengths and weaknesses in the state of the art, and accordingly suggest some foci for future development in this important technology area.

For more information, contact IET