Introductory assignments in CS Theory ask students to construct instances of various computational models (such as finite automata, regular expressions, context-free grammars, or push-down automata) for a given language. Verifying the correctness of their model instance is challenging for beginner CS Theory students since the concepts are abstract and there are infinitely many possible inputs. A popular JFLAP software allows students to visualize the running of their instance on a specific input. A recently developed server extension to JFLAP computes the equivalence of a student’s instance to the instructor’s solution and, if they are not equivalent, it returns a ``witness string,'' an input string on which the student’s construction and the correct solution differ.

We report on comparing student performance and perceptions in three course sections of CS Theory that differed in the tools the students used for model-construction assignments: no JFLAP, vanilla JFLAP, and JFLAP + server (with immediate correctness checking). In Spring 2022, 59 out of 86 students consented to include their data in our research. We compared the performance of the three sections on 28 questions (7 DFA problems, 2 NFA problems, 11 regex problems, 6 CFG problems, and 2 PDA problems). We found that while the grades were comparable among the sections, the section using JFLAP + server credited the instantaneous witness string feedback with their increased understanding of the concepts.