CTAT Detector Library

Example detectors

All detectors below can be found here.

Gaming-the-system detectors

• Cross-system detector of gaming-the-system

  • Description: Cross-system gaming detector from Luc Paquette, Michal Moskal, and Ryan Baker
    • Paquette, L., Baker, R. S., Carvalho, A. D., & Ocumpaugh, J. (2015). Cross-System Transfer of Machine Learned and Knowledge Engineered Models of Gaming the System. Lecture Notes in Computer Science User Modeling, Adaptation and Personalization, 183-194. doi:10.1007/978-3-319-20267-9_15

• Gaming-the-system

  • Description: [coming soon]

---

Detectors of student help-seeking behavior

• Help model

  • Description: [The Help Model from Vincent Aleven, Bruce McLaren, Ido Roll, et al., (e.g., see Aleven et al., 2006; 2015)]
    • Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2016). Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1), 205-223.
  • Free parameters: note that this demo version has some free parameters that can be tuned (e.g., thresholds on the duration that must elapse between attempts in the tutor in order for an attempt to be flagged as "deliberate")
  • Outputs: strings representing assessments of students' help-seeking behavior (e.g., hint avoidance, non-deliberate attempts, hint abuse, as well as 'acceptable' and 'preferred' hint-use according to a normative model of hint use)

• Help model (variant: prefer that student attempts step first, before asking for a hint)**

  • Description: [A variant of the Help Model from Vincent Aleven, Bruce McLaren, Ido Roll, et al., (e.g., see Aleven et al., 2006; 2015)]
    • Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2016). Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1), 205-223.
  • Free parameters: note that this demo version has some free parameters that can be tuned (e.g., thresholds on the duration that must elapse between attempts in the tutor in order for an attempt to be flagged as "deliberate")
  • Outputs: strings representing assessments of students' help-seeking behavior (e.g., hint avoidance, non-deliberate attempts, hint abuse, as well as 'acceptable' and 'preferred' hint-use according to a normative model of hint use)

---

Bayesian Knowledge Tracing

• Bayesian Knowledge Tracing (BKT)

---

Detectors of unproductive persistence

• Wheel Spinning

  • Description: A sliding window variant on the wheel spinning detector proposed by Beck and Gong.
    • Beck, J. E., & Gong, Y. (2013, July). Wheel-spinning: Students who fail to master a skill. In International Conference on Artificial Intelligence in Education (pp. 431-440). Springer, Berlin, Heidelberg

• Predictive Similarity:

  • Description: [coming soon]

• Predictive Stability:

  • Description: [coming soon]

---

• Idle

  • Description: outputs alerts in response to fixed, user-specified thresholds on student idle time
  • Outputs: custom message after each specified time-interval
    • the current format is a comma-delimited string, where the first element is a boolean stating whether the student is over some author-specified idle-time threshold, and the second is one of a set of author-specified milestone times that the student has exceeded
      • (e.g., "0, > 0 s" indicates that the student is not considered idle for long enough to trigger an event/alert, and "1, > 1 min" indicates that the student has been idle for long enough... specifically: the student has been idle for more than 1 minute)

• Error count on current step

  • Description: keeps a running count of attempts on each step in a tutor interface
  • Outputs: the running count of attempts on the most recent step the student has worked on
    • (e.g., if a student has tried 30 times on a particular interface element, the output will be "30")

• Recent correctness rate (sliding window)

  • Description:
    • (https://www.fi.muni.cz/~xpelanek/publications/mastery-detection.pdf)

• Recent error rate (sliding window)

  • Description:
    • (https://www.fi.muni.cz/~xpelanek/publications/mastery-detection.pdf)