Human Oversight and Overreliance
Ensuring human oversight in AI systems prevents overreliance on technology and maintains educational integrity.
What procedures do the vendor, program, and instructor have in place for human review and content quality assurance?
No technology is perfect but by asking the right questions about a technology and carefully considering our capacity to effectively leverage it, we can maximize impact.
Ask
Human Oversight Mechanisms:
What mechanisms do you have in place to ensure human oversight over AI-generated content and decisions?
Feedback and Improvement Process:
How do you incorporate user feedback into continuous improvement and updating of your AI tools?
Consider
Professional Judgment:
How will you maintain a balance between AI assistance and your own professional judgment in educational decision-making?
Review Process:
What processes will you establish to regularly review and assess the quality and appropriateness of AI-generated content?
Human Insight:
How do you plan to incorporate human insights and feedback into the continuous improvement of the AI tool’s performance and output?
Example
An adult education program introduces an AI tutor for integrated education and training to help adults develop English literacy in in-demand occupational training. The AI tutor offers reading support to augment an occupational training manual. Instructors monitor the AI interactions, provide supplementary instruction when learners encounter complex issues, and offer deeper insights and problem-solving strategies based on their lived experience during regular sessions. Learners develop foundational literacy and workplace skills, benefiting from both personalized AI guidance and expert human instruction.
The AI Integration Framework was adapted, with permission, from the "Top 6 Questions for GenAI EdTech Providers" resource developed by AI for Education.
This is one dimension of the AI Integration Framework.