Critical voices in Higher education especially is worried what this might mean for young learners and how you will ever be able to control content. Recent research indicates that potentially one in six (18%) of 13 to 16 year olds are being served unsuitable or harmful content by AI platforms, highlighting an emerging concern for teachers and policy makers. The AI is equipped to create adult-oriented content which clearly conflicts with accepted values and safety protections that should be anticipated in an educational environment.
In turn, “content moderation”, “digital safety” and "AI governance" have gone from technical jargon to key terms driving use of AI in education. There is a growing demand for AI technologies in schools — and with this greater need for educational content must come safeguards that protect vulnerable students from exposure to harmful material. Yet only around 25% of schools have put in place stringent AI safety filters still leaving many vulnerable to viewing NSFW content wrongly posted. This gap underscores the need for a more integrated approach to govern AI use in educational settings.
An egregious example of this happened recently in South Korea, when an AI-powered learning app served up porn to students. This debacle invited wide criticism, for which a sharper content regulation was enforced by the government on such AI based educational tools. This real-world example shows what happens when AI systems optimised for learning do not filter out inappropriate material well enough and really, it is a warning to organisations worldwide.
Media scholar Sherry Turkle cautioned, "Safety and ethics need to be at the forefront if we use AI in education." The difficulty lies in producing AI systems that help learning while following the most stringent of ethical guidelines. It accomplishes this by not only deploying state-of-the-art filtering algorithms to identify and stop harmful or NSFW content but also ensuring the learning environment is focused, secure and as supportive for students as it can be. Despite a 30% increase in companies investing in moderation technology over the past few years, they are still sophisticated tools that most industries have been learning to use effectively; education is new at this.
In addition, it is important to meet the challenge of digital literacy. Educating children on responsible AI engagementCombat potential misuse by students through effective education It makes sense for schools to include courses on the limits of AI and its dangers, so that they have at least some knowledge to be prepared when interacting with digital environments. Progress has been made, yet the Gaggle report found ahead of Rio that over half (51 percent) of educators are asking for more resources to be included in their digital safety training and preparedness — meaning you could say there is an undercurrent within the sector which remains ill-prepared tackling with some really quite nuanced challenges NSFW character AI can lay down.
While this may not seem like a huge win for the education sector, the implications of NSFW character AI are ungodly influential when you think about it in relation to bettering us all. Given how technology will continue to change, schools need more and better controls over what they bring in with them. To maintain trust and safety in learning environments, it is important to verify that AI tools being used in classrooms are aligned with educational goals, but also expose students only to appropriate content.
Learn more about nsfw character ai;