AI safety for kids a top concern for COPPA compliant AI startups

June means the start of summer is upon us, and as teachers put the 2024-2025 school year behind them, many are wondering what the year ahead will bring regarding President Trump’s AI in Education executive order. 

Signed in April, the order aims to provide the country’s students with “early training” in AI as well as access to AI tools through public-private partnerships in K-12 education. Thus far, AI use in classrooms has had mixed reviews. 

While some school systems have rolled out district-wide training like Miami-Dade Public Schools in south Florida, survey data released in October of last year demonstrated that just 1 in 4 teacher colleges and institutions had provided training on using AI tools for teaching.

Even more unclear is what AI education and appropriate use looks like for younger students. There has been no shortage of stories of older high school and college students using AI to cheat, but it’s noteworthy that the executive order specifies use in K to 12 classrooms.

But AI tools are dependent on data, and the most popular tools, like ChatGPT, are designed to collect as much as possible – technically making them illegal to use with students under 13 without clear parental consent. 

This is due to the Children’s Online Privacy Protection Act (COPPA), a federal law designed to protect the online privacy of children. Originally established in 1999, the rule was updated earlier this year to better reflect the current realities of today’s highly-connected world – namely with even stronger protections. 

Enforcing those protections, however, isn’t always easy. In fact, a 2022 study found that 96% of the apps used in U.S. K-12 schools shared children’s personal information with third parties – and that was before the mass proliferation of generative AI. When it comes to AI-powered tools, very few achieve COPPA compliance. 

Those that do acknowledge the challenge. 

Ivan Crewkov is the co-founder and CEO of Buddy.ai, an AI tutor for kids under 12, reaching over 20 million students annually from all over the world. The Buddy.ai app is centered around Buddy, an AI-powered talking animated robot who leads users through interactive lessons and games designed to teach English as a foreign language. 

Crewkov and the Buddy team recently announced it was officially COPPA compliant through the kidSAFE Seal Program, an independent safety certification service and seal-of-approval program designed exclusively for children-friendly websites and technologies.

In response to their recent COPPA-certification, Crewkov said, “My co-founder and I created Buddy with the goal to help our own children learn English. When we started, we could not have imagined the scale of development ahead of us. Although this was a lengthy and challenging process, attaining kidSAFE and COPPA compliance was important to us as parents — we wanted to build AI technology that prioritized safety and privacy complemented by design.”

According to a press release shared by the Buddy.ai team, building an effective tool that was capable of reaching COPPA compliance required building their own AI technology from the ground up. A process that required a team of speech technology experts over four years of development and $13 million in VC fundraising. 
Today, the Buddy.ai app has been downloaded over 50 million times and is used by students all over the world, including the US, Europe, and the LATAM and MENA regions, among others.

Disclosure: This article mentions a client of an Espacio portfolio company.