The Question You Should Be Asking

You're about to use a new AI tool with your class. Before you do, you should be able to answer these important questions: Where does student data go when I use this tool? Who can access it? How long is it stored? Can the company use it for other purposes? And what are my legal responsibilities?

If you cannot answer these questions, you are not being intentional about protecting your students. The uncomfortable truth is that most teachers cannot answer them, and many schools lack clear policies requiring educators to ask. This article walks through what you actually need to know about student data and AI tools. It may not be as exciting as pedagogy discussions, but it is far more important. Data protection is not just a privacy issue. It is an equity and trust issue.

What Data Are You Actually Giving Away?

First, it is important to understand what counts as data. It includes more than most people realize. When you use an AI tool, you are sharing two main types of information: content data and metadata.

Content data refers to the actual student work, such as essays, homework answers, and discussion responses. Metadata includes details about who did what and when, including student names, ages or grades, school names, submission times, completion duration, and interaction patterns. Both types matter because together they tell a detailed story about your students.

When you use a cloud-based AI tool, you are sending this data to someone else's servers. The company gains access to it. Their employees might review it for training or quality purposes. The data becomes accessible from anywhere, which brings both convenience and security risks. It could potentially be breached, and the company could even go out of business. Most importantly, you are no longer in full control. You have transferred that control to the company and must trust their policies and practices.

AI companies handle educational data in different ways. Some use student data to improve their models, sell anonymized versions to others, keep it indefinitely, or share it with partners. Others delete it after the school year, avoid using it for training, and maintain stricter access controls. Most teachers do not know which category their tools fall into because these details are often buried in lengthy legal documents.

Companies sometimes claim they only use aggregate or anonymized data, such as patterns showing that seventy percent of students struggle with fractions. While this seems less sensitive, even aggregate data from small or demographically distinct schools can still be identifying. The process of aggregation also begins with collecting individual data first.

The Legal Landscape (Simplified)

Understanding the legal basics is essential. FERPA, the Family Educational Rights and Privacy Act, protects student records. It requires secure handling of records, gives parents access rights, and restricts sharing identifiable data without permission. Schools should have contracts with vendors that clearly define data usage, security standards, retention periods, and audit rights. Unfortunately, many schools lack these contracts, and some teachers use tools without school knowledge, creating legal risks.

COPPA applies to children under thirteen and requires parental consent, transparency, and security for data collection. Many AI tools may not fully comply, so extra caution is needed with younger students. Various states have added their own privacy laws, such as California's CCPA, New York's requirements, and others in Colorado, Connecticut, and beyond. If students or families are in Europe, the strict GDPR rules may also apply, requiring explicit consent and offering strong individual protections.

Questions to Ask Tool Vendors (The Checklist)

Before using any AI tool with students, ask vendors direct questions. Do you collect student data? What specific data do you collect, including names, content, metadata, or behavioral information? How long do you retain it? Do you use student data to train or improve your AI models? Can you share it with third parties? What security measures are in place? Do you offer a FERPA-compliant contract? Can I request deletion of data, and how quickly? Do you have a school-specific privacy policy? Have you experienced security breaches? And can schools audit your compliance?

Practical Steps You Can Take Right Now

You can take several practical actions immediately. First, anonymize student work before sharing it with AI tools. Instead of including names and specific details, describe the work generically, such as a fourth-grade essay about families. This simple step protects privacy while still allowing the AI to provide useful feedback.

Second, work with your school administration to develop a clear policy on AI use and student data. The policy should specify approved tools, data handling procedures, required contracts, parent consent processes, and breach response plans.

Third, communicate transparently with parents when using tools that involve student data. A simple email explaining the tool, its purpose, what data is involved, and the protections in place helps build trust.

Fourth, keep good documentation, including privacy policies, contracts, usage dates, and parent communications. Finally, conduct regular audits once or twice a year to review the tools you use and any changes in their practices.

When the Tool Doesn't Meet Your Standards

If a tool's privacy practices are unacceptable, you have several options. You can build your own solution by using school-controlled platforms like Google Classroom and manually transferring anonymized content. You can use AI only for your own planning work rather than student-facing tasks. Some privacy-focused or open-source alternatives exist, though they may require more setup. And sometimes the best choice is simply not to use the tool at all. Saying no to protect student privacy is a valid professional decision.

The Real Issue: Power and Trust

At its core, using an AI tool means trusting a company with your students' valuable information. You are depending on their security, policies, and ethics. This creates a power imbalance, especially since students and families often have the least influence. Being thoughtful about data means acknowledging this reality and taking steps to protect students. It requires asking difficult questions and sometimes choosing not to use certain tools.

One More Thing: This Is Evolving

Privacy laws, company practices, and technology continue to change. Stay informed by reading policies and asking questions. Push your school for better practices and stronger policies. Individual teacher awareness combined with institutional support creates meaningful progress in protecting student data.

Get personalized ethics guidance: Visit AI Ethics Advisor

Navigating AI assessment decisions requires balancing multiple ethical considerations unique to your context. Themis provides personalized guidance for your specific assessment challenges, helping you make informed decisions that protect both efficiency and fairness.

Learning AI ethically isn't just about what you know—it's about the values you carry forward. Our collection features thoughtfully designed apparel and accessories that reflect your commitment to responsible AI use. From tees and sweatshirts to hats and everyday accessories, each piece is a conversation starter about the technology we're building and the future we're shaping together.

Explore the Collection