AI Risk and Bias Review
This page describes Pearl Education’s governance approach to identifying and mitigating AI-related risks, including bias, accuracy, safety, and data-use concerns, before and after AI features are deployed in the Products. Pearl’s AI risk and bias review process is designed to be consistent with the data-use commitments in MSA §6 and §7.1 and the Student Data protections in DPA Article IV.
Why This Matters in K-12
AI tools that surface student data to educators and administrators carry real stakes for real students. A system that inaccurately flags some student groups more than others, or that produces unreliable insights, can affect how educators allocate attention and support. Pearl takes this responsibility seriously. AI features in the Products are designed to inform professional judgment, not replace it — and to do so fairly and accurately across the student populations Pearl’s customers serve.
Pre-Deployment Review
Before releasing an AI feature that processes Student Data or Customer Content, Pearl conducts an internal review. That review addresses the following areas:
Purpose and Data-Minimization Check Does the feature use the minimum data necessary to deliver the intended result? Could a less data-intensive approach achieve a comparable outcome? Is the processing consistent with the purpose-limitation in DPA §4.1?
Fairness and Bias Assessment Are there known or foreseeable ways the feature could produce disparate outputs across student subgroups — for example, by race and ethnicity, gender, grade level, or program type? Is the training data (for AI features Pearl builds rather than configures from third-party models) representative of the populations the feature will serve?
Accuracy and Reliability Evaluation Is the feature sufficiently reliable for its intended use case in a K-12 educational context? Are known error modes or confidence limitations communicated to Authorized Users?
Safety Review Could the feature generate outputs that are harmful, misleading, or inappropriate in a K-12 setting — including outputs about students that could mislead the educators and administrators who act on them? Is the feature scoped to advisory and informational use, rather than autonomous action?
Human-Oversight Design Is the feature designed so that AI-generated outputs are surfaced to a human decision-maker before any action is taken? Could the feature be misconstrued as a final determination about a student?
Data-Minimization Controls
Pearl’s AI features are built to operate on the minimum Student Data and Customer Content necessary. Controls include:
- Restricting AI features to Customer Content that is relevant to the feature’s stated purpose
- Not using AI features to process Excluded Data categories (DPA Schedule 1, Part C) — for example, biometric records, health/medical records beyond accommodations Customer elects to record, or government identification numbers
- Not retaining Student Data beyond what is needed to provide the Services (DPA §4.1; MSA §7.1)
- Not using identifiable Student Data, curriculum content, session recordings, or other Customer Content to train AI models (MSA §5.7; DPA §4.2)
De-identified and aggregated data used for service improvement or research is processed under Pearl’s documented de-identification standard before any such use (MSA §6.3; DPA Art. V), separating it clearly from identifiable Student Data.
Human Oversight and Human-in-the-Loop Design
Pearl’s AI features are advisory. AI-generated outputs — such as program alerts, engagement summaries, or natural-language analytics responses — are presented to administrators and instructors/tutors for their professional review and decision-making. Pearl does not use AI to make autonomous, consequential decisions about students (for example, automatically restricting access to services, generating binding performance assessments, or determining educational placements without educator review).
Pearl staff maintain oversight of AI feature behavior in production. When customers or Authorized Users report anomalous or unexpected outputs, Pearl investigates and takes corrective action where warranted.
Post-Deployment Monitoring
Pearl’s intent is that post-deployment monitoring for AI features follows the same discipline as security monitoring for the core product — with defined escalation paths and periodic review.
Alignment with MSA §6 and §7.1
Pearl’s AI risk-and-bias governance is designed to be consistent with its contractual data-use framework:
- MSA §7.1 — AI features process Customer Content solely on Customer’s behalf to provide the Services; outputs are for Customer’s use. Risk and bias review ensures features stay within this scope.
- MSA §5.7 / DPA §4.2 — Identifiable Student Data is never used for AI model training; AI features are not used to enable the sale of Student Data or its use for targeted advertising. Pre-deployment review confirms each new feature complies.
- MSA §6 / DPA Art. V — De-identified and aggregated data used for research or service improvement follows Pearl’s documented de-identification standard and is shared with research partners only under data use agreements that prohibit re-identification. This use is separate from and does not affect the no-identifiable-training-data commitment above.
Responsible AI Contact
To raise concerns about Pearl’s AI practices, a specific AI-driven output, or potential bias in a Pearl AI feature, contact responsibleai@poweredbypearl.com, which routes to Pearl’s privacy team (privacy@poweredbypearl.com).
