Skip to content

Latest commit

 

History

History
29 lines (21 loc) · 2.42 KB

DataLeakage.md

File metadata and controls

29 lines (21 loc) · 2.42 KB

Data Leakage Risks in ML/AI Systems

Description

ML/AI systems are prone to data leakage, which can occur at various stages of data processing, model training, or output generation, leading to unintended exposure of sensitive or proprietary information.

Extended Description

Data leakage in ML/AI systems encompasses more than unauthorized database access; it can occur subtly when models unintentionally expose information about their training data. For example, models that overfit may allow inferences about the data they were trained on, presenting challenging-to-detect risks of potential data breaches.

Potential Mitigations

  • Data Masking and Encryption: Protect data at rest and in transit with encryption and mask sensitive details when feasible.
  • Access Controls: Deploy robust access control systems to ensure that only authorized personnel can reach sensitive data and models.
  • Regular Audits: Carry out frequent audits of data access logs and model outputs to uncover any leaks.
  • Differential Privacy: Apply noise to datasets or model outputs to prevent the re-identification of individual data points.
  • Data Minimization: Limit the use of data to what's necessary for training and operations to minimize the risk of leakage.
  • Monitoring: Set up real-time surveillance to identify abnormal data access patterns or potential security incidents.

Related Weaknesses

  • CWE-200: Exposure of Sensitive Information to an Unauthorized Actor: Denotes the risk of accidentally revealing sensitive data.
  • CWE-359: Exposure of Private Personal Information (PPI): Highlights the dangers of leaking personal data.

Impact Analysis

  • Financial Impact: Data breaches can lead to significant fines and are particularly costly in heavily regulated industries or areas with strict data protection laws.
  • Reputation Damage: Trust issues stemming from data leaks can affect relationships with clients, partners, and the wider stakeholder community, potentially resulting in lost business.
  • Legal and Compliance Implications: Non-compliance with data protection can lead to legal repercussions and sanctions.
  • Operational Impact: Breaches may interrupt business operations, requiring extensive efforts to resolve and recover from the incident.
  • Intellectual Property Risks: Leaks in certain fields could disclose proprietary methodologies or trade secrets, offering competitors unfair advantages.