Notes from Lecture on Cyber Security and AI Products

Introduction

• Career Advice: Finding a good mentor is crucial for career growth.
• Event Host: Steven Wong, co-organizer for Product Tank BV.

Guest Speaker Introduction

• Mjeri Vatla: Seasoned product leader with experience at Amazon and Google.
  • Specializes in cyber security, infrastructure design, regulatory compliance.
  • Passionate about empowering diverse backgrounds in tech.

Presentation Overview

Aim and Goals

• Focus on basic understanding of AI/ML and cybersecurity.
• Address challenges, threats, and secure development practices in AI product development.

Cyber Security and AI Integration

• Impact of AI/ML: Revolutionizing industry, enhancing cyber security measures.
• **Importance of Privacy, Security, and Compliance:
  • Protecting algorithms, models, and hosting infrastructure from malicious access.
  • Privacy is critical due to extensive data usage in model training.

Core Aspects of Cyber Security in AI/ML

Data Security

• Protect data used for training and inference.

Model Security

• Safeguard models from tampering and exploitation.

Infrastructure Security

• Secure environments where AI/ML models operate (cloud or on-prem).

Access Control

• Implement mechanisms to control access to models/data.
• Utilize authentication & authorization protocols (IAM, Kubernetes, etc.).

Integrity Verification

• Ensure data and model integrity throughout their lifecycle.
• Importance of verifying outputs to prevent misuse (e.g., malware generation).

System Resilience

• Build resilience against disruptions and attacks for continued operation.

Regulatory Compliance

• Adhere to regulations like GDPR and CCPA for data protection and privacy.
• Example privacy-preserving techniques: Differential Privacy, Federated Learning.

Common Threats and Challenges in AI Products

• Data Poisoning: Malicious data introduction during training.
• Unauthorized Access: Breaches targeting confidential data.
• Model Theft: Exploiting models to replicate functionality.
• Algorithm Bias: Resulting in unfair outcomes.
• Operational Security Risks: Insecure APIs, lack of logging.
• Third-party Integrations: Potential vulnerabilities from third-party services.

Secure Development Practices

• Data Security: Strong encryption and access controls.
• Privacy Practices: Data anonymization techniques.
• Continuous Monitoring: For anomalies and security incidents.
• Complying with Standards: Implement industry best practices from organizations like NIST or OWASP.

Frameworks for AI/ML and Cyber Security

• Examples of applicable frameworks:
  • Privacy by Design: Embed privacy from inception.
  • Fair Information Practices Principles (FIPP): Ensure transparency, accountability, and data minimization.
  • NIST Framework: Manage privacy risks through structured processes.

Ethical Guidelines

• Follow ethical AI development practices emphasizing transparency, accountability, and fairness.

Case Studies and Real-World Examples

• Activision: AI-based phishing attacks.
• TaskRabbit: Data breach attributed to AI-enabled botnet.
• Y Brands: Ransomware attack using AI for targeted data theft.
• Deepfakes: Rising concern in misinformation and public manipulation.

Future Trends in Cyber Security for AI

• Shift towards proactive measures and leveraging AI for threat hunting and vulnerability prediction.
• Ethical development emphasizing compliance and responsible AI use.

Q&A Segment Highlights

• Differences between traditional cyber security measures and those for AI systems.
• Steps to ensure AI solutions are secure and their integrity.
• Advice for aspiring cyber security product managers on building relevant experience.

Conclusion

• Emphasize the need for continuous learning in cyber security and product management.
• Encourage leveraging resources and frameworks when developing AI products.