Jordan works as a red-team security analyst for a technology…
Jordan works as a red-team security analyst for a technology company that deploys large-scale AI-driven content-moderation services. These AI systems receive user-generated text, images, and videos, and automatically classify or filter harmful content. Jordan is concerned about whether the system is robust and safe, especially given the increasing number of adversarial inputs designed to bypass AI moderation. Jordan would like to conduct penetration testing by crafting adversarial prompts and manipulated inputs, similar to fuzz testing, that intentionally attempt to break or deceive the AI moderation algorithms. However, the AI service is currently deployed in production, actively moderating content for millions of users in real time. Jordan’s tests would be executed directly against the live system. (a) [6 pts] Discuss the ethical concerns that arise from performing this type of testing on a production AI moderation system with real users. Consider impacts on users, system integrity, privacy, service availability, and professional responsibilities. (b) [6 pts] Discuss how Jordan could design or conduct the testing in an ethical and responsible manner while still achieving the security-testing goals. Provide concrete steps or safeguards (e.g., environment separation, permissions, data protections, audit procedures).
Read Details