AI systems that use context — conversation history, user profiles, location data — to personalize outputs introduce serious ethical risks. Context shapes what information people see, what decisions are made about them, and how their data is retained and potentially misused, often without their meaningful awareness or consent.

Context in AI refers to any information beyond the immediate input that shapes the model's output: prior conversation turns, user demographic data, browsing history, location, stated preferences, or inferred attributes. While context enables more relevant and personalized AI experiences, it introduces layered ethical risks. The first is surveillance by design: AI systems that retain and reference personal context are, functionally, persistent monitoring systems. Users often don't understand what's remembered or how it influences outputs. The second is contextual manipulation: when an AI knows a user is anxious, financially stressed, or politically motivated, it can — intentionally or through optimization pressure — tailor outputs that exploit these states rather than serve the user's genuine interests. Third is context pollution: inaccurate or outdated contextual data leads to persistent errors in personalization that users may not be able to identify or correct. Fourth is consent asymmetry: organizations collecting contextual data have far more information about how it's used than the users who generate it. Addressing these risks requires explicit context disclosure, user control over what's retained, regular context audits, and design principles that prioritize user agency over engagement optimization. These aren't theoretical concerns — they are active issues in deployed consumer AI products today.

What Are the Ethical Risks of Context in AI Systems?