Debiasing Embeddings: Why Simple Fixes Don't Work

Early approaches to embedding bias tried to 'project out' bias directions from vector space — remove the gender axis, remove the race axis. Research now shows these techniques are superficial: bias is distributed across the embedding space and resurfaces in downstream tasks even after apparent removal.