When protections outlive their purpose: Managing defense systems at scale