By itself this is a reasonable question.

It is a problem thay can however be solved (or at least, solved well enough) by normal boring mundane things like "democracy" (yes I know about Arrow's theorem).

Right now the problem is our understanding of what "alignment" even means with an artifical mind: we literally can't do that yet, we are currently so bad at this that we are failing to balance "follow instructions" with "don't break the law" even though we've got a system that actually knows what the law is and that breaking the law is bad (we saw it writing notes to itself about this).

[dead]