51 papers out of 4841 accepted papers is 1%. If these were the only papers, then I would say the review system actually doing pretty well. Considering how noisy the review processes are, I don't see this as a big problem on its own.

We can't stop people (or agents) from submitting low quality papers in the current system. We can improve the review quality, but at a human cost.

If hallucinated citations are making it to top conferences, do we just have to accept that no one is willing to do the work to ensure that our research is grounded in reality? Perhaps the volunteer peer review system is broken and we need people who are paid to carefully check the citations