I agree with you here. But look at it from a spectator's perspective, and not from a referee's perspective.
Most fans, don't know each and every law in the book. They watch the game, and talk about that what they see in front of them. With technology and replays, the fans see more things which they didn't in the past.
While I do agree with you that most of the Ref-bashing is unwarranted, there are way too many instances nowadays where bad calls goes unpunished. Scrum resets is an area that frustrates me the most. And I wonder how many there will be if we implement a "stop the clock" approach regarding scrums. If the ref stops time during the binding setup of the scrum, up to the point where the scrumhalf puts in the ball, there will be a lot less resets, because no team will benefit from wasting any time. The ref can still blow for a penalty if the scrum is collapsed, or early engagement, but there will be no time lost.
But scrum resets is something that the players, coaches and the referees should focus on, and get a more simplified approach to ruling on the scrums. I'm sure no coach will agree with the ref if their prop gets penalised, when he's not the one collapsing the scrum. Jonathan Kaplan has been very verbal about this matter. And he says it's an area that needs a lot more focus from the Referees panel.
Bad calls is what makes fans extremely angry. They feel their team are being mistreated, and that "why should I bother going if the game is spoilt by bad calls" -approach is warranted. Without bad calls, the ref doesn't paint a target on his back, and fans, coaches and writers will have no choice but to revert to the team and the players to take the blame. We humans love it to point the finger at someone else. If referees ensure that they make the correct calls according to the laws, then no matter what anybody says, their decision is warranted. I'm sure each and every person in this world wants that comfort that "I know I was right" feeling. There is no judgement, and there is no self-doubt, which results in confidence.