From my experience it's often not the abilities of Sunday league referees that draws the most criticism but their attitudes, there are a couple in our league who are such self important tossers and usually end up ruining the game one way or another.
There's one in particular, youngish lad probably somewhere between 18-22 I'd guess, who I don't think has ever managed to referee one of our games without dropping an absolute clanger, now that's bound to happen at amateur level just it's likely that some pissed up lump with spanner one into his own net but he has no humility to accept that he made a mistake, he makes no attempt to try and explain any of decisions regardless of how he's approached, he makes a rod for his own back. Last time he reffed a game of ours it was a complete farce, he started by telling us no-one was allowed to wear gloves other than goalkeepers, he booked one of our players for jumping out of the way of a vicious tackle claiming he'd 'jumped into the man', he awarded 3 penalties, one was a completely fairly competed header following a throw in, another saw a player launch himself over the keeper collecting a ball but somehow got a penalty, following this moment a magic a man who'd last been anywhere near the ball some 40 yards from goal was sent off for apparently conceding the penalty?!
When you've got officials like that the game quickly becomes a nonsense, it's not like he's just overly strict so you can reign things in to account for it, he's completely all over the place so you've no idea what's coming, plenty obviously get rattled by this and then his manner of dealing with people only makes things worse until it becomes a team v the ref, or sometimes both teams against him, affair.