I reckon literary agents get very hard done by, especially when it comes to harsh comments about what they choose to represent, or not, as the case may be.
Ability to sell an author's book to the publisher is a huge issue! People forget that agents have mortgages to pay and children to feed. They'll be homeless with hungry children crying for food if they don't sell their clients' books.
Certainly we must all write the book that's in us -- we could hardly write someone else's book -- but it has to be within the envelope of what other people want to read, which is what the big stores will stock, which is what the stores will order from the publishers, which is what the publishers will buy from the agents, which is what the agents will offer to represent.
If I were a literary agent, I'd be a whole lot more ruthless and demanding than the bunch doing the job now. (And the Publishing Gods preserve me from such a fate.)
If you were an agent, and your next meal depended on selling the books you chose to represent, then what would you do?