Personally, I like them all. Season 3 and 4 weren't my favorite seasons, but it's not Ra's or Damien's faults. Ricardo Diaz on the other hand, well...he was fine at first but in my opinion, by the end of season 6 he overstayed his welcome.
Other than that, Slade's my favorite bad. Merlyn was the first but still pretty cool. Adrian I feel is overrated, but it didn't ruin season 5 for me.