My coach has a rule on my hard bike sets: if I reach a point where I can't keep my heart rate up anymore, it's time to go home.
I had five sets today. On the fourth one the heart rate was waning. It was not terrible, but it was not heading upward in the direction I wanted it to. The legs were screaming and did not have the ability to drive the heart rate up anymore.

I think I made the right decision, but I think my head is getting in the way of smart training. Quality over quantity. And who knows, maybe this is the way to keep from getting injured!?