If he can walk (or cycle) a decent distance without collapsing then take him.
I hadn't been on holiday with my parents since I was 15, my mother died when I was 42 and by the time my father was 74 I decided I should go on holiday with him. Not living in the same city as him I had no idea of his general level of health, well how it had deteriorated. I took him to Cuba for two weeks, but it was at the airport that I realised what a struggle it could end up being, usually a 10 minute walk for me to the terminal took us over 30 minutes, he just couldn't walk very far at all without having to take a rest.
We had our up and downs in Cuba, but I remember fondly seeing him ride a horse for the first time, watching him devour ice cream like a 14 year old child, we caught up on a lot of talking, with hindsight there are things I could have organised better to help him (I should have guessed even though I didn't actually know of beforehand) and we did have a great time. Within two years he died, so it turned out to be one of the best things I did, for him and for me.
So, yes take him.