I think two people are talking here about riding a horse. It could be also a motorbike.

One of the guys must just have been complaining about the horses or vehicles being unsuitable to ride. I have added a few more words to meet the mood of speech of the original sentence.

"Then don't get on it and you will not fall. It's that simple man. Look at me. Do I fall down?"