You don't always find yourself facing audiences who are smiling the whole time. This is natural. As a speaker it is vital to prepare well and believe in what you have to say. You can't please every person in every crowd, but your message can matter much more than if you were only aiming to entertain.
