“We’re fine. I don’t know what y’all doing but find something else to do because we’re fine," the "Family Feud" host said at Invest Fest Sunday