"I'm like, 'Judy, I don't know. Listen, if it didn't work out, you're the last person I want to piss off,'" the "Saturday Night Live" alum recalled saying.