The "Scream Queens" star revealed he got "angry" whenever anyone approached the actress for an autograph because he felt it interrupted their time together.