“It was really hard. I know we both did quite a bit of damage to each other," the "Suite Life of Zack & Cody" alum revealed on "Call Her Daddy."