"Mr. Abercrombie called me almost every night," the hotel heiress writes in her new memoir, noting it "took decades" to see herself as a "victim."