The infamous talent manager knew his "new life" was going to cost him his "relationships and friends" amid an alleged mass exodus of his A-list clients.