City subjected to false rumors from Trump loses residents integral to community over mass deportation fears