Checking the phone number never was useful. It is my understanding that it is as easy as sender email to spoof, but without any of the protections that exists for emails.

The only way to be sure of who you are talking to on the phone is to call back a known number.