Easy call the number you know or go to the website you trust.
When I get a call from my bank and I am not expecting it and don't reconfinze the number, I ask for there department name and some way to indentify them and I call the number I know is my bank and go from there. If it is a scam, bringing it to my banks attention means they might help alter other possible victims. They have a vested interest not to allow your account to be compromised too.
Perfectly reasonable concern, but I also think easy to solve. If conversation was not intimated by you conclude and initiate the way you know you trust.