I find this to be super annoying as well. I have an iPhone and it does the same thing. I can be listening to a different app and answer a call when I hang up music starts.
Best option is to leave the radio off. Things like directions and messages will still alert through the radio