Contacts saved on the SIM remain with the SIM and swapping will not transfer the contacts onto the new SIM you have swapped to. Contacts saved on the phone will remain.