Answering the phone disconnects Infinity?

Anyone got a clue what I could try to stop this from happening? Not every call does this but usually the calls that arn't local.

Try putting a dsl filter on the phone.

Almost certainly, that your installation was done incorrectly/poorly. Ring them up and complain till they send an engineer to fix it.