Back in the GPO days ... you would be charged line rental including the first phone and if you wanted a "non-standard", different coloured phone, or the "Trim Phone" or an extension, or external bell, or ... there would be an additional monthly charge!
Registered hard of hearing used to get a bell FOC.
Doubt it is causing noise though, unless the noise is a constant hum caused from unbalance.