Their wording is wrong, if no fault is found with their wiring, but the fault is found to be on something which is your responsibility then they may raise the £180 charge.
It is NOT the case that an engineer attending and finding no fault will raise the charge, if this were the case, then no intermittent issue would ever get looked at.
In this case it needs to be looked at carefully on the DSLAM logs first, before assuming a fault at your end. Though ruling out your hardware not talking very well to the DSLAM is very important.
All modems if they see a transient fault will retain the sync caused by this transient fault until someone intervenes to reboot the modem. Thus it may not be a fault with DSLAM or router, but simply some random outside noise.