So even though the cable runs outside my house they are not willing to consider!
It seems daft, but you are not alone in that boat - I've not seen a single case where BT are willing to rearrange a network to make it viable for a single house.
We have recently seen a case where a whole new cabinet has been inserted, splitting an old one (presumably like was done for your two cabinets). In fact, the cabinet numbering near me suggests exactly the same happened here years ago, though both cabinets have been converted. But that is what a network re-arrangement requires - whole new cabinets.
One problem of handling you individually is that, if they did this for you, then the set an expectation for your neighbour. And then for his neighbour, and so on.
From their perspective it is better for the entire cabinet to be done... and right now, the best prospect of that is to harness some BDUK funds.
Note: Just because the cable goes past your house doesn't mean that they're capable of diving into it to extract 1 pair for use with you. The cable carries tens or hundreds of pairs, and can only realistically be broken into at the cabinets. Then smaller cables (but still with many pairs) distribute out to the DPs. It is only here where you can make use of a pair to a property - but of course there must be a spare one around.