Probably expected, I'm afraid. The vast majority of modern UPSs, especially the affordable ones, don't generate nice clean sine waves (which would have little harmonic content and hence little RF noise). Downmarket ones use what the vendors generously call a stepped sine wave which will splatter RF noise in all kinds of ways. Others generate pseudo sine waves using pulse width modulation, which may be almost as bad.
Random thought: how serious are you about this? Running direct from DC rather than UPS is perhaps an option to avoid the RFI if you are serious and technically capable (or have a budget

). It should be reasonably easy for the router, but running a PC and monitor won't be so easy. A laptop, on the other hand...
In my box of bits at home I have the parts for a 300W MOSFET-powered pure sine wave inverter (nb inverter not UPS), from a design in Elektor many years ago. Being sine wave it will be electrically cleaner, but being sine wave it will be inevitably inefficient, and the switched mode ones are both more efficient and cheaper (no expensive transformer in many cases). I haven't actually built it yet, but maybe there are similar designs still around. A 300W 12V toroidal transformer was hard to find then, it may be harder today.
Take JonRennie's advice first, keep things as far apart as possible. Get a few clip-on ferrites from Maplin and put them near to the UPS on cables attached to the UPS, and so on. Maybe filter the LAN cable from the PC? Just play around and see what helps.
If your UPS has a "surge filter" for the phone cable, ignore it. Direct from wall socket to router, as best you can.