Strange issue with IP allocation and stability

So, bit of a strange issue here, I'm not even sure which forum to put it in.

Been using 4G from SMARTY (Three network) as my home broadband for 2 weeks, it's working really well with most websites and services. Some websites seem to have issues with loading content in the background (ie. Twitch.tv will buffer every minute or so, despite a stable 50Mb/s connection), which goes away when using a VPN. I assumed it may be down to the public IP address changing (something that cannot be disabled), so I created a small test site to track changes to the IP (can be viewed here).

It seems the IP address changes with pretty much every single new connection (ie, every 1 second or so), which is a lot more frequent than I would have expected. Keepalive seems to keep sessions open with the same IP for about 30 seconds. Is this normal and expected behaviour with LTE?

Second part, it appears that on all desktop browsers (Chrome, Edge & Firefox) that background AJAX requests frequently have issues, either timing out or failing due to 'ERR_CONNECTION_RESET'. Interestingly, Chrome on Android does not have this issue at all, connected to the same 4G router via WiFi. Any ideas what's going on and how this could be resolved (without needing to connect to a VPN 24/7)?

Screenshots:
Desktop (Chrome): https://i.imgur.com/Tr4slDE.png
Android (Chrome): https://i.imgur.com/a2ExPrH.jpg (via WiFi, using same 4G SIM & connection)
Twitch network debugging: https://i.imgur.com/oI3VEX0.png

EDIT: The timeouts seem to occur quite less often when testing with another desktop PC, it seems my PC may be the culprit. No ideas on why, as using a VPN or alternate WAN connection makes the problem go away.

Edited by thebigdolphin1 (Thu 12-Aug-21 22:56:43)

Okay, disregard - I'm confident this is an issue with my Windows installation or NIC driver. Other PCs (and even a linux distro on the same machine) don't experience these timeouts or connection failures. If anyone has any ideas on what could be the cause, please do let me know as I still haven't found a fix.

That internet connection is *not* normal or stable. A public IP address that is changing on every page connection attempt and/or every single second is categorically not behaving correctly.

Clearly your machine when running windows is having issues with it. The other machines and OS's appear to be able to "paper over the cracks" in the poor quality of the connection.

You may be able to tweak some setting on your windows box to help, perhaps increasing the MTU size for example, but the root cause appears to be the erratic/unstable public IP address allocation.

I'll have to put the SIM in my phone and try a different mast to see if it's just a local Three issue, though after speaking to some others it seems Three/SMARTY aggressively reassign IP addresses. Worth noting that once a connection has been established, the connection retains its public IP for the rest of its lifetime, its just each connection may have different public IPs independent from each other. I've noticed it causes a few problems with sites that log you out on IP changes, or CDNs that host content using an IP-based policy.

I have a feeling it's something to do with my NIC's driver software, they put a bunch of QOS stuff in there and I can't seem to find a generic driver that fixes the problem. Unfortunately I can't find much online as it's now a discontinued product.

Badly configured (CG)NAT can result in a different IP for every new connection, it does help distribute client traffic but breaks many services, most notably Sky on-demand.

Three seem to have a lease time of one minute so renew every 30 seconds, using the 3internet APN to obtain a public IP the address is mostly renews successfully whilst the LTE connection is established, infrequently the renewal is NAKed and a new address assigned.

What happens when you use a different APN?

Others may know better but I can't see how an issue on your LAN would cause the WAN IP to change.

Can't be done with SMARTY, they manually override the APN (you can enter any value and it works the same).

I'm not suggesting the local NIC is causing the IP to change, but that it's the reason connections keep timing out or dropping on my machine. The WAN IP obviously changes to Three's IP assignment policy, but no other devices seem to share the connection issues.

In reply to a post by thebigdolphin1:

I have a feeling it's something to do with my NIC's driver software, they put a bunch of QOS stuff in there and I can't seem to find a generic driver that fixes the problem. Unfortunately I can't find much online as it's now a discontinued product.

What's the TCP offload setting for the NIC driver (not the same as Windows)?

Heh, forgive me a little here but this is going into territory I'm not familiar with. Looking at the NIC's device properties, 'TCP Checksum Offload (IPV4)' (I assume that's what you're after?) is set to 'Rx & Tx Enabled'.

Yep set to 'Disabled'. Also set Large Send Offload V2 (IPv4) to 'Disabled'.

Reboot machine. Go back to these settings and ensure settings have remains disabled.

Test again

I have no experience of SMARTY but I just counted the public IP change over 20 times in less than 1 minute on both your Android and desktop screen shots, surely this isn't right, you really need to try the sim in a different device like a mobile phone to see what happens.

Edited by deleted (Sat 14-Aug-21 22:59:43)

Nay that doesn't seem to change anything unfortunately 😕

I did notice that after changing settings the NIC stopped working and refused to be re-enabled ('code 10'), and logged a few meaningless errors visible in the Event Viewer - but it's back working after a restart. Seems like there might be a driver issue, I'll have to see if I can find another that's compatible but I've had no luck so far.

What NIC is it?

Just tried with my phone's SIM (Three) and it's also experiencing IP changes every second or two, so it's not just a SMARTY issue. I'll go out and try a different mast tomorrow, but I have a feeling that's just how it's designed to work with Three's network. I'll also put a support request in and see what they say about it.

Ah sorry, should've said; Killer E2400 Ethernet. Seems Intel recently bought over the company and no longer offers drivers and support for the device as it's around 7 years old now. Tried installing a driver I downloaded 2 years back and Windows seems to ignore it - the install process goes through (or sometimes freezes indefinitely), but the driver version remains the same - perhaps it's the same version as I have now. EDIT: Yeah it's the same version as Microsoft provide, just looked in the INF file. Not sure if any later versions exist.

Right now its showing as version 9.0.0.42 provided automatically through Windows, dated 2016.

Edited by thebigdolphin1 (Sat 14-Aug-21 23:28:34)

Did you say the same machine & NIC running linux was fine?

The other thing to do would be to try another NIC with windows.

Yeah, same PC works fine running Linux from a bootable USB, so it's not a hardware issue.

Ha, you read my mind mate; just realised I had an old PCI WiFi NIC lying around (it's not anything great, one of those cheapo things off Aliexpress) and gave it a try with the Ethernet disabled. Seems the NIC isn't at fault then - timeouts and connection reset errors are also occurring through that card.

I'm wondering if some VPN or packet monitoring software is interfering with it — I thought I disabled them all when trying to fix this the other week, but maybe not. I'll try uninstalling Wireshark and my VPN apps to see if that restores any stability.

Edited by thebigdolphin1 (Sat 14-Aug-21 23:50:13)

In reply to a post by thebigdolphin1:

I'm wondering if some VPN or packet monitoring software is interfering with it

Extremely likely culprits, especially if they are not fully uninstalled (rather than just switched off). Some VPN software embeds itself quite deeply into the windows networking stack and can screw things up.

Frustrating, uninstalled about 8 different network-related apps and still no luck with that.

Next step I'm gonna start killing programs one by one to see if anything improves, then I might make a fresh install of Windows on a second drive to see if that suffers the same problems (I assume it wont).

Same hardware running linux doesn't have an issue -> therefore the issue must be with Windows/NIC driver and/or other software under windows.

Clean W10 install to second drive and boot from there, sounds like a good plan to conclusively prove it.

In reply to a post by thebigdolphin1:

Just tried with my phone's SIM (Three) and it's also experiencing IP changes every second or two, so it's not just a SMARTY issue.

For interest, just checked the public IP on my phone over a 10 minute period and it remained the same so it must just be the way Three\Smarty is setup

In reply to a post by dect:

For interest, just checked the public IP on my phone over a 10 minute period and it remained the same so it must just be the way Three\Smarty is setup

You've probably come to the conclusion I did on Thursday

In reply to a post by thebigdolphin1:

Next step I'm gonna start killing programs one by one to see if anything improves

Security software, and drivers can't be "killed" as they don't run in user land but interface with the kernel. They need to be uninstalled.

If you have any third party security products, start with those.

Super weird but somewhat good news at least. Fresh Windows install works completely fine, so there's something cocked up in my current installation. Ideally I don't wanna have to reinstall and reconfigure anything, but I'm honestly out of ideas on where to look next. Tried disabling AV and Firewall (both built into Windows) and that's shown no improvements.

That’s quite positive actually. A slow process but if you add back software one step at a time you will eventually find out what ‘broke’ it.

The most important part of that being "one step at a time" .

I think I'm just gonna copy my files and get rid of this current installation, I've spent far too long trying it the other way around (removing suspicious apps/drivers one by one) and getting nowhere. I have a feeling its just some weird config or driver left behind from years ago, I've been using the same Windows installation for 10 years now and it's just been upgraded over time.

I'm surprised my system hasn't shutdown (yet) from all the services and drivers I've killed.

Edited by thebigdolphin1 (Sun 15-Aug-21 18:40:02)

Good move. Windows does well with a clean slate and 10 years is a *very* long time on the original build.

You really should upgrade to Windows 10 as well. If you're running a 10 year old version, it's probably Visa or 7 right.

I didn't want to upgrade to W10 from my trust W7 setup, but now I have, I couldn't go back.

You can upgrade for free as well usually.

https://www.theverge.com/2020/1/14/21065140/how-to-u...

Nah I meant it started on 7, and has been upgraded to 10 over the years without doing a complete reinstall. I mostly wanted to hold off for the release of Windows 11 next year, but I'd rather just get it over and done with now as it's causing problems.