It is a good point, it may be that with a faster link the router is having more problems tracking the NAT table, and falling over due to that.
Alas routers are very bad in offering diagnostics in this area, try with a small number of connections and then increase them, i.e. learn what the router can cope with.
P2P with its use of UDP and many simultaneous connections to different places on the internet is a big issue for routers, and similar at the ISP level