What you should do is tke the switch out of the equation.
Connect your PC or Xbox direct to the ADSl router and turn it on. This will then gets its IP address etc using DHCP and should work at the 'best' speed. Mane sure you are happy with the repnse/game play. With these all powered on, then put the switch between them, making sure not to turn off the xbox etc... Just links the cables as needed, then retest. If the speed goes down and is not working correctly, then this is either the switch (try different ports) or one of the cables (try different ones). If it is slow like this, and it should be ok with no difference, then the switch/cable is faulty.
IanD