For me it's simple the Xbox controller is bulkier and heavier and has a nice weight to it, The PS4 controller doesn't feel right to grip to me because both sticks are at the bottom and you have to space your hands further apart, It feels too light all of which work better with smaller girly hands lol.
Also I heard that the 2 sticks are at the bottom on the Sony controllers to make up for the lack of stick in the downstairs (read trouser department) of the users.
I'm joking of course about the bottom bit but the top bit stands, I think most people actually call it girly as the Xbox controllers are much bulkier and heavier than the Sony counterparts (especially the horrible DS3 without the actual shocks)