As long as your device was in "Flight Mode" then it should be fine.
The somebody who was sitting next to you was just being a busybody as Pipexer mentioned.
That's why Flight Mode present on most modern portable devices - It disables all of its communication technologies (3g .etc) and its WiFi capability so it doesn't interfere with Aircraft equipment.
As long as it is in this mode, it won't make a difference whether if it's on or off because it doesn't have the ability to transmit or receive any data or cause a disturbance to anything on the aircraft (Apart from maybe certain people who get over-worried about such things and don't know how they work).
If it was such a bad thing, then I'm sure we would have seen numerous articles over the internet every day regarding such an issue.
-------------------------------------------------------------------
Downstream Upstream
Connection Speed: 24276 kbps 1211 kbps
Line Attenuation: 16.5 db 6.4 db
Noise Margin: 1.1 db 6.2 db