Remote storage

I'm looking at signing up for Amazon's S3 service for remote storage of my photos. I have a few hundred gig to put up but was wondering If anyone had any alternative suggestions that are worth considering.

My main concern is that with the current economic climate, I don't want to just choose the cheapest solution; I want to make sure that my photos are still where they are supposed to be in 2 years time!

I don't plan to use any remote service in a customer-facing context - it is for me to store my photos and maybe get to them if I need one when I'm not at home and that's it. I'll still be keeping them locally as well, but decided a remote storage solution was cheaper than building a full-on RAID in the pc (I just have a couple of 750gb disks and a 500gb usb drive for photo storage atm)

I don't have any practical experience but I wouldn't trust any remote storage company. A couple of high-capacity external disk drives should be all that is required.

Hard disks are famously quoted as being in one of two states - crashed, and about to crash I do have local storage but want something as an addition to this. I do get the odd occurrence of needing an image for a potential customer/client whilst I am somewhere other than home so an internet-based storage solution seems a good option

I have never experienced a hard disk failure on my PCs, but then I do tend to steer clear of bleeding-edge technology, and yes, I do have back-ups.

P.S. A cross-post in Free Chat should elicit some responses since quite a few posters there use on-line storage.

In reply to a post by micksharpe:

P.S. A cross-post in Free Chat should elicit some responses since quite a few posters there use on-line storage.

and some randomly stupid answers

Just use a raid 1 (mirrored) or raid 5 (parity) set at home. It's all they'll be using on a remote server (if the use it at all).

Edited by deleted (Fri 06-Mar-09 13:49:52)

and some randomly stupid answers

I'll have you know we're highly intelligent and very helpful.

We just get carried away at times.

In reply to a post by micksharpe:

and some randomly stupid answers

I'll have you know we're highly intelligent and very helpful.

We just get carried away at times.

yeah and you know I do try to help in this and the hardware forums

I costed RAID up last year and decided it wasn't worth the money tbh

Err from what you've said about how much data you have to backup (2x750GB + 500GB) you'll be paying amazon ~£200 a month for storage. You could mirror all those disks for that.

I'm pretty sure a figure of 200GB was mentioned - not 2TBs.

Local storage is fine as long as you don't get a local disaster.

Remote access could be provided to the local storage but an external provider could be better as then they have to deal with security.

However, don't have any experience of using them as I don't store anything that is more signfiicant than a major disaster that would cause my 3 local copies of everything to be destroyed.

EDIT: Additional comment. I am assuming that this is the US service? That might affect speeds of data access but also the US don't recognise the data protection of other countries for data stored over there. So, if you have data (photos) which you would consider sensitive then be careful about the terms of how that is handled under US law.

ANOTHER EDIT: Forget the comments about US hosting as I see they do EU as well.

Edited by ian72 (Fri 06-Mar-09 16:21:55)

In reply to a post by ian72:

I'm pretty sure a figure of 200GB was mentioned - not 2TBs.

Which would be £20, which would get you a sata raid controller for the 2x750GB disks already owned. Still cheaper.

Local storage is fine as long as you don't get a local disaster.

True I suppose. Somewhere like rapidshare would be cheaper for less than 500GB, although perhaps not as secure.

You dont even need to buy a raid controller. You could just do a software raid. The £20 raid cards dont offer anymore performance over a standard software raid anyways.

They dont have the hardware to perform the mirroring on them, its only when you start getting to the £80+ cards you start to see a proper controller card where it is the hardware on the card doing the mirroring.

Personally I wouldnt trust a company with 200GB of my data if it was important also the problem of accessing the data download/upload would be quite time consuming.

I would go for the raid option and keep my data at home. If remote access was needed then I would setup either a vpn or an ssh tunnel with a share running inside it.

In reply to a post by jonathanHodd:

I'm looking at signing up for Amazon's S3 service for remote storage of my photos. I have a few hundred gig to put up but was wondering If anyone had any alternative suggestions that are worth considering.

My main concern is that with the current economic climate, I don't want to just choose the cheapest solution; I want to make sure that my photos are still where they are supposed to be in 2 years time!

I don't plan to use any remote service in a customer-facing context - it is for me to store my photos and maybe get to them if I need one when I'm not at home and that's it. I'll still be keeping them locally as well, but decided a remote storage solution was cheaper than building a full-on RAID in the pc (I just have a couple of 750gb disks and a 500gb usb drive for photo storage atm)

I have been using Flickr for several years with no problem.
http://www.flickr.com/
There are two types of account available, a free account in which there is a maximum display of the last 200 photos uploaded, but, no access to the original high resolution photos and the Pro account which costs $24.95 a year, has unlimited storage and display and you can access your full resolution photos. You have options to categories of people who can see your photos from everyone to making them completely private so that only you can access them. Codes are available through every photo page to enable you to embed the photos at any size and link back to the photo page.

Flickr is primarily a photo sharing site, but, it can and is used for archiving as well. As with all on-line sites you are not advised to rely on it as your sole archive, anything can happen in the Internet so you should keep a physical back-up at home either on HD or CD/DVD or any other reliable media.

Edited by deleted (Sun 08-Mar-09 12:04:06)

I am a great believer in a good quality (Expensive) raid systems, they can be great. I have been using RAID in various shapes and sizes since the late 80's.

I have however had a couple of very bad experiences with a cheap raid controller, and would not go there again.

However RAID only covers you for a hardware disk failure it is not a complete solution, it does not cover you for "finger trouble or O/S hickups" deleting data, or local disaster (fire or theft). You are thinking along the correct line in having an offsite copy of your data.

My current solution is to have the primary local storage on a 3ware raid controller and use Jungledisk with Amazon S3 as offsite backup for our critical data. We have about 60GB there. It just seems to do what it says on the tin, and I hope that Amazon will be around for a while.

With a standard MAX ADSL line we can upload just over 2Gbytes per day, but our daily upload is usually a few hundred MBytes. One criticism would be that the download speed appears to be far less than our ADSL line speed.

Tim.

Just my take on things.

Two backups. One internal on a dedicated hard drive (you can even disconnect it if it's very rarely used). The other backup on an external USB drive.

Personally I'm not a fan of remote storage, but that's because I've never used it. I'm also less keen on RAID 1 or 5, since it involves familiarity of RAID systems, trials of backup and restore, and regualr refreshers or you forget how to do it when the time comes.

With my preferred system, you have easy to access data, with little or no experience required.

Thanks very much for all of your replies, you've given me lots to think about, but I'll just clarify a couple of things:

The images are currently stored on a couple of 750gb internal disks, nowhere near full and this will continue to be my primary store going forward.
My 500gb MyBook is considered a secondary backup, and once that's full, I'll buy another. It's not on very often, but on often enough to make sure nothing sticks solid

What I am missing at this point in time is a way to get to my images whilst away from home, and also some form of off-site storage. Although a home RAID is an option for some protection against hardware failure, it has no protection against burglary or other home-based catastrophe, although I guess a pc laden with disks would be pretty heavy I'm no stranger to mucking around with hardware, and my unix box used to be water-cooled for a long time, back in the days when I had 5 external ips and a whole suite of computers at home (I've somewhat rationalised and simplified my collection now, 'just' 2 pcs + laptop)

I might be a little paranoid, considering just one of my images has been syndicated to HBO and ESPN for a significant amount of money, I'm keen to make sure I have a sensible level of protection from disaster, at a sensible cost

I'm quite tempted to try using my flickr account as a store - it's actually something I feel daft for not thinking about myself, it'll cost me no more money and it doesn't have to be 100% reliable - I will just use it as a 3rd-string backup strategy

Once again, thanks for alll taking the time to consider and reply to my query

In reply to a post by jonathanHodd:

Thanks very much for all of your replies, you've given me lots to think about, but I'll just clarify a couple of things:

The images are currently stored on a couple of 750gb internal disks, nowhere near full and this will continue to be my primary store going forward.
My 500gb MyBook is considered a secondary backup, and once that's full, I'll buy another. It's not on very often, but on often enough to make sure nothing sticks solid

What I am missing at this point in time is a way to get to my images whilst away from home, and also some form of off-site storage. Although a home RAID is an option for some protection against hardware failure, it has no protection against burglary or other home-based catastrophe, although I guess a pc laden with disks would be pretty heavy I'm no stranger to mucking around with hardware, and my unix box used to be water-cooled for a long time, back in the days when I had 5 external ips and a whole suite of computers at home (I've somewhat rationalised and simplified my collection now, 'just' 2 pcs + laptop)

I might be a little paranoid, considering just one of my images has been syndicated to HBO and ESPN for a significant amount of money, I'm keen to make sure I have a sensible level of protection from disaster, at a sensible cost

I'm quite tempted to try using my flickr account as a store - it's actually something I feel daft for not thinking about myself, it'll cost me no more money and it doesn't have to be 100% reliable - I will just use it as a 3rd-string backup strategy

Once again, thanks for alll taking the time to consider and reply to my query

What might be worth doing is creating two online accounts. One with low res photos (with perhaps watermaking) and use that for a web portfolio for your clients. and one private one which has the hires back ups ....

Mick,

I know this thread is a bit old but I'm just interested as to why you don't trust online data storage / backup providers?

I personally use humyo for live online data storage, I also use our own offsite data backup service.

For business, all business critical data is stored on redundant arrays, then backed up offsite.

Backup remotely, locally and to friends/family/work computers ;O)

http://www.crashplan.com

Any of the remote storage sites will be fine. If you are worried then use 2 and duplicate all images across them and pick ones in different countries to be doubly sure!

I use smugmug for photo storage. They have an unlimited storage option and are pretty reasonable on price. Their pro with galleries and the whole shebang is $150 pa which is fairly reasonable. I am on the normal unlimited plan which I think is around $60 pa. Don't know what the file size limits are. They have been increased recently but if you are into large TIFFs then the pro might be your only option. You can set galleries to be invisible and password protected so you don't need to share any photos if you don't want.

Build a Windows Home Server and you have the added functionality of being able to dial-in from outside. Or just use LogMeIn on a standard machine.

What's wrong with remote desktop or vnc?

two words that will never be combined in my house - windows and server

i used to have 5ips and a collection of boxes all with their own name, some of which were on 100% of the time (one linux box was even water cooled for a laugh) and accessible by me from work, but I've simplified my collection (and my life fwiw) these days and have no interest in going back down that route

Remember that for VNC you need to know the remote end IP address (also, need to configure router at that end to forward traffic to a particular PC).

LogMeIn will allow a connection easily for a user with dynamic IP and multiple PCs using DHCP (which is how many people set up a home network, not always using fixed IPs on the LAN). It is a lot easier to talk a new client through setting up LogMeIn on the PC with a problem than explaining how to reconfigure their router to forward to the specific PC, fix the IP so that PC won't switch addresses, etc.

Some IP scanners try to probe for VNC since it is easy to test ports 5900/01/02... and 5800/02/03... Cannot comment on pros/cons of remote desktop, not used it enough.

In reply to a post by jonathanHodd:

Hard disks are famously quoted as being in one of two states - crashed, and about to crash

What do you think remote storage companies use to store things on exactly?

seb

In reply to a post by seb:

What do you think remote storage companies use to store things on exactly?
seb

Nothing quite like quoting out of context is there One would hope that any reputable storage company would be using arrays and backups etc. I would rather pay for a service like that than go buy all the hardware required as it would best fit the income model of my photography

It's all moot as I'd mentioned in a previous post that I'm uploading them all to flickr. As my employers are having images off me each week for free, I'm evening things out by using company bandwidth to upload the images to flickr so I can provide them with what they want, when they want it.

In reply to a post by jonathanHodd:

Nothing quite like quoting out of context is there One would hope that any reputable storage company would be using arrays and backups etc. I would rather pay for a service like that than go buy all the hardware required as it would best fit the income model of my photography

Firstly, you can buy a 2 disk NAS array easily with two 2TB drives in RAID1.. Of course you have risks in that (e.g. fire) and I'm not suggesting an online service woudln't be good, but I've seen so many ISP infrastructures that I rarely believe the sales talk on their websites..

I wouldn't be that sure they take backups either.. That would cost a lot to do (more disks needed.. or lots and lots of tape..)

Best option would be to use local hard drive plus online storage


seb

I thought they kept the bits and bytes of data floating in a cloud by magik.

In reply to a post by camieabz:

I thought they kept the bits and bytes of data floating in a cloud by magik.

Imagemagick