Curious what everyone else is doing with all the files that are generated by photography as a hobby/interest/profession. What’s your working setup, how do you share with others, and how are you backing things up?
As OP, I should probably answer, too.
Normal workflow is:
- Raw images import from SD card to network drive (physically on spinning hard drives in a NAS in my home).
- Lightroom catalog is on my laptop, where I actually do all processing and editing, using a network connection with the raw files stored on the NAS.
- Exports go to local folder on the laptop, to be shared however I share.
Travel/offline workflow is:
- Raw images import from SD card to external SSD.
- Lightroom catalog is on my laptop, and I can do all editing there as part of the same catalog even when I don’t have access to my NAS (or when my connection is slow).
- When I get home, I manually copy the files from the hard drive to the network drive, and then update the library file location in Lightroom to point to the new location.
My raw backup solution:
- Cron job backs up raw files daily to an external drive hooked up to my old always-on Mac Mini.
- Mac Mini syncs with a cloud backup service that charges per machine rather than per terabyte, so I can take advantage of the pricing that works out to be $8/month for about 8TB of data.
My post-processed backup solution:
- All exported, post-processed pictures get copied to Google Photos, for all the functionality of cloud syncing/sharing, face recognition, etc.
- All photos taken directly on my cell phone are also automatically synced to Google Photos.
Things I’m generally considering:
- I’m always on the lookout for de-googling
- I’m also all for making the workflow simpler/easier/more reliable between my files stored locally on the laptop, on the external SSD, and the main NAS storage drives.
- I might switch the NAS storage to SSDs instead of hard drives, but that’s lots of money just for a little bit of extra speed.
- I might want to switch my cloud backup to live cloud features instead of a cold backup, but I like the pricing I have with cold storage.
- I’m wondering whether it’s possible to have an automated solution that always keeps the most recent 50GB of raws on my local laptop, maybe 500GB of raws on a networked SSD, and the rest of my files on networked hard drives, while seamlessly updating the older files to the slower locations. I’ll have to think on this some more.
Just curious, where are you able to get 8TB for $8/month?! That’s awesome!
It’s an elaborate workaround for Backblaze’s pricing. What I was alluding to in my comment was that they charge $8/computer for backup, but without any limit to the amount of data they’ll sync. They won’t let you sync network drives, but they will let you sync external USB drives, so syncing a network drive to an external drive allows you to basically sync that external data to a cloud backup.
That way I have 3-2-1: 3 copies, on at least 2 devices, on at least 1 remote location. For $8/month (but also, some up-front investment in physical stuff I had to buy).
Ah! Gotcha, good workaround! :)
I am a hobbyist photographer. My files come off the SD card from the camera to a external SSD that I use with Lightroom. That drive and the Lightroom Catalog backups get synced to a local NAS that I’m running at home. The NAS gets regularly backed up to a drive that goes to another location. I have two drives for off-site backup so there is never a time when there isn’t one off-site. I’d like to replace the off-site drive with cloud of some type. I was previously bandwidth limited but now that I’m not I need to do it.
I feel like I haven’t figured out a good sharing solution yet. I usually just use Google Photos to create and album and share that with people. I used to use OwnCloud but I’ve moved away from that.
For backing up I have a cron job that use rsync to transfer files to a cloud storage location on rsync.net. It’s not the cheapest around but it has been 100% reliable. Since I am also handling backups for my family’s data, I depend heavily on it. Also, it is standards based so no proprietary clients or any of that nonsense. You have a choice of standard clients like: rsync, borg, sftp, scp, rclone, restix, and git-annex.
Stay away from commercial pay-for backup and restore programs. At least one commercial exception is the free version of Veeam, which I use to backup and restore the OS disk. The free version of Veeam doesn’t require you to go hunting for your product registration key when you attempt to restore on new hardware, assuming your old hardware died. (The same can hold true on same hardware where you only replaced your hard drive.) It is a major pain to not have your product registration key available to you when you are trying to perform a restore. You probably won’t. In this situation, many years may have pass buy and folks may not remember where they stored the key. It’s even possible for some people to never find the key. In that case, those folks have lost access to their data forever, unless the company is willing to give you a key for free. Otherwise, you have to buy the software a second time and hope the newest version (which could be 8 years newer) is still compatible with the old version’s database format (should be). Feels a little like ransomware doesn’t it?
Free software is fine as long as it has a good reputation. rsync and Grsync come to mind.
For photography, I just want replicas of my files… in a flat file system. Good old NTFS, … you know, the standard Windows, or Mac, or Linux file system. I don’t want a software front end and backup and restore pitfalls between me and my images.
~[Editorial: I mirror but I don’t recommend mirroring to others. Instead I recommend just regular copies. See why in the disclaimer below. A mirror is just a replica but it is different technically than just copying files. A mirror will also delete deleted files. That’s what I want but it is dangerous and prone to user error. Instead, just copy them and keep all versions on the second and third drives. This way nothing gets deleted by accident. It’s messy but safer.]~
So I mirror the photography drives onto other drives, two others, and keep one at an off site location and rotate it in and out. When one drive fails, no problem. Just buy a new drive and mirror one of the other two and I’m back in business. No backup and restore software to deal with at all.
Why? Because Raw images are already compressed, and so are Jpegs, and Tiffs don’t compress well. There is nothing to be gained by compression used in standard backup/restore software… and much to be lost… namely access.
For windows I actually use Robocopy for photography data. It’s included with the OS and if you do your research you’d probably find what I did, that Robocopy is the most robust file copying program out there. And it can perform the copies while the system is in use, etc. Highly reliable. And then, as a front end for Robocopy, I use RoboMirror. RoboMirror makes Robocopy easier to use. Otherwise, Robocopy has no front end of its own.
A warning / disclaimer about mirroring A mirror of your data is not a backup. It is just an exact copy. What this means is that there will be no way to backup different versions of the same file over time. It also means that your pictures are susceptible to user-error. Mirroring also mirrors deleted files, meaning, if a file is missing from the source, and it exists on a target, the deletion will be replicated on the target. This means that you have to be damned sure that the data you are copying from has the full set of files because if not, you may be unknowingly deleting any files that exist on the target but that don’t exist on the source drive. If that’s what you want, then it is fine. Just make sure the source drive has the full complement of files before you do a restore operation. I do this by making sure the source and target sizes are about the same, wherein the difference in size is explainable by the new files on the source drive.
This is what I want because it deletes deleted files that just take up space. But again I don’t recommend it for others because of the potential pitfalls. For others I recommend regular copies. RoboMirror allows for this, i.e., copying and not mirroring; it’s just a check box.
Here’s an example of a disaster that, while it wouldn’t happen with backup and restore software, it would happen with mirroring (and why I recommend copying instead of mirroring). Let’s say you have a software-related error on your photography drive. Let’s also say the error wipes out half of the photos you’ve ever taken. Further, let’s say you aren’t aware of it. You update a few photos no problem. Then you do a mirror (not a backup, nor a straight copy of the new files). What would happen? You will have just wiped out half of the photos on your backup drive. Your only hope then is to realize what’s happened and restore everything from the third drive.
Hobbiest here.
I import to a folder on my desktop, run the photos through photo mechanic for culling/sorting, and transfer a NAS for long term storage. The NAS syncs with OneDrive using the 1 TB that comes with office 365. Haven’t run into the limit yet, but when I do I’m not completely sure what I’ll do.
I’m going to attempt culling and sorting via iPAD, but I’m kind of dubious that it will be anywhere as fast a photo mechanic. If anyone is already going the iPAD I’m interested in hearing what software you’re using.
Files moved to PC after a shoot, weekly sync to NAS, monthly backup from NAS to two portable hard drives that are rotated.
Lightroom not supporting network drives is a nightmare, could do with a way to automatically sync the catalogues back and forth but not aware of a good one.
Running lightroom cache from an SSD and images stored on a HDD, might move to all SSD in the near future.
