New Hi-Res cameras - file size issues

[gurtch]

• I am a senior non-pro, seriously involved in photography since my first darkroom in 1959. I switched to digital capture around 2000. The files were inferior to my medium format scanned negative files. In 2002, I purchased a Canon 1DS, (full frame 11 mpixel), for about $8,000; it was state of the art. For each camera upgrade the files got bigger. My workflow was, and still is: keep recent raw files and Photoshop processed files on my PC internal drives, (both SSD drives, one for the operating system and programs, and one for active files). I eventually transfer the files to duplicate external storage. When I started out I used CDs, of which I have 157 pairs. The files got too large so I switched to DVDs, of which I have 92 pairs, then to Blueray discs of which I have 8 pairs. The camera files continued to get bigger, and I switched to duplicate external hard drives. I currently use a Sony A7RIV, and Fuji GFX 100S, and the cameras use lossless compressed files, but they are still huge. I know storage gets cheaper all the time, but is it harmful to allow the external drives to compress the already lossless compressed files? To complicate matters, I have another larger external drive, and Backup Program (called AShampoo), to once a week back up both internal SSDs, as well as one of the two external permanent file drives. If I were starting out today, and much younger (I am 84), the cloud storage, for a monthly fee would probably make sense. To give you a rough idea of my files, I have about 900 Photoshop "finished files", ready to print and frame from digital capture. Also about 150 likewise Photoshop finished files from scanned negatives. The A7Riv raw files are about 61,000 kb, the GFX 100s files are about 112,000 kb. Typical Photoshop finished files are about 352,000 kb for the Sony, and 600,000 for the Fuji. I will also post on the Computer Board.
• Thanks a million
• Dave
• Web site: www.modernpictorials.com

 

[Derek Zeanah]

So, an external device applying compression won’t compress anything that’s already compressed; or at least not much. Either way don’t worry about it.

what you probably need is a network storage device from Qnap or Synology. There are lots of options and variables, but at the end of the day you will have a dedicated storage device that can store a lot, will have lots of drives configured so if one (or more) fail you won’t lose data, and it can automatically upload your data to Amazon or Backblaze (encrypted if you prefer) so if your house burns down your data is safe.

I'm happy to answer questions or advise.

 

[gurtch]

Thanks Derek. Too late to backwards cloud upload all I have stored in the past to various cds, DVDs, external drives etc. But maybe I should look into using cloud storage for new files, and just "retire" what I have already backed up to a third larger 4TB drive which includes every thing on my two internal drives.
Any suggestions for cloud storage?
Thank you again.
Dave

 

[Derek Zeanah]

I prefer Backblaze - last time I compared it was affordable and simple.

 

[gurtch]

Thanks Derek.
Dave

 

[docholliday]

Pro or not, if your files are important to you, then you should follow the standard 3-2-1 rule of storage: have 3 copies of your data, 2 of them never touched (one-way add), 1 offsite. I'm a developer as well as a commercial photographer and an IT director for multiple businesses, so I move a *lot* of data on a daily basis. My main box has around 20TB of RAID 50+2 SAS storage with all my active data on it. That is backed up to an on-site NAS around with 100TB over 10GB-T, which is then compressed for off-site "cold" storage over the internet.

The reasoning behind the 3-2-1 design is that you only ever touch one copy, the fastest one, and usually on your box. The second copy is kept on-site for fast restore and backup. If you accidentally delete a whole folder, it's quicker and easier to pull from the on-site right back over to the live data. The third is offsite for safety, but is usually over a slow medium, like the internet, so it happens once a night. The third copy is directly compressed and pulled from the second copy, never the first.

The second copy can be simply an hourly differential mirror of the live, a daily backup, or a periodic copy of all changed files from the primary. The third is additive only - it will have copies of files you've even deleted locally. There's times where you'll delete something, and realize days later that you needed that or did not mean/notice/expect the deletion.

I have some 500K+ "finished" PSD files, ranging in size from 50mb to 900GB (and a few that are 1-2TB). Some of these are personal projects, most are client imaging. I keep these so that a client can call upon me to retrieve a past file for a current campaign. These, to me, are "free money", as they've already been edited, approved, and known by the client. For my personal work, I have film scans from back in the my early days shooting MF/LF all the way to current massively large digital files.

My offsite is utilizing the cold storage of the S3 Glacier Deep Archive on the AWS. It's cheaper since it's rarely accessed, has free inbound transfer, and cost $0.00099 per GB to store. I've written my own code that does the work to get it there, but in reality, any offsite backup will be fine matching your budget and needs. Some services offer unlimited storage for a fixed cost, like Carbonite, and that's the way to go. Remember that the issue for offsite is the retrieval - if you lost a file now and needed it immediately, it may take a while to download it back. In the case of the Glacier, it can be 12 hours before the file is even ready for retrieval. For consumer backups, it can take an hour or so to index, pull, and download all your stuff. But, it's better than losing all your work!

Size-wise, expect each subsequent store to be at least 2x the size of the closer, faster one. If you have 4TB of local storage, your second should be at least 8TB (2 full copies worth of backup versions), and the cold store should be at least 16TB (2 full copies of the intermediate backup). These are storage capacities, not data sizes. Just because you have 1TB of data on a 4TB drive doesn't mean that you won't eventually grow that data. You don't want to get to a point where your backup quits working (somethings without being noticed) because it ran out of space.

Never, never, ever, trust DVD/BR/CD media. Never. It'll corrupt or lose your data one day when you least expect it. I did a test a few years back with some friends where we all burned known data files (specific data patterns) onto different media types and brands. We made multiple copies on a Plextor at 8x for the best burn. Validated and tested each burn, then compared the burns to the source files to ensure "perfect" data matches. We then placed some in safes, others in cases on a shelf, and even a set in a disc case in the trunk of a car during autumn (a data tracker showed the hottest it ever got in there was 93F). We tested the discs after letting them sit for 6 months. They all had errors. All of them. One of the discs that was in the trunk even reported as "blank media" and allowed me to re-burn the disc! If the errors existed on a real data set, it would've corrupted an image somewhere while others would've been fine and been unknown until you tried to access that one specific file.

The foils on media are heated by a laser and altered to produce the pits and landings. Heat, gravity, and other factors can cause these to lose integrity and give ROMs difficult targets to retrieve your data. They're great for transferring, temporary storage, or as an additional backup, but should never be relied upon as the primary backup!

Also, be careful using anything like USB flash or SSDs for long term storage. Flash media can die/has died unexpectedly and is much harder to recover data from. I had a client that backed up piles of imaging, as well as their Quickbooks backups to a 256GB USB flash which they kept in their safe. One day, their desktop got hit by lightning and they went to retrieve data from the USB. The process of downloading all the files got the drive hot and it failed about half way through. SSDs are the same way. Great for running your box, acting as a workspace/scratch disk for your current project, but if anything happens to the drive, your chances of recovering any data vs old spinning rust is much lower (and costly).

 

[Derek Zeanah]

Doc Holliday nailed it. To simplify it for those who don't want to get too deep into these topics:

Step One: Make a place to store your data.

Your data is stored on your computer already. What I'm talking about here is a place that sits on your network whose job it is to store data and make it accessible. I default to Qnap brand because of features they supported years ago, but make doesn't matter. I strongly recommend buying a pre-built device that's easy to manage like a Qnap or a Synology. TrueNAS boxes are better, but I chose a Synology for my dad who's 76 and has a tech background because there's real value in simplifying this stuff to the point where you can't accidentally screw things up.

Here's an example of one of these things that sits on your network (I chose this because I have one sitting in the other room):



The red box is a place for 5 hard drives - if you install 10 TB drives, then you can configure these to be

• RAID-5 which leaves you with ~ 40TB of storage, and the system won't lose data if one drive fails, but if two fails you'll need to restore from backup. One drive's worth of capacity is devoted to redundancy.
• RAID-6 which leaves you with ~ 30TB of storage, and you're configured so you can lose up to 2 drives, but a third failure will cause data loss. The equivalent of two drives are devoted to redundancy.

Now, the green box is for SSDs, and we don't need to go into that complexity here. You can use it to speed things up automatically though.

(The details don't matter - a two-drive box would be cheaper and would likely work for your needs, but they make these things in all sizes with all sorts of features.)

Anyway, you connect to a device like this via regular internet or (preferably) 10-gig Ethernet or Thunderbolt. Configure a volume on this device, map it to a drive on your computer (call it the R: drive, for 'remote'), copy your existing data to it, and that data is now protected. Tell your RAW processing program to make a copy on the R: drive each time it imports a new file and you'll be sure to have at least enough of your data on this box to recover if you forget to move the finished files over manually.

Step Two: Make Weekly Offline Backups

I've got a shelf of external hard drives, and every Monday I plug one of these into my Qnap and a backup automatically fires off to move data to the external drive, then dismount the drive. This is my second copy of the data - if lightning strikes the power wires right outside my house and overpowers every UPS and all my electronics get fried, I still have my data as it existed last Monday, and the Monday before, and the Monday before, because I store these disconnected from my computers. Malware that encrypts everything? It won't get data on external drives that aren't connected.

But I can lose data to theft, flood, or fire, so I also....

Step Three: Make Automated Off-Site Backups

Qnap, and Synology, and others support doing this. If you want to use Amazon S3 like Doc uses, or BackBlaze like I do, you can configure your Qnap to backup to your cloud of choice automatically however often you think is appropriate. And you can use some services (like Amazon Glacier) with a setup so that data is saved but never deleted from the remote cloud storage. Want to encrypt it to make sure nobody else can read it? Just make sure you remember your password if your building burns down, but that's built in too.



As far as I'm concerned this is the 'correct' way to manage your data. Centralize it, and perform two different types of backups, and automate what you can.

 

[RobbieAB]

To answer the question asked: in theory compression can make the resultant archive larger than the original, but for your purposes, that is vanishingly unlikely. It probably won’t gain you very much, but it won’t cost much either.

To address the broader points, while Derek and Doc are correct about how backups should be managed in theory, I suspect they are slightly over engineering the solution. From your post, it sounds like you have a little over 1000 images, with 600MB as the largest image. This is less than 1TB total! They are correct about multiple copies, I notice you are already doing that. Cloud storage is an option, but it needs to be priced correctly: it can very easily get expensive if you set it up wrong, with your data as hostage to continuing payment! There are also questions around restore speed, but that is probably not a major concern here.

What jumps out to me is that 1TB is comfortably within the territory of reasonably priced external hard-disks and SSDs. Off-site backups can simply be an external storage device in a friends house. You have a system that seems to work for you, I wouldn’t rush to change it dramatically!

If the 1TB estimate is wildly off, obviously m6 comments may not apply.

 

[docholliday]

There's actually a fourth step that everyone always forgets...and it comes back to bite them at some point. That fourth step is to test recover your backups at least once a year if not more often. I have coded an automated test that runs every 91 days in my program that does my backups, but it can be done manually. Pushing to a backup is no good if you pushed bad data, wrong data, or corruption happens in the path. The easiest way to do this is to try a full restore from the farthest backup from the live data.

You can simply take an extra drive and once a year, restore from that cold backup. The point that the third backup is based on the second, which is based on the first means that any data corruption in the chain will appear at that farthest backup. By trying a restore of that coldest backup using the most barebone process, then attempting to access a sampling of data, you "prove" the data out that 1) the data is good, 2) you can actually restore it in case of recovery necessity, and 3) establish a time needed to perform the task.

Once a year, I'll put a bare metal box on the bench, load a cheap blank drive, and do a base install of Windows. From that, I'll download the bare drivers needed, install my recovery software, and pull the data backup.

This test box drive then gets Ghost'd to an image as my annual "4th backup", as well as ensuring that I can access all my data should I need to. Nowadays, it's easier to just spin up a VMWare instance and do the same restore, but the concept is the same - try to do a full disaster recovery as if your life depended on it.

 

[docholliday]

To answer the question asked: in theory compression can make the resultant archive larger than the original, but for your purposes, that is vanishingly unlikely. It probably won’t gain you very much, but it won’t cost much either.

To address the broader points, while Derek and Doc are correct about how backups should be managed in theory, I suspect they are slightly over engineering the solution. From your post, it sounds like you have a little over 1000 images, with 600MB as the largest image. This is less than 1TB total! They are correct about multiple copies, I notice you are already doing that. Cloud storage is an option, but it needs to be priced correctly: it can very easily get expensive if you set it up wrong, with your data as hostage to continuing payment! There are also questions around restore speed, but that is probably not a major concern here.

What jumps out to me is that 1TB is comfortably within the territory of reasonably priced external hard-disks and SSDs. Off-site backups can simply be an external storage device in a friends house. You have a system that seems to work for you, I wouldn’t rush to change it dramatically!

If the 1TB estimate is wildly off, obviously m6 comments may not apply.

The issue with compression is two folds - 1) it can actually make already compressed data larger and 2) it can make data recovery of failed volumes more difficult.

The current price point for drives is 4TB as of this point of 2021. That means that buying a 1TB is actually wasting money as you can do more and keep longer a 4TB for a few more sheckels than that 1TB.

It's not overengineering - it's years of practical application and dealing with the faults of people who "underengineered" their setups, lost data, and then came crying to me hoping for a miracle. Many of them were photographers in studios who had years of client images that weren't backed up sufficently.

The unspoken rule in data safety is that any manual interaction to produce a backup is a bad solution. The weakest link isn't the hardware, it's the human that's often too busy to create the backup, who forgot to insert/extract the backup, who shoved the wrong drive into their backup slot mistakenly, who thought that a padded box was enough to protect the drive they took offsite, or who misconfigured their backup and missed a critical folder.

Again, for most people, it's no big deal. Their images and data just isn't *that* important to them. As I opened my post above - pro or not, if you actually care about your images, you need to do a proper backup and spend the necessary money to do it. If you don't care or they just aren't as important, then don't even bother to waste a dime on backing up and just shove them into a cloud somewhere blindly for nickels or keep it on an old USB drive in a "fire-proof" safe. You'll probably never need that backup or it'll be so out-of-date and out-of-sync that it won't be worth anything when you do.

I gave a lecture about image archivalness and the one thing that always catches the attention of the photographers in the audience is that old images in grandma's shoebox are actually safer than most digital photographer's image stores. Even if those photographs get wet, they can be somewhat recovered. If a fire breaks out, you can always grab the box and run out a door. 20 years later you can fix the fading. But bad backup habits in the digital realm can mean a complete loss with one errant keystoke, cable trip, or cold drink near your precious images. Even if you don't lose it all, bad bits can't be as easily repaired as an tint from bad fixing or stains on a print. Oh, and if you "archived" well, you had an automatic back up in those days - the negs!

 

[earburner]

Data backup is a nightmare, having a good strategy is needed. Accounting for the following is needed, Onsite hardware failures, onsite malicious problems, remote controlled onsite malicious problems, Offsite hardware issues, offsite company failures. So from your hard drive failing, your building burning down, a disgruntled member of staff wipes everything as they walk out the door, a virus encrypts your local copy, nas copy and cloud copy. I once heard a story about a cloud provider going bankrupt and the administrators charged $10,000 for customers to gain access to their data... Everyone has covered this well, the only additional thing i do is make a copy of everything on a new external drive every 6 months a put it in storage, so in the event of a total loss, i don't lose everything

 

[docholliday]

Data backup is a nightmare, having a good strategy is needed. Accounting for the following is needed, Onsite hardware failures, onsite malicious problems, remote controlled onsite malicious problems, Offsite hardware issues, offsite company failures. So from your hard drive failing, your building burning down, a disgruntled member of staff wipes everything as they walk out the door, a virus encrypts your local copy, nas copy and cloud copy. I once heard a story about a cloud provider going bankrupt and the administrators charged $10,000 for customers to gain access to their data... Everyone has covered this well, the only additional thing i do is make a copy of everything on a new external drive every 6 months a put it in storage, so in the event of a total loss, i don't lose everything

Exactly...the amount of variables that can result in total failure gets worse each day. I found that the worst offenders are Mac users who swear that all their stuff is mysteriously "automatically" backed up (or think that Time Machine is a backup solution) and younger business owners who think that Google Drive/MS Onedrive/iCloud is a viable backup solution.

My once a year test restore serves two purposes - 1) test to ensure that the backup is intact & viable and 2) becomes that last-line-of-defense copy in the event of total systemic failure. Ya can't cover it all, but you can sure the hell try!

I've heard the same story about that provider going out and charging massive amounts. There's also one that a friend of mine had clients fall victim to where the really, really cheap provider, who had great service for years, suddenly went poof overnight and disappeared without a trace taking all the customers data out. No warnings at all, but it seems they were already broke and just shut down. Those customers didn't even have the option to pay for their data and the contract covered enough that there wasn't a bit of liability.

I also had a portrait photographer referred to me that "backed up" their work using their lab's submission service for prints. The lab cleaned up their FTP upload drive one day and when their SSD disappeared off their box failing miserably, they went to get the files from "backup". Needless to say, they were crying when they called me. Sorry, I can't make data appear out of thin air.

 

[MGrayson]

Another PoV. I've tried using NAS boxes three times. They just don't like the Mac OS, or changes to the Mac OS don't get reflected in a timely manner, or I don't know what. These solutions always get frustratingly error prone within a year. This is just MY experience, and obviously, it works fine for many many others. What DOES work for me is an old Mac Mini running as a file server in another part of the apartment. There's a large disk attached to it and all the computers on my network back up there (CCC, not Time Machine) as ONE of their three methods. There are another few backup disks attached to my desktop, and then BackBlaze. All backup disks are 8TB-10TB spinners.

 

[tcdeveau]

Lots of good info here. BackBlaze looks pretty attractive, I should look into that as another option.

I've taken the approach of using a RAID 5 array for my main catalog which is cloned periodically with CCC onto at least two additional physical drives stored in different physical locations. I've been toying with the idea of a mac mini in lieu of a NAS as well but have been too busy/lazy to look into implementing it.

With COVID I've been getting lazy and not as diligent and may rethink/revamp my strategy. I'm a low volume shooter so thankfully don't have a ton of a data (only single-digit TB...for now).

 

[docholliday]

Consumer grade NAS is very error prone and buggy, usually because of useless features or optimization for streaming/media. The other issue is that they usually want to connect via SMB and that has it's issues too. Some of them I've found it's actually more reliable to set up the FTP and run an FTP batch to as it's more properly configured. I have mine configured for iSCSI and mapped as an actual drive instead of a network share and it works great. However, it's not a consumer NAS as it's actually a rackmounted "true" NAS for datacenter usage.

Any computer running as a file server is going to be just as good as a NAS for backup. In reality, you really don't need NAS or RAID for that second layer backup - a slower, low heat and power drive is just as good. Remember that it's just a copy of your real data. For most people, taking a few extra hours to restore from a slower backup isn't a problem, and a slower archive-grade drive will last longer than a "desktop" or performance rated drive. The RAID is more appropriate for the live, hot data, as it's your first line of defense against failing disks and allows you to continue working uninterrupted while you wait for a replacement drive. Actually, if you have + 2 on your RAID configured, you should already have 2 hot standbys that kick in which is actually what you're waiting to replace.

 

[tcdeveau]

Consumer grade NAS is very error prone and buggy, usually because of useless features or optimization for streaming/media. The other issue is that they usually want to connect via SMB and that has it's issues too. Some of them I've found it's actually more reliable to set up the FTP and run an FTP batch to as it's more properly configured. I have mine configured for iSCSI and mapped as an actual drive instead of a network share and it works great. However, it's not a consumer NAS as it's actually a rackmounted "true" NAS for datacenter usage.

Any computer running as a file server is going to be just as good as a NAS for backup. In reality, you really don't need NAS or RAID for that second layer backup - a slower, low heat and power drive is just as good. Remember that it's just a copy of your real data. For most people, taking a few extra hours to restore from a slower backup isn't a problem, and a slower archive-grade drive will last longer than a "desktop" or performance rated drive. The RAID is more appropriate for the live, hot data, as it's your first line of defense against failing disks and allows you to continue working uninterrupted while you wait for a replacement drive. Actually, if you have + 2 on your RAID configured, you should already have 2 hot standbys that kick in which is actually what you're waiting to replace.

What's your take on enterprise vs. consumer drives for backups?

I've been going with enterprise drives for my cloned backups

 

[docholliday]

Lots of good info here. BackBlaze looks pretty attractive, I should look into that as another option.

I've taken the approach of using a RAID 5 array for my main catalog which is cloned periodically with CCC onto at least two additional physical drives stored in different physical locations. I've been toying with the idea of a mac mini in lieu of a NAS as well but have been too busy/lazy to look into implementing it.

With COVID I've been getting lazy and not as diligent and may rethink/revamp my strategy. I'm a low volume shooter so thankfully don't have a ton of a data (only single-digit TB...for now).

Be careful with large disks and RAID 5. The possiblity of a second disk failing while doing a parity rebuild is very high, especially if the disks are the same brand, model, and batch/vintage. I've actually seen a few different RAIDs drop out during rebuild with 10TB drives on 5+1 causing full array loss and subsequent cold restore from backup onto a completely new array. Needless to say, it was not rebuilt with RAID5+0 when I came into the problem. I wouldn't run less than a RAID6+1, but would actually prefer 50+2 for full protection. At the least, RAID 10+1 or 6+2 would be best and utilizing drives from different manufacturers, batches, and models.

 

[docholliday]

What's your take on enterprise vs. consumer drives for backups?

I've been going with enterprise drives for my cloned backups

For backups, consumer grade drives are acceptable, but they aren't meant for 24/7 workloads. So, if you use those to save a few bucks, be sure to do that 4th step I mention - test your backups and preemptively replace those drives. It's just a midspan backup, as you should've protected the primary drive and also have a final offsite cold store...

I've used old consumer drives pulled out of well-abused USB storage as backup, as well as refurb/used enterprise drives. It's about having that third backup and a proper RAID in the first. If one of these drives died, it's no big deal since the third layer is still holding. Actually, I've never really bought new drives for my personal lab/studio/gear. I always buy old server pulls or refurbs. Even in my RAIDs, the SAS drives are server pulls. 50+1/2 or 60+1/2 with "cold" spares sitting on the shelf because I got a case of the drives for about 1/3 of a new drive.

Enterprise drives are worth the extra pennies because they are meant for 24/7 workloads and have a lower BLER rate. They are also usually created with stiffer chassis and better internal components. They're also louder, typically faster, produce a hair more heat, and consumer a few watts more power. Then, there's the design of the drive itself. The consumer drives usually use inferior formatting to produce high capacity whereas enterprise drives have better designs. It's like for SSDs where consumer media is TLC/QLC and enterprise is MLC or SLC, some consumer "NAS" drives use a shingled layout and that can have it's own issues as the drive gets older.

The biggest difference between the drives I've noticed is that consumer is about cheap and somewhat performant, with no longevity expectations whereas enterprise is about consistent performance, reliability, capacity, and predictability "at any cost". Look up enterprise SAS SSDs, like the Samsung PM1643a or Western Death Ultrastar DCSS530 if you want your eyeballs to pop out at the price!

 

[RobbieAB]

The current price point for drives is 4TB as of this point of 2021. That means that buying a 1TB is actually wasting money as you can do more and keep longer a 4TB for a few more sheckels than that 1TB.

At no point did I say only buy 1TB of storage, I said 1TB is comfortably in the capacity of the “sweet spot“ drives.

Essentially, large, high capacity multi-device systems are not needed for capacity reasons, and introduce an unneeded complexity for this size point.

It's not overengineering - it's years of practical application and dealing with the faults of people who "underengineered" their setups, lost data, and then came crying to me hoping for a miracle. Many of them were photographers in studios who had years of client images that weren't backed up sufficently.

Given my point above, I would argue that any solution that incorporates a 5 drive NAS such as Derek proposed IS over-engineered for this scenario. Even a 2-drive NAS is questionable given there is nothing in the original post to suggest that local network storage is needed or desired.

I would suggest 2-3x “sweet spot” drives, using the fastest USB option you can get, and swapping them around for the original poster. 1-4TB should be large enough. This is basically the workflow that has been in use for years, and is ok, if you are disciplined enough to keep it working. The refinement I would add, as everyone else is suggesting, is off-site, but for the original poster’s requirements leaving a disk with friends or family should be enough.

 

[docholliday]

At no point did I say only buy 1TB of storage, I said 1TB is comfortably in the capacity of the “sweet spot“ drives.

Essentially, large, high capacity multi-device systems are not needed for capacity reasons, and introduce an unneeded complexity for this size point.



Given my point above, I would argue that any solution that incorporates a 5 drive NAS such as Derek proposed IS over-engineered for this scenario. Even a 2-drive NAS is questionable given there is nothing in the original post to suggest that local network storage is needed or desired.

I would suggest 2-3x “sweet spot” drives, using the fastest USB option you can get, and swapping them around for the original poster. 1-4TB should be large enough. This is basically the workflow that has been in use for years, and is ok, if you are disciplined enough to keep it working. The refinement I would add, as everyone else is suggesting, is off-site, but for the original poster’s requirements leaving a disk with friends or family should be enough.

Ah, well, to each their own. I'll tell ya that 1TB is definitely not the sweet spot of price-vs-size right now. That is currently at the 4TB mark for SATA spinning rust. For SSDs, yes 1TB is the high mark of reasonable. File sizes are getting bigger and did you see the rule-of-thumb I mention above where your backup space should be at least 2x your storage? Well, if the OP is already at 1TB, what's going to happen when they start to add more images? That'll become 2TB soon enough which puts your minimum backup at 4TB, the current price break point. Prices are kinda whack right now, but the break point is still noticable - I did just order a case of 50 10TB Ultrastars this morning for a server upgrade.

As far as complexity, how does having extra capacity with data protection introduce "unneeded complexity" over a single drive for loss? If you believe that mirroring 2 disks is questionable, I'd hope that your data is well backed up offsite. The worse thing you can do is put the backup on the same computer as the original data. What would you do if a surge/lightning/running kid causes that box to die? Both your data and backup could be lost promptly. USB backup devices are the second most unreliable method next to removeable media. They are usually in plastic cases with no fans or ventilation and weren't meant to take hours long transfers unless you purchase a more expensive model.

As far as being disciplined, that's all fine-and-dandy as I said above, if you can maintain that. But, life gets in the way and the weakest link is the human factor. Plugging in the wrong drive, forgetting to do it altogether, or "I'll do it tomorrow" is more common than you'd think. If it works for you, great. You've been lucky. Try telling that to the people whose "workflow has been in use for years" that lost all their data on that one bad day.

Again, if your data isn't important to you, then fine...do whatever you think is best or has worked. But if it is important, you'd want to spend the extra time, effort, and "over-engineering", as you call it, to protect your work. Even being able to restore an earlier version of the data can be a nightmare as you may not be able to reproduce the exact same edits you've made to an image since that backup was created. For anybody selling prints, that can be a problem. I know that I wouldn't want to spend days re-editing some of the shots I've done, even for personal work.