ChrisAnn
Junior Forum Member
Posts: 227
Open to constructive criticism of photos: Yes
|
Post by ChrisAnn on Feb 6, 2018 20:14:10 GMT
This is academic at the moment for me as I'm not anticipating getting a new computer this year.
However, I've just done a backup through the Organiser on to an external hard drive. To save space I've done an incremental backup and also because when I tried to do a full backup I was told that some of my photos were missing. I knew they would be missing as earlier photos are kept on a separate hard drive (I have 36,000+ of them) to save space on my C: drive. I connect both hard drives using the same usb port and, therefore, can't connect the storage drive at the same time as using the backup drive. The incremental backup didn't seem to mind, no doubt because the last two years' of photos are still on the C drive.
When I come to restore the backups on to a new computer would the restore process know about the incremental backups and also which order they are in or would I be asked to provide the name of the file? Also, when I come to do the next incremental backup do I link it to the current incremental backup or the original backup file?
Another question, if I manually, outside of Elements, changed the name of the backup folders to something more logical (hindsight is a wonderful thing) would that upset the process?
|
|
|
Post by michelb on Feb 6, 2018 22:02:47 GMT
Hi ChrisAnn, If I understand well, your problem is: - not enough space on your computer drive - part of your library is kept on an external drive - you have only one port available - for the backup you are using another external drive.
Could you specify the size in GB of each of your external files? The total size of your library (roughly equal to the sum of your full backup + incremental one)? If the external drives are big enough there should be no problem.
I would absolutely avoid incremental backups to prepare for a migration on another computer. (On my present setup, I have a USB hub powered externally for 5 slots, that might be helpful in your situation)
If you have some photos 'missing' during a full backup, the backup will restore anyway, but obviously not the missing files. It's better to try to reconnect them before, but you can reconnect after as well.
You know that you can drag and drop folders with all their subfolders from one drive to another? That means you could migrate the files on your computer to the hard disk holding the other part of your library? That's something you can do with a single slot available. Maybe not as safe as the backup and restore process, but the incremental backup has risks... The big advantage is that when you get a new computer, you simply have to copy your whole catalog folder from the computer to the same hard drive and store it in the same external drive than your library of photo files. Your new computer will be able to use the copy of the catalog with all files connected.
To keep it simple, forget incremental backups. If you can find the USB hub, do a full backup. You'll be safe until you want to upgrade. If not, moving the part of the library from your computer to the external drive with the other part is the solution. What may be problematic after the move is how to create full backups periodically with a single slot? If your external drive is big enough, it can hold both the library and the backup. Not ideal if your external hard drive crashes. Maybe you could find some relative or friend willing to copy the backup folder on your second external hard drive?
|
|
ChrisAnn
Junior Forum Member
Posts: 227
Open to constructive criticism of photos: Yes
|
Post by ChrisAnn on Feb 6, 2018 22:48:12 GMT
Thanks Michel - I think I understand what you're saying. I will have to read it again in the morning when my braincells are more alert.
Are you saying that when the time comes for me to migrate to another computer I really should move all my photos back on to the C drive and then do a complete backup?
Or is it better for me to attach the storage drive to a different slot from the one I use for the backup drive? By doing that will the backup then backup all the photos, not just those on the C drive? I think there is another usb slot on the computer box, but it's at the back which makes it more difficult to use. I always move my photos via the Organiser so that it knows where to find them. If I put the storage drive into a different usb slot (I think there may be one on the monitor which would make it easier to get at) will they have a different address and therefore confuse the Organiser?
|
|
|
Post by Sepiana on Feb 6, 2018 23:53:28 GMT
ChrisAnn, thanks for starting this thread!
Thanks to our in-house Organizer guru, Michel! You have just given me the answer to a question I had.
I would absolutely avoid incremental backups to prepare for a migration on another computer.
|
|
|
Post by Bailey on Feb 7, 2018 0:15:27 GMT
Hi ChrisAnn. I agree with michelb about incremental backups and I don't use them at all for any purpose. Firstly because I suspect you don't save any space compared to doing full backups. I suspect the sum of the files sizes of the original full backup plus subsequent incremental backups will be larger than one full backup because of the small overhead in disk space each incremental backup will have and secondly, because doing a full restore could be messy if you have a large number of incremental backups. The only benefit of incremental backups, imho, is that they save time because they back up files created/modified only since your last full or incremental backup. I am a big fan of the KISS Principle and so let me explain my backup/restore strategy and hopefully it might give you some food for thought. The one big assumption I making here is that at some point you will be able to store all your photos on a single disk drive on your pc and that you have an external disk drive large enough to hold one full backup of all your photos. 1. I do full backups of the entire catalogue periodically onto an external HDD via a usb cable connection directly to a slot on my PC. On my PC it doesn't matter which usb slot I use to connect the external HDD. Because the full backup normally takes 5-6 hours nowadays and I don't want to be doing anything else on the pc during the backup, I run the backup overnight. All my photos are stored on the C: drive on my pc in subfolders under MyPhotosWarehouse. Personally, I would avoid using usb hubs, and that is just me, because I have read and heard elsewhere that devices connected to hubs might not work properly in some circumstances. But that is totally up to you and your experience. And that's it - one single step. A very clever person once told me there are only 2 kinds of hard disk drives in the world - ones that have failed and those that will fail. Given that fact, when my pc will eventually fail (today, tomorrow or whenever), my strategy is then fairly simple as well. 1. Plug my external HDD into my laptop and restore the full backup to my laptop until I get a new pc. One final tip (I am digressing a bit here) - I never reformat my camera SD cards or delete raw files from them that have been copied to my PC until I have done a full backup via the Organiser. This ensures having 2 copies of at least the raw files at all times since the last full backup - one on the camera card and one on the pc. HTH
|
|
|
Post by michelb on Feb 7, 2018 8:30:46 GMT
Hi bailey and Sepiana, We are absolutely on the same line . An historical comment about incremental backups in the organizer: The way they have been implemented in the organizer is typical of the time they have been programmed. We should keep in mind that the common storing physical medias at that time were CDs, not yet DVDs and certainly not external USB drives (more than fifteen years now!). It was common to store our photo library to several CD/DVDs with a single catalog on the main drive. For backups we had to use several CD/DVDs even for a full backup. So, we had to be 'organized' to name the series of backup CDs. Typically, we could store the photo files both on the computer itself and on various CDs, hence the necessity to implement how to manage 'offline' media with 'previews' stored in the computer (another legacy feature). Today, we can ignore incremental backups and 'offline' media... well not quite . For instance the distinction for importing from 'camera or card reader' or importing from 'files and folders' is no longer totally obvious. Photo files on a NAS are considered "offline". You can use a USB stick or a card reader and choose either way (downloading or keeping "offline" with optional previews...) External USB drives are always considered just like internal ones. It's now common to 'import' photos from your smartphone by syncing to Google Photos or using Dropbox. No cables, no card reader, there should be another category "import from the Cloud"! So to "keep it simple", let us ignore the "legacy features" and follow bailey's advice . Comparison between incremental backups and full ones: - They all contain a copy of the database; they contain new and changed files: smaller size - Since they all contain the database, that seems redundant, but it's not. The big advantage is that you are able to restore to the situation of any of the incremental backup. - The backup time is still long. Using other syncing utilities is often much faster (I do use Windows Synctoy) but they only keep the latest situation. - The big problem is that many (most?) users don't understand and test the restore process and it fails when you absolutely need it. The backup folders should be named clearly; you should always start with the latest one and follow the prompted order, newest to oldest (full backup). The idea is very smart but not understood. The latest backup contains the correct database and backup.tly management file. What's new or changed is restored and marked 'done' in the last list of files in the database. You process the next newer backup: only new and changed files in the remaining list are present. They will be restored if the previous step has not yet restored them. Files not in the list and present in the incremental backup will be ignored: that means that they have been deleted. Then same process with the next older backup. This explains why you have to start with the newest backup to ignore deleted files and avoid restoring the same file several times.
|
|
ChrisAnn
Junior Forum Member
Posts: 227
Open to constructive criticism of photos: Yes
|
Post by ChrisAnn on Feb 7, 2018 9:34:57 GMT
Phew! There's a lot of info on this thread. Most of which I understand and, with a bit of concentration can implement. Thanks so much folks. I can see that I need to rethink my storage and backing up procedure and now is a good time to do it when it's too cold to get out into the garden and, more importantly, when it's not a matter of urgency.
I'm so glad I thought to ask the experts and thanks so much.
|
|
|
Post by Andy on Feb 7, 2018 11:35:02 GMT
I agree with the prior posters, but want to add to the discussion of backups - always factor in the potential of catastrophic loss at your residence. A USB hard drive backup isn't going to save you in the event of a fire/flood/etc. I highly recommend doing what Michel and Bailey recommend and then go an additional step of having a cloud based backup that you can recover from anywhere. Another alternative is to take one of those backup USB drives and bring it somewhere else - work, family, friend, etc.
|
|
|
Post by Bailey on Feb 9, 2018 3:17:08 GMT
Hi Andy, Yes an off-site backup is always a good idea if it is practical to do so. My catalogue is too large to upload to the cloud and also I have never been keen on cloud backups unless the files are encrypted. Anyone will be hard pressed to convince me that unencrypted files on the cloud cannot be viewed by at least some systems admin/support people looking after the cloud server. Other off-site backup options are not really very practical for me. I recall a friend of mine told me a few years ago now that he buried his backup HDD in a box in his backyard. I suppose that would most likely save me from fire but not from flood Anyway, given the statistical probability of fire, flood, earthquake etc where I live is microscopically small, having a single backup is within my risk tolerance level. But each to their own though.
|
|
|
Post by michelb on Feb 9, 2018 8:13:53 GMT
Hi Andy, Yes an off-site backup is always a good idea if it is practical to do so. My catalogue is too large to upload to the cloud and also I have never been keen on cloud backups unless the files are encrypted. Anyone will be hard pressed to convince me that unencrypted files on the cloud cannot be viewed by at least some systems admin/support people looking after the cloud server. Other off-site backup options are not really very practical for me. There is much to say about online backups. That option may be excellent, adequate or unpractical depending on your needs. My own criteria are: 1 - Library size - more than 500GB, 70 000 items. Only 10% are useful for sharing or interesting to leave to the family 2 - upload/download speed: I have a very good fiber connection: (35,8 / 39,4 Mb/s now) 3 - safety/privacy/longtime reliability: must be taken into account 4 - sharing: I often need to access my useful (generally recent) pictures from other computers and devices. The problem is that you no longer have the organizer catalogs to search for items. Lightroom is better, but not perfect. The ease of use of online syncing sites (OneDrive, Dropbox, Google Photos...) makes it very easy to retrieve smartphones pictures without any cable. Sharing a collection for an event only requires a link to selected people. So, I have decided to use online services as sharing tools, not full backup ones. I have created a selection of files to be shared - family history - recent events etc. They are exported as jpegs. I could resize them for optimal viewing size on displays or good 4 x 6 in. prints. The collection is synced (I prefer Dropbox, but OneDrive or similar would do). It is organized by a folder hierarchy and I 'write metadata to files'. Even with a slow local connection speed, it's accessible everywhere on my smartphone. - I use Dropbox to sync that collection on my computer. When I have taken photos with my smartphone, I select the files to sync to the Cloud, then from the synced folder on my computer I do import in my catalog (no need for cables or card reader). - My Wife is now shooting a lot with her good Android phone. She uses the saving option of all her shots to Google Photos. From the computer, I can open her Photos library and download them very fast. Same result. I still think that for real backups, several external drives in different location are the best safety for me. At least two dedicated to organizer periodical full backups and one for using Microsoft Synctoy for syncing after each editing session.
|
|
ChrisAnn
Junior Forum Member
Posts: 227
Open to constructive criticism of photos: Yes
|
Post by ChrisAnn on Feb 9, 2018 8:33:32 GMT
I've just done a complete backup through Organiser on to the same hard drive where the majority of the photos are stored. (I've remembered that it is 1tb size so there is plenty of space.) What I forgot to do before starting the backup (and there was no prompt) was to make a folder to put it in so all the files are sitting straight on the I: drive. which is making it look very messy. I guess it wouldn't be a good idea for me to make a folder and move everything into it.
My other concern is that I have two completely different catalogues - the main one is personal, the smaller one is for my charity work. I've backed them both up into the same space but I can only see one backup.tly file. Would the second one have overwritten the first one? The two catalogues do have different names.
|
|
|
Post by Bailey on Feb 9, 2018 10:31:32 GMT
I've just done a complete backup through Organiser on to the same hard drive where the majority of the photos are stored. .. .. My other concern is that I have two completely different catalogues - the main one is personal, the smaller one is for my charity work. I've backed them both up into the same space but I can only see one backup.tly file. Would the second one have overwritten the first one? The two catalogues do have different names. Doing a backup onto the same HDD where the original files are stored really defeats the purpose of doing backups because when the hard drive fails you have lost the original files and the backup. The backup really should be made on a separate HDD. Regarding backing up multiple catalogues, I would back them up onto the same separate hard drive but in different folders. eg...On the backup HDD I would have a folder called something like MyPhotosWareHouse_backup. Then create a separate subfolder under MyPhotosWareHouse_backup and output each catalogue's backup to its respective subfolder. The subfolders could be called the same as the catalogue name that is backed up to that subfolder. HTH
|
|
|
Post by Bailey on Feb 9, 2018 10:37:19 GMT
Hi Andy, Yes an off-site backup is always a good idea if it is practical to do so. My catalogue is too large to upload to the cloud and also I have never been keen on cloud backups unless the files are encrypted. Anyone will be hard pressed to convince me that unencrypted files on the cloud cannot be viewed by at least some systems admin/support people looking after the cloud server. Other off-site backup options are not really very practical for me. . . So, I have decided to use online services as sharing tools, not full backup ones. . . That's all I use them for as well
|
|
|
Post by michelb on Feb 9, 2018 11:59:42 GMT
I've just done a complete backup through Organiser on to the same hard drive where the majority of the photos are stored. (I've remembered that it is 1tb size so there is plenty of space.) What I forgot to do before starting the backup (and there was no prompt) was to make a folder to put it in so all the files are sitting straight on the I: drive. which is making it look very messy. I guess it wouldn't be a good idea for me to make a folder and move everything into it. My other concern is that I have two completely different catalogues - the main one is personal, the smaller one is for my charity work. I've backed them both up into the same space but I can only see one backup.tly file. Would the second one have overwritten the first one? The two catalogues do have different names. You can use the same external drive, but you have to take care not to mix backup folders, catalogs and folder trees. It's not ideal for speed nor safety, but that may help for a migration process. Especially if you have several catalogs, you have to distinguish two situations: 1 - The files managed in both catalogs are totally distinct: no big problem. You may restore twice on the same master folder. That master folder will be shared from both catalogs. Your restored catalog folders will be separate as before. 2 - The files are at least partly common to both catalogs. The safe solution is to restore each catalog on a different master folder. That will create duplicates for common files, but there is no risk in managing the files in a catalog, the other one will stay correctly connected with its own set of files.
|
|
ChrisAnn
Junior Forum Member
Posts: 227
Open to constructive criticism of photos: Yes
|
Post by ChrisAnn on Feb 9, 2018 12:05:32 GMT
Thanks Michel, it is so lovely to have contact with someone who knows what he's talking about. With hindsight I think I will make two folders and then do the backups again and save them into their respective folders. I'm so glad I've asked about this before it becomes something which is crucial and needed immediately. I've got this thread saved for future reference (when I've forgotten everything I've learned over the last few days.)
|
|