Murphy Mac - Screencasts and Tutorials » Page 'Time Machine: Some Details'

Time Machine: Some Details

Share

Time MachineThere are a couple things I’ve noticed about Time Machine that are worthy of mentioning. Yes, Time Machine is a great application and the Mac world will be a better place for it. And despite its admirable job of hiding all the nitty gritty from sight, the nitty gritty is still back there, making things happen. You can try to simplify backup — but the fact remains, backup isn’t simple.

Renamed Files and Folders

I’d read some speculation that Time Machine might be very, very smart. That it might somehow tag your files in a way that it knew when you renamed or moved one. The hope here was that Time Machine wouldn’t copy a renamed file to your backup drive and store it twice. Not so. Time Machine does make the copy.

Example: A big folder with hundreds of images gets backed up at 3PM. You change the folder name at 6PM. The entire folder and all its contents get copied to the backup drive. Yes, all those files now reside on your drive twice. Once with each name. If you kick off Time Machine manually and watch in System Preferences you can see the files getting copied. Now the same set of files is claiming twice the amount of space you might want them to occupy.

Files not Included in Daily Backup

The second thing concerns the backup stored on a daily basis. Remember, Time Machine keeps an hourly backup for the last 24 hours. And a daily backup for the last month. What might surprise some people is which backup is selected as the daily backup. It’s the first one of the day. In other words, if you go into Time Machine on Friday and fly back through its space-time continuum to Monday’s backup looking for a file you created during Monday Night Football — it won’t be there. It will be in the daily backup for Tuesday, assuming you didn’t delete it Monday night.

It’s not a big deal. And let’s be clear, I’m not saying it’s a flaw either. If it makes you feel better you can think of Tuesday as the daily backup for Monday. But there’s something important to be aware of.

Let’s assume you keep your Time Machine drive plugged in all the time. If you create a file at 6pm on Monday and delete it at 10pm it will be on the hourly backups created Monday evening. But it won’t be on the daily backup for Monday or Tuesday – or any other day. When those hourly backups drop off the file is gone.

This might never matter to you. But if you’ve gotten much more confident with the Delete button since you started using Time Machine this is something to be aware of.

The bottom line: Files created and deleted the same day might never make it to the land of daily backups. You’ve got about 24 hours to recover such files if you need them. If you want to make sure a new file gets rolled into a daily backup and thus have it accessibile for a longer time frame: don’t delete it until after the first backup the next day.

| Permalink

30 comments to “Time Machine: Some Details”

  1. I’d assumed the ‘renaming a folder’ thing worked like that but I hadn’t known about the daily backup thing — thanks for that!

    I haven’t really given much thought to Time Machine since I plugged my drive in and clicked “Yes”, although I did stop it from backing up system files — I think I read how to do that on MacOSXHints.com.

  2. I skip system files too. And Fusion!

  3. Hi Murphy,

    New to Leopard, Time Machine and the whole backing up thing so please excuse my ignorance, but why should I exclude system files and applications from TM?

    Thanks,

    Dave

  4. dave –

    I’m not saying you should. I’m using a relatively small drive. So I don’t have lots of space for files that could be replaced with a reinstall.

    If your Time Machine drive is large enough I’d include the system files.

    In my case I’m only interested in backing up files that are otherwise irreplaceable.

  5. Good post! I had no idea about which hourly backup goes into the daily one. I, personally, love the Time Machine auto backup. It makes it very easy and very… well, Apple like!
    Of course, as I write this my external hard drive failed and I’m working sans backup. I’m considering what I want to do: either get a new HD for my enclosure or buy of of those G-Drives. What to do, what to do.

    BTW, I didn’t include new Time Capsule because I do not like the fact that if the hard drive fails, I’m not sure what Apple will charge me to get a new one in there.

  6. I think the “Daily” backup behavior makes perfect sense….

    So the last hourly backup from “Monday” begins at…. midnight. Which is technically “Tuesday.” But that’ll be the one that contains the most updated Monday info (FIles created @ 11:59pm). The only issue is that it’s referred to as the backup from ‘Tuesday.’ You eluded to this in the article, but I thought it was worth sharing WHY it makes sense and isn’t a fault of the program – it’s doing the right thing, though perhaps they should have called that midnight backup ‘Monday’ instead. But logically, it’s correct in the timing strategy.

  7. Waiting for the Daily Backup missing files “feature” to be spun by the Blogsphere into something akin to Safari’s “Private Browsing”…

    Don’t want your wife in your Time Machine? Simply delete risky content same-day, and — Boom! — Time Machine covers your tracks…

  8. I am a bit shocked at your results in that they are specifically the opposite of what Apple advertises. Are you really sure that the behaviour you describe is what is happening?

    The renaming of the folder example is supposed to induce a copy of the folder but not the contents (using hard links). Apple has also *specifically* stated that the hourly backups are all smooshed together so that no changes or deletions of any files made throughout the day are lost.

    I don’t know who to believe now. :(

  9. Jeremy

    I’ve seen quite a few posts on various forums that report the same results I got.

    The links you’re referring to – that’s something else. When you use Finder to look at your Time Machine drive you can look in folders for all the different backups. And the files appear to be in all of them. In other words, a file created on Monday appears to be in the Tuesday and Wednesday folder. But it’s not stored in all those places, it’s linked. Because the file didn’t change.

    My example deals with folder and files that have changed. The main point: ONE folder name changing results in all its contents being copied again. Some people won’t like that, but it’s not unusual behavior as far as backup goes.

  10. Thanks, I had known about the renaming thing, but not the daily thing.

    I have a large enough drive that I let system get backed up, but have decided to exclude the /Library/Updates folder because of how I’ll use Time Machine in conjunction with Software Update. I’d be downloading updates right when they’re released, but wait maybe a week (until I’m sure they’re safe) before installing them and I don’t want those update installers getting backed up (who does?, you can re-download them if you need to).

  11. this is pretty much what I have discovered after lots of testing. Further, the weekly backups SEEM to be based on the date of the first backup, you get a weekly every 7 days after that. A month is really 30 days too.

    When the TM disks fills (I did it in less than 2 weeks with a disk 3x the size of the internal disk), TM starts “thinning” the oldest backups. That means that the first to go is the INITIAL backup. Anything that hasn’t changed since then is lost in the sands of time except enough of the system to make a bootable restore. Do not expect TM to be an archival backup.

    See more musings at

    http://www.girr.org/mac_stuff/backups.html

    – gws

  12. Hard links make it look like you have several copies of a file, but they actually all point to the same data on disk. So I would assume that even when you see two copies of the folder, the files exist only once on disk. OTOH, if you rename or modify a large disk image…

  13. philippe –

    Your assumption is not correct. Read the whole post. I tested this and concluded the renamed folder had its contents copied again. Not hard links. People see hard links in each backup for files that didn’t change, or have their parent folders renamed. but when a folder is renamed its contents are copied. I verified by checking used space before and after on the drive. Try it.

  14. i have new mac since dec 07 and now decided to buy external drive and use time machine. question. i have a lot of photos and have not deleted any. they are both now on my ex-drive and computer. can i start deleting bad photos from my computer since they are already on my ex-drive or will they be deleted from my ex-drive as well once time machine runs? thanks for your guidance.

  15. marcos – as long as they made it into a daily backup they’ll be on your external drive for a while. But anything you delete from the computer will eventually be deleted from the external.

    maybe you should make a long-term archive of your photos on another drive or copy them to dvds. Or both.

  16. Thank you! That helps!

  17. Murph – your description of folder copying makes no sense. A hard link is a pointer to a block of data on the disk. It doesn’t matter to the hard link whether it’s in directory A or B – it still links to the same block of data.

    If what you are describing is happening I can only assume it’s a bug in TM and will be fixed in one of the next releases. There is no technical reason for this to happen.

    The daily backup behavior is subtle but on the other hand it’s what you would expect. The point is not to get the “most recent” from monday, but to do daily backups. What time of the day the backup is from is irrelevant – any kind of bad situation you describe with a monday morning backup can also happen with a monday night backup. The only thing that matters is that the backup happens regularly every 24h.

  18. You’re entitled to your opinion nikster, but the more information people have about backup the better. If someone is relying on something being rolled into the backup for that day it won’t necessarily happen. If people delete something they created that day thinking Time Machine has it they could be wrong. If you think it’s irrelevant I don’t think you understand the post.

    I don’t think you understand how Time Machine uses links. The fact remains that the data is being copied again and taking up MORE space if you rename. Since the total amount of free space is reduced by renaming something than I don’t see your point at all. Call it whatever you want – but I actually tested it. If my results were wrong I think readers of the Ars article linking to me would have pointed it out. They didn’t.

  19. I am curious if somebody had the issue that an external drive which is backed up by TM has crashed?
    I currently have this situation and I am wondering how I can get the data back up after I plugged in a new drive. Time Machine itself does not recognize the new drive as a replacement of the old of course so they must be a way of telling TM to restore the data to a specific new destination which is unknown to TM before.
    Also I would be curious what behavious TM shows when it has restored the data to a new destination and if as described by some earlier “forgot” the previous backup data.
    I am looking forward for some helpful posts!
    Regards from Berlin,
    Holger

  20. If you are worried Time Machine deleting something important (like photos perhaps), you can just copy them to the drive outside of Time Machine and thus they will be safe. It would make TM start deleting old stuff sooner if you put them on the same drive obviously because their would be less space but it’s an idea to get around the whole daily backup thing. I would only do it for the important stuff.

  21. Have you done any work analyzing the hard links that TM creates? I’m looking at my backups and it really looks to me like TM is NOT always creating a new hard link to an unchanged file… I seem to have a LOT of files in my backups that have just a single hard link, implying TM thought it was a modified file. And when I say a lot, I’m talking on the order of 1.5 million files per daily backup. Things like .gif files in the Address Book’s help folder, which seem unlikely to change. Unless I’m confused about the way it uses hard links… but I think a file that is unchanged for a year should have 365 hard links, one per daily backup, right?

  22. My mistake, the actual number should be 101 hard links, 24 for the last 24 hours, 29 for the last month, and 48 for the weeklies before that. But you get my meaning.

  23. I haven’t looked at the hard links. But to see if I understand what you’re saying, let me paraphrase to see if we agree….

    Let’s pretend Time Machine only does daily backup files: Then yes, for a given file that didn’t change over a year I would expect to see a hard link for each day.

    My testing dealt mostly with moving files. I think a lot of people expected Time Machine to know a file had been moved and store it only once in the Time Machine archive. At the time I tested that wasn’t what I was seeing. IF the file was in Folder A on Monday and Folder B on Tuesday it would end up being stored twice, until enough days went by for the Folder A one to drop out from age.

    A lot of people misinterpreted what hard links are doing when I first posted this.

  24. Yep that’s what I meant. And on my Sept 26th backup for example, I confirmed that a relatively old text file in my user directory had 78 hard links, so on some files it appears to be working pretty well. Yet out of that whole backup (1042867 files) there were 956597 files with 1 hard link.

    I haven’t done any more extensive analysis yet, but on the face of it that looks to me like Time Machine is NOT in fact being very efficient about skipping unchanged files.

  25. Any one used Time Machine to restore to a different Mac model? eg. Mac Mini to iMac

  26. Hi

    i just copied all my files to my imac ( appx 300gb at 9.00 pm on 13th night) took backup to tmachine . Disconnect my mac on 14th from network then format and reinstall it on 16th. with time machine restore at value 14th nov 10.00am but when i chk all my data it was not there all. i got my partial data on my mac but not full, like i made few more folder which are missing. can u let me know how i can recover to them

  27. I just renamed a folder with 26 gig in it. I had only done a few Mb of work that day and time machine has backed up 26 gig. I googled around after feeling a bit peeved and found this post. Anyone doubting this just try it. It’s easy to test. Make time machine do a backup. When it finishes rename a folder with a few more gig in it and then do a backup right away. Click on the time machine spinning icon in the menu bar to see how much data is being copied right away! No so smart there, methinks…

  28. “Good post! I had no idea about which hourly backup goes into the daily one. I, personally, love the Time Machine auto backup. It makes it very easy and very… well, Apple like!
    Of course, as I write this my external hard drive failed”

    Jesus Christ, you seem to be proud APPLE’s hard disk FAILED?! APPLE WAY, you say?? Do you know that APPLE inserts the CHEAPEST JUNK into their products? No quality control whatsoever. I had Micron Millenia PC for 9 (NINE) years before it couldn’t boot any more (I bought it at max RAM of 128MB at that time). It worked perfectly all that time, I did not even bother to clean all that dust around it. I just bought hyper expensive iMac and – drive failed. WTF??? I AM SWITCHING TO BUSINESS CLASS DELL OR HP PCs WITH WINDOWS 2003 ENTERPRISE ON IT. AND IF I NEED ANY “APPLE FEEL” I’LL INSTALL FUSION WITH LINUX MINT AND, IF I AM CRAZY ENOUGH, SNOW LEOPARD AS WELL. THAT’S IT. NEVER AGAIN I’LL BUY “APPLE FEEL” HARDWARE. I STILL HAVE MY THINKPAD T43 WORKING LIKE DRAGON FOR MY MOBILE NEEDS.

    My mistake was when I bought iMac I did not check complaints all over the internet. There are literally thousands and thousands of complaints. My rough estimate is that 50% of machines fail at around 15 to 18 months

  29. Excellent point brought up by Murphy, and I think many people (myself included, initially) did not get the point of this post.

    Murphy stated CLEARLY that TM is doing the “right” thing by saving the midnight copy. Whether midnight refers to day 1 or day 2 is not the point Murphy is making (but he did confuse people a little by saying first backup / last backup of the day). He simply wants people to be aware that the data is not backed up “forever” simply because it has gone through the hourly backup. It must go through the midnight backup if it’s to be kept for the remainder of the month, and also the midnight backup at the end/beginning of the month if it’s to be kept for longer periods of time. It’s a good point that many people not familiar with backup might not have thought about.

  30. I love your website and would recommend it to everyone!!!

Leave a comment