Export thread

When Windows alerts a user to scan a drive, is the location of the suspect sector/sectors stored in an accessible location and in a format understandable to the average user? If so, would it be acceptable practice to tell SpinRite to focus on the portion of the disk containing those sectors rather than running it on the entire disk?

Jim

If the error indicated is a bad sector on the disk, one presumes such errors rarely come in singles... if a drive is giving any hard[ware] error, it's a warning to take it out of service and replace it. Hardware is cheap and your data and your sanity is more expensive. If it's a soft error (filesystem related) then SpinRite will provide no help.

Grizbud said:

When Windows alerts a user to scan a drive, is the location of the suspect sector/sectors stored in an accessible location and in a format understandable to the average user?

Click to expand...

I don't believe so. I think all that Windows does is set the "dirty" flag on the drive, so, on the next boot, it knows that a disk check is required.

A somewhat threadjack:

Bad Sectors and the Frozen Lake analogy by Jim Williamson

When talking with others about a bad sector and a recommendation to replace the storage device, I offer them this short chat:

Think of the bad sector on your drive in this view: You're walking on a frozen lake and you fall in. The spot where you fell in was either ice that was not strong enough to support you or you fell into an ice fisherman's hole*. Either way, you fell in, got wet and weren't happy. Now, it would be nice if we could know what type of ice was right around where you went in. Was it thin ice where today, two steps back, it held you up and in a few days it'll be weaker and you'll fall in again - or was the ice good and thick all around the hole and this was a localized spot that won't grow larger? While it would be nice to know if this was a one-time-event, we ultimately won't and, for safety, we should replace the drive now.

* they were fishing for some LARGE fish.



Addition to the topic: Is there an easy way to find out the \\directory\filename that might be in use (have been in use) by a bad sector? Assume NTFS file system.

First, AlanD, despite the pranging I will probably receive for daring to say "thank you" to you for answering my original question ... Thank you. Of course, writing a reply to you is, in the opinion of some, wasting storage space when using Emojis is their preferred method. Regardless .... Thank You.

And now for something completely different.

Wow, holy cow, and a bunch of other expressions. I'm simply amazed by these responses. You, of course, understand the implication of your replies. It appears you are saying Steve will be wasting his time creating SpinRite 6.1 and 7.0, should move on to something "more valuable," and we should simply junk our old hardware. While it probably is of no real concern, the original message in this thread did not say "bad sector," but "suspect sector/sectors." Here, all along I thought finding a way to continue to use existing hardware and not contribute to the dumping ground of electronic waste was a good thing. Foolish me. Ok, I'll fold my tent and disappear into the night.

I did not say that creating Spinrite is a waste of time. I answered your original question, " in a format understandable to the average user?". Steve and SR are not the average user.

What SR does is check every sector, and when it finds a bad or suspect one, goes to work on it. The drive will have a number of "spare" sectors, when it finds a bad or suspect sector during a "write" operation, it will mark the sector as bad and swap in a spare. But this only happens when you try to write to the sector. What SR does is read and then rewrite every sector to force this re-allocation to happen. The extra magic is that it tries far more times than normal disk software, and makes an intelligent deduction of what the original data might have been rather than just giving up and saying "that sector is bad and I don't care what the data was".

Grizbud said:

When Windows alerts a user to scan a drive, is the location of the suspect sector/sectors stored in an accessible location and in a format understandable to the average user? If so, would it be acceptable practice to tell SpinRite to focus on the portion of the disk containing those sectors rather than running it on the entire disk?

Jim

Click to expand...

@Grizbud / Jim: Thanks to SpinRite's heritage (in 1987 a 10MB drive cost $5,000 and no one had spares lying around), SpinRite has always taken the position of fixing an entire drive. The world has obviously changed a great deal since then. In the short term, the next SpinRite - v6.1 - will again be fast enough to practically scan and repair the sectors of an entire drive. I just benchmarked a 2TB Seagate Barracuda “spinner” at 183.3 megabytes/second using the forthcoming hardware drivers that SR v6.1 will have. At that rate, SpinRite will be able to perform a full scan of a 2TB drive in 3.031 hours. And if/when SpinRite encounters any trouble on the drive, it will drop into a new higher-speed repair mode before proceeding on to the rest of the drive.

In the future, SpinRite will be aware of our newer filesystems in the same way it's aware of FAT file systems. That will allow it to operate at the file recovery level. Users will be able to ask for the recovery of specific files to be recovered, and it will have extensive file system recovery / rebuild capabilities.

But, for now, even the next release with its blazing drivers, will finally again be practical to use on a whole-drive basis.

In my experiences with computer repair, Windows very rarely warns a user about a failing drive. Reason seems to be that it'll only warn when the drive itself sets a 'FAIL' status for its overall-health self assessment. I liken that assessment to the 'dummy light' on a old (pre mid-90s) car. In the overwhelming majority of cases the light would only turn on when there was OBVIOUSLY something VERY wrong and the car was either DEAD or SPUTTERING to a complete stop. That seems to be how the HDDs treat that 'passed' or 'fail' status.

So rather than rely on that 'dummy light', I primarily rely on Reallocated, Pending and Uncorrectable sector counts in the SMART stats. For the drive/model HDDs (such as laptop-Toshiba) that exclude reporting Pending and Uncorrectable, I look at Power On Hours and compare that value to the most recently reported error. If it's been like 9,000 hours since the last error, then it's likely fine. But having no errors doesn't mean it's fine either; like pending/uncorrectable stats, manufacturers don't always enable error logging. I doubt it's coincidence, that where manufacturers have decidedly avoided reporting pending and uncorrectable counts, also seem to have the least reliable devices. For these devices, (and more especially absent of SpinRite,) even having 1 reallocated is justification for pitch because we have no clue what the pending and uncorrectable counts are.

Instances where WINDOWS alerted the user to run a scan, in most cases was a result of WINDOWS having run-upon some corruption in the file tables. Some cases the corruption was caused by an abrupt shutdown; for examples power loss or system halt (BSoD). While file table corruption is most often caused by failing sector(s), that is not specifically what Windows is warning about.

The way ChkDsk works, running a surface scan over failing clusters most often does little good, no good, or even makes matters worse. ChkDsk does not rewrite as it reads. ChkDsk also seems to ignore failing/slow clusters. As a result, in the overwhelming majority of cases it basically waits until it's too late for it to read the data before it even marks a cluster as 'bad'. Then it just ends up relocating misread data. Not good. At that point it's too late to run data recovery. The misread data was correctly written (relocated), so there's nothing to recover. Meanwhile the other slow/failing clusters surrounding the one it just relocated (at the filesystem level) are left alone. All around bad.

SpinRite on the other hand notices and compensates for slow/failing sectors, works VERY hard to recover bits at the sector level (rather than cluster), and has the drive hardware perform the reallocation. Reformat a drive with clusters marked bad by ChkDsk, and those flaky sectors may be put back in use. SpinRite on the other hand does its reallocation at the hardware level via firmware. So those bad sectors won't be put back into use, even with a repartition and quick format. Showing you where those bad clusters are in the drive map, also enables you to partition around the bad spot; something I've taken advantage of in the past. I've used other sector rewrite tools in the past. In terms of data recovery, they function much like ChkDsk, simply relocating misread data.

I always explain CHKDSK to non-techy users as:-

1 It is checking that every file on the disk is in the Index (FAT or MFT)
2 It is checking that every file in the Index exists on the disk.

It will try to correct any mismatches, but does not look at the contents of the file for readability or accuracy.

DiskTuna said:

............... drive .... should be ........ clone[d] ... using tools that expect and are designed to handle read issues. .................

Click to expand...

OMG that reminds me: @Steve PLEASE add cloning capability to your product. I've wanted to say that for YEARS. Frankly, the other cloning products suck. They're just cloning misread data. File system awareness would even allow the user to target specific files for copy to USB media. Of course that means increasing the price. (wink, wink)

Only attempt it even once you have cloned the rest of the drive, that way you probably have the bulk of the data should the drive decide to fail completely. So cloning strategy should initially be aimed at stressing the drive as little as possible, and steer away from troubled areas.

Click to expand...

This is known as a multi-pass cloning strategy. Seems like when drives are that tipsy, they barely (if at all) last the first pass and don't survive the second. The SMART self-assessment is nearly always 'fail' by then. In other words, it's not just some bad patches on the platter; there's some other thing(s) going on with it. (servo, r/w head, power, memory, etc) Had one drive that wouldn't work unless I used my finger to place a very small amount of pressure on the armature. Weird. Anyway, multi-pass and with my experiences, in every case the data from the first pass was too corrupt to be at all usable. With NTFS file recovery scenarios, it seems that recovering the file table is critical or the rest of the data is virtually useless. The file table not only contains file allocation data, but the first 4k chunk of nearly (if not) every file itself. (and many files are <4K)

With regards to recovering data from bad sectors: If a sector is truly bad then you can't. You can not trust the data the drive gives you. ..............

Click to expand...

There are of course cases where the data was written incomplete or incorrectly to start with. But if that were the always case it wouldn't be possible for SpinRite to succeed where other products fail. The way those tools often work is, keep reading the sector a set number of times and hope a single result passes CRC; basically a more aggressive ChkDsk. When it doesn't get a result that passes CRC it seems to just relocate the corrupt data while resetting the CRC code. SpinRite is way different. Remember that data reads are not just a static good/bad result. There are levels of degredation; each read can produce slightly varied results. What I've noticed is, often the degradation seems to skip sectors in a near-pattern like manner. At the file system level, it's rejecting that entire 4K cluster over CRC failure. But the reality is, there may only be one out of those four sectors that actually need fixed. (depending on hardware and filesystem, there may be four sectors per cluster) Statistical analysis over hundreds of rereads at the bit or byte level can yield a result that is close enough to work.

DiskTuna said:

Incorrectly written data can not be recovered.

Click to expand...

@DiskTuna I may be misunderstanding what you're saying, and I may be about to compare apples to oranges, and I know you were talking about flash memory. BUT, this statement is not true for spinners. SpinRite's claim to fame is that it will attempt to re read defective sectors (or clusters?) up to 2000 times bringing the read head in from different angles and at different speeds. It compiles a statistical database from the different reads, and can often retrieve all but a few bytes of the sector. Many times, this will be enough to make Windows happy enough to read the files in the sector. If this is totally off base compared to what you were discussing, I apologize. Also, I don't know if SpinRite's aggressive reading would do any good on flash since it would, presumably return the same bad data each time. It is also true that it is often better to get corrupted data back than no data. It depends on the situation and the data.

May your bits be stable and your interfaces be fast. Ron

@DiskTuna I see that you have professional level experience whereas mine is prosumer. And, I wasn't trying to be controversial with my prior comment. Thanks for providing that clarification. What you say is very interesting and brings up some thought provoking concepts as to what SpinRite could and couldn't do. It is, in fact bending my brain a bit. I'm trying to think through the permutations of what if a similar event happens during a power failure or something. Not having much luck so I may just have to go to lunch. But, it is a known fact that SpinRite can often recover "unrecoverable" data. Don't feel obligated to spend lots of time on this. It is not urgent. But it is interesting.

May your bits be stable and your interfaces be fast. Ron

It could be argued that SpinRite would recover what was on the disk. The fact that the data on the disk is incorrect, doesn't change that. It is the old "Garbage In, Garbage Out" principle.

Presumably cabling errors could also cause this.

DiskTuna said:

I agree it's interesting! And I will not make it a secret that I am somewhat skeptical when it comes to unexplained recoveries. When it comes to reading let's call it bad sectors, in the end we have to work with what the drive gives us. Often the drive gives nonsense. Applying statistics to nonsense still will produce nonsense. I mean I really see how Spinrite can be useful. But in it's current form not for data recovery, even if alone for the fact it tries to recover in the original location. And yes, I do know it can work. years back I wrote a DOS tool called DiskPatch which also had a read/write surface scan feature. More than once were people able to revive drives using that by simply provoking the drive's sector reallocation with far less re-reading and analysis.

Click to expand...

@DiskTuna I'm afraid I'm going to have to differ with your assertions. SpinRite is in fact, one of the most unique and effective data recovery solutions on the planet, at least for spinners. I guess it's still being determined what it does for flash. There is, to my knowledge, nothing like it. I don't mean any offense by this, but if I may ask, are you a Security Now listener, and if so for how long? There's no wrong answer. But I would simply point out that @Steve and @leolaporte are now up to podcast 799.


I've listened to almost all of them. Almost every one has one or two testimonials of people who've brought hard drives back from the grave with SpinRite. You could even search the transcripts for the testimonials. I couldn't figure out a way to do that with Google and limit to this section of the GRC website. This page, and the others which are linked on it, explains the operation of SpinRite.


SpinRite 1) disables auto reallocation, 2) erroneous data reading - here's a quote "
But rather than ignoring the data from a bad read, SpinRite uses its unique "hardware level access" (which no other utility has) to read whatever data the drive was able to get from the bad sector. SpinRite begins assembling a database of this bad data, which will be used by SpinRite's "Dynastat" data recovery system.

It often happens that during this re-reading and data collecting phase, one perfect read will be received just by trying to read the sector many more times than other software ever bothers to", 3) head repositioning, 4) dynastat analysis, 5) accept partial data.

I'll leave it up to you to read the details and the other links. But, in my opinion, like I said, there is no other program like SpinRite. It has a proven history of valid IMPOSSIBLE data recovery on mechanical drives. As you have seen, it also appears to be beneficial for recovery on flash drives, albeit probably through different mechanisms.

I will admit, and @Steve says, that if there is a hard (media) error, or a controller error, then software won't be able to recover it.

May your bits be stable and your interfaces be fast. Ron

SpinRite 6 has been successfully recovering data using advanced techniques for 16 years. Don't know about previous versions. In 2004, as far as I know, most of the tools you mentioned didn't even exist. If they are now doing what SpinRite does, they're just copying the original. Give credit where credit is due. If they are doing such things, it's about time they got on board. @Steve started it all. The following link is from @Steve 's website - a review from 2004.


May your bits be stable and your interfaces be fast. Ron

PS - http://www.deepspar.com/mobile/about-company.html DeepSpar disk imager BEGAN development 2 years after the prior review that I quoted. And, SpinRite had already progressed from version 1 through version 5.

May your bits be stable and your interfaces be fast. Ron

The fact is that @Steve was a pioneer, if not the pioneer, in data recovery when most others were just wannabe's. Whether he's currently industry standard has no bearing. He was data recovery when data recovery wasn't cool. These are things you appear to be disputing. And, I imagine he knows more than the rest of us what can and cannot be done from the interface of a storage drive. And yes, that's changed in 16+ years as the drives took control over more and more of their internal operations and hid more and more of it from the interface. It's also true that some of the documentation was written years ago.

But, OK, we can stop questioning @Steve 's claims and zoom to the present and continue exploring what can be done with the current crop of drives.

May your bits be stable and your interfaces be fast. Ron

@DiskTuna Look. I have no wish to fight, and there's no need to. But, this forum belongs to Steve Gibson of Gibson Research Corporation, IE @Steve . By "us" I meant people on the forum, and Steve's Security Now podcast fans, and SpinRite users. I and many here have known him for 15 years through his podcast. I don't know him personally. But, I've spent a lot of hours listening to him explain security issues, technology, and a good bit of talk about hard drives, data recovery, and SpinRite. He essentially invented the world of hard disk data recovery when almost no one else was doing that. I've heard enough of his commentary to know that he's a solid engineer who doesn't make exaggerated claims about what he, or his software, can do. On several occasions, including:

DiskTuna said:

Don't claim lower level access to hardware than any other tool in the world when this demonstrably isn't true.

Click to expand...

you have made allegations, or implied, that Steve is making exaggerated claims. I assure you that, whatever he said in his documentation, at the time he said it, was true to the best of his knowledge. If technology has changed, then we can investigate to determine how that new technology can interact and benefit from the old world of SpinRite, which is being updated as you know. I would like to suggest that you stop disparaging SpinRite, or Steve's work, or his claims. Let's move forward and learn more about the world of data storage and how to protect all our precious fragile bits. But, bear in mind that spinners are far from gone. Even some of the old ones like 4 GB ones are still chugging along. So, there's still lots of work for SpinRite to do. And, as you've seen with SSD's, but it's also true for spinners, SpinRite has a very cool therapeutic effect on the drives, helping to prevent data loss from ever occurring in the first place.

So, here's to saving the data! Forward!

May your bits be stable and your interfaces be fast. Ron

Very well. We'll just move forward and try to save all the data we can, by whatever means we can.

May your bits be stable and your interfaces be fast. Ron