Hardware config for SpinRite to recognize drives above 2.2GB

[brado2049]

You need another computer where you can install the drive to be tested as an internal SATA drive.

Thanks for the comments.

Agreed. This was the net of the research / comments of mine earlier in this conversation. I had been barking up the wrong tree (both now and a while ago), in thinking that the issue was my hardware. In my case, it wasn’t. My hardware adequately supported large hard drive capacity, if the software did. SpinRite didn’t in my config (and I’m referring to my latest testing over USB, not testing done in the past). I was under the impression testing over USB was possible. For the most part, it is not — you must have internal SATA connection. My lesson learned.

To that point, an update: I’ve been talking with the Zima Store folks about CSMWrap, and they were interested in that — they’ve got a sale that just started, so I decided to give this a greater college try: I ordered a ZimaBoard and a Zimaboard2 (the Z2 won’t ship until August). Once those arrive, I’m going to try to get SpinRite working on both, the ZimaBoard out of the box, and ZimaBoard2 with CSMWrap.

I have also spec’d out a mini-ITX config if I have to go that route, as I’ve been wanting an open-bench config in general for miscellaneous hardware testing. But I’m going to hold off on that until after the ZimaBoard tests.

 

[jlee]

The rare exception would be a system with a more modern BIOS that can address space beyond the 2.2 TB limit.

Well, the BIOS Enhanced Disk Drive Services (EDD) spec was released back in 1995. It allows up to 64-bits of disk addressing, far more than SCSI or ATA. I doubt there's any BIOS around today that doesn't support it, and SpinRite will use EDD when it detects the BIOS supports it. The 2.2TB limit is almost certainly not because of BIOS limitations these days, or even in the last 20 years.

 

[peterblaise]

@jlee, as far as I understand, the BIOS limit relevant in this thread limits USB access.

Not IDE ATA SATA.

- - - - -

Some really old-by-now BIOS, especially PATA-generation, never expected to 'see' any drive larger than 28-bit LBA topping out at 137.4 GB, though later PATA climbed to 750GB via 48-bit LBA before SATA overcame PATA, yet some folks bridged 1 TB SATA to PATA in some systems.

Not USB.

New BIOS - actually BIOS with ISA sockets - 'today' is mostly to replace legacy systems targeted for industrial installations still running Windows XP or earlier with dedicated custom programming that can't be migrated to more-modern equipment, but the facility is in need of new systems to replace ageing systems that have become unreliable or impossible to maintain, see:
https://www.google.com/search?q=Who+sells+new+legacy+BIOS+computers+with+ISA+sockets?

'Today', systems generally have moved on to UEFI, and not BIOS, and the few new systems with a Compatibility Support Module (CSM) have, in my experience, been crappy at BIOS, FreeDOS, and SpinRite 6.1.

Hence this thread.

I believe the challenge in this thread is for folks to find any semi-vintage computer that uses BIOS with 48-bit to 64-bit LBA and has SATA sockets and or permits adapters to attach a drive under test directly as SATA.

So, yes, BIOS Enhanced Disk Drive Services (EDD) facilitated access to 64-bit Logical Block Addressing (LBA), but computer vendors in that generation never expected USB thumb drives to do anything other than transfer small amounts of data between computers via 'sneaker net', so they never paid for any research and support to make USB boot and access fully functional or standard through the BIOS and DOS.

It was a long time before external USB HDDs became a 'thing', and they were for backup, not booting.

It even took a while for USB to boot with anything more than a small drive and a single-purpose utility on board, vendors having become accustomed to diskette and CD reproduction houses for full operating system distribution, where a gazillion copies of boot USB drives with full Windows was unimaginable.

So, yes, they built USB as a boot method, but I have one BIOS that is so stupid that it marks ALL USB drives that are present at boot as the size of the largest USB drive under 137GB, so if I leave a 16GB and 256GB USB drive inserted, they both get turned over to FreeDOS and SpinRite 6.1 as 16GB drives.

I have other computers where the BIOS only sees one USB drive, so I can boot to SpinRite, but it is not shown any other USB drive in the computer.

USB sux.

So the challenge is to test used computers to find one that can behave for the user's goals, and testing USB drives via SpinRite 6.1 is a significant challenge, partly due to the dominance of UEFI that does not turn drives over to SpinRite 6.1, and partly due to legacy BIOS being so variable - no vendors depended on USB drives doing anything predictable under DOS, instead, letting Windows bring it's own drivers, so BIOS never got sophisticated, USB wise.

- - - - -

That being said, I suggest plugging the USB drive in question into ANY Windows computer and running free ValiDrive on it, see:
https://www.grc.com/validrive.htm

ValiDrive is like a mini-SpinRite 6.1 Level 5 under Windows, and I run ValiDrive on ALL USB drives, including 20TB drives in USB adapters.

Of course I do!

So, @brado2049, the original poster wrote:

CSM BIOS support and able to boot off USB. I have tried directly attaching to the motherboard on a desktop, and also connecting SATA drives externally via USB. I have bought and tried 6-7 different adapters and hard drive cradles which purport to support larger hard drive geometry. All show my drives at 2.2TB, even though if I boot into Windows they all show 4TB​

Yes, CSM and BIOS and USB sux.

But Windows 'sees' the drives just fine via USB . . . so run ValiDrive on them, and get back to us with the results - charts please!

Thanks.

 

[DanR]

Well, the BIOS Enhanced Disk Drive Services (EDD) spec was released back in 1995. It allows up to 64-bits of disk addressing, far more than SCSI or ATA. I doubt there's any BIOS around today that doesn't support it, and SpinRite will use EDD when it detects the BIOS supports it. The 2.2TB limit is almost certainly not because of BIOS limitations these days, or even in the last 20 years.

It is apparently a legacy spec that is no longer commonly used.

Google Search

To quote Google's AI:

No, BIOS Enhanced Disk Drive (EDD) services are not commonly used in modern computer systems
,
EDD is a legacy specification that provided a way for the BIOS to interact with disk drives and make information available to operating systems, particularly older ones like MS-DOS.

Why EDD is less common now

• Modern Interfaces: The rise of newer interfaces like SATA and NVMe has shifted the focus away from older standards like the conventional INT 13h interface that EDD built upon.
• UEFI: The transition to Unified Extensible Firmware Interface (UEFI) in modern computers has also reduced the reliance on traditional BIOS standards like BIOS Boot Specification (BBS), which is related to how the BIOS handles the boot process.
• Operating System Advances: Modern operating systems have their own robust methods for accessing and managing storage devices, often bypassing the need for BIOS-level services like EDD once the OS has booted.

While EDD might still be present or enabled in some systems for backward compatibility, its role in the day-to-day operation of modern computers is significantly diminished.

I am not qualified to either agree or disagree.

 

[peterblaise]

Really, @brado2049, run free ValiDrive under
Windows on the drive attached as USB in an
adapter, and show us the results.

https://www.grc.com/validrive.htm​

ValiDrive is like a mini-SpinRite Level 5,
checking 576 regions across a drive.

Thanks.

 

[peterblaise]

"... To quote Google's AI ..."​

Wow, we really have entered the next
generation.

Good word, 'generation', because at first it was
AI-enhanced, merely quoting web sites that
responded to the search terms, but it was not
generative AI, that is, Google did not appear
to be inventing responses without web
sources.

Now, in only a few years since Google started
AI-enhanced features, I'm starting to see what
appears to be generated contents, as if
conversation, correcting my inquiry, and
redirecting me to rephrase.

I think we are going t have to be scrutinizing
of making decisions based only on Google's
AI-enhanced search summaries.

I hate to include Google's list of links, but it
may become a requirement in support of
whatever we think we are sharing, as if it's
authoritative.

Google AI-enhanced search results are like a
mini-Wikipedia [ Article ] on demand
WITHOUT the [ Talk ] page behind showing
our work and contrarian points of view, and
without human intervention.

And we really have to highlight our own
hands-on experience.

So, what's it gonna take to nail down what is
going on in the challenge:

Hardware config for SpinRite to recognize ​

drives above 2.2GB​

?

 

[brado2049]

Thanks for the additional conversation @DanR and @peterblaise — looks like there’s some additional possibilities of things to try — greatly appreciated!

I wanted to give a status on what I’ve been up to. This may be no new information for many, but I imagine there are some who might be helped (it helped me), but for everyone I wanted to confirm that this was an affordable and quick route to a working SpinRite hardware rig.

I bought both a ZimaBoard 232 and a ZimaBoard 2. The latter is not shipping until August, and I confirmed does not support CSM / Legacy boot. However, I’m going to test some other possibilities on it once received (e.g. CSMWrap). However, I received the ZimaBoard 232 a few days ago, and today opened the box and gave it a try.

I can happily report, right out of the box, no BIOS configuration changes or anything related required, was able boot to my USB drive (created as prescribed in the SpinRite doc) and run SpinRite. Further, it did indeed recognize the full 4TB capacity of the SATA hard drive I had used for prior tests in this forum thread (the full capacity of which had not been recognized by SpinRite over USB).

This is probably worthy of a blog post in the near future, but for now, I am including a few screenshots, and this summary note: if you are looking for quick path to a dedicated SpinRite rig for SATA drives (other options if you leverage the PCIe expansion), for the total of ~$105 (which included the ZimaBoard 232, Mini-DisplayPort to HDMI adapter, and SATA Y-cable), you’re in with this config. SpinRite estimated 7.26 hours for a Level 3 on my 4TB drive.

I hope this helps someone. I’ll post again if / when I get a blog post written and have some interesting testing info on ZimaBoard 2 once received.

 

[peterblaise]

I'm imaging the "Yeehaw!" shout when the
4TB measurement appeard on SpinRite's
screen!

;-)

Thanks for sharing - the resolution is so important.

 

[brado2049]

@peterblaise -- your imagination would have indeed been accurate. I was thrilled when I saw the full capacities of a few different drives come up. In finally getting to run a long-running SpinRite operation, I inadvertently discovered something I had forgotten (much to both my humor and annoyance), that older OSs weren't plug-and-play. In the screenshots I shared in a previous post, I had this rig attached to a monitor with four inputs, and after about 4 hours of running a Level 3, with a LOOOOOONG way to go, I switched the monitor input to another computer to do some work. Oops...no way to get the display back, switching back and even disconnecting / reconnecting the display didn't work. I was completely blind to progress. LOL...well dedicated monitor employed now....

 

[Tazz]

@peterblaise -- your imagination would have indeed been accurate. I was thrilled when I saw the full capacities of a few different drives come up. In finally getting to run a long-running SpinRite operation, I inadvertently discovered something I had forgotten (much to both my humor and annoyance), that older OSs weren't plug-and-play. In the screenshots I shared in a previous post, I had this rig attached to a monitor with four inputs, and after about 4 hours of running a Level 3, with a LOOOOOONG way to go, I switched the monitor input to another computer to do some work. Oops...no way to get the display back, switching back and even disconnecting / reconnecting the display didn't work. I was completely blind to progress. LOL...well dedicated monitor employed now....

Have a look here: https://forums.grc.com/threads/turn...ank-when-i-turn-it-back-on-next-morning.1418/

Specifically the second page. It sounds like the same thing you're experiencing.

 

[brado2049]

@Tazz — thanks for that — yep, appears that was pretty much it. I connected my ZimaBoard to a dedicated monitor (no change of input) and ran SpinRite again, and now powering the monitor off / on has worked fine so far.

 

[jlee]

It is apparently a legacy spec that is no longer commonly used.

To quote Google's AI:

No, BIOS Enhanced Disk Drive (EDD) services are not commonly used in modern computer systems
,
EDD is a legacy specification that provided a way for the BIOS to interact with disk drives and make information available to operating systems, particularly older ones like MS-DOS.

All that is saying is that the legacy BIOS is no longer used to access disk drives, which we know to be true. But EDD is still supported by all BIOS CSMs today that I know about.

SpinRite definitely uses EDD if it's present, take a look at

https://grc.com/groups/spinrite.dev:53707
https://grc.com/groups/spinrite.dev:36848

 

[jlee]

@jlee, as far as I understand, the BIOS limit relevant in this thread limits USB access.

Not IDE ATA SATA.

- - - - -

Some really old-by-now BIOS, especially PATA-generation, never expected to 'see' any drive larger than 28-bit LBA topping out at 137.4 GB, though later PATA climbed to 750GB via 48-bit LBA before SATA overcame PATA, yet some folks bridged 1 TB SATA to PATA in some systems.

Not USB.

To get to a USB drive via the BIOS you have to use the BIOS interface. It used to be that the BIOS interface was limited to 8GB. Thus even if you had a SCSI or some other drive that allowed you to address LBAs past 8GB, 8GB was all that you could do through the BIOS. A big reason for EDD was to get past that. Every BIOS today supports EDD, so the BIOS interface limit is not a factor in accessing USB drives.

That's the front end of the BIOS, the BIOS interface. You also have to worry about the backend which here is the actual USB drive. The BIOS takes commands from the BIOS interface and has to issue the appropriate commands across the backend interface, which on an external USB drive is the USB interface.

USB drives themselves use different protocols across the USB cable to access the drive. The oldest is SAT, which uses SCSI commands. There are different SCSI read and write commands which allow larger LBA ranges and transfer sizes. The smallest SAT allows are the SCSI READ (10) and WRITE (10) commands, which can address 32 bits or up to 2.2TB. But I'd be surprised if any BIOS today implementing SAT to access USB drives only supported READ (10) and WRITE (10) commands. And no USB drive of greater than 2.2TB is going to limit its support to only READ (10) and WRITE (10) SCSI commands.

There are also more modern USB protocols for accessing external drives and AFAIK they all support more than 2.2TB.

All of which is to say that modern BIOSes should allow you to access any external USB drive greater than 2.2TB.

 

[peterblaise]

Re: "... modern BIOSes ..."​

At some point they all became legacy ad hoc, subcontracted, least
common denominator, so there are no 'modern', nor 'standards'.

They've moved on to UEFI, which is proving equally random,
unpredictable, and frustrating, but usually talk to NVMe ( though
SpinRite does not ).

So long as the computer booted for the vendor, they were done, fini,
moved on, no additional BIOS features promised or supported.

As mentioned, an HP tech asked my why I wanted to boot from USB.

!​

In other words, they did not consider it a feature worth supporting, so
if it fails, they do not care, and have no remedy.

Hence the hunt for used gear that actually does what we want, and
no 'specifications' help, only hands-on testing, and that's only good
for the current test, the next test may prove the gear is inappropriate,
hence having multiple old BIOS-computers at hand.

- - - - -

Our own @Paul F wrote programs to test USB and replace limited
BIOS access with less-limited SCSI access, hoping to get past BIOS
quirks, and then SpinRite had access to the entire drives, if slowly.

Amazing, but ultimately easier and more reliable to use another
computer and or another attachment method.

So, got a stack of used computers to keep on hand, "just in case"?

;-)

 

[peterblaise]

Re: "... SpinRite definitely uses EDD * if it's present ..."​

But [ for us poorly supported end users who are hunting down old
BIOS systems, before acquisition, for us, there is ] no way to tell if EDD
is present or fully supported and functional.

So, "... buy it and try it ..." is the slogan of the day.

Got a stack of used computers to keep on hand, "just in case"?

;-)

- - - - -

* From the web: In the context of BIOS drive access, EDD stands for BIOS Enhanced Disk Drive Services.
Here's a breakdown of what that means:

• Enhanced Disk Drive Services (EDD): EDD is an extension to the legacy BIOS INT 13h interface, which traditionally used Cylinder-Head-Sector (CHS) addressing for disk access. EDD was developed to address limitations of the old system, particularly in handling larger drives and newer interfaces.
• Purpose of EDD: EDD enables the BIOS to communicate more effectively with the operating system (OS) about connected disk drives. This includes providing details such as:
  • Bus Location: Whether the drive is connected via PCI, SATA, USB, etc.
  • Interface Details: Information about the interface (e.g., ATAPI, SCSI).
  • Drive Parameters: Capabilities and geometry of the drive.
• Key Improvements with EDD:
  • Larger Drive Support: EDD helps the BIOS support hard drives larger than the 528 MB limit of the conventional INT 13h interface.
  • Logical Block Addressing (LBA): EDD replaces CHS addressing with LBA, which simplifies data access and supports much larger storage capacities. According to Lenovo, "LBA works by assigning a logical block address to each block of data on a storage device... allowing the system to easily locate and retrieve data without the complexity of traditional addressing methods like cylinder-head-sector (CHS)".
  • Information for the OS: The information gathered by EDD is exported to the OS, allowing it to make more informed decisions about how to interact with the drives. This can include information needed for boot processes or for handling different SATA modes (IDE emulation vs. native SATA).

In essence, EDD acts as a bridge between the BIOS and the OS, allowing them to better understand and utilize modern disk drives with their increased capacities and diverse interfaces.

 

[Casper]

Hi A laptop setup that works with Spinrite 6.1 on a 14 Tb desktop drive.

ThinkPad T400 laptop released in 2008 with an Ultrabay Slim swappable hard drive /DVD drive slot that connects directly to the motherboard. I was able to use the Ultrabay Slim slot cradle to connect a 2.5 inch SATA drive or a 3.5 inch SATA drive using a SATA 22Pin Male to Female Data Power Extension Cable 50cm

SATA 22Pin Male to Female Data Power Extension Cable 50cm | eBay UK

Compatible with 22 Pin SATA connections such as SATA hard drives, SATA optical drives, SSHD etc. 22 Pin SATA Male to Female. Extend your SATA connection with this extension cable. Iron buckle for secure connections.

www.ebay.co.uk

plus a 12v molex connector off an external drive hub.
https://www.amazon.co.uk/Docking-Tccmebius-TCC-S868-UK-External-Enclosure-Red-Black/dp/B07MVFPJ5K
The laptops boot screen was unable to detect the 14Tb drive when connected via usb using the usb hub but when connected via the Ultrabay slot it detected it as 14 Tb.
The laptop could also detect smaller 1 TB 2.5 and 3.5 SATA drives via usb using the usb hub but the Smart data was unavailable and the time to complete was very much longer.
I am very happy even if it takes 39 hours to do a level 3 on this 14Tb drive.

 

[jlee]

Re: "... SpinRite definitely uses EDD * if it's present ..."​

But no way to tell if EDD is present or fully supported and functional.

Incorrect. EDD defines a function "Check Extensions Present" which can be used to determine if EDD is supported. You load the following registers with

AH 41h
BX 55AAh
DL BIOS device number

and then execute int 13h. If the EDD extensions aren't present, it causes an error (i.e., the carry bit is set with AH set to 1) since it's an illegal request under the old int 13h definitions. If EDD is supported (i.e., the carry bit is clear) you get a bit map in CX that tells you more about which parts of EDD are supported. Thus before doing anything SpinRite can use this test to determine whether or not to use EDD.

 

[peterblaise]

Cool that the EDD specification implementation promises predictable
accessibility and enumeration.

I updated my prior post-15793 to say:

"... But [ for us poorly supported end users who are hunting down
old BIOS systems, before acquisition, for us, there is ] no way to tell
if EDD is present or fully supported and functional. So, "buy it and
try it" is the slogan of the day. Got a stack of used computers to keep
on hand, "just in case"? ;-) ..."​

Has anyone seen EDD play out in the environment in question?

From https://www.grc.com/groups/spinrite.dev:36848

Subject: Re: Next Steps​

Date: Sat, 2 Apr 2022 10:48:33 -0700​

From: Steve Gibson <news007_@_grc.com>​

​

Following up on what Milton Scritsmier wrote...​

​

> Here are another two interesting tables:​

> ------------------ Mini Benchmark -----------------​

> Spinner Large Buffer 127-Sector Buffer​

> ------- ------------------- -------------------​

> 500GB ATA: 53.264 53.383 127.150 127.175​

> BIOS: 126.514 126.480 126.570 126.491​

> 1.0TB ATA: 85.313 85.574 160.725 160.738​

> BIOS: 106.130 116.492 105.726 103.448​

> In the two columns for "Large Buffer" what does that mean with​

> the BIOS rows? Aren't you still limited to 127 sectors with​

> BIOS int 13h? Or are you using EDD?​

​

You're right. We're still limited, though that would not be ​

clear from my labeling. And even EDD, believe it or not, despite ​

having a 16-bit field for the sector count, remains limited to ​

127 sector transfers max. It gives us greater sector number ​

range, and in the final EDD v3.1 we also get a 32-bit pointer​

to all of the lower 32-bits of RAM. But, incredibly, never​

more than 127 sectors at a time.​

​

> Yes, this appears true for SATA drives, both HDD and SSD. But​

> keep in mind that when you get to modern NVMe drives it may no​

> longer be true. The problem is that these drives can transfer​

> gigabytes per second. If you limit yourself to 64KB commands​

> the I/Os required to reach these speeds becomes prohibitive.​

> For example, if a NVMe SSD drive is capable of 4GB/sec reads,​

> it takes 131,072 64KB I/Os per second to handle that speed. I​

> don't know if any PC processor can handle that issuing​

> commands one at a time.​

​

Right. And I later verified that on a different machine large ​

buffers WERE definitely faster. And as for NVMe, I've ​

deliberately kept SpinRite's throughput measurements 64 bits for ​

exactly that reason. ​

​

We know that SATA (III) has peaked at around 600 MB/sec. So its ​

bytes/sec will always fit into 32 bits. But not the future. ​

​

> There is a new Windows API called "DirectStorage" that​

> Microsoft is about to release that is designed to mitigate​

> this issue. See​

> https://devblogs.microsoft.com/directx/directstorage-is-comin​

> g-to-pc/ especially the discussion about I/Os per second on a​

> modern NVMe SSD. It's not clear if they're issuing larger​

> commands but they definitely are talking about taking​

> advantage of the NVMe's much larger command queuing feature.​

> ​

> Allyn Malventano on this week's TWiT​

> (https://twit.tv/shows/this-week-in-tech/episodes/868 ) at​

> about 1:50:00 in talks about modern NVMe SSDs and​

> DirectStorage. It turns out that SSDs using NVMe and the​

> upcoming PCIe 5 interface will be able to do 14GB/sec reads!​

​

I meant to get back to finish listening to Allyn last week.​

I caught the show setup but not much into the show itself. ​

___________________________________________________​

/Steve. https://dev.grc.com/Steve/spinrite-v6.1/-/wikis/home​

I dunno - anything informative there?

Thanks.

 

[peterblaise]

And from https://www.grc.com/groups/spinrite.dev:53707

Subject: Re: AMI BIOS work completed​

Date: Sat, 20 Jan 2024 10:45:16 -0800​

From: Steve Gibson <news008_@_grc.com>​

​

Following up on what Scott F wrote...​

​

> I thought 2.2 TB was a limitation for ALL drives that SR​

> accesses via the BIOS, regardless of firmware vendor, because​

> an MBR disk can't be larger than that, and BIOS systems only​

> know how to read MBR disks directly.​

​

The early BIOS vendor Phoenix Technologies led the way as far ​

back as 1995 with their release of the Enhanced Disk Drive (EDD) ​

Specification which switch the BIOS from passing values in ​

registers to passing a pointer to a parameter block. While that ​

parameter block still had a transfer length limit of 127 sectors ​

due to the 64K DMA transfer limitation of the PC's hardware, it ​

specified a 64-bit linear sector field... as far back as 1995!​

​

So ever since then, there's hasn't really been any BIOS driven ​

limitation on drive size.​

​

Now... DOS, on the other hand, is the interpreter of the MBR and ​

you're correct that the MBR strictly imposes a 32-bit limit on ​

partition length and location.​

​

> Steve, wasn't this limitation one of the driving forces to​

> create your own drivers and bypass the BIOS?​

​

In practice it might well be that actual BIOSes ARE self-limited ​

to 32 bits. But the spec doesn't impose that limit.​

​

The two driving factor for bypassing the BIOS have been SPEED ​

and low-level access. The innovation that SpinRite 6.0 brought ​

was that while it still used the BIOS for its bulk transfers, it ​

used direct ATA access for recovery and all sector-level work.​

​

And the BIOS =definitely= imposes that 127 sector transfer limit ​

since it has no concept of newer hardware that's capable of ​

using flat 32-bit transfer counters and addresses.​

________________________________________________________________​

/Steve. https://dev.grc.com/Steve/spinrite-v6.1/-/wikis/home​