FYI SpinRite's Development Roadmap

  • Be sure to checkout “Tips & Tricks”
    Dear Guest Visitor → Once you register and log-in:

    This forum does not automatically send notices of new content. So if, for example, you would like to be notified by mail when Steve posts an update to his blog (or of any other specific activity anywhere else), you need to tell the system what to “Watch” for you. Please checkout the “Tips & Tricks” page for details about that... and other tips!

    /Steve.
  • Forums Outage Fixed (Obviously!)
    Just a note that the glitch following yesterday's
    webserver certificate update was found and fixed!
  • A Patch for SpinRite 6.0's Division Overflow
    Please see my blog posting for the whole story!

moo

New member
Sep 27, 2020
2
0
I would: the secret access code is: wait! ;)

Unfortunately it doesn't exist yet. What does exist is a beta version of a disk speed test, and that should be released in the next week or so. Check back here in the new year, or stay tuned to Security Now.

You can never predict Steve and what will come up with issues as he develops and tests, but I wouldn't count on seeing a SR6.1 release any time before, say, June 2021 at the earliest.

Also, 6.1 will not have UEFI support. That will be 7.0, and that won't get released for at least a couple years if things go as they have been.
Thank you for your response, I appreciate it.
 

Scott

Member
Sep 18, 2020
10
5

Roadmap Suggestion: Tackle SAS drives next after 6.1

(posting on forums.grc.com as well as in newsgroups)

Summary:​

@Steve ,my suggestion is to add SpinRite capabilities for SAS (Serial Attached SCSI) drives before moving on to other capabilities like USB or UEFI.

Rationale:​

SAS drives are high performance drives used in enterprise-class storage, often in RAID arrays. I expect there would be a large market from enterprises for SpinRite in this space, as it builds upon SpinRite's known strengths in conventional magnetic recording (CMR) mechanical drives, and would (I think) take much less time to build than other proposed features. Additionally, the SCSI drive work needed for this could be reused as part of the USB feature addition.

Why address SAS prior to other planned features:​

This work is in the traditional “sweet spot” of SpinRite, addressing a still-burgeoning market of mechanical drives. It would take less time than other planned work, would work in a FreeDOS environment, and add tremendous value for enterprises. Steve, I could see you having a separate “Enterprise” version of SpinRite (and charging more) for these capabilities if you wanted.

Quick Technical Background:​

Drives:​

SAS drives are high performance drives, with models on the market as CMR mechanical drives as well as SSD. Mechanical drives have rotational speeds of 7.2K RPM, 10K RPM, or 15K RPM, and when connected via the current SAS-3 spec, achieve throughput up to 12 Gbps; the mechanical drives are clearly higher performance than AHCI / SATA drives, and are unsurprisingly much more expensive.

Host-Based Adapters (HBA):​

The HBA allows communications between the PCI bus and the drives. Unlike parallel SCSI, drives are not daisy-chained, but cables can be split and expanders can be attached to cables. Most HBAs allow for 4, 8, 16, or 24 direct-attached drives, and with the use of expanders, up to 1,024 drives can be attached to one HBA. HBAs allow for both internal and external connections to SAS drives. SAS allows for cable lengths of 10 meters vs 1 meter for SATA / 2 meters for eSATA.

HBAs come in two flavors:
  • HBA with hardware RAID built-in, or
  • HBA without hardware RAID (HBA in "IT" or "Initiator Target" mode).
In order to fully address the drive's capabilities, the driver would be written to work with an HBA in IT mode.

Most SAS HBA models have a BIOS and can be accessed under FreeDOS as well as UEFI. Unlike AHCI for native SATA, there is no industry standard for driver protocol to HBAs; different manufacturers use their own driver model, but I believe Steve could write a driver for just one vendor's family of cards; see next section for why.

Why only write a driver for one vendor's HBA cards?​

SAS drives run in RAID arrays and are hot-swappable. Any suspect drive would be pulled from its RAID array and then connected to a standalone "SpinRite" PC to run repairs. That PC could have the specified vendor's card in it. After repair, the drive would be reinserted back into the RAID array.

LSI Logic (now Broadcom) has the lion's share of the market; in addition to their own branded cards, they make most of the OEM cards (Dell, HP, IBM), and they use an industry standard protocol called MPI (Message Passing Interface) for communications, so it would make sense to me to target this vendor's family of cards.

Work:​

I believe there are 3 chunks of work:
  1. Write driver for the SAS Host Based Adapter. In addition to the vendor technical documentation, BSD-licensed driver source code in C is freely available and can be copied or studied for royalty-free use in closed-source commercial products like SpinRite.
  2. Add SATA drive capabilities. SAS HBAs work with both SATA and SAS drives, so this should be a port of the SATA driver work you’ve already done, using the SATA Tunneling Protocol (STP).
  3. Add SAS drive capabilities. This would translate your drive logic into SCSI commands, and be new work, using the Serial SCSI Protocol (SSP).

Code Reuse:​

  • As noted, the SATA drivers would hopefully be a port of the existing SATA work
  • The SAS drive capabilities, using SCSI, are new. However, the USB has two protocols for communication to USB drives:
    • USB Mass Storage Bulk-Only Transport (older), and
    • USB Attached SCSI (introduced with USB 3 but usable with USB 2 devices)
The UAS work, when it comes to pass, would be able to leverage the SCSI work done for SAS drives.

Reasons not to do this:​

This would not benefit home users, as very few home setups (even those with home NAS devices) use SAS drives. But I would think plenty of medium size and larger enterprises would be eager to have SpinRite capabilities for their on-premises storage systems.


Anyway, my 2 cents (or 200 cents, thanks for bearing with me!). Note I am not recommending this work for my own benefit, I'm semi-retired, but in thinking of what Spinrite 6.x and higher would be addressing, this seemed like a pretty big hole.

References:​

Great background thread on history of Broadcom (LSI Logic) family of HBAs is here, along with MPI commands:
https://forums.servethehome.com/ind...d-raid-controller-technical-discussion.24119/

BSD-Licensed source code (freely usable or copyable in closed source programs):
https://github.com/freebsd/freebsd-src/tree/master/sys/dev/mps/mpi

Wikipedia SAS:
https://en.wikipedia.org/wiki/Serial_Attached_SCSI

Wikipedia USB Attached SCSI:
https://en.wikipedia.org/wiki/USB_Attached_SCSI

Broadcom main product page:
https://www.broadcom.com/products/storage/host-bus-adapters

SAS cable selection info (they are directional, so cable types matter!!)

https://www.servethehome.com/sas-sa...8088-8470-8482-8484-single-device-connectors/
 
Last edited:

Scott

Member
Sep 18, 2020
10
5

Roadmap Suggestion: Tackle SAS drives next after 6.1

(....... CUT .....)
And based on conversations in the newsgroups, this probably doesn't make sense as a priority -- the R in RAID stands for Redundant, so RAID arrays will not need the data recovery features of SpinRite; though resurrection of a wonky drive is nice and saves some money, it may not be the most urgent need for SR
 

Argonot

Member
Sep 18, 2020
8
3
@Steve, will the new SpinRite be able to work on multiple hard drives simultaneously? I think that if SpinRite is able to work on several hard drives simultaneously, it would make it more attractive to companies with a lot of hard drives. I'm thinking for example of Backblaze that at the end of 2020 had 150,757 hard drives on their data centers. For a company like that I think it would be useful to run SpinRite on the drives that they plan to deploy, so that they minimize the failures of the deployed hard drives, and run it on the hard drives that fail, because if they choose to replace a drive "it can take up to a couple of weeks to rebuild a drive, especially large drives on busy systems".
 

Steve

(as in GRC)
Staff member
Feb 1, 2019
294
918
68
Southern CA, USA
www.grc.com
@Argonot : Not yet, but Yes! eventually. Simultaneous operation on any/all of a system's drives is absolutely on the roadmap. But none of the current SpinRite code — nor its user-interface — was designed to support simultaneous operation on multiple drives. So... your idea is absolutely right, but I cannot get there until I'm able to spend more time getting SpinRite ready for it.
 

Steve

(as in GRC)
Staff member
Feb 1, 2019
294
918
68
Southern CA, USA
www.grc.com
SpinRite's Development Roadmap — 4th Release

The link above is to the short 2-page PDF summary of the current, just updated, plan for SpinRite.​

What happened, is that over the past three or four days, while I've been working on SpinRite v6.1, I've also been working to lay out a clear plan for the future past SR v6.1. After extensive research into many possibilities, there was one clear winner for the way to go forward: Dropping 16-bit DOS and switching to a 32-bit real-time operating system which can boot from either BIOS or UEFI.

SpinRite's next home for the rest of its long life will be the On Time RTOS-32 operating system. It is clearly the right solution for a great many reasons. So, SpinRite's Roadmap has been updated to explain the implications of this decision.
 

jdoverton

New member
Sep 30, 2020
2
1
This user does not have permission to use the HTML BB code.
The link above is to the short 2-page PDF summary of the current, just updated, plan for SpinRite.​

What happened, is that over the past three or four days, while I've been working on SpinRite v6.1, I've also been working to lay out a clear plan for the future past SR v6.1. After extensive research into many possibilities, there was one clear winner for the way to go forward: Dropping 16-bit DOS and switching to a 32-bit real-time operating system which can boot from either BIOS or UEFI.

SpinRite's next home for the rest of its long life will be the On Time RTOS-32 operating system. It is clearly the right solution for a great many reasons. So, SpinRite's Roadmap has been updated to explain the implications of this decision.
Steve,

I can't thank you enough for your contributions to many aspects of my understanding of security and drive maintenance. I just had to upgrade my PC (the last one was about 10 years old, still serviceable now that I reformatted the drives and reinstalled Windows - so my wife uses it). However, the new PC is UEFI only. I cannot get readspeed (or anything else) to boot that doesn't support it. I was so looking forward to readspeed and I will try to resurrect a retired machine to use it, but having waited so long for 6.1 that supports ACHI natively, I hold little hope of seeing 7.0 soon. Can't wait for 6.1 and its speed enhancements if nothing else. I just have to hope my new PC's disk holds out until 7.0.

Put my vote in for advancing UEFI support.
 
  • Like
Reactions: Steve

orthopodvt

New member
Dec 30, 2020
3
0
I run a mixture of PC's and Mac's, so having UEFI support would be nirvana. I just keep my fingers crossed that none of my Mac drive crap out....
 

Ianc

New member
Oct 6, 2020
3
0
I'd vote for UEFI support too. Having just bought a (Win10) laptop for my photo editing, it only supports UEFI booting and I feel very vulnerable not being able to run spinrite on it.
 

Kmcdonald

New member
Nov 8, 2020
4
0
Having USB 3.0 to SATA docking station and Multifunction HDD Docking station and well as an external IDE drive enclosure and several external drives, I am interested in how well they will work with SR 6.1 and 7.0 since 6.0 doesn't detect any drives connected to USB ports. My current computers are old enough to support BIOS boot and both boot-up Readspeed fine from USB. (Note: I moved Spinrite 6.0 to the same USB stick that I installed Readspeed on and it runs fine from there for scanning the internal drives).
 

Steve

(as in GRC)
Staff member
Feb 1, 2019
294
918
68
Southern CA, USA
www.grc.com
@Kmcdonald : I'm afraid that v6.1 probably won't perform any differently than v6.0. But my plan is to get v6.1 out as quickly as possibly, then I've designed a solution to get moved over to v7 as quickly as possible. Since v7 won't offer anything other than UEFI support — being primarily a port of v6.1 from 16-bit to 32-bit code and OS — people might choose to wait until v7.1 to upgrade from v6.1, since it won't be until v7.1 that I'll be able to get USB working natively. /// That's the plan at the moment.
 

Steve

(as in GRC)
Staff member
Feb 1, 2019
294
918
68
Southern CA, USA
www.grc.com
Just out of curiosity, will v6.1 still fit on a 1.44 MB floppy?
Yes. Easily. The 16-bit DOS SpinRite executable, and the DOS OS itself, will both easily fit into a single diskette with lots of room to spare. The SpinRite executable will be somewhat larger, but probably not much.

For SpinRite v7, I don't yet have any idea how large the 32-bit OS and 32-bit SpinRite will be.
 

LikesCookies

Brian Tillman
Sep 23, 2020
19
4
> people might choose to wait until v7.1 to upgrade from v6.1, since it won't be until v7.1 that I'll be able to get USB working natively.

I'm not being mean or insulting, but that should be able 2030, then, judging on how long it took to get from the start of 6.1 development until now. I'm not sure I'll still be alive. Oh, well, anything worthwhile is worth the wait.
 

DanR

Dan
Sep 17, 2020
238
77
I'm not being mean or insulting, but that should be able 2030
Pessimist! ;)

The real work on building SR 6.1 started about two months ago. All that came before was laying the groundwork, although unquestionably SpinRite related. And then there has been some notable non-recurring effort to decide upon and then create the development environment for subsequent SR builds.

SR 7.0 UEFI boot ought to be a fairly quick proposition, i.e. SR 6.1 with UEFI boot.

SR 7.1 USB will take longer due to the driver development required and the many variations of USB to be considered.

SR 7.2 NVMe may be a shorter effort with few variations?

Steve should be able to complete all of this by 2029! Easy! :D

Seriously, I would speculate on SR 6.1 and 7.0 being released this year. 7.1 maybe by the end of this year, but very possibly still a work in progress.

It's all just speculation. There are just too many unknowns and potential Gotchas to do otherwise.

Just my $0.02
 
Last edited:

PUAlumni

New member
Jan 4, 2021
2
0
Being a fan of this software since V5 (I know, not THAT long ago), it's somewhat crippling not being able to use SR or RS, since newer hardware is UEFI only.
 

Similar threads