Not a bug Future version feature questions

[Societus]

Good evening! I've been a fan of SpinRite for years, having been a user at my previous job I was looking at getting into it for myself after a while not doing much technical work.
I am in the process of setting up a SAN for my current office, which juggles many plates, but is large enough to justify having its own SAN. As such, I have some questions that may not come up too often, but feel would be pretty useful to add to the roadmap at some point.

1) Are there any plans to add SAS HBA support? The last time I tried to access disks on one, they didn't detect any drives so just assumed it was not doable at the time.
2) Connected to the first question, is it feasible to pass a SAS HBA (presuming SAS support gets added) on a hypervisor to a VM to run SpinRite without tying up a whole machine? I have 2 chains of disk shelves that are a big mirror, and being able to offline one chain to run a semiannual level4 of 64 drives, or have a really convenient way to thoroughly bulk-test new drives for consistency while keeping the rest of my system working on other things?

Keep up the great work, looking forward to 7.0!

 

[miquelfire]

The way I have seen SANs work, you don't actually have direct access to the drives. My understanding of them is that the SANs themselves may not allow you to run software that has direct access to the HBAs inside. The only access you have at best is the same as a hardware raid controller that exposes virtual drives (after whatever RAID level you used to create them), and at worst, another layer that is something like the virtual drives you can create in VM software (you know those VDI/VMDK/etc files that emulate a drive).

I think the best case is to remove a single drive, test it, and return it so the SAN can rebuild the RAID. If you're using hot spares, it can rebuild while the test is happening, and if the speed is the same, by the time you put the disk back in, you can take out another disk and continue on.

Mind you, I have only used SANs that offer iSCSI access, and none of them had video output.

 

[Societus]

The way I have seen SANs work, you don't actually have direct access to the drives. My understanding of them is that the SANs themselves may not allow you to run software that has direct access to the HBAs inside. The only access you have at best is the same as a hardware raid controller that exposes virtual drives (after whatever RAID level you used to create them), and at worst, another layer that is something like the virtual drives you can create in VM software (you know those VDI/VMDK/etc files that emulate a drive).

I think the best case is to remove a single drive, test it, and return it so the SAN can rebuild the RAID. If you're using hot spares, it can rebuild while the test is happening, and if the speed is the same, by the time you put the disk back in, you can take out another disk and continue on.

Mind you, I have only used SANs that offer iSCSI access, and none of them had video output.

In the traditional sense you would be right. If I were running a consumer/commercial SAN build I would have little to no control over the things that would provide direct disk access. What I am doing is hosting a SAN using conventional server hardware and HBAs in order to manage my own iSCSI, NFS, CIFS, and Ceph targets/outputs. So in a way it is a really, really big NAS, but it has no global network access, only p2p direct connections to the initiator machines that would be using it. The simplest example of this is my rear office machine. It's 2 mirrored servers with 256GB each of RAM, whose primary job is to drive the 3 disk shelves connected to them that hosts content for my managed backup and forensic recovery service. A third server is my hypervisor and connects to the storage boxes, and runs VDIs for the rest of the building running thin clients.My thought was having an HBA on that hypervisor that can be pased as a PCI device and connected to a specific JBOD solely for the occasional run of SpinRite. Nearly every drive I use outside of my personal desktop is SAS now, so have little use for the software anymore, but would like to be able to, since it was a great early warning system and occasional savior.

 

[miquelfire]

Sounds like you're using TrueNAS or something like it. If that's the case, I don't see why what you're suggesting wouldn't work. Even though it's SAS, you might be able to use 6.1 anyway. And if not, the BIOS on the VM software you want to use might be good enough (even if slow). I don't know if AHCI works on SAS drives. AHCI might only work on SATA drives because it's part of the standard controlling IDE and SATA. AHCI might be generic enough that it works with SCSI drives as well.

 

[Societus]

Sounds like you're using TrueNAS or something like it. If that's the case, I don't see why what you're suggesting wouldn't work. Even though it's SAS, you might be able to use 6.1 anyway. And if not, the BIOS on the VM software you want to use might be good enough (even if slow). I don't know if AHCI works on SAS drives. AHCI might only work on SATA drives because it's part of the standard controlling IDE and SATA. AHCI might be generic enough that it works with SCSI drives as well.

AHCI can't be set on SAS devices, which is why I was hoping that support for SAS drives would be added someday to SR. It would also be a little different if I tried to input each raw block device as an AHCI drive, because the paravirtualization layer would likely alter the effectiveness or even make the use of SpinRite dangerous. At least that is how I understand the benefit of SpinRite being so low level.

 

[Scott]

AHCI can't be set on SAS devices, which is why I was hoping that support for SAS drives would be added someday to SR. It would also be a little different if I tried to input each raw block device as an AHCI drive, because the paravirtualization layer would likely alter the effectiveness or even make the use of SpinRite dangerous. At least that is how I understand the benefit of SpinRite being so low level.

I suggested SAS / HBA support some time ago:

https://forums.grc.com/threads/spinrites-development-roadmap.32/post-3523

Would require development of a driver that is definitely not in SR 6.1, and I don’t think exists in RTOS, but similarity to other SCSI type drivers might make development easier. Maybe @Steve will do it for SR 7 if he thinks there’s a corporate market that would pay for the feature.

 

[Kyra@TWiT]

I would really like to see some HBA support, and also throw my hat in for NVMe support sooner rather than later.

LSI Logic compatible PCIe HBA cards are a cheap way to expand the number of SATA drives on a standard PC. No need to use them as RAID controllers, each drive can be seen by the BIOS or UEFI individually and indistinguishable from a drive plugged in straight to the motherboard. This is how I use HBA so I can have more drives on my server. RAID-like functions are done at the OS and file system level with much more flexibility than doing it in hardware.
I bought SpinRite 6.1 to run checks on my server drives. The server sat unused for a year since a storm brought down part of the roof on the it and water poured everywhere.

Though the BIOS could see the SATA drives just the same as if they were plugged into the motherboard, SpinRite could only reach them in "BIOS" mode, which means very very slow.

I had no choice but to run SpinRite on a different PC with the cables dangling out so I can test drives one by one. It sees the drive as AHCI but it still seems slower than the drives have ever been.

NVMe drives are everywhere now. I don't think anyone who knows enough to use SpinRite would not use an NVMe drive on their next computer if they're not already.

Sadly I came across this document which made for some grim reading: https://www.grc.com/miscfiles/GRC-Development-Roadmap.pdf
It's dated January 17, 2021, and says the next step would be SpinRite 6.1 which was released not that long ago. If the document is current, NVMe support won't come until version 7.2, and HBA or SAS support are not in the roadmap at all.

It took 16 years between 6.0 and 6.1. I hope that's not a predictor of the time it'll take till 7.0, and on to 7.2. Many of us will probably go through several hardware iterations by then.

I don't know if @Steve has changed plans, but I strongly believe NVMe support should have priority over USB since NVMe is the primary drive in many computers/laptops.

In the meantime, and this might help the OP: I plan to hedge my bets, use SpinRite on some of the drives and rely on the server's redundancy to protect me in case a non checked drive fails.

 

[DanR]

Sadly I came across this document which made for some grim reading: https://www.grc.com/miscfiles/GRC-Development-Roadmap.pdf
It's dated January 17, 2021, and says the next step would be SpinRite 6.1 which was released not that long ago. If the document is current, NVMe support won't come until version 7.2, and HBA or SAS support are not in the roadmap at all.

Kyra@TWIT,

The SpinRite Roadmap for SR 7.x development is indeed out of date. @Steve's current anticipation is that SpinRite 7.0 will now include the following:
- UEFI/BIOS dual boot capability, for booting and running on both newer UEFI and older legacy BIOS systems
- Full native driver support for external USB drives
- Full native driver support for NVMe drives

In late 2022 Steve acquired a software package called RTOS-32, which will provide a pure 32 bit environment for SpinRite 7.x development. He was able to quickly demo the feasibility of UEFI/BIOS dual booting. He also noted that RTOS-32 comes with plain vanilla drivers for USB and NVMe, which he feels he can easily customize for SpinRite's needs. This, plus the limited and often problematic USB support in SpinRite 6.x, led to Steve's announced decision to combine the SR 7.0-7.1-7.2 items in the current SR Roadmap into a single SR 7.0 release.

When will this SR 7.0 development start? Unknown. I am hoping by mid 2025. That remains to be seen.

How long will SR 7.0 development take? A BIG unknown. I am thinking 2-3 years. Again, that remains to be seen.

Meanwhile . . .

One of the many items on Steve's currently full plate of side projects is a quick limited capability SpinRite 7ish Windows app. Per Steve's comments (and my quite possibly wrong perceptions ) it could include the following:

- Run as a background app
- Mostly (if not solely) for scanning external USB drives
- With native USB driver support
- Would take exclusive control of the USB drive being scanned (as ValiDrive v1 does now)

No idea when this might occur.

I cannot speak to HBA, SAS support.

 

[Bplayer]

Steve dropped this bombshell (to me) in the grc.thinktank newsgroup a few days ago:

Thanks. We'll be back to SR7 sooner than originally expected since I'm pretty much 100% certain that the next SpinRite will be a full native Windows app. Then, once we have that, I'll port that over to the Win32 compatible RTOS-32 for the "Pro" version that's bootable.

 

[Kyra@TWiT]

@DanR When I first saw incoming mail from GRC I thought Steve had sent this week's Security Now's email super early. It was not the case, but your post filled me with hope, but wait, there's more!

Steve dropped this bombshell (to me) in the grc.thinktank newsgroup a few days ago:

Thanks. We'll be back to SR7 sooner than originally expected since I'm pretty much 100% certain that the next SpinRite will be a full native Windows app. Then, once we have that, I'll port that over to the Win32 compatible RTOS-32 for the "Pro" version that's bootable.

Holy <bleep>! That's <bleep>ing amazing! Someone needs to erect a statue of @Steve RIGHT NOW! *Jumps with joy*

 

[DanR]

Steve dropped this bombshell (to me) in the grc.thinktank newsgroup a few days ago:

Thanks. We'll be back to SR7 sooner than originally expected since I'm pretty much 100% certain that the next SpinRite will be a full native Windows app. Then, once we have that, I'll port that over to the Win32 compatible RTOS-32 for the "Pro" version that's bootable.

Right!!! That is what I had in mind when I wrote the second part of my previous post.

However: It is one thing for SpinRite to take complete control of and then scan an external USB drive as a Windows app.

But: It is an entirely different thing for SpinRite to try to scan a drive that is currently in use by Windows and/or other Windows apps. That situation would create lots of conflicts and restrictions for SpinRite. I have no idea how Steve might be addressing this.

 

[Scott]

Right!!! That is what I had in mind when I wrote the second part of my previous post.

However: It is one thing for SpinRite to take complete control of and then scan an external USB drive as a Windows app.

But: It is an entirely different thing for SpinRite to try to scan a drive that is currently in use by Windows and/or other Windows apps. That situation would create lots of conflicts and restrictions for SpinRite. I have no idea how Steve might be addressing this.

This would only be useful to work on drives that can be taken offline, so that excludes the boot drive and internal drives that are in use such as those with swap files on them.

One could create a “Windows to Go” USB boot drive and run windows SR 7 (“WinRite”??) and address the internal drives on the PC