Something fun about operating systems

  • SpinRite v6.1 Release #3
    Guest:
    The 3rd release of SpinRite v6.1 is published and may be obtained by all SpinRite v6.0 owners at the SpinRite v6.1 Pre-Release page. (SpinRite will shortly be officially updated to v6.1 so this page will be renamed.) The primary new feature, and the reason for this release, was the discovery of memory problems in some systems that were affecting SpinRite's operation. So SpinRite now incorporates a built-in test of the system's memory. For the full story, please see this page in the "Pre-Release Announcements & Feedback" forum.
    /Steve.
  • Be sure to checkout “Tips & Tricks”
    Dear Guest Visitor → Once you register and log-in please checkout the “Tips & Tricks” page for some very handy tips!

    /Steve.
  • BootAble – FreeDOS boot testing freeware

    To obtain direct, low-level access to a system's mass storage drives, SpinRite runs under a GRC-customized version of FreeDOS which has been modified to add compatibility with all file systems. In order to run SpinRite it must first be possible to boot FreeDOS.

    GRC's “BootAble” freeware allows anyone to easily create BIOS-bootable media in order to workout and confirm the details of getting a machine to boot FreeDOS through a BIOS. Once the means of doing that has been determined, the media created by SpinRite can be booted and run in the same way.

    The participants here, who have taken the time to share their knowledge and experience, their successes and some frustrations with booting their computers into FreeDOS, have created a valuable knowledgebase which will benefit everyone who follows.

    You may click on the image to the right to obtain your own copy of BootAble. Then use the knowledge and experience documented here to boot your computer(s) into FreeDOS. And please do not hesitate to ask questions – nowhere else can better answers be found.

    (You may permanently close this reminder with the 'X' in the upper right.)

coffeeprogrammer

Well-known member
Jul 19, 2021
169
14
So I just wanted to share something that has made me quite happy lately. I’ve been wanting to learn more about operating system. I typically try to break things down to there simplest concepts. I don’t have the ability to spend lots and lots of time on this research, but I do have weekends and some times after work. The most beneficial resources so far has been a Udemy course I have purchased called “Developing a Multithreaded Kernel From Scratch”. Much of what I have learned is still a bit jumbled up in my mind. I am going to have to use additional resources to gain more insight to what I have started to learn from the course.

Some of what I have learned includes the interrupt vector table for 16bit code. So what I think this is, is a table that is pointers to code that runs when a particular interrupt happens or is received on the PIC/CPU. From what I have learned it is located at the beginning of memory and holds 256 interrupt locations at 4 bytes each. Making it use the memory from 0000:000H, 400H or 1024 bytes. The locations or offset have been predetermined for particular uses. Interrupt zero is the divide by 0 interrupt, meaning that if code divides by zero then the code that is pointed to by the interrupt vector table for interrupt zero will run when that divide by zero occurs. From what I can tell there are CPU interrupts, basically exceptions, and hardware interrupts, which handle events from hardware like keyboards, COM ports, hard drives and mice. On x86 system there are two 8259 PICs, a master and slave. I don’t understand how the PIC’s interrupt vector table offsets overlap with the CPU interrupts, but they do, somehow. The lidt is the x86 instruction that lets the program load an interrupt descriptor table to (not sure about this) a special interrupt descriptor register IDTR. Again is all the 16-bit code.

What I have learned from 32-bit code, protected mode, is that there is a similar structure the IVT called the interrupt descriptor table. I’ve been thinking one thing that it might help with managing multiple processes and the switching between them, but I am not sure if that is correct or how it would. It serves the same purpose as the IVT, expect in 32 bit protected mode. The code that the IDT points to is called an interrupt service routine and these ISR must be an important part of device drivers for OSs like Windows and Linux and bare metal type things like spin rite.

In the Udemy course the author is using QEMU to run the os and uses GDB to debug it. I don’t know much about QEMU and this is the first time I have used GDB. But it seems to me that a person would not need much memory to do the kind of research in QEMU and there must be a way to do a memory dump and see the IVT or the IDT with any interrupt service routines. I am not sure if the author takes this direction in the course to have the programmer see their code in the debugger. I think if you used symbols you could see C code when debugging live. The course does remind me of the work, or least knowledge Steve has and uses in the development of spin rite.

I guess lastly I just wanted to comment on why, because this stuff is cool, at some point (not sure how soon), I want to able to identify the code in Linux and ReactOS that sets up the IDT. This is among other areas I want to be able to identify in those kernels/Oss code bases. Anyway hopeful someone will at least be happy with the link I have provided, it really is a great course. Feedback/comments/additional insights very welcome. Thanks. :)







 
I guess lastly I just wanted to comment on why, because this stuff is cool, at some point (not sure how soon), I want to able to identify the code in Linux and ReactOS that sets up the IDT.
Agreed. I had a quick look at ReactOS. I think you'll find what you are looking for in kinit.c.
 
Agreed. I had a quick look at ReactOS. I think you'll find what you are looking for in kinit.c.
Yes I should have known that. The kiinit.c also is where the boot loader gives control/loads the kernel.

KiSystemStartup is the function. I wanted to debug from this function onwards but I could not because the ReactOS kernel has not yet setup debugging at when KiSystemStartup runs. However that said I think I figured out something. If I use QEMU to run reactos rather than vmware I can halt the CPU (I think) right after post and right when the boot loader is starting. If I can do that, then I can debug KiSystemStartup or anything else in the startup process. I’ve got to have symbols of course because assembly really would not be any use. I’ve been trying to do just this for about two hours, I got QEMU installed for Windows and I can run the ReactOS live cd no problem, but when I try and build with symbols using the

configure.cmd -D_WINKD_:BOOL=TRUE -DGDB:BOOL=TRUE -DSEPARATE_DBG:BOOL=TRUE -DKDBG:BOOL=FALSE


like on the ReactOS wiki, it will not build so I am stuck without symbol files. This has been for using GDB as the debugger. I did a little bit of googling and I think you can connect WinDBG to QEMU. I did have a build a few months ago for ReactOS with PDB symbols for WinDBG and I could debug things, but after KiSystemStartup. Hopefully halting the machine early will give me the early boot. Wish me luck.
 
Super cool update, I did after several hours of trying on windows as my host system, then a ubuntu 22.04 vm , I had to download ubuntu 20.04 (RosBE support) and I finally can debug ReactOS right from the boot loader. I had to load the symbols manually and I know very little about GDB commands, keep that good luck coming.
 

Attachments

  • free_loader_debug1.PNG
    free_loader_debug1.PNG
    115.8 KB · Views: 147
Made a youtube video of debugging reactos under linux. I wanted to use Windows but I had a few problems. I think this could help anyone wanting to debug reactos. I start at freeloader up until KiSystemStartup

 
  • Like
Reactions: fcgreg
Well another update, I don’t know if anyone out there has ever been interested in operating systems like I have, but I was thinking of a way to debug ReactOS in an IDE rather than doing everything via command line, I am somewhat familiar with JetBrains software, so I gave Clion a try. Clion does not have any problems connecting to a QEMU base remote GDB debugger and then setting the break points like you would any other ide. I still am loading the symbols manually and I have not figured out a way to load them automatically, but it works well. I can’t think of any reason this would not be possible with the Linux Kernel as well, in fact, that is what the Jetbrain’s documentation demonstrates. So I think it could be an effective way to learn more about OS internals.

As I don’t know much about how the processor does things with the GDT and all those related things, I am trying to read the Intel Software Developers Manual, volume 3 “System Programming Guide”. I really wish there was a good video course I could buy because I don’t have unlimited energy to read all this. If any one knows of any good youtube or video courses that would cover the same thing as the ISDMv3, please let me know. Thanks.
 
I found a few courses on youtube that are just what I was looking for. I am linking to the channel that is crowd funded, after watching a few I think I will give them some money. If you like security now then you will like these, diagrams right from the intel software developers manual. Maybe Steve can mention it on the podcast and give their funding a boost. Glad I found these, much easier than taking to time to read it all.