I've had this lil dream for a while now, i've been doing research, and i know how to bootstrap, how to switch to protected mode(sort of) etc..
I cant understand how any small OS development gets done. where are the protocols for everything??
Say, i want to interface a HDD controller, to set up a DMA transfer from sectors 2 through 4 to the memory - HOW?
I know i could just use BIOS calls, but they only work in real mode.
I can't find any details on this stuff, not even the location of peripherals(ioports). all i know is that a parallel port is usually at 0x378(i think). maybe im just rubbish with google...

Another thing:
how do you do floating point in an x86 ASM. just to do a simple sum like 3.5*2.

This is really annoying, i want to make an OS.
can someone help? maybe a site which has all of the details of the IBM PC and internal peripherals?

There are not any existing software protocols for yet to be designed operating systems. When you design an operating system you design the software protocols. The first thing that you need is a cross compiler. The cross compiler works on an existing hardware/os combination and produces machine code for the machine that you are creating an operating system for.

You have to get the hardware engineering specifications and design into the I/O supervisor of your new OS how to handle HDD controllers.

You get that from the CPU specs of how each instruction works. In the case of x86 Intel also provides an assembly manual which provides the same information in a form that is probably more readable to a programmer. Although that is not always the case. On the old IBM 360 series I found the hardware specs manual easier to read than the assembly manual.

----------------
Steve Stites

This question comes across as: "I want to write a new operating system in 1 month, I have no idea where to start, and I want some neat reference book that gives me step-by-step instructions."

First, every bit of hardware that you will be concerned with has a spec somewhere (even if it was reverse-engineered by someone.) You simply have to find the things you need one step at a time.

Second, I would think that studying the source code for something like Linux would be a good place to start. There are also textbooks on the design of operating systems.

First thing when you're dealing with hardware at the lowest system level is to find documentation of the pieces of hardware you have: processor, memory controller, all peripherials. Everything should be written in the docs (warning: sometimes the details are not disclosed, in such case I'd just search for different hardware, if that's just a hobby).

What's wrong with Linux? I mean, the Linux kernel is the result of about 15 years development with hundreds of people, building on something which was designed in the late 1960-ies. You could say it is pretty mature, isn't it? Why would you try and re-invent the weel?

Or is it super-underpowered hardware you have to write your OS for, like a Motorola 6809 or an Intel 8051, try OS-9.

Really, all modern MT OS-es share the same philosophy and are roughly designed like Unix. No, I didn't mention DOS or Windows, those are not real OS-es.

Try also the book Fundamentals of Operating Systems by Andrew M. Lister which describes how an OS works, before even getting into one line of code.

jlinkels

Andrew Tanenbaum's book Operating System Design and Implementations isn't a bad book for getting started writing an OS. Tanenbaum is the original author of Minix.

I would think there are still HUNDREDS of resources on developing OSes on the net. Your biggest help will come from the alt.os.assembly and alt.os.development groups on Usenet (google groups), their archives will have EVERYTHING you need.

Because he wants to. How ironic! It's the very thought that created Linux in the first place, and probably what people told linus torvalds when he had it.
But I do wish that what we know of today as "free", "hackerly" and "open source" wasn't a reimplementation, and one of a corporate product, that too 30 years old.

A book called "the undocumented PC" could help you lots if you can find an old one somewhere.

You do floating point by using fpu instructions. You'll need an opcode list or a reference manual. http://ref.x86asm.net/index.html

Books on operating systems usually talk about things at a higher level - like memory models, rather than the specifics of, say, switching from real to virtual mode on an x86 and how to use the memory page table, although specifics are often given to illustrate a case.

For everything else, it gets to be something of a nightmare. You need to recognize all the crucial controller chips and initialize them - for example, the IDE controller (so you can get your HD), the PCI controller so you can at least detect devices on the PCI bus, and you have to at least put the video chip into a legacy compatibility mode (usually VESA). The last time I wrote a driver for a PCI controller it took me an entire week to read the datasheets, write, test, and debug the driver.

Once you have some understanding of how a modern UNIX operating system works (aside from Andy Tannenbaum's book there are also other books with Linux as the example), go build and install Linux, BSD, MINIX3, and HURD. That will get you familiar with the various tricks needed to build each; hopefully you will see the differences in the kernel build parameters and you can look at the gcc source code and see what the compiler does differently for each of these systems. At that stage you will have a good idea of how to do the initialization, what order to do things, and how to finally get onto chaining to the 'init' program. Here's a question you should be able to answer before you get too far: how does the Linux kernel execute the 'init' bash script in the initrd image - how does the kernel know it's not the binary 'init' program? Now the other big thing - you need a filesystem.

The operating system provides a variety of calls which allow executable code such as the 'glibc' library to do its job of writing text to screen and reading input devices like the keyboard and mouse. You need to read the documentation for a specific compiler and its C library to work out what calls must be implemented and their order of priority. Also, you can follow links from the Open Group web site and obtain a free copy of the latest POSIX standard - that documents kernel calls which must be implemented to help maintain code portability between operating systems.

On top of all that, the operating system needs to be a little flexible and allow a user to load the driver for a particular piece of hardware. Linux does this via 'kernel modules' and a userspace tool (udev) is notified by the Hardware Abstraction Layer to create the device node entries. The external interface of a module and how a module is dynamically loaded into the kernel is a topic of its own.

Another topic to be worked out early on is the format of an executable file (the latest version of ELF is what GCC uses by default). Since most executables will not be statically linked, you also need a dynamic linker/loader.

So don't try to do everything at once - have fun - start out with printing 'hello world' and waiting there until someone presses the 'reset' button. Then switch to a different memory mode and print out register information to demonstrate that you've successfully switched modes, then begin with the initialization and print out diagnostics as you go along. You may consider creating a simple serial terminal output so that there's some hope of another machine capturing messages before things go bad.

In short: expect to do an awful lot of reading and learning to write an OS for something like an x86. You could always write an OS for a much simpler machine - say the Rabbit4000, a mutant of the Z80. It is a simple enough machine that you can write a decent OS entirely in assembler (or even byte code!) in just a few months. Tweaking a compiler to work with your OS is a different problem though ... but such a simple machine which runs the program from flash memory does have its advantages: no dynamic linker since the kernel and payload program are already in memory.

Thanks for the massive reply.

it seems the situation is as bad as i thought(no real standard - if you wanna make a successful OS you need about a million drivers to support everything).

I have messed around with boot sector programs that do stuff like HEX and BCD displays of registers.

one thing does POP(bad pun) into my head:
Why can't you simply copy the required subroutines(pre-written for that hardware) from the BIOS into somewhere, and then switch to protected mode.
All you should need to do is look up the address of the BIOS call that you want in the Interrupt Descriptor Table, and copy to main memory byte-by-byte until you hit a return from interrupt instruction.
replace the return from interrupt with a return, and you have yourself a subroutine to use in protected mode(you might need copy the entire BIOS, because of jumps in the code).

on a related note - where is the BIOS mapped to in main memory? Is the IDT pointer pointing to somewhere in there?

I'm considering doing something with a home-made computer.
I was thinking of the z180(z80180) as the CPU, and i could use some of that fancy MRAM as memory AND backing storage. The only problem is that it isn't very dense, and is expensive(1MB of it = 2x512KB chips = ~£38($76).
Also, I need to make an EEPROM programmer first.

The BIOS routines are meant to work, not to work well. Your suggestion can be made to work to some extent, but there is no telling apriori where routines are in BIOS and what other routines they refer to and what code uses flat memory and what code uses relative jumps - and this stuff changes between every BIOS and every revision. And of course, the BIOS is a secret! The firms providing the code don't want anyone doing anything with it.

Well, you can use generic low-level stuff supported by most devices. Like VESA for video (forget about acceleration), or even 13h mode, etc.

AFAIK BIOS HDD-related calls(interrupts) work nicely in Win98/Win95 - even in protected mode. But I'm not sure if interrupt handlers provided by BIOS or MS-DOS.

I suggest to start digging linux kernel and internet.
As for google read this.
Also check this google query for insights:
And if BIOS call has more than one return instruction or calls subroutines with "relative" addresses that won't work (98% possibility, I think). Also see BIOS license, copying and reverse-engineering BIOS code might be forbidden.

http://www.pcguide.com/ref/mbsys/res/addr-c.html
http://www.frontiernet.net/~fys/rombios.htm
No offense, but if you really want to write something like OS, start searching information yourself, or you won't ever finish it. Questions like "how do I multiply two numbers in ASM" are way too simple if you are attempting to create your own system.

2leedude
Generic recommendation. Start with something very simple. Forget about DMA transfers, virtual memory, network and even floating point instructions - since you can't write Linux alone, everything that is not required for making working OS should be avoided. Stick to 386 CPU instructions or lower (you may even try to develop ROM for ZX-Spectrum, this will be more realistic), and use only one video-mode. You may want to use real-mode only. Take a look at FreeDOS - making something similar to this will be much easier. Or check minix3. Or search for other attempts to write OS and study them.

P.S. I've heard about one or two people that were using some custom-written "kind-of-OS" on their systems. Information not verified, and those "OS" were probably very generic, maybe something built on top of MS-DOS, etc.

There is Free BIOS: http://www.coreboot.org/
Looking at its source might help.

None taken. but just for the record, i know how to multiply 2 numbers, i am not a complete ASM noob. I was referring to the whole SSE thing which i now understand after more reading(you could be right about the first bit). I also learned that the x86 cpus still have the x87 FPU integrated, so I dont need SIMD.

dammit, good point, i forgot about that stuff.

I can write stuff in linux. I've messed with device drivers(for my own devices)/AI stuff.
anyway, i like a challenge.

Also, about the rabbit 4000. Thanks for making me aware of this interesting piece of kit.

and the homebrew computer:
Maybe i should use an ARM...or an 8086...or a z80...or my own design with an FPGA...or even with TTL chips like that magic-1 cpu.

either way...i need an EEPROM programmer.
But my PCB software's(gEDA pcb) autoroute/auto placement feature isn't very effective. So my current EEPROM programmer layout needs like 3 layers if i leave everything to automatic routing/placement.
Maybe i should develop a better algorithm for it, like Particle Swarm Optimisation maybe.

Geez...i DO have too much free time.

ok, im gonna go now, usually when i brain dump like this, i need to think about stuff for extended periods of time.

Thanks for the wisdom, everyone. And for the personal abuse, ErV.
Lol, jkjk.

Lee