What tools you use?

[albeva]

Wondering what tools/enviroment you guys use to build/develope your OSs and what's good and/or bad about your current setup? I'm asking this because for the last few days I've been more messing with my tools/enviroment than actually code the kernel and wonder if others have had similar adventure with it.

As for me I use code::blocks IDE.

At the beginning I used DJGPP and batch based build (and yes from tutorial). However I quickly ran into trouble with batch build (dos command line args can't be too long, it still continues to compile if an error occured and no IDE can pick up easily the output/support batch building) So I wrote a makefile (kinda adventure on it's own for me). and configured code::blocks to compile directly using djgpp (required some hacking) BTW if anyone tries to do the same then

Code: Select all

@echo %* > c:\djgpp\bin\gcc_params.txt
@c:\djgpp\bin\gcc.exe @c:\djgpp\bin\gcc_params.txt 2>&1
@del /Q c:\djgpp\bin\gcc_params.txt

this can save you a lot of time. Just save it as gcc.bat and let the IDE call this instead

Well this setup worked for a little while but was very awkward to use - since I use vistax64 it means no DOS. So I got a VM running windows 2000. I compiled my kernel in it then copied the file to vista, made a disk image and then started VM. Very tedious.

So I tried to compile things with mingw - what a waste of time that was! After a lot of banging my head against the table and looking for help I finally settled for cygwin and compiled my own gcc (I still can't understand why cross tools can't be precompiled?). So this worked out much better.

The way I made my disk images was also rather stupid - I got a working binary with grub installed and used WinImage GUI to copy and paste a file into it Lot of pointing, clicking and dragging. And annoying popups! ...

Well I used this setup for a little while. Until I ran into trouble with placing my kernel to a high memory area. After lot of guess work and head scratching I found out that old grub I had did not support differing linear and virtual addresses in the binary.

Now of course I had to go and figure out how to create a GRUB disk. And I swear my next project will be a SIMPLE COPY PASTE GUI tool for creating a simple grub bootdisk containing a simple kernel binary. I had to boot into linux, spend hours trying to figure and work things out (I am thick headed sometimes) reading the tutorials. Finally I managed. I am rather proud of that! Here is a small script I used

Code: Select all

umount   /dev/loop0
losetup -d /dev/loop0

mkdir /mnt/myfloppy

dd if=/dev/zero of=floppy.img bs=512 count=2880
losetup /dev/loop0 floppy.img
mkdosfs /dev/loop0
mount -t msdos /dev/loop0 /mnt/myfloppy
cd /mnt/myfloppy
mkdir grub
cp /usr/lib/grub/stage[12] /mnt/myfloppy/grub
echo "(fd0) /dev/loop0" > /mnt/myfloppy/grub/device.map
grub --device-map=/mnt/myfloppy/grub/device.map /dev/loop0

echo "title   ZineOs Kernel 0.0.1"  >  /mnt/myfloppy/grub/menu.lst
echo "root    (fd0)"                >> /mnt/myfloppy/grub/menu.lst
echo "kernel  /kernel.bin"          >> /mnt/myfloppy/grub/menu.lst

cp /home/albert/ZineOs/kernel/bin/kernel.bin /mnt/myfloppy/

NOT the most elegant but it does it's job

So now I am back in windows. And dicided to tweak my build flow a bit. Here's a small batch script I use to copy compiled kernel binary to a disk image and execute the VirtualPC

Code: Select all

winimage bin\floppy.img /H /Q /I bin\kernel.bin
"virtual pc" -quiet -pc zineos -launch

Plus some extra trouble I had because of linux is 64bit and linker didn't want to link things together...

But now finally I almost got it perfect. I optimized both GRUP (boot directly -don't show the menu) and Virtual PC to minimize startup time. Installed colorgcc (I highly reccoment to anyone who hasn't got one yet!) I think I am almost content with the current setup. Although I will want better debugging options. I guess bochs is the way to go?

So what's your story? You got any cool/small/nifty scripts to ease your life?

I post here my makefile as well just in case someone finds it useful or is trying to solve similar problems I had. I think it's also suitable to compile the tutorial kernels.

Code: Select all

#
# Simple makefile to build ZineOS kernel
# @todo Add platform checking (allow compiling under linux)
#       issue an error if compiled under windows (mingw)
#       djgpp *could* also compile. IF elf support exists
#

# Is it Cygwin crosscompiling?
# else it's linux
ifeq ($(TERM),cygwin)
	GCC_PREFIX	= i586-elf-
else
	GCC_PREFIX	=
endif

# The output file
BIN			= bin/kernel.bin
#Linker
LD			= $(GCC_PREFIX)ld
LDFLAGS		= -s -T link.ld --oformat elf32-i386 -melf_i386
# NASM assembler and files
ASM			= nasm
ASMFLAGS	= -f elf32
ASMMAIN		= start.asm
ASMSRCS		= src/$(ASMMAIN) $(filter-out src/$(ASMMAIN), $(wildcard src/*.asm))
# FreeBASIC compiler and files
FB			= fbc
FBFLAGS		= -nodeflibs -w pedantic
FBSRCS		= $(wildcard src/*.bas)
# C/C++ compiler and files
GCC			= $(GCC_PREFIX)gcc
GPP			= $(GCC_PREFIX)g++
CINCS		= -I./include/libc -I./include
CFLAGS		= $(CINCS) -pipe -Os -g -m32 -Wall -nostdinc -nostdlib -fno-builtin -fleading-underscore -nostartfiles
GPPFLAGS	= $(CFLAGS) -fno-rtti -nostdinc++ -fno-nonansi-builtins -fno-exceptions
CSRCS		= $(wildcard src/*.c)
GPPSRCS		= $(wildcard src/*.cpp)
# ASM Files *MUST* be first (ASMMAIN)
OBJECTS		= $(patsubst src/%.asm,obj/%.asm.o,$(ASMSRCS))\
			  $(patsubst src/%.bas,obj/%.bas.o,$(FBSRCS)) \
			  $(patsubst src/%.c,obj/%.c.o,$(CSRCS)) \
			  $(patsubst src/%.cpp,obj/%.cpp.o,$(GPPSRCS))

# Build targets
.PHONY : all clean link

# Rule for ALL
all : $(BIN)

# Rule for CLEAN
clean :
	rm -f $(OBJECTS) $(BIN)

link:
	$(LD) $(LDFLAGS) -o $(BIN) $(OBJECTS)

# Compile C sources
obj/%.c.o : src/%.c
	$(GCC) $(CFLAGS) -c $< -o $@

# Compile C++ sources
obj/%.cpp.o : src/%.cpp
	$(GPP) $(GPPFLAGS) -c $< -o $@

# Compile FB sources
obj/%.bas.o : src/%.bas
	$(FB) $(FBFLAGS) -c $< -o $@

# Compile ASM sources
obj/%.asm.o : src/%.asm
	$(ASM) $(ASMFLAGS) $< -o $@

# link together and create the binary
$(BIN) : $(OBJECTS)
	$(LD) $(LDFLAGS) -o $(BIN) $(OBJECTS)

[quok]

I hack on my OS code from any of 3 or 4 different machines. I've got a reasonably fast desktop computer that's usually in Linux, but the wife sometimes boots to Windows (and then I'm too lazy to reboot it to linux just to code). I also have a IBM Thinkpad T42 that runs linux, and a macbook pro 17". The desktop (when in linux) and my macbook also have an OpenBSD VM on them.

In Windows, I use Cygwin & Nedit primarily. On Linux, it's Kate, on the Mac it's Smultron, and on OpenBSD it's strictly vim. I also use vi/vim on the other machines as necessary. SVN is also a must.

Since I dev from all these different environments, a cross compiler is simply a must have. I spent the time to retarget GCC and GNU Binutils properly for i?86-pc-quokos (instead of i586-elf) and x86_64-pc-quokos. In the process, I also changed some of the defaults for GCC. For instance, DWARF-2 debugging info is the default, instead of STABS (which is the default on i386 elf based systems, unless overridden like I did), and GCC outputs CFI for all instructions even on i?86, not just x86_64. For now I have compiled compiled using --without-headers as I don't yet have a decent userspace C library, but I also made sure the stack protector is enabled, and my kernel provides hooks for the stack protector functions. The kernel panics when the stack check guard value isn't what is expected. I've standardized on binutils 2.18 and gcc 4.2.4 since when I started retargetting GCC, 4.3.0 was simply too new and 4.3.1 wasn't out yet. I don't plan on updating until after 4.4 is out, even though 4.3.0 has some warnings I'd like to use and I now think that adding a decent i?86-pc-myos type target is very easy. (There's a wiki article on it at http://wiki.osdev.org/OS_Specific_Toolchain, but I have gone a bit more in depth. I plan on updating that article one of these days when I have some time to devote to it.)

Since I have gone as far as making a cross compiler for my OS dev hobby on a bunch of different machines, I now require the use of it in my Makefiles. They look for i686-pc-quokos-gcc or x86_64-pc-quokos-gcc, as well as make sure __quokos__ is defined by the compiler. If any of those don't exist, I #error out with a scathing remark.

There are a couple of other people who send me code and patches from time to time, so I spent a rather significant amount of time coming up with a very complete and strict set of coding standards, which initially were based off OpenBSD's style(9) man page, but have evolved quite a bit. There is also documentation on how to build the retargetted cross compilers, and I keep all the relevant patches in my SVN repo as well.

For debugging, I occasionally use bochs. I find that I'm much more productive with a graphical environment for debugging however, so I use Insight coupled with qemu these days. Insight is a Tk-based GUI that is fully integrated with GDB. It's available at http://sourceware.org/insight/. The website isn't updated very much, but a new insight release is always available on the ftp site within a few days of a new gdb release. My only gripe about it is that it is very slow when running on windows.

I disliked the idea of needing to mount a loopback file to create floppy images to test with, simply because it usually requires root privileges, and again the command is completely different on various platforms. In my coding style document, I have a statement something like "the entire source tree must be buildable by a non-priveleged user. If you require root for something, you're doing it wrong." So to comply with my own coding standards, and because it is much easier, I use cd images. This requires having a mkisofs command, which isn't' available on Mac OSX by default, so I just compiled it there. Now I've got the idea of not relying on external tools as much as possible, and I recently learned of the hdiutil program on OSX, so I'm going to use that on OSX instead of mkisofs in the near future.

My kernel uses the not-so-uncommon idea of an initrd, which I have chosen to implement as a tar archive. Being that there are more than a few different archive formats for tar, and not all tar programs support them all (nor use the same options to select differing formats), this proved somewhat problematic. I ended up writing my own tar program to solve this. It is rather simplistic in that it can only create an archive, only supports a USTAR formatted archive, and takes two arguments: the output file, and the name of a directory to descend and use as input for the archive. I really wish I could have just used pax, but it's not installed by default everywhere just yet.

At first I had a rather simple makefile because that's all that I needed. It was not at all unlike the one in your post, albeva. I managed to set up proper build environments by detecting what platform was being used and making adjustments accordingly. But lately my needs have changed, and actually requiring a cross compiler (and a very specific one at that) made my life a lot easier. I found make, with all of it's implicit rules to be rather slow for my needs, and I couldn't run parallel makes on my source tree to build it any faster, as my Makefile wasn't thought out well enough. I looked at switching to cmake but decided against it because, again, it's not available on most machines by default. So I spent probably 2 weeks or so learning all there is to know about GNU Make (I had previously only known enough to get by), and completely rewrote my build system. Using features of Linux's kbuild system as inspiration, I turned off all implicit rules and default variables, and started from scratch making sure that my Makefiles, and my code, could easily be built using parallel makes. I also use recursive makes. I track my own (auto-generated) dependencies (so I can print out why something got compiled in case I was wondering about it), and as part of those dependencies I track my compilation commands. If CFLAGS changes, files get recompiled. I offer "pretty printed" output from make, the actual command being run (default make behavior), or nothing at all. CFLAGS is easily changed on a per-directory or per-file basis. Once GCC 4.4 is out, I'll be able to allow that on a per function basis as well. I pass everything through GCC so I can use preprocessing statements anywhere, even in linker scripts. This really makes for a nicely integrated source tree. I also rely on incremental linking ... all objects in a directory get linked together in to a single .o file. Then that .o file gets linked together with other .o files into a single .o file in the parent directory, and so on, until there's just a handful of object files (one for each of the 'top level' directories in the src tree) that need final linking to generate the kernel image. This does slow things down a little bit but it's not much to notice at all. I plan on writing a little tutorial on writing this sort of mostly scalable recursive build system. I'll probably throw a rather permissive license on the Makefiles themselves and document the crap out of it so others can use it too. There's a bunch of other features that I haven't even mentioned yet, and won't because this post is already too long.

Lately I've been refactoring some of my code to bring it in line with new ideas and goals of mine, and also with cleaning up some things that I'd like to release seperately. My kernel module loader is a self-contained ELF loader that handles all the parsing and relocations. I'll probably contribute it to the wiki, along with my ideas on how to control which functions are made available to the loaded modules. Currently I allow any function in the kernel, but I'd like to be able to limit modules to a certain API, as well as some other things.

A very very long time ago I remember looking at OSKit, which now appears dead. I'm sometimes tempted to build my own OSKit-like set of libraries under a more permissive license than the OSKit licensed GPL, and not using the COM model that OSKit did as well. I know there are other projects out there that are like this, and I'm starting to look at them rather than duplicating work, but I may just come back to this anyway since I sometimes can have a serious case of "not invented here" syndrome. LibOSDK comes to mind, but that project's goals are quite different from mine, and I'm not a fan of the LGPL either.

[inflater]

What I use? Batch files, FASM, Notepad. 'Nuff said.

[Solar]

bash, vim, make, gcc, svn, gdb.

Pro: Doesn't require any additional setup / project files, readily available on any machine. Good exercise in basic skills.

Contra: No auto-completion, no "go to declaration".

[xyzzy]

In terms of editors, it's either GEdit or Nano on Linux, or Smultron on OS X. I've never developed on Windows. I've gone a similar way to quok with cross-compiler - I have a i686-pc-exclaim/x86_64-pc-exclaim cross-compiler and a script that will build it for you.

Currently I'm using SCons as a build system, but I may consider returning to Make sometime. I like having object files built in a different directory to the source code, and there are some obscurities with how SCons does that.

For source management, SVN. I used to use bzr but I found it to be rather annoying at times, although I do miss having a decentralized VCS...

[AndrewAPrice]

I use Cygwin/GCC, Cygwin/BASH, Visual Studio 2008, Visual Assist X, MagicISO, VirtualBox.

Visual Studio calls BASH which runs a script similar to:

Code: Select all

#!/bin/sh

PATH=/bin
cd /cygdrive/c/Users/Andrew/Desktop/perception/kernel

GPP='i586-elf-g++'
LD='i586-elf-ld'
echo loader.asm
nasm -f elf -o loader.ao loader.asm

for FILE in *.cpp
do
    echo $FILE
    $GPP -c $FILE -nostdlib -fno-rtti -fno-exceptions
done

echo linking
$LD -T link.ld -o kernel.bin loader.ao *.o libgcc.a

cp Kernel.bin ../bin/kernel/kernel.bin

If successful, VS calls a post build event which copies the binary on to the disk image (using MagicISO). The run event fires up VirtualBox

So, a typical "F5" (a.k.a. build>run) is a fully automated process of compiling any projects that have been modified, build the image, and start emulating it.

Visual Assist X has perfect autocorrection (not like the broken slow IntelliSense that generates 80mb database files). I have 2 columns open in Visual Studio. A typical layout is the source on the left, and header on the right.

Pros: Really comfortable/familiar environment (auto complete, syntax highlighting, refactoring and macros tools). One step build and run.
Cons: VS can tell when there's an error in the build process, but won't take me to the line. No debugging support of any kind.

[jakh]

vim, gcc, make, bash,

[AJ]

Hi,

I use cygwin with cross-compiled (i586-pc-elf, x86_64-pc-elf) G++/Binutils for my builds (along with NASM for my assembling). Thanks to many on this forum (and particularly Solar's Makefile tutorial on the wiki), the build process is pretty much automated whatever I do to my source / source tree.

I edit with Eclipse 2008 release (Ganymede) as my IDE which is build-configurable, but I still prefer to have the Cygwin Bash window open and manually type my build commands. For floppy virtualisation I use VFD for Windows and use a combination of Bochs, VPC2004, VPC2007, VirtualBox and real hardware for testing. I develop on an XP and Win2k machine but both are set up identically from the build point of view (and I sync with SVN).

Cheers,
Adam

[Brynet-Inc]

A wrench and a hammer.

[souradipm]

Off-topic: Lol to Brynet-Inc

On-topic: I use DJGPP to compile the kernel, via a batch file. I use Notepad++ as my editor, and use Bochs and VirtualBox to test my os, occasionally on real hardware.

[Alboin]

gcc, vim (Scribes if I'm in the mood.), gdb, make, valgrind, and, if I'm using some version control system, git. For my OS, I use a hacked together make-like system that I made. I basically keep changing it to what my laziness prescribes.

[Telgin]

The actual coding environment for me mostly consists of Notepad. I haven't made it past ASM code so far, and I don't need highlighting or anything for ASM. I'll probably stay with it even for the C++ code later though, because I can't get used to the Linux text editors, and if I use Visual Studio I'd be too tempted to try and compile it within VS, and that's not gonna work... Maybe Notepad++ or whatever it's called would be a better choice.

Anyway, I also use Cygwin with Bash as the pseudo Linux environment so I can use the GNU compiling tools. NASM for assembling, and crosscompiled G++ when I get to that part. I use shell scripts in Bash to compile, link, and insert files into a HDD image I use in Bochs. The image was created and is updated using a tool I created in Visual Studio (C++) to work with images of my filesystem. More elegant than blindly copying stuff to a floppy and trying to boot off of it.

I kind of wish I could just do all of this easily within Windows though. Bash doesn't get along with me at all, and I'm sick of having permission problems running scripts that I can't figure out how to solve. Compiling a version of Bochs to support SMP and 64-bit was horrifically difficult... I couldn't get the Linux source to compile to save my life because of those script errors. The Windows version wasn't much better though, it wouldn't compile without modification, which I find very strange. Oh well, maybe I didn't break it in the process.

[elderK]

bash, binutils, cvs, gcc-core, make, ratpoison(as a wm and to stop vermin attacking both my coffee and cigarettes...) and anything vi-like (elvis, vi, vim, nvi, cream) that is installed.

all imaging programs I use are my own.

~K / zeii

[Troy Martin]

Wordpad and NASM.

No, seriously. When you're writing in 16-bit (hell, even 32-bit) assembly, minimalisticness sticks to you.

[PatrickV]

Nasm and Visual Studio.NET for coding my asmebly language