Trade: Mistakes for Knowledge

[montrom]

I've been at it for a couple years now, and along the way I hit a few snags and from them I have learned a few things. Anyone care to share some insights you've gained from making mistakes while developing your project? Should make for an interesting read. I'll get started:

Function Names: I discovered how useful it is to name your functions using this pattern: <name of file>_function_name(). It's useful because there is no question where this or that function resides. You instantly know which file to look in and that is helpful for when others are working with your code and also for when you or someone else is debugging new code. This mostly decided in the design phase.

Design Phase: I discovered that design is something that should be done on paper first. I made the newbie mistake to rush right into it, and I finally discovered that I needed to redo (almost rewrite) most of my code, because I decided on a different design midway through, and I am still fighting with all the problems these changes have produced. And, each time I veer in a new direction, change destroys pieces of old logic. The amount of time I spend fixing those mistakes, sometimes rewriting entire files of code, all this could have been avoided.

Comments: I discovered that using /* */ to comment blocks of code is much better than //, for one reason: As long as there is another block of code below the first block, having the same /* */ comment style, if you ever wanted to comment out the first block, then all you would need to do is remove the */ from the block's comment, and the first block will then be totally commented out. Speed.

Drivers and Apps: I discovered that it is stupid to mix your drivers and applications into one file. This is what you do when you're testing, make sure to strip out those codes and to put them in another file when your driver is done. You would severely limit your driver otherwise. Don't even use a printx inside the driver.

TODO LIST: I discovered that a todo list is worth its weight in gold if done properly. I keep a spiral notebook that I use to keep track of what needs to to be done and in what order. I keep notes about what bugs I have and a description of how to replicate it, etc. I have never been more organized and productive in my life. Thank you, todo list.

Testing 1, 2, 3: I discovered that every test should be done at least twice if the first test failed. Sometimes bugs work themselves out after the second try. And, if they do, then you are given a pretty good idea of what happened. Perhaps some flag wasn't set initially but is set after the first time your function is called. Now you know that you forgot to set that flag and you wouldn't have known that had you stopped testing after the first failed attempt.

I'll post back more if I think of anything else.

[NickJohnson]

Function Names: I discovered how useful it is to name your functions using this pattern: <name of file>_function_name(). It's useful because there is no question where this or that function resides. You instantly know which file to look in and that is helpful for when others are working with your code and also for when you or someone else is debugging new code. This mostly decided in the design phase.

I think that's a bit extreme. If you really need to make sure that all functions are unambiguously placed, you can kill two birds with one stone here and save some space. If every function is prefixed with the name of the subsystem it belongs to or the type of data it operates upon (e.g. memory_alloc(), syscall_exit() or thread_free()), which is good organization imho, then those functions can be grouped in files bearing the same name (e.g. memory.c, syscall.c, and thread.c respectively.)

[montrom]

Hi, that is how I once did it. But, I found that as my project became larger, that I was building code into files that could have been placed somewhere better so I would move it, but then a month or so down the road, I would forget where I placed it and if I needed to find it, I would have to go searching through many different files and folders and this was time consuming. So, doing this has helped me a lot. I suppose your way would be fine with small projects, but those that grow to something more complex would benefit from this most. And, since an OS is most likely to someday become more and more complex, it's probably best to adopt the method early on. Just my humble opinion.

[gerryg400]

1. I think the main thing I have learned over the years is - be consistent - always try to do things the same way.
e.g. All my for loops look the same
for (i=0; i<n; ++i) etc.

2. Make yourself a function template and follow it - I do something like this.
- assign a default return value
- test parameters and bail on error
- perform functions that don't have side effects, allocate memory or modify global variables
- perform functions that do have side effects, allocate memory or modify global variables
- if any errors cleanup the previous step
- return value

3. Let the compiler do the optimisation.

4. Make sure your code is testable and debuggable. You should know where you're going to put the breakpoints and how you're going to do the tests while you're designing the code.

5. Only write/test/debug code once. If you've got a working linked list 'class' (applies to c as well), never write another one.

6. assert costs nothing (if NDEBUG'd out) at runtime and provides a commentary if correctly used.

Just my thoughts....

- gerryg400

[Solar]

I second the "consistency" call, and I don't buy the /* */ over // part.

// isn't for disabling code, it's for giving comments. If you use /* */ for disabling code regions, use it for this purpose exclusively. Disabling the closing of a comment to get a code-disable is confusing at best.

If you use /* */ for commenting and want to disable code, use #if / #endif.

So much for the criticism; but I also have some advice:

Build your toolbox. Once I start typing the same compound statement two or three times, I make it an alias or a shell function and put it in my ~/.bashrc (Linux or Cygwin, doesn't matter.). Like this:

Code: Select all

# search string in headers
hdrfind ()
{
    if [ $# == 1 ]; then
        find . -type f -name \*.h -exec grep -H "$1" {} \;;
    fi
}

# search string in headers and source files
srcfind ()
{
    if [ $# == 1 ]; then
        find . -type f \( -name \*.cpp -o -name \*.h \) -exec grep -H "$1" {} \;;
    fi
}

# dump a stacktrace from a corefile
stacktrace ()
{
    if [ $# != 2 ]; then
        echo "Usage: stacktrace <executable> <coredump>";
    else
        local exe=$1;
        local core=$2;
        gdb ${exe} --core ${core} --batch --quiet -ex "thread apply all bt full" -ex "quit";
    fi
}

What is useful strongly depends on your environment, and the above code is hack-ish, not polished end-user-tool quality. It doesn't have to be. The message is: Don't waste your time typing trivial things, automate them. (And keep the code around so you can transfer it to other systems as necessary.)

[montrom]

Two more:

Driver Dev: I discovered that it was much easier to develop drivers in VGA mode. So, when I developed my GUI, I found that I needed to write a mouse driver (for example) and that doing so was difficult trying to test and debug it in the GUI. So, I stripped out my VBE code and created a complete new environment that was like my GUI environment, except it was much easier to debug code in. And any changes that I made to the kernel would not effect my real project.

JavaScript: I discovered that it was quicker and much easier to write some of my code in JavaScript first. I'm talking more about the logical parts of my drivers and design. Such as the scroll bar, or my browser window, even something math related. For example, I worked out the logic for both my browser (how it will work with icons and a mouse) and the scroll bar (how it will work when 10 icons are there and how it will work when n amount of icons are there, and what it will do when I drag the track bar or click the up or down arrow buttons, etc) using JavaScript and HTML. And, it honestly made the entire process much quicker since I didn't need to compile my kernel, assemble my loader, wait for the VM to load, wait for the VM to load my loader, wait for my loader to load my kernel, invoke the browser/scroll bar, reboot, and repeat.

[Solar]

And what should I do? I don't know JavaScript at all...

[montrom]

Hi, since JavaScript is so much like C, it can be easily converted to C and back. If you wanted to learn it or get better at it, the Internet is filled with JavaScript references, this website is the best source: http://www.w3schools.com/. And, despite it being a fast way to code something logical for your OS, it wouldn't hurt to learn it, since most of us have our own blogs and websites. And, if you know it and you're a C programmer, then why not use it to your advantage?

And, I now realize that you may have been hinting to the fact that virtually the same thing can be accomplished in Visual Studio or something, but even then JavaScript holds some clear advantages in regards to speed and ease of use. Such as:

• The setup time is like 3 seconds, you basically create a file and start typing code. You don't have to fire up the IDE, include headers, etc.
• Casting is sometimes necessary but rarely needed in JavaScript. This makes it possible to focus more on the logic rather than the C language specifics like casting.
• There are several files created using an IDE. There is only one file created when you code in JavaScript.
• If your code crashes in C, the computer may crash with it. If the code crashes in JavaScript, literally, nothing happens.

There is probably more, but I'm getting tired.

[gerryg400]

Just got a new one, having wasted an hour of my life -

7. Make sure you put a guard page at the bottom of the kernel stack of each core. (I can confirm that an array of 1000 void* 64bit pointers quickly fills up a 4k stack !!!)

- gerryg400

[xenos]

Function Names: I discovered how useful it is to name your functions using this pattern: <name of file>_function_name(). It's useful because there is no question where this or that function resides. You instantly know which file to look in and that is helpful for when others are working with your code and also for when you or someone else is debugging new code. This mostly decided in the design phase.

For those who prefer C++ (like me), I would modify this a bit and say: One class per file. If you have a class named Foo, put in in Foo.cpp. If you are then looking for Foo::Bar, you know where it is.

Some more things I found useful (and which are not purely osdev specific):

Multiple test environments: I found it quite useful to have a multitude of test environments for running my kernel. For example, I use several Bochs builds (32 / 64 bit, SMP / UP...), QEMU, AMD SimNow! (SMP / UP), different VirtualBox machines and sometimes even MS VirtualPC. If the code runs fine on most of them but fails on a single one, there must be something going wrong. This way I found quite a lot of problems in my code that were silently ignored by some of these virtual machines. (Of course, testing on real hardware is another important point, but takes more time.)

Build and run script: I made a simple script to perform all steps necessary to build the different versions of my kernel and run them using the different environments mentioned above. It calls autoconf, automake etc. if necessary, configures with the desired kernel options (i686, x86_64, with or without ACPI / SMP support...), builds the kernel and drivers, creates a boot floppy... This simplifies the build process a lot.

Syntax highlighting: It makes it a lot easier to see what some code does, or to find something in your code, if you use an editor with syntax highlighting. I wouldn't want to miss it.

[Artlav]

Userspace: It's worth writing your userspace in such a way that it can be easily runned on the host OS, where debugging is much easier.

Anticipate: If something can go wrong, it will go wrong someday. A lost pointer down in the kernel because of unexpected return can be worth years of trying to fix the floating fallout bugs in userspace. "I'll add the code to check for that highly unlikely condition later" attitude guarantees that it won't be added.

I would have to go searching through many different files and folders and this was time consuming.

Correct me if your case is different, but doesn't all that take is typing the function name into the search box and getting the result?

[montrom]

Hi, think of it like from the perspective of your mail person. Run outside and remove the numbers from your address, now your address is just whatever it would be without the numbers like (123 Street, has become just Street). If your entire community was a population of about ten, then sure it would be no problem to search for your name and then from that we could get your full address. However, now imagine the same thing in a metropolitan area, having millions of people. Sure we could still search our system for these addresses but it would just be easier and overall faster if we began including the house number with the address (i.e. 123 Street v Street). And, for that first community with only 10 people, I would imagine perhaps its wise to adopt the big city ideas now while they are still small, before to do so becomes a real chore when they get bigger.

[NickJohnson]

And, I now realize that you may have been hinting to the fact that virtually the same thing can be accomplished in Visual Studio or something, but even then JavaScript holds some clear advantages in regards to speed and ease of use. Such as:

• The setup time is like 3 seconds, you basically create a file and start typing code. You don't have to fire up the IDE, include headers, etc.
• Casting is sometimes necessary but rarely needed in JavaScript. This makes it possible to focus more on the logic rather than the C language specifics like casting.
• There are several files created using an IDE. There is only one file created when you code in JavaScript.
• If your code crashes in C, the computer may crash with it. If the code crashes in JavaScript, literally, nothing happens.

I can see a lot of advantages of JavaScript as a prototyping language, but none of those really are advantages. C setup time is about equal if you're not using an IDE and takes only one file, casting is only really necessary with low level C code (or floating point stuff), and any reasonable OS shouldn't let any code crash the computer, C or no. In contrast, JavaScript is dynamically typed, garbage collected, and has closures and objects: these all make it a good prototyping language.

[montrom]

Thanks to everyone who has contributed thus far to this thread. So far we have covered many of the basics, except:

Real v Fake Environments: Fake Environments are where you develop the driver. Real Environments are where you test them.

Backups: Make backups just as often as you make significant progress. Either place your new code online or offline somewhere. Do not replace any confirmed to be working code with unproven new code without first testing the new code on real hardware. This is especially true once you've begun developing a GUI. You'll do most of your development in an emulated environment, but all final cuts should be made in a real environment, where it will one day be forced to survive.

[Solar]

A word on the "backup".

Use version control software. Strong contenders today are git, Mercurial, and SVN. But don't be afraid to use even lowly RCS if in a pinch. (I use RCS regularily when hacking a script or a config file.) Nothing beats the ability to go back in time to check when something broke, and which change broke it.

Use a issue tracker. Mantis, Bugzilla, Trac - again the tool doesn't matter much as long as you use it.

(Of course you could use full backups of your source tree and a CHANGES.txt file instead. That would still be better than nothing, but those tools I mentioned above are there for a reason - they add huge amounts of productivity.)

Control your changesets. One changeset should address one issue only, and should be describable with a single line checkin comment. If the issue is too complicated for a one-liner, create a ticket in your issue tracker, document things there, and reference the ticket number in your checkin comment.