CompilerDev?

[kutkloon7]

[...]

I am quite sure that many OS developers are also interested in compilers

I actually have a sort of compiler in my OS source tree that gets built and then builds parts of my OS (same with a runtime linker, of which I have two now)

Cool! Is there a specific reason why you do it this way? Smaller build time for the kernel itself?

[Combuster]

Cool! Is there a specific reason why you do it this way? Smaller build time for the kernel itself?

The "kernel" is only worth one assembler command. The tool turns data definitions into code that reads and writes the data, as well as header files for various languages and implementation styles, so that there's only one place where the OS's primitives are used directly.

[kutkloon7]

Cool! Is there a specific reason why you do it this way? Smaller build time for the kernel itself?

The "kernel" is only worth one assembler command. The tool turns data definitions into code that reads and writes the data, as well as header files for various languages and implementation styles, so that there's only one place where the OS's primitives are used directly.

I couldn't think of a good reason bu that makes a lot of sense. Cool approach

[Muazzam]

You're Right there is no (Standard) Websites for Compilers Development. I think Operating system development is much easier than implement a language translator. In my opinion Compiler development is "Science" while Operating system development is "Engineering". Compiler development requires knowledge of Discrete mathematics, Automata theory, Logic etc. I am interested in writing an Assembler not high-level language compiler. Because there is no need of higher level languages (for me).

[AndrewAPrice]

I have the basics of a compiler down (lexers, grammars, abstract syntax trees, linking, simple stop-the-world garbage collector, SSA) - I took a course at university and figured the rest out online. But, information online is very scarce. There are research white papers, people's blogs, slideshow notes from college courses online - but they are usually really indepth and upclose - and you wish someone could stand back and give you the overall picture of how it fits in, or they're talking high level (how do I implement this!) and you understand the theory, but you really wish someone would kind of hint (not do it for you, but point you in the right direction to get started) at how you would implement this algorithm in a low level language.

Everyone seems to be pointing to the dragon book as the go-to reference guide, so I ordered myself a copy to see what the fuss is about.

[no92]

Compiler development requires knowledge of Discrete mathematics, Automata theory, Logic etc.

Sorry for contradicting, but that's not what I think. It took me about 1h of brainless coding to get a working "Hello World" compiler, without reading anything about the topic. I'm not rich enough to be able to pay 260€ for the dragon book (that's good as I want to start from scratch without being influenced by other designs).

At some point, somebody who has some knowledge about this topic should create a wiki page explaining where and how to start, as (appearently) quite a few in the OSDev forums want to start CompilerDeving too.

[kutkloon7]

Compiler development requires knowledge of Discrete mathematics, Automata theory, Logic etc.

Sorry for contradicting, but that's not what I think. It took me about 1h of brainless coding to get a working "Hello World" compiler, without reading anything about the topic. I'm not rich enough to be able to pay 260€ for the dragon book (that's good as I want to start from scratch without being influenced by other designs).

At some point, somebody who has some knowledge about this topic should create a wiki page explaining where and how to start, as (appearently) quite a few in the OSDev forums want to start CompilerDeving too.

Depends, writing a predictive recursive-descent parser is pretty simple. When you go into some more advanced topics (I think especially optimization) you're going to need some theory, I guess (I haven't really coded a compiler, but that's what I read ).

[AndrewAPrice]

There is this wiki page.

[no92]

I created a tutorial on the wiki (calm down, it's a user page), waiting for reviews.

[Combuster]

Code: Select all

$code = array(
	new Token("f", array("hello_world")),
	new Token("{", array()),
	new Token("printf", array("%s %s!\\n", "Hello", "World")),
	new Token("}", array()),
	new Token("printf", array("Test\\n")),
	new Token("hello_world", array())
);

This is really a kludgy and non-extensible way to start with things. It works for very limited cases, but there's a reason why people use abstract syntax trees, because you want to store logical blocks, rather than a mixture of random typeless symbols, which will make you put all sorts of unrelated code into blocks where they're not relevant (like the if (curly_brace) ignore(); style of code you already have).

How you actually want to write that is more like the following:

Code: Select all

$code = array(
  new Function("hello_world", array(/*no arguments*/), array(
    new CallExpression("printf", array(
      new StringExpression("%s %s!\n"),
      new StringExpression("Hello"),
      new StringExpression("World")
    ))
  ),
  new CallExpression("printf", array(new StringExpression("Test\n"))),
  new CallExpression("hello_world", array())
)

From there, it's much easier to add additional language features. Adding integers to this structure is simple, as is 1 + 1, as well as nesting functions in arguments such as printf("%i", getch()); Try to find an obvious way to do that with your original structure

[no92]

Thanks for the reply.

I'll probably reimplement it the way you proposed it, thanks alot . Any other suggestions/improvements (MessiahAndrw, I'm talking to you!)? It's still a WIP, so feel free to directly edit it, if you're sure your code/explanation/etc. is better.

[kutkloon7]

I have the basics of a compiler down (lexers, grammars, abstract syntax trees, linking, simple stop-the-world garbage collector, SSA) - I took a course at university and figured the rest out online. But, information online is very scarce. There are research white papers, people's blogs, slideshow notes from college courses online - but they are usually really indepth and upclose - and you wish someone could stand back and give you the overall picture of how it fits in, or they're talking high level (how do I implement this!) and you understand the theory, but you really wish someone would kind of hint (not do it for you, but point you in the right direction to get started) at how you would implement this algorithm in a low level language.

Everyone seems to be pointing to the dragon book as the go-to reference guide, so I ordered myself a copy to see what the fuss is about.

That is so true! For the dragon book, I think you can see it mainly as a reference. It is probably one of the more comprehensive works on compiler development. Don't expect too much hand-holding or implementation details. There are some sections devoted to 'the bigger picture', but the main part of the book consists of a systematic treatment of the available techniques and algorithms.

By the way, no92 and Combuster, really nice to see that people finally are creating something! It's probably not perfect on the first try, but actually starting to write something is sometimes the biggest problem. I'm willing to contribute, but I think I should get some more experience with building compilers first...

[AndrewAPrice]

I'll probably reimplement it the way you proposed it, thanks alot . Any other suggestions/improvements (MessiahAndrw, I'm talking to you!)? It's still a WIP, so feel free to directly edit it, if you're sure your code/explanation/etc. is better.

It's looking good. I've thought about writing a section on language design, since I designed a compiler for my own language.

[no92]

By the way, no92 and Combuster, really nice to see that people finally are creating something! It's probably not perfect on the first try, but actually starting to write something is sometimes the biggest problem. I'm willing to contribute, but I think I should get some more experience with building compilers first...

Heh, the first time I did write a compiler was yesterday ... nothing needed except for motivation and some basic theory

EDIT: I updated the tutorial to include Combuster's suggestions

[kutkloon7]

By the way, no92 and Combuster, really nice to see that people finally are creating something! It's probably not perfect on the first try, but actually starting to write something is sometimes the biggest problem. I'm willing to contribute, but I think I should get some more experience with building compilers first...

Heh, the first time I did write a compiler was yesterday ... nothing needed except for motivation and some basic theory

Yeah, I did dive in the theory some time ago, but it just got me stuck with more questions.
What language am I going to parse? An existing one (or a subset or an extension based on an existing one) or a new one (what does the new language look like? Should I write a formal EBNF scheme first)? Do I want to write a recursive-descent parser or a table-driven one? What do I want to parse to? Machinecode, ASM, bytecode, intermediate code (what format? SSA, triple)...