Why is free(ptr) preferable to free(ptr, length)?

[vvaltchev]

@vvaltchev: I'm thankful for your detailed explanation too. I'm thinking of a single-language OS. If this language is strongly typed, your point #1 would apply to 100% of allocations. I'm also happy to hear your buddy allocator wasted surprisingly little memory even before usage optimization.

You're welcome
It would be great if #1 could apply to 100% of the cases. But sometimes, enforcing that is "impossible" or inconvenient.
A few examples.

1. How to allocate strings? You can always allocate their maximum size but, are you OK with that?

2. Suppose you're implementing a container (e.g. ring buffer). Container's code is used by a variety of callers, in different subsystems, and the buffer size is a parameter. How can it become known at compile-time? Unless you wanna do the things C++ style, by having everything in headers (e.g. templates), the buffer size cannot be known at compile-time. Note: C++ templates are VERY bad, especially in kernels because they duplicate the code. "Compile-time" is often good, but sometimes terrible. It's a waste to duplicate your rb_write() for example, for every different buffer size. And it's not only about code size, it's about performance as well. The bigger the text section is, the less instruction cache locality there will be. That means more icache misses, and slower code execution.

3. Suppose you have to allocate a buffer which size is decided by the user space. It's impossible to have a compile-time value. OK, it's not impossible, but you'd have to allocate page by page. In that case, you won't get a contiguous chunk, though. Therefore, you'll have also to map all those pages somewhere in the virtual memory, in order to have your contiguous buffer. That's pretty slow.

Anyway, what do you mean with a "single-language OS"? I'm curious.

[eekee]

After I posted, I realized I hadn't thought it through. The sort of language I was thinking of would keep size data with each array, so from the user programmer's perspective, it would be equivalent to free(ptr). I guess that answers all of the points. (I hate C++ templates too.)

A single-language OS as I'm thinking of it has a single language for almost everything from kernel and driver development to the user command shell and scripting. This is quite possible with Forth. I'm intending to implement a Forth-like language with high-level data types. Incidentally, older and simpler Forths effectively require the length to free memory, but ther memory management is extremely primitive. More sophisticated Forths have ALLOCATE and FREE which work like malloc() and free(), the latter not requiring a length.

[thewrongchristian]

(I hate C++ templates too.)

I feel I have to step in here and defend C++ templates a bit.

In the context of allocation/free, while it makes no sense to template the size of an array being allocated, it very much does make sense to template the type of the array being allocated.

In my C based code, I have a FIFO, for example, which can be sized at construction. Now, to keep the code duplication at a minimum, I only support a single word size (the native word size) for this FIFO, and while FIFOs can have variable size, they're all the same type. My keyboard driver puts scan codes into such a FIFO, which is wasteful because I end up putting byte sized scan codes into a FIFO of (currently) 32-bit words. But I don't have many scan codes backed up (the FIFO is small), there is only one of them, and don't waste that much memory.

But had I used C++ templates, I could have a single body of code to support both byte and word FIFOs, with no code duplication. OK, the compiler will expand that to duplicated object code, but it's a price I'm happy to pay for the reduced maintenance and bug burden of not having duplicate source code. I'm only likely to have byte and word based FIFOs, so it's not a big code burden.

And for the record, I would use free(ptr) rather than free(ptr, length) if I indeed used free(). But I don't, 'cos I use GC for the general heap, and a bump pointer based allocator for temporary allocations that are released en-masse.

[vvaltchev]

After I posted, I realized I hadn't thought it through. The sort of language I was thinking of would keep size data with each array, so from the user programmer's perspective, it would be equivalent to free(ptr). I guess that answers all of the points. (I hate C++ templates too.)

A single-language OS as I'm thinking of it has a single language for almost everything from kernel and driver development to the user command shell and scripting. This is quite possible with Forth. I'm intending to implement a Forth-like language with high-level data types. Incidentally, older and simpler Forths effectively require the length to free memory, but ther memory management is extremely primitive. More sophisticated Forths have ALLOCATE and FREE which work like malloc() and free(), the latter not requiring a length.

That's interesting, thanks for sharing
I'm just wondering, aren't you a little afraid of writing both an operating system and a compiler at the same time? Operating systems are already tricky per se to debug. If chances are that the bug might be in the compiler itself, it becomes a nightmare, at least for me. Unless, of course, you're planning to get your compiler very mature and tested before start writing your OS.

[vvaltchev]

I feel I have to step in here and defend C++ templates a bit.

But had I used C++ templates, I could have a single body of code to support both byte and word FIFOs, with no code duplication. OK, the compiler will expand that to duplicated object code, but it's a price I'm happy to pay for the reduced maintenance and bug burden of not having duplicate source code. I'm only likely to have byte and word based FIFOs, so it's not a big code burden.

Actually, your reason for using C++ templates is 100% valid. The problem with templates in the kernel comes when they are used everywhere, like in the userspace (and it's a problem there, too). Modern C++ is all about templates and not everything gets inlined and folded magically as C++ evangelists claim. It's very tricky and it's case by case: a tiny change can make a ton of template metaprogramming code that use to "disappear" in the binary, to be actually emitted. I could talk about that, forever.

The case you're talking instead, is totally legit. Sometimes, in C we have no other choice than copy-paste code to support different types or use a macro and then instantiate that macro for each type. That's EXACTLY what C++ templates do. If you use them in that case, you're paying zero. You explicitly want different instantiations of that code, because that's the only efficient thing to do, or because you simply have no other choice. An example like yours? My implementation of itoa(): I used a macro, then at some point I decided that using a C++ template is much better. Despite my kernel is written in C, there are a couple of C++ files like: https://github.com/vvaltchev/tilck/blob ... n/itoa.cpp

[PeterX]

Sometimes, in C we have no other choice than copy-paste code to support different types or use a macro and then instantiate that macro for each type. That's EXACTLY what C++ templates do.

Thanks for mentioning that. I was not aware of it. I hate templates, too, but this use-case makes really sense.

Greetings
Peter

[vvaltchev]

Thanks for mentioning that. I was not aware of it. I hate templates, too, but this use-case makes really sense.Peter

You're welcome. You know, with time I've learned that the good/bad dichotomy cannot be used, most of the time. Technologies are not always "good" or always "bad". There are instead a bunch of trade-offs. Even a "crappy" thing, if it's used a lot of people, must have some reasons to exist, at least one. But, that reason can be totally outside of the tech realm as well. For example, the given "crappy" technology is "good" for some companies, at some stage of their life, just because there are many devs who can use it. Or, a "crappy" technology, even if it's pretty inefficient, can allow developers to quickly write a given software and generate money. We're not discussing weather I like that or not, just that's a fact.

The same principle applies to the interface of free() [both interfaces have their reason to exist] and to C++: the world is full of crappy C++ code, but that doesn't mean the language is inherently bad, neither that is inherently "good". It can be good in certain contexts like every other technology. Of course, there's always our personal taste, but that's another story. I like C so much, but I'd never use it for writing a script. No matter my "taste", python will always be 100 times better for scripts than C

[Gigasoft]

Nothing wrong with templates, I like them. It is of course also easy for bad programmers to use templates to express their incompetence. At work, someone managed to write a data logging function template containing a good sized chunk of logic that would be expanded thousands of times, the reason being that he needed to access a couple of different static data members (a sequence number and a type ID constant) for each different kind of report, of which there were many.

When used properly, templates enhance productivity without incurring a corresponding size enhancement.

[PeterX]

You know, with time I've learned that the good/bad dichotomy cannot be used, most of the time. Technologies are not always "good" or always "bad". There are instead a bunch of trade-offs. Even a "crappy" thing, if it's used a lot of people, must have some reasons to exist, at least one.

That's why I use C++ and Linux despite the fact that I actually hate them:

a) I had looks at the Linux source code and I don't like what I see. But they managed to produce a stable and powerful kernel and have a large "ecosystem" of user applications.

b) And C++ is widely used which matters for my free software projects because I want them to be reused and hacked by other programmers. And of course C++ is (besides a lot of other stuff and a lot of crap) C with OOP, which is nice.

Greetings
Peter

[nexos]

I never cared for C++ or Linux. I use C for coding, and plan on switching to OpenBSD in the future.

[vvaltchev]

That's why I use C++ and Linux despite the fact that I actually hate them

I'm sorry to hear that you hate Linux. I'm not gonna advocate in favor it, because it's a matter of personal taste, like iOS vs Android. Just, allow me to be a little sad
Do you know this epic essay? https://www.dreamsongs.com/RiseOfWorseIsBetter.html
In my experience, developers kind of actually divide between those two groups. Do you dislike Linux because you prefer the "MIT approach"?

[PeterX]

I'll open a new thread for this.

Greetings
Peter

[rdos]

True, but applications have less choices than the kernel that also can control page tables & physical memory mapping.

If you're allocating large page-aligned objects in the kernel, you probably don't want to use the same malloc/free as you do for arbitrary sized objects. Feel free to, but I think it'll make your implementation more difficult. It might be better to call your Virtual Memory Manager directly (e.g. "void* AllocatePages(size_t num_pages)", "ReleasePages(void* first_page, size_t pages)") and just leave comments that this is allocated via the virtual memory manager rather than mallloc.

Most of my kernel is in assebly, and I have no real equivalent of malloc and free. Rather, I have a byte aligned allocater that allocates a selector and a page aligned allocator that can both allocate linear memory and map them to selectors. I do support C and use it in some rather complex drivers, like hid, ACPI, the audio codec, and the TTF font manager, however the critical code is still in assembly there and C is only used for handling the complexity of the related specifications.

Does your kernel allocate objects that are larger than 1 page in size? Mine doesn't (I'm building a microkernel) so I'm curious if you do.

Some devices require larger schedules, so yes. I have more like a monolithic kernel.