optimalisation

[antoni]

While implementing memory allocator, I came across optimization troubles. Most of them were irrational and couldn't be explained otherwise. Unfortunately, after removing 02 flag, result changed, but it was still senseless. So, by inserting and removing 02 in selected places, I received various results, none of which was good. Have you any ideas how to deal with this problem? What flags do you use mostly? Here are my standard flags:

Code: Select all

-ffreestanding -mcmodel=large -mno-red-zone -mno-mmx -mno-sse -mno-sse2 -O2

+ g for GDB sometimes

Also, how should I debug this program? Should I turn off optimization and then debug it, or should I debug it with optimization?...

PS:
I didn't even use O3... There is enough trouble with O2.

[bzt]

Unfortunately, after removing 02 flag, result changed, but it was still senseless.

Does this mean your code is non-functional with -O0? Then you have a bug for sure.

So, by inserting and removing 02 in selected places, I received various results, none of which was good. Have you any ideas how to deal with this problem?

This usually happens when you have UB in your code. The optimizer is not perfect, but in the last couple of years I've found only one bug, meaning it is quite good. It is more likely the strange behaviour is a result of UBs in your code.

As for the flags, I suggest to specify the -mcpu or -march, that means a lot to the optimizer. Also add "-fno-builtins", because gcc likes to replace your memcpy implementation with a memcpy call Sometimes it could do this for malloc too. Btw, do you really need the large code model?

Also, how should I debug this program? Should I turn off optimization and then debug it, or should I debug it with optimization?...

You can do both. Normally you would compile with -g -O0 for debugging, and -O2 for production, but only if you have no UBs, and -O0 and -O2 compiled code does exactly the same. Until then feel free to use -g -O2 and debug that (it is harder a bit, because functions get taken apart and inlined, but works).

Cheers,
bzt

[Octocontrabass]

Also add "-fno-builtins", because gcc likes to replace your memcpy implementation with a memcpy call Sometimes it could do this for malloc too.

"-ffreestanding" already includes "-fno-builtin". If you have an example where it does not, please report it as a bug to the GCC developers.

GCC may automatically replace code that looks like memcpy, memmove, memset and memcmp with function calls, regardless of which options you pass to it. The only guaranteed way to prevent this behavior is to write your memcpy/memmove/memset/memcmp functions using assembly (inline or external), since GCC won't analyze that code. (There are other ways to prevent such optimizations, but I don't think any of them are documented, so there's no guarantee they'll work with future versions of GCC.)

[bzt]

Also add "-fno-builtins", because gcc likes to replace your memcpy implementation with a memcpy call Sometimes it could do this for malloc too.

"-ffreestanding" already includes "-fno-builtin". If you have an example where it does not, please report it as a bug to the GCC developers.

GCC may automatically replace code that looks like memcpy, memmove, memset and memcmp with function calls, regardless of which options you pass to it. The only guaranteed way to prevent this behavior is to write your memcpy/memmove/memset/memcmp functions using assembly (inline or external), since GCC won't analyze that code. (There are other ways to prevent such optimizations, but I don't think any of them are documented, so there's no guarantee they'll work with future versions of GCC.)

Hah, my apology, you're correct. Freestanding includes no-builtin, and indeed it does not prevent optimizations to emit libc calls. I was wrong about this, sorry.

Cheers,
bzt

[thewrongchristian]

Also, how should I debug this program? Should I turn off optimization and then debug it, or should I debug it with optimization?...

You can do both. Normally you would compile with -g -O0 for debugging, and -O2 for production, but only if you have no UBs, and -O0 and -O2 compiled code does exactly the same. Until then feel free to use -g -O2 and debug that (it is harder a bit, because functions get taken apart and inlined, but works).

Cheers,
bzt

The gcc manpage recommends -Og with -g, which enables optimisations that don't interfere with debugging.

Personally, I tried it and couldn't get on with it. I found the code didn't run in the source code order, so the optimisations involved were changing the program flow which is awful for debugging, so I leave optimisation off in general.

Every now and then, I will compile and run with -O3, to make sure my code still runs as expected, and bugs when enabling any -O options are a sure sign of relying on undefined behaviour.

[bzt]

The gcc manpage recommends -Og with -g, which enables optimisations that don't interfere with debugging.

Personally, I tried it and couldn't get on with it. I found the code didn't run in the source code order, so the optimisations involved were changing the program flow which is awful for debugging, so I leave optimisation off in general.

Yes, same here. I believe if you have problems with your code, then it is the best to debug with optimizations off. But when -O3 gives you different behaviour, then you have no other choice than to debug that particular optimized binary to find your UB.

Every now and then, I will compile and run with -O3, to make sure my code still runs as expected, and bugs when enabling any -O options are a sure sign of relying on undefined behaviour.

That's a very good attitude!

Cheers,
bzt

[nullplan]

I have pretty much sworn off GDB altogether, compile everything with full optimization and debug symbols, and if things go wrong, add copious amounts of debug statements. Yes, there is a bit of an observer effect, but with GDB, there is a much bigger one. GDB has lead me astray more than once, and its complexity is through the roof. So, since it is one big block of complexity I can do without, I have simply stopped using it.

Other than printf debugging, I do use lowlevel tools (watch points are super useful), but only ones that actually show what really is going on, not like GDB show me what would be going on if C was BASIC and the compiler was an interpreter. Favorite userspace debugging tool: strace.