It's surprising to see so much debate about when or where UB "happens".
Unfortunately this article doesn't refer to the best tool to root out UB - UBSan. If you write C or C++, you should have a UBSan+ASan build that is configured to terminate with an error when it encounters violations. You'll save yourself a lot of hair-pulling later. Then you don't have to wonder about the nebulous topics like what "line" the UB "happened" on.
For some compilers, when they add an optimization that capitalizes on an aspect of UB, they will also add a corresponding warning. So make sure you have warnings enabled -- and in most cases you'll want -Werror because generally no one scrutinizes the build output unless there's a failure.
I really hope that Windows Clang would soon support UBSan. It already supports ASan and is really helpful, but sometimes you need to go the extra mile to catch the most hard-to-find errors.
Yeah they are and that's too bad. HWASan is another option that reduces the overhead but would only work on a core that had MTE (which your microcontroller is not as likely to have).
gwp-asan (like electric fence) is another option for invalid access to heap-allocated buffers. Requires an MMU and creates additional TLB pressure w/some allocation overhead, but no instrumentation overhead and no access overhead.
The UBSan+ASan combo is great! Rust's miri is another good one.
This post is already fairly long (~10min read) and I didn't want to make it any longer. It's also already getting flamed in predictable ways ("nonsense! UB has to be reachable to be a problem!") so I didn't want to add more ways to get flamed for it (:
> Here's the list of guarantees compilers make about the outcomes of undefined behavior:
> That's the whole list. No, I didn't forget any items. Yes, seriously.
I’m a little disappointed there isn’t an empty <ol> or <ul> in the middle there. I imagine screen readers would announce something like “list, zero items”.
Post author here. I'm very curious about the screen reader idea. If you or someone else is able to confirm what a good screen reader might do currently / what it might do with an empty list element, I'll happily update the post to whatever is the better behavior.
I wish there were a best practices doc + a matching linter for accessibility things like this.
I just tested "<ol></ol>" in VoiceOver on Mac and nothing gets read. Different screen readers are notoriously inconsistent though.
> I wish there were a best practices doc + a matching linter for accessibility things like this.
This sounds like it was meant to be a fun tweak/joke for screen readers but it's accessible without.
So it's not practical to automate a lot of accessibility checks like this one because it depends on context and there's subjectivity to it, similar to visual design and text readability. The best way to get more of a feel for this is to try a popular screen reader for your OS, do the tutorial, and try it on the page yourself.
> "If my program contains UB, and the compiler produced a binary that does X, is that a compiler bug?" It's not a compiler bug.
True, if the language spec is the only thing in the universe you care about. But there are better and worse things compilers can do in the presence of UB, and the big one is warning you about it when it possibly can.
There's another big discussion on the front page about signed integer overflow in particular, and gcc screwing the developer over because it was UB when they thought it was just implementation defined:
> gcc screwing the developer over because it was UB when they thought it was just implementation defined
No, the developer thought "UB" produced an integer with an undefined (but still valid integral?) value. They misunderstood what UB is, not whether or not something was UB.
Both "UB" and "IB" are terms defined in the language standards, and it's obvious that's what we're talking about here.
x86 defines what specific instructions do, nothing so crass as "signed integer overflow" - that's part of why the language standard has to define it as such.
Yeah, except that they effectively aren't part of the language standard. Possible behavior is to ignore UB, behave in a documented manner, or issue a compiler error. Not remove other code or any other nonsense optimizing compilers do.
I'm not gonna wage a holy war against optimizing compilers, but let's not pretend that this is a language standard issue.
> x86 defines what specific instructions do, nothing so crass as "signed integer overflow"
I don't understand what you mean. The x86 standard defines what happens if an `add` instruction is performed and the result overflows, possibly affecting the sign bit (what happens is that the OF flag is set, and the target register is set to the two's complement representation of the sum modulo 2^bitlength).
That's irrelevant. A C compiler targetting x86 has no obligation at all to handle signed integer overflow in the same way the processor natively does. Because it is not an assembler - it is compiling against the theoretical C virtual machine.
This is for the same sort of reason that unsigned overflow does have a well defined meaning in C, and will always do exactly that even for targets that do not natively do that (if necessary it would emulate that behaviour in software).
It kinda has an implied obligation to not screw with its users. And doing the same optimizations to fold bloated Cpp templates properly messes with my C code.
If it would result in different behavior in the event of an overflow (which I think is generally not the case), then no. A typical for loop (using <) can easily been seen to not overflow, so this is a rare case anyway. Loops using signed comparison with <= where the compiler can't rule overflow out should result in a compiler or linter warning.
The inequality argument would seem to apply only to for loops with constant iteration counts, yeah? The compiler can't know that "i < n; i++" won't overflow if "i" could start out higher than "n". (wrong)
I am not an expert on compiler optimization, but having reasoned through a few cases where a compiler is _unable_ to correctly make an obvious-seeming optimization, I tend to think that compiler writers have good reasons for the things they want to be able to assume.
If "i" started out higher than "n", the loop would exit immediately and there would be no increment. More generally, if "n" is an int, then even if n is modified each iteration, the "i < n; i++" still can't overflow because it will always exit at INT_MAX if not before. If "n" is something else like a long then that's likely a mistake that falls under "the compiler should warn if you do this".
If there were good reason for wanting to assume this, I would expect someone to have presented it; responses I hear just take the form of a single (usually contrived) example where more performant code is generated, then insisting that all other factors (e.g. being predictable/unsurprising, acting like the underlying architecture, mitigating bugs) are irrelevant. Just as GP did.
If there weren't good reasons for making the assumptions that modern compilers do, I'd have expected users of C and C++ to have converged on a more conservative compiler, at least in some significant number.
It’s well-defined on x86, but the compiler is allowed to insert INTO instructions (which trigger an interrupt on overflow), which is effectively UB (as the interrupt routine is outside the scope of the C language).
> But there are better and worse things compilers can do in the presence of UB, and the big one is warning you about it when it possibly can.
The problem is that this is really hard to do without producing either too much or too little warnings, as undefined behaviour is usually input-triggered; and the compiler usually doesn't know all constraints on the arguments that the programmer knows about.
For example, the function f() in the article of the other discussion is perfectly well-defined for any argument x <= 0x402010, as in that case no overflow happens. The compiler doesn't know whether all callers ensure the argument is valid (e.g. this might be a library function, or the invariants might not be expressable in a way the compiler can understand). This is a lose-lose situation for the compiler: it can warn, but people will find the unnecessary warnings annoying and turn them off, while if it doesn't warn people will complain about perceivedly bad optimizations.
Of course there's a point to be made that the real problem here is that the C standard makes signed integer overflow undefined behaviour, but that's not gcc's fault and can be easily worked around with -fno-strict-overflow or -Wstrict-overflow.
> This is a lose-lose situation for the compiler: it can warn, but people will find the unnecessary warnings annoying and turn them off, while if it doesn't warn people will complain about perceivedly bad optimizations.
This could still be greatly alleaviated with the use of asserts: A compiler could default to the most conservative set of assumptions for an input value - e.g. that a parameter of a global function assumes the full value range of its type - unless the programmer provides more assumptions through an assert statement.
In debug builds, you could compile those asserts to actual runtime checks, so the program can be appropriately tested. Then, in production builds, the asserts would simply be skipped.
The second improvement would be warnings about "nonensical" statements (from the compiler's POV) that will be optimised away. E.g., many of the most egregious examples of UB are from code where the programmer inserted a bounds check to protect against UB at another location - and then the compiler optimized away that same bounds check as a "nasal demon" action, caused by the UB it was supposed to protect against. In those cases, it would be useful to warn "hey, this bounds check can never fail because that other statement implies that either we're in bounds or there is UB. You might want to check for UB here."
We're re-litigating that other thread... but the function in the other thread had a conditional (trying to check for overflow after the fact) which would never be true, regardless of x. And it could statically prove this.
And that proof, the compiler then decided to use to remove the code, ignoring the red flag that the programmer undoubtedly had tried (and failed!) to achieve something with it. I would have preferred a warning here. I don't turn off "condition is always true"-type checks.
While I agree that in general a warning about "condition is always true/false"-style checks is useful, it's also a bit tricky to do without false positives. With macros and inlining (and in C++, templates) you can easily end up with conditionals of which any given instance is always true or false, but that aren't superfluous. Portability is another example. Depending on your environment and code style hitting these might be exceptional and warning-worthy, or something that happens all over the place.
I also see this come up a lot in code generation and simd code. On the code generation it's way easier to just push all optimizations to the compiler, including constant propagation (i.e. easier to just mark an upstream boolean as constexpr true, instead of adding your own constant propagation). For simd it's common in my experience to do lots of (if size > X), where size and X are both known at compile time but might vary by target.
No, they aren’t. If they don’t make changes that are observable outside the loop, they can be assumed to terminate. ThTs a compile-time thing. https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1509.pdf (not guaranteed to be equal to the final wording of the standard):
“An iteration statement that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression-3, may be assumed by the implementation to terminate.”
The way to think about UB is that it’s like a false premise. Anything and everything can be derived from a false premise. Compilers can end up with arbitrarily false conclusions if the program contains a false premise.
The other important thing to realize is that UB is generally a runtime condition and cannot be detected statically, in the general case. That’s usually why it’s been made UB in the first place.
This post is very good. One thing to add, is that if you code has not entered a state of UB yet, but will at some point, then all bets are off. The phenomenon is generally known as time traveling UB.
Compilers assume your code doesnt contain UB. Example:
If(x == 0)
printf(“hello”);
y /= x;
The compiler can assume x is not zero because it assumes the user will not cause UB, therfore the if statement and printf can be removed.
No, no, no. UB is a global property of your program. "All bets are off" even at compile time.
"Anything at all can happen; the Standard imposes no requirements. The program may fail to compile, or it may execute incorrectly (either crashing or silently generating incorrect results), or it may fortuitously do exactly what the programmer intended."
UB is a property of the source code, or more specifically a property of how source code should be compiled into machine code. It means that there is no constraint on the machine code which will be produced if your source code has certain properties.
"If any step in a program’s execution has undefined behavior, then the entire execution is without meaning. This is important: it’s not that evaluating (1<<32) has an unpredictable result, but rather that the entire execution of a program that evaluates this expression is meaningless. Also, it’s not that the execution is meaningful up to the point where undefined behavior happens: the bad effects can actually precede the undefined operation."
Many people in this thread are confused and think that unreachable UB (either statically or dynamically unreachable) compromises the entire program. This is not true, but your comment helps me better understand how people reached this conclusion.
Reachable UB (either provably reached at compile time or dynamically reached at runtime) can retroactively invalidate the correctness of previous statements. But this does not mean that unreachable UB compromises the correctness of well-defined executions.
You are indeed correct. I would add that there are some instances of UB that actually UB at translation time: for example ODR violations and some preprocessor dark corners. At least the latter is getting stamped out.
The C++ standard calls these "ill-formed, no dioagnosrics required" instead of UB, although it's not always consistent. It is indeed more or less "translation-time UB". The difference is exactly that affected lines don't need to be in the execution path to make all executions undefined, and that the compiler/linker may fail the translation.
int main(int argc, char **argv) {
int a = atoi(argv[1]);
int b = atoi(argv[2]);
return a + b;
}
contains signed integer overflow in some executions but not others. Can you explain what effect the counterfactual signed integer overflow is allowed to have in executions that do not contain signed integer overflow?
Out-of-order execution isn't the reason for this. The C standard assumes an abstract machine that allows OoOE, of course, but even with strict in-order hardware UB can hit you at any time, even before you'd think it could. That is because the C standard doesn't limit UB to any constraints like "following lines" or "subsequently executed instructions". Independent of the hardware, the compiler is allowed quite a bit of reordering of instructions. The standard just requires that (some) effects of those executions are ordered as written, but that doesn't include UB.
So if you have UB in your future path of execution, the compiler might just do whatever _right now_.
It still has to be executed by the abstract machine for the given set inputs for the whole execution to be undefined.
You are right that once the execution is undefined, you can't reason about the behavior. This includes ordering of side effects, or side effects od operations occurring before the line with UB.
But uttering UB in dead code making execution UB is utter nonsense. Otherwise any execution of any program that just utters __builtin_unreachable() would be undefined.
That's sounds plausible, but isn't. First of all, out-of-order execution has nothing to do with unreachable core - it just means that reachable code sometimes gets executed in a different order than the binary shows. It is speculative execution which can lead to unreachable code actually being executed.
However, even when processors execute code speculatively, they guarantee that no observable effects of executing that code will happen. So, if you have something like `if false {*NULL=7/0}`, the processor might attempt to execute that line speculatively, but it will revert any change it made, and it will not trap regardless of any other flags.
> This is nonsense. The line with UB has to be reached for the execution to be undefined.
Check the footnote at item 14, it links to a page which shows one example of how a line with UB can be reached (basically: the optimizer can reorder the code so that parts of the line with UB will run much earlier than you'd expect).
Unless your code is actually commented out (actually dropped from the program), it can still (maybe) execute.
I think the main reason for this is an if(testSomeState()) { //code with UB } block being optimized to just //code with UB.
Functions which are unreachable, maybe I agree, they can't be executed (without UB somewhere in reachable code). I'm not sure that's true though - if you have UB in your hardware (not your code), you can still execute that code. This is why (at least I was taught) never to leave dead code around. There's this better thing for that called source control you may have heard of...
IF you have perfect hardware, and IF you have perfect logic/compiler and IF you have no UB in your program, then maybe you can guarantee that dead code is really dead, but I'm not really sure you want to heavily define dead code like that.
Just remove it. Especially if it has UB in it to cause more fun bugs...
If UB happens¹, anything can happen. So if the check can only happen after UB, or leads to UB, it can be removed. That isn't the case in your example – testSomeState() can return false (and if it can't, it getting optimized out is unrelated do the code it guards). Points 14-16 from the article are just nonsense, UB is a runtime concept and dead code doesn't run unless you already triggered UB¹.
I checked the footnote. It describes a case where invoking UB (transmuting "3" to a bool) can cause dead code to become live. But transmuting "3" to a bool is itself UB. So this fails to prove that execution becomes undefined without reaching a line containing UB.
"The line with UB" becomes a very vague notion well before the program even executes. You can see this by dumping the disassembly of your program. The compiler can determine when values are defined and used and will emit code that satisfies the semantics of your program (according to the language specification). This often means that expressions within the same statement get evaluated at significantly different times, hoisted outside of loops, etc.
The assembly dump does not matter for this analysis (except for finding buggy compilers!) - the C abstract machine itself applies all side-effects before each sequence point.
You have to understand that in the modern understanding of UB as applied by compiler writers, the compiler has every right to assume the program never triggers UB. This means that lines which would be UB if a variable had certain values can be used as proof that the variable doesn't have those values, thereby eliminating checks.
For example, if a program looks like this:
void foo(int* const p) {
if (p!=NULL) {
bar(p);
}
*p = 1; //this line would be UB if p could be NULL
//so we are free to assume p is not NULL in this scope
}
It will very likely be optimized to this:
void foo(int* const p) {
bar(p);
*p = 1;
}
And even if bar() itself had a line checking if p was not NULL, that will get ellided as well if bar() gets inlined.
This is a good example of how UB on a later line can have an impact on earlier lines, or even in other places in the code.
Note that it's not executions which trigger UB, it is source code programs that do.
I don't think this proves the point, because foo() will not invoke UB unless it is called with a NULL argument.
It is true that when foo() is called with a NULL argument, the UB will be realized before bar() has had a chance to run (EDIT: this doesn't actually appear to be true for the given code sample, neither Clang nor GCC elide this NULL check: https://godbolt.org/z/v9v74c3jd, which makes sense as bar() could call exit()).
> Note that it's not executions which trigger UB, it is source code programs that do.
> Furthermore, if every possible execution of a given program would result in undefined behavior, the given program is not strictly conforming.
A conforming implementation must not fail to translate a strictly conforming program simply because some possible execution of that program would result in undefined behavior. Because foo might never be called, the example given must be successfully translated by a conforming implementation.
> It is true that when foo() is called with a NULL argument, the UB will be realized before bar() has had a chance to run. But the line with UB has already been proven reachable prior to the actual dereference, so the UB has in a sense already been invoked.
The point of those numbered lines was to refer to how people normally think about UB. Someone seeing that function may be tempted to say that it's not UB to call with a NULL value as long as the dereference doesn't happen (say, bar() could call exit() or it could run an infinite loop). What you're saying is exactly what the article is saying: the fact that a line with UB is unreachable in practice doesn't mean that it can't affect your program.
The standard talks about executions in the sense of theoretical executions by the C abstract machine, not actual executions of the compiled binary. I thought the other comment was rendering to the latter notion of execution, but perhaps I was wrong and it was also talking about the former.
> Someone seeing that function may be tempted to say that it's not UB to call with a NULL value as long as the dereference doesn't happen (say, bar() could call exit() or it could run an infinite loop).
I would expect that a call to exit() inside bar() would prevent eliding of the NULL check. If the program calls exit() before the dereference, it does not invoke UB and therefore should run correctly. I tried your example and Clang and GCC do not actually elide the NULL check in this case, probably for this reason: https://godbolt.org/z/cnWPz3o59
> The standard talks about executions in the sense of theoretical executions by the C abstract machine, not actual executions of the compiled binary. I thought the other comment was rendering to the latter notion of execution, but perhaps I was wrong and it was also talking about the former.
I don't see how this would change the analysis. Running the binary triggers actual executions of both the abstract machine and the physical binary.
The fact that bar could exit or not return doesn't matter (although it was indeed my initial line of reasoning) as if the value is null bar won't be called and UB happens; hence the value can't be null and bar can be called unconditionally. The value is still dereferenced only after calling bar.
Still gcc doesn't optimize the check away so it is possible there is some other line of reasoning that prevents the transformation.
Now look at a large code base, with lots of checks for inputs. With O0 it's usually straight-forward to read, but with heavy optimizations applied it can become a real nightmare. And in that scenario all it takes is some reordering of the UB before the DCE pass can prove the code to be unreachable (which might not happen on the first DCE pass, because the value analysis/constant propagation didn't work its magic, yet). Also static program analyses can be both forward and backward.
I'm afraid it's not nonsense, that's why it's a common misunderstanding ;)
No, the commenter is right, at least for Rust (can't say for other languages). UB is a property of a particular execution of a program, it happens when code is executed on the abstract machine. As the blog post linked from the article (point 14, footnote 6) states:
> Right now, we have the fundamental principle that dead code cannot affect program behavior.
If you write i+1 anywhere on your code, the compiler is free to assume i is different from the maxumim value and rewrite your code with that in mind.
If you are not satisfied, you can read the article's references (there are 2 for unreacheable code). Or any article about UB here for the last 20 years.
No, that transformation is straight-up buggy unless the compiler can also prove that n > 0 (ie by inlining the bar() function into a context where n is always positive).
If you had *p == 0 before calling bar(p, 0) then *p had better not be 1 after!
The first form has no undefined behavior. Since p may be NULL, in general the compiler is not allowed to make that transformation unless it can prove bar never gets called with p=NULL and n=0 (which it probably can't).
If the compiler has some special knowledge of the target architecture which makes the second form behave as if the first form was executed (such as if NULL is a valid address on the target architecture), it may make the transformation, but that still wouldn't cause undefined behavior, because it must behave as if it had the first form.
Yes, thank you. It's clear these points are incorrect if you consider the purpose of undefined behavior. Compilers are allowed to assume undefined behavior is never invoked (which may be useful for optimization and other reasons); it is a promise from the programmer that they will not invoke undefined behavior specifically so that the compiler may assume they don't. Things are often made undefined behavior because the assumption is useful, but hard or impossible to prove in general with static analysis, so it requires a programmer who can consider the context of the code to assure the assumption always holds. If undefined behavior is invoked, some of the compiler's assumptions are wrong, so anything could happen (see the "principle of explosion"[1]). If no undefined behavior is invoked, all of the compiler's assumptions are correct, so the program must behave correctly.
For example, imagine a C programmer writes a function that adds two `int` values. Hypothetically, with some inputs, the addition may overflow. That would be undefined behavior, so the compiler assumes the function is never called with such inputs, and optimizes accordingly. The programmer knows this, and assures the function is never called with those inputs. Since the only assumption the compiler made was that the function wasn't called with input which would cause the addition to overflow, and the programmer assured that assumption is correct, the function always behaves correctly.
Consider if the same function's inputs were provided at runtime from stdin. The behavior of the whole program is undefined if invalid inputs are given, but if only valid inputs are given, the program must behave correctly.
Everyone's getting confused about the word "reach". If the execution of a program triggers UB at any point then its behavior is undefined at all points, even before the UB was triggered. But the compiler does have to respect and preserve the behavior of any completely UB-free execution of your program, even if other executions (say with different input) could trigger UB.
So we could say, unreachable UB is fine, but if you do reach it, then it doesn't matter when you reached it.
Yeah the format of the article makes this mistake extra confusing. The statement in point 14 is in fact true, but the whole premise is that all these statements are supposed to be false.
> 14. Okay, but if the line with UB is unreachable (dead) code, then it's as if the UB wasn't there.
This one came as a surprise to me. Is this really (unconditionally) true? And if so, why? Seems trivial to me to ensure that UB that will never under any circumstance execute will not affect the functioning of an otherwise correct program.
The compiler isn't required to know if code is unreachable. For example, `if (fermats_last_theorem_is_wrong()) {*p == 1} ` will imply to a modern compiler that p is not NULL in the scope where it is defined, even if we now know that this code will never be reached (assuming that function is correctly written).
Or, here is a better example:
for(int i=0; i > n;) {
}
*p=1;
Here the compiler will assume that the loop terminates, since non-terminating loops without side-effects are UB in C++.
Then, since the loop terminates, p will be dereferenced, so it can be assumed that p is never NULL, so any NULL checks for p in this context can be ellided.
This remains valid even if you call this function with an n such that *p=1 is unreachable.
If a some translation unit defines int fermats_last_theorem_is_wrong(void) { return 0; } then a well-defined program can execute that sequence even with p equal to NULL.
The compiler is required to successfully translate a program that contains unreachable UB, and such a program is conforming if undefined behavior is never reached. This is explicitly spelled out in https://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_109.html
> Furthermore, if every possible execution of a given program would result in undefined behavior, the given program is not strictly conforming.
A conforming implementation must not fail to translate a strictly conforming program simply because some possible execution of that program would result in undefined behavior. Because foo might never be called, the example given must be successfully translated by a conforming implementation.
*p is undefined if p is NULL. But as long as that statement is never reached when p is NULL that is not a problem. That is trivially true in this case, but the same applies even for more complex relations between the conditional and the undefinedness of the statement.
Or put another way, in
int * p = 0;
if(fermats_last_theorem_is_wrong()) {
p = malloc(sizeof(int));
*p = 0;
}
if(p) {
puts("fermats last theorem is wrong\n");
}
if(fermats_last_theorem_is_wrong()) {
*p == 1;
}
if(p) {
puts("fermats last theorem is wrong\n");
}
the compiler is not allowed to remove the if(p) checks and unconditionally print that fermats last theorem is wrong unless it can prove that fermats_last_theorem_is_wrong() is true.
In fact, something having undefined behavior and the compiler being allowed to assume that it is unreachable are effectively the same thing. Which is why the spec for std::unreachable() and the compiler-specific intrinsics that preceded it is that it invokes undefined behavior and not more complex wording.
Yes, my example with Fermat's last theorem was completely wrong, not sure why it seemed plausible to me when I wrote it, it's obviously just a guard, and it obviously is not UB to have a pointer write behind an If...
Presumably (and really don't take my word for this - I'm a Java bod) the p dereference needs to come before the p NULL check for the p NULL check to be ignored. (Otherwise, how can a NULL check protect a dereference.) And therefore, the NULL check would be unreachable as well, and therefore never executed and not relevant.
I am not certain of the language lawyer side of it, but no reasonable (or even moderately unreasonable) compiler would omit that check because of the guarded behavior; way too much code would break, probably including that compiler's standard library.
> For example, `if (fermats_last_theorem_is_wrong()) {*p == 1} ` will imply to a modern compiler that p is not NULL in the scope where it is defined, even if we now know that this code will never be reached (assuming that function is correctly written).
No it won't. Otherwise all precondition checks would be meaningless. The compiler can only assume that it isn't null when fermats_last_theorem_is_wrong() is true. If the compiler cannot reason about fermats_last_theorem_is_wrong() then it can't just ignore it.
> for(int i=0; i > n;) { } *p=1;
> Here the compiler will assume that the loop terminates, since non-terminating loops without side-effects are UB in C++.
Infinite loops are UB precisely because otherwise the compiler would NOT be able to reason about this. So yes, here the comiler can assume that p is a valid pointer - but only because it can also assume that *p=1 is NOT dead. This falls entirely under the umbrella of behavior not being defined at all after you execute undefined behavior (the infinite loop).
> This remains valid even if you call this function with an n such that *p=1 is unreachable.
Only because the loop itself is undefined behavior in that case.
That is not an infinite loop, it has an exit condition on i > n.
An infinite loop is not UB. It is well defined. Just the definition of a terminating loop is a bit funny.
> An iteration statement whose controlling expression is not a constant expression, that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression, may be assumed by the implementation to terminate.
Plus:
> An omitted controlling expression is replaced by a nonzero constant, which is a constant expression.
The compiler may only use as-if rule here if it can deduce the value is constant or limited. It can (but does not have to) assume the loop will end due to the conditional and lack of the other features. Which is still defined behavior. (since C11 at least, probably earlier)
> An iteration statement whose controlling expression is not a constant expression,156) that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression-3, may be assumed by the implementation to terminate.157)
> 157)This is intended to allow compiler transformations such as removal of empty loops even when termination cannot be proven.
This makes infinite loops UB unless they fall under one of these exceptions (constant controlling expression, performs I/O, accesses volatile objects or atomics).
Not particularly. It does not allow the compiler to assume true infinite loops can terminate.
It only allows the compiler to assume loops with non-constant expressions can terminate. (And a bunch of other exceptions where it cannot.) A true infinite loop has a constant expression as the conditional and cannot be optimized away based on this rule.
What this rule allows the compiler to do is to optimize away loops where number of iterations are known ahead of time if it has a non-constant expression condition.
> It only allows the compiler to assume loops with non-constant expressions can terminate. (And a bunch of other exceptions where it cannot.)
That is basically what I said: loops can be assumed to terminate, except for an enumerated set of exceptions where infinite loops are allowed.
You seem to be defining "true infinite loop" to mean "while(1)" or "for(;;)" exclusively. But "for (int i = 0; i < 1;)" is also an infinite loop, and that one is UB since the controlling expression is not constant.
> What this rule allows the compiler to do is to optimize away loops where number of iterations are known ahead of time if it has a non-constant expression condition.
If the number of iterations are known ahead of time, then it is not an infinite loop so I don't see how the rule would apply.
> Infinite loop with a constant expression is defined behavior.
Yes, because a constant controlling expression is one of the enumerated exceptions to the rule against infinite loops.
> You're showing potentially infinite loops with non-constant expression as condition.
"for (int i = 0; i < 1;)" is an infinite loop, full stop. It is not potentially infinite, it is infinite under all possible executions. It is UB because it does not fall under one of the enumerated exceptions where an infinite loop is allowed.
I think we are saying the same thing, the only difference is that you seem to be defining "infinite loop" to only include loops with constant controlling expressions, whereas I am defining it to mean "any loop that does not terminate."
Yes, the part about Fermat's last theorem I wrote is actually obviously wrong, I don't know what exactly I was thinking when I wrote it.
The example with the loop though is more to point out that our native understanding of "unreachable" may be different than the compiler's. The same problem could occur without invoking UB if we un-conditionally generate a SIGKILL to our own process and then dereference a NULL pointer - the compiler doesn't know what SIGKILL does, so it doesn't know the UB is unreachable in practice, so it may reorder etc.
> the compiler doesn't know what SIGKILL does, so it doesn't know the UB is unreachable in practice, so it may reorder etc
It is the reverse. If the compiler doesn't know what raise(SIGKILL) (or really, any other function it cannot see through) does, it has to assume that it can potentially never terminate, abort or otherwise change the control flow, so it can't safely reorder across it. And even if it does return, it could still inspect memory and observer the violation of the as-if rule.
That's why for example pthread_mutex_lock/unlock worked correctly for the most part even before C++ got a proper concurrent memory model.
That (or more general 13-16) also does not make sense to me. Most UB (although perhaps not all, so it may be true for some obscure UB, or ones with constant arguments) are undefined for specific dynamic conditions, so it is not 'line with UB' but more 'line with UB under specific program state', e.g. expression 'a / b;' has UB if b == 0, 'a->item' has UB if b == NULL, or 'a * 4' has UB if a is int and large enough to cause overflow. If the expression is not executed, the dynamic conditions are not there, and therefore there is no UB.
The compiler may make optimization decisions based on the presence of UB so the code only needs to be present.
The compiler can't know that an arbitrary part of a function is unreachable if, for instance, that code path is controlled by a parameter or global state since that equates to solving the halting problem.
The undefined behavior can still affect code generation even though the program gets translated.
It's just that the generated code may not do exactly what you expect it to do because the presence of undefined behavior allowed the compiler to make assumptions which may surprise you.
DR 109 allows that a program may be strictly conforming even if some possible executions of the program invoke UB:
> A conforming implementation must not fail to translate a strictly conforming program simply because some possible execution of that program would result in undefined behavior.
This text specifically allows for the case that a program is strictly conforming even if there is a possible execution that invokes UB. If a program is strictly conforming, it must produce the correct behavior.
I would challenge you to show an example where GCC or Clang will break the correctness of a program on account of UB that is not reached during program execution.
Here is an example where GCC and Clang specifically respects the correctness of a program as long as it does not reach the UB: https://godbolt.org/z/befWah77W
The linked article argues that making storing invalid values in the bool undefined (rather than just using such bools) is important because otherwise the compiler is not allowed to move the code (without adding appropriate checks that it would have been reached). The compiler can only make that transformation if it has no effect on any cases where the code wasn't reached at all, which includes not introducing any new UB in those cases. Predrag conclusion that this means having UB in dead code matters is simply wrong.
The blog post does not support the assertion that unreachable UB which is not executed is a problem. It starts by saying
> The Rust compiler has a few assumptions that it makes about the behavior of all code. Violations of those assumptions are referred to as Undefined Behavior.
Then it says
> In other words, even just constructing, for example, an invalid bool, is Undefined Behavior—no matter whether that bool is ever actually “used” by the program.
The blog post is talking about how just creating an invalid/trap/niche representation for a value is UB. The act of creating the value is the UB here, not the usage. So, in this case the UB is most definetely reachable and executing (The act of creating the invalid value).
I have another falsehood: most programmers believe that people working with C/C++ are constantly battling against UB all day long, five days a week, and that they are irresponsable and careless. That they should live in fear and terror, because "unsafe" is around the corner, and that their programs are not mathematically proven and that they might containt an explotaible bug because, you know, every other C/C++ program opens a socket to the internet or is a "sudo" kind of tool.
Most people driving cars with an active recall are unaffected by the problem the recall is supposed to address. They can drive on blissfully unaware that something could go wrong. They might even enthusiastically recommend that others purchase the car they drive.
For a few people, they'll experience a failure, possibly a dangerous one.
And this is the root of the problem. The fundamental concept of using a language prone to UB exposes all programs to a small risk. In isolation, the risk is small. In aggregate, the problem is real.
The same applies to vaccines and medicine drugs. Should we stop giving them to people? "For a few people, they'll experience a failure, possibly a dangerous one.".
Should we try our best to minimize the risks? Absolutely. But we are talking programs here. We shouldn't measure every program with the same ruler. Not all programs need to be MISRA compliant when they don't need to.
My Reddit app crashes several times a day, and I guess there is no immediate danger.
There are no viable alternatives to vaccines and medicine in general (ones that actually work, that is). There are alternatives to C -- for most cases.
> My Reddit app crashes several times a day, and I guess there is no immediate danger.
That's because your app is heavily sandboxed and probably (on Android, at least) running in a VM which enforces memory safety and isolation from other apps.
If each crash had, say, a 1% likelihood of eating a large portion of your data (or leaking pictures or whatever) you'd probably care a lot more about it.
The problem highlighted by the parent poster is that all these little risks add up and often end up being catastrophic (e.g. heartbleed) because so much of what we run is interconnected via the network (or just data in files).
It can be true that most programmers working in C or C++ are spending most of their time solving real problems that are still most effectively solved by C or C++, and at the same time, that even the absolute top C and C++ programmers regularly misapprehend UB itself or whether their own code contains UB in a way that causes a serious issue for the program.
It does seem odd to just "lump unspecified behavior and implementation-defined behavior together" in a side-note under the heading of "implementation-defined behaviour", when that term has a precise technical meaning in the C and C++ specifications.
I think that note just means "this post isn't about unspecified or implementation defined behavior". I'm sure you could write articles about each of those.
Post author here, and yes this ^ is precisely what it's intended to mean. There are also other more exotic flavors of behavior, and the post isn't about those either.
I tried to cover as much ground in the post as possible, but the post is already a 10min read and covering unspecified / impl-defined behavior would have made it a 20min read instead :)
I did get that eventually, but I missed the side-note in my first read through, and it was lumping two concepts together under the name of one of them that I found confusing.
If, in your description of the three buckets, you called the middle one something else like "Platform-dependent behaviour" or whatever, I think that would have been less weird. You'd still only need one short paragraph to say it's compiler/OS/hardware dependent, covers implementation-defined and unspecified, and is not the focus of the article.
I have a suggestion - it may be completely off the wall, but hear me out.
C needs a mechanism to declare in a source file that certain behaviour must be defined, where that behaviour is undefined in the C standard but the target architecture behaves the defined way. Then, when the source is compiled on the target architecture it works as expected, but when compiled on something else it refuses to compile.
For example, unsigned ints. It'd be great to be able to declare "I define int to be 32 bits and to overflow like a sensible 32-bit int - treat this as defined behaviour, and refuse to compile on a system where this isn't true." The number of platforms where this would fail to compile would be extremely small, and at least they would give a sensible compile error on systems that are weird.
Not cleanly, but if you assume 2s complement and wrap all signed addition in a macro function you could cast to unsigned before the add and cast the result back to signed.
Suggestions like this are unfortunately quite infeasible (they fall under the hat of "define all the behaviors"), and the talk I linked in the post goes into why: https://www.youtube.com/watch?v=yG1OZ69H_-o
He's not trying to define all behaviours, he wants a static assertion that the compiler will compile certain behaviours in a specific way. Seems very different.
That's why I said "falls under the hat of" :) I promise the talk is relevant.
`-fwrapv` is an attempt toward doing such a thing for signed integer overflow specifically. I bet it's a pain to maintain because it significantly increases the surface area of the compiler that needs to be tested.
I don't believe doing this for more cases is feasible at the level of the compiler. Things like CHERI where the hardware and OS help are much more viable, in my book: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/
Signed integer overflow flag is relatively painless to maintain, because compiler needs to be able to track it for unsigned integers anyway. That is, signed integer was modeled with a pair of flags (interpret bits as signed, can't overflow) and unsigned integer (interpret bits as unsigned, wrap on overflow), so signed integer under -fwrapv is just (interpret bits as signed, wrap on overflow).
I think it's pretty clearly feasible in the specific example GP gave, since -fwrapv does this for GCC and clang. In principle it should be do-able from code level with e.g. 'pragma gcc optimize("frapv")' (but the gcc docs for this say that this feature is for debugging only and doesn't always work).
"goes into". https://youtu.be/yG1OZ69H_-o?t=1039 You mean this slide of the Joker with title "Do I really look like the kind of language that would define its behavior?"
As far as I can tell there is no reason LLVM couldn't compile a document like "On x86 division by zero throws a #DE exception. On ARM division by zero returns zero. On x86 shifting by more than 32 bits shifts by (cnt mod 32) bits. On ARM shifting by more than 32 bits clears all bits. [...]" that specifies all undefined behavior. The current "nasal demons" behavior is preferred in the name of performance on some SPEC benchmarks that nobody was asking for, screwing OS writers and anyone who expected sane language semantics.
Could you elaborate on 13-16? I have no idea what you could possibly mean, and the linked blog post doesn't help. As far as I know UB needs to execute for it to apply and make a program invalid. The linked rust blog post is about how in rust, creating some values in and of itself is UB (It's not that there is unreachable UB later on that uses the invalid value, it's that there was already UB executed earlier in the program in the creation of the invalid value). I know there are some "global" UBs that do not need to "execute" at runtime, but those can simply be thought of as executing at compile time.
I think it's silly to tell people what to do when you don't know what they're doing. For all you know, they could be generating code from a design tool that already does that static analysis.
What's nice about Ada is you have choices, and the ecosystem facilitates creating and using such tooling.
I know what you're referencing, but it's not obvious to people who don't already know about it.
And in a way, I think you did say something deep. Web developers often have no idea that problems like this were solved a long time ago, and it lowers their expectations for what should be possible with the tools of their industry.
If the program knew the intention of the programming we wouldn't need programmers to begin with. Undefined behavior may be obvious to the architect but not to the architecture.
Good point. Also UBsan and Asan are great additions to any C/C++ workflow. But if it's actually part of the code, I'm forced to use it even if I'm lazy, versus an external tool that I can forget to run (or forget to add to my build system).
Leveraging what the compiler gives you is like that saying "the best camera is the one you have on you".
I love constexpr because it allows me to write compile time unit tests inline with the library code, with static_assert. The implied UB check is gravy.
While theoretically true, there are a whole bunch of undefined behaviors that are not caught in constexpr context in practice. In particular unsequenced operations on the same object are typically not caught.
It's still super useful though, and more common language UBs are caught.
It is. Or if it is unfeasible to implement then it can be considered to be a defect in the standard.
Consider that something like (++*ptr1) + (++*ptr2) may or may not be UB, depending where the pointer points to. The required additional bookkeeping might make compile time evaluation unpractically slow.
Alternatively the sequencing could be always defined between any two subexpressions within constexpr context.
My least favorite thing about "falsehoods X believe about Y" articles is that they make a point out of not backing up or demonstrating any of their claims.
Fortunately, this post includes a link to an excellent 45min talk on undefined behavior which does go into a fair bit of the detail you were looking for. I encourage you to click it -- it's right in the post's introduction.
The purpose of this post and other "falsehoods" posts is to shine a light on the existence of a misconception or set of misconceptions. By that point, there are usually plenty of other high-quality posts about "why / how / when" etc. but for a variety of social dynamics reasons those posts by themselves did not have the wide distribution and readership they deserved. After shining a light on the misconception, interested readers are generally able to continue the learning process both through the sparked discussion and through doing their own searching.
It is not explained why points 31 to 36 are valid, unlike HDL synthesis tools, C compilers are pretty deterministic, same input same output, except timestamps, etc.
If an UB adopt X behavior one time, a reproducible build will take same X behavior the next time.
Of course I am not negating the fact that this undefined in the first instance nor condoning the UB usage.
That is the observed behavior of the compiler not the defined behavior. (and while there’s some logical reason to assume that will likely continue to hold, the list is falsehoods programmers believe about undefined behavior, not observed behavior.)
(Post author here.) Agreed with the parent comment. To restate the same thing in another way in the hope of avoiding confusion:
It isn't a bug in the compiler if the compiler is non-deterministic in the presence of UB. Such behavior does not violate any guarantee provided by the compiler or language spec.
In fact, it's still not a bug if the compiler is non-deterministic, full stop. Many (most?) compilers have non-deterministic behavior due to a variety of factors. To get a sense of what's involved in getting determinism, look up some articles on why it's difficult to get reproducible builds working. It's a lot harder than it might seem at first.
Determinism is a QoI issue. It is an important QoI feature though and I think most compilers meant for production use at least strive to implement it. I guess things like parallel, distributed compilation make it quite hard.
> C compilers are pretty deterministic, same input same output, except timestamps, etc.
"Sometimes gcc will opt to use a randomized model to guess branch probabilities, when none are available from either profiling feedback (-fprofile-arcs) or __builtin_expect. This means that different runs of the compiler on the same program may produce different object code." (from the gcc manual)
I was curious about that as well. If it is just an extension of 37-40, then that makes sense - since UB may or may not effect different runs of the same executable, then certainly the same is true of different runs on different builds of the same executable. But that is a confusing way to put it - it sounds like it is saying that reproducible builds are not possible, which I am skeptical of.
And if each of those outcomes had a percentage associated with them, I suspect that most undefined behavior would cluster around less than 5 outcomes matching the expectations of even the most junior C programmer.
The bunch of amateur bards trying to mythicize undefined behavior really need to find something better to do with their and our time…
Divide by zero was one that I always believed would give segmentation fault…but that is totally not the case; I’ve gotten Infinity as a return once… I forget what compiler gives what…
I thought this was going to be an actually useful post about the details of UB, when it happens, and what it generally looks like, but instead it's just more of the same often repeated "if you write a program with UB your compiler will literally summon dragons and wipe your hard drive and kill you in your sleep."
The truth is that practice diverges from theory, and, in practice, it is actually useful to talk about UB as a thing that happens and not as some mystical aura the compiler can sense or not. Leading to, in practice, almost all of these points being, if not flat-out wrong, at least unhelpful or misleading to the average programmer. For example:
> 14. Okay, but if the line with UB is unreachable (dead) code, then it's as if the UB wasn't there.[1]
The linked post ([1]) doesn't even showcase this phenomenon, it shows something completely different, where ignoring a 3 transmuted into a bool causes more UB. The problem was that you already had UB to begin with when transmuting a 3 into a bool. I want to see a godbolt output where the claimed:
if (false) {
run_ub();
}
actually causes problems.
> 28. At least it won't completely wipe the drive.[2]
This is just unhelpful for anyone trying to learn more about UB. UB can call parts of your code that would otherwise be dead, which is a useful piece of information to know. And if you literally write "rm -rf /" in these dead sections, then yeah, I guess UB is technically wiping your drive, but stating it like that just gets in the way of anyone trying to learn more about UB and just perpetrates this unhelpful theory that UB can literally do anything it wants, such as...
> 30. At least it won't start playing Doom if the program didn't already have the Doom source code in it.
Really?
> 31-32. If a UB-containing program "worked fine" previously, recompiling the program without any code changes will still produce a binary that "works fine." Recompiling without code changes and with the same compiler and flags will produce a binary that still "works fine." Recompiling as above + on the same machine will produce a binary that still "works fine."
In practice, compilers are generally deterministic. I see no reason this should be the case. Don't link me a sadistic compiler that randomizes code output.
Finally,
> The moment your program contains UB, all bets are off.
is blatantly untrue. All guarantees are off, but I can bet what most programs with UB do, and it is still useful to be able to reason about what programs with UB do. For example, I can tell you what:
int main(int argc, char \*argv) {
int *p = NULL;
*p = 4;
}
does on O0 on my machine, and most linux machines, despite the fact that it is untrue for "Any kind of reasonable or unreasonable behavior happening with any consistency or any guarantee of any sort."
You guys are engineers right? Software engineers? Why don't you do what all engineers are supposed to do and that is use experience and intuition to know when theory applies in practice, and when it doesn't?
UB can easily wipe "the drive" on my microcontroller by sending a wrong SPI command to the flash chip, or by messing with write unprotected memory mapped built in flash.
Welcome to kernel code. I believe one could make it play DOOM by clever malice in this case.
Yeah, I never said it couldn't, I said the more helpful thing to say is that UB can cause what looks like dead code to not be dead. If that dead code includes, an rm -rf, or, say, a function to enable/disable arbitrary pins on your chip, then yes, that dead code might be called, and with undefined parameters too. But saying UB will wipe your drive is just misleading and almost wrong.
> When a C or C++ program triggers undefined behavior, anything is allowed to happen in the program execution. And by anything, I really mean anything: The program can crash with an error message, it can silently corrupt data, it can morph into a colorful video game, or it can even give the right result.
>> Falsehood #28. At least it won't completely wipe the drive.
> This is just unhelpful for anyone trying to learn more about UB. [...] And if you literally write "rm -rf /" in these dead sections, then yeah, I guess UB is technically wiping your drive, but stating it like that just gets in the way of anyone trying to learn more about UB and just perpetrates this unhelpful theory that UB can literally do anything it wants
The example is contrived, yes. But even if the source code doesn't literally contain "rm -rf /", if the program contains a buffer overflow vulnerability, then someone can craft an input to inject arbitrary code/commands into the program execution and still effect a deletion anyway. So yes, UB really does mean anything can happen.
> Undefined behavior: Anything is allowed to happen, and you might no longer have a computer left after it all happens.
and
> The moment your program contains UB, all bets are off.
Does this include that a compiler, when it can prove that a program it compiles contains UB, erase your disk as part of the compilation process? i.e. without you ever running the compiled program.
> Does this include that a compiler, when it can prove that a program it compiles contains UB, erase your disk as part of the compilation process? i.e. without you ever running the compiled program.
No, but technically compiler would not violate specification by compiling program that on running will scan your disk for secret data, publish it online, apply ransomware to your disk and deliberately destroy your SSD.
is allowed to row hammer your RAM and inspect and modify the states of other programs, but is it allowed e.g. to delete your file system through a system call?
The C specification doesn't say anything about what a compiler may or may not do during compilation. It only specifies how the compiled program should behave.
So, yes, the C specification allows a standard-conforming compiler to erase your disk during compilation. It even allows it to do that if your code doesn't contain UB.
I'm halfway through watching that Chandler Carruth video the article linked to, and I feel like I'm getting lectured at. It's making me not want to ever write c++ ever again (Unfortunately I'll have to be writing c++ for a while longer)
These days it is not quite unknowable, if you always runs your programs with the address sanitizer and UB sanitizer on. They aren't perfect but they would provide a pretty good indicator of whether there was any UB or not.
I wouldn't call some cases of UB like overflowing a signed int or running into a dangling reference "actively pushing the compiler". I'm having a hard time believing you didn't ever run into these or similar issues having "programmed for a long time".
For other cases (e.g. aliasing, alignment), I'd agree that one is rather safe as long as the dangerous tools (e.g. reinterpret_cast) are not used.
It's the same for me. Well, I've hit segfaults when dereferencing invalid pointers (including NULL). I've had some surprises with shifting and signed integers as well. But I'm not sure that I've hit strange behaviours late in development. All that stuff is just, oh, I got that wrong, let's fix it. At most - oh I hadn't known that detail about signed arithmetic.
If the compiler removes an important check or branch like in the post, and it isn't noticed in development chances are the code is dead, or not that important. And I'm not saying this to downplay the risks. A big risk is that a particular instance of UB is exploited only by a newer version of the compiler, after the code is already finished. But in general, the language remains a remarkably productive one that lets me get very close to where I want to be, very quickly.
Is it possible to know for sure that it's all correct and will continue to be correct with newer compilers? Not 100%, but maybe it's still a good tradeoff in some environments.
> I cannot say I have ever written anything that ran into or caused undefined behavior. And I have programmed for a long time.
Assuming you're working with C or C++, you've never:
- Dereferenced a pointer?
- written a loop with a signed index?
- Accessed an index in an array or a vector?
- Used a variable?
- Used an optimising compiler?
Compilers use undefined behaviour in all of the above scenarios as much as possible, and to do what you likely intended. That's not to say _all_ UB is as reasonable, but a significant amount of what you write relies on some implicit assumptions about UB for a compiler to output reasonable code.
Compilers use reasoning based on UB to optimize and simplify code. But that's not a complete argument why it's highly likely that a given code exhibits UB at run time.
In a similar vein to "dereferenced a pointer", "null check on a pointer" is one of those things that often gets optimized out without the developer realizing it.
> I cannot say I have ever written anything that ran into or caused undefined behavior. And I have programmed for a long time.
If during that long time you were writing C or C++, it's very likely that you did but you may not have attributed the bugs you were fixing to Undefined Behavior. You may have considered them either to be your bugs or "compiler bugs that you worked around."
I have written in C/C++ and gotten my fair share of null pointers or broken things but all related to logic or my own programming errors. But to me those are just that logic errors or my own crappy code errors.
Back when I worked on VAX/Alpha (I am old) I absolutely had to get down and read assembly/binary to figure out the compiler was doing what I thought. And in some cases had to work around compiler bugs.
But for most of the more recent things I have written I just have not run into these types of errors. I am really boring or I write code that is boring ha.
It is fascinating that people study this stuff and interesting to read about, I just do not run into it.
> I cannot say I have ever written anything that ran into or caused undefined behavior. And I have programmed for a long time.
Perhaps this is true, or perhaps you are just unaware of the undefined behavior you have relied on.
What I can say for sure, is that all of the non-trivial C++ programs written by a large team which I have worked on in my career have invoked some undefined behavior. Usually, the compiler does do "the right thing" despite the error, but sometimes it blows up in our face, and causes incredibly hard to debug bugs, such as a pointer appearing to simultaneously point to two different addresses, because an array was declared as two different sizes in two different translation units.
Even when I spot the instances of undefined behavior, it's often hard to convince other developers that, although this seems to behave correctly now, it could break in unexpected, hard to debug ways at any time in the future.
The biggest problem with UB, as interpreted by C/C++ compilers is the unfounded belief that the spec says that you can assume any path that leads to UB is defacto incorrect and can be ignored. That's the bogus logic required for steps 13-16 to be true.
That interpretation of the specification language is clearly bogus as that means more or less all C and C++ programs can be compiled to a single return instruction - when compiler devs introduced this new new interpretation of UB, it turned out they had to then go through and add a bunch of exceptions for basics like memset, memcpy, and memmove could work without falling afoul of their invented nonsense.
This is an absurd reduction of the current situation.
> That interpretation of the specification language is clearly bogus as that means more or less all C and C++ programs can be compiled to a single return instruction - when compiler devs introduced this new new interpretation of UB, it turned out they had to then go through and add a bunch of exceptions for basics like memset, memcpy, and memmove could work without falling afoul of their invented nonsense.
I've never heard of anything like this, have you any sources for this happening?
Theres a big difference between "the GCC mailing list is full of language lawyers" and "memcpy doesn't work because UB and the compiler Devs fixed it by adding an exception for it".
> I think aliasing rules messes up memcpy if you are dogmatic?
I'm _not_ dogmatic or a language lawyer and aliasing is a total pain in the ass, but that's a whole different topic to compilers getting fundamentals wrong because of language lawyering.
I think memcpy is fine because it operates on a special type that is allowed to alias anything. Strict aliasing has other fun issues like the impossibility of implementing malloc-like functions or the lack of any requirement for access to non-char members of structs to work at all[0].
memcpy() and memmove() are language-level functions. They are allowed to do things that cannot be implemented in user code. The C standard library's implementation of these functions have tight integration with the compiler to ensure that it can implement all the required semantics without triggering undefined behavior.
Comparing unrelated pointers is unspecified, not undefined. It's also an implementation detail of memmove. It can use a magical total order comparison of pointers that is otherwise unavailable to users of the language.
memset, memcpy, and memmove, etc all depend on UB.
memset and memcpy read and write padding bytes, which is UB. If you try to say memset/cpy is a "primitive operation", then you're still stuck with the "correct" struct initialization (memset, bzero) being UB if you did something like:
This is a necessary consequence (in the formal logic sense) of the "principle of explosion"[1] for optimizing compilers. If the compiler assumes some undefined behavior does not occur, but in fact that undefined behavior does occur, all of conclusions it makes which depend, directly or indirectly, on the contradicted assumption are invalid.
Since, in the process of optimization, optimizing compilers make conclusions about parts of a program based on other parts of the program which may be executed later, undefined behavior in one part of a program causes invalid conclusions about other parts which may be executed earlier. Therefore undefined behavior cannot be contained to just parts of a program executed after a particular part, if the compiler tries to optimize parts of the program based on parts that may be executed later.
It's surprising to see so much debate about when or where UB "happens".
Unfortunately this article doesn't refer to the best tool to root out UB - UBSan. If you write C or C++, you should have a UBSan+ASan build that is configured to terminate with an error when it encounters violations. You'll save yourself a lot of hair-pulling later. Then you don't have to wonder about the nebulous topics like what "line" the UB "happened" on.
For some compilers, when they add an optimization that capitalizes on an aspect of UB, they will also add a corresponding warning. So make sure you have warnings enabled -- and in most cases you'll want -Werror because generally no one scrutinizes the build output unless there's a failure.
I really hope that Windows Clang would soon support UBSan. It already supports ASan and is really helpful, but sometimes you need to go the extra mile to catch the most hard-to-find errors.
This is all fine and dandy, but UBSan and ASan are memory hogs that just won't fit in my single digit MB microcontroller.
So the only way to run them is too use an emulator, and that means I cannot sanitize drivers.
Yeah they are and that's too bad. HWASan is another option that reduces the overhead but would only work on a core that had MTE (which your microcontroller is not as likely to have).
Actually it is an enhanced armv7 which might work. Though officially HWASan requires armv8.
Besides that we have Zephyr stack and heap overflow detection, but it is not quite as accurate as ASan, and does not handle cases UBSan does.
gwp-asan (like electric fence) is another option for invalid access to heap-allocated buffers. Requires an MMU and creates additional TLB pressure w/some allocation overhead, but no instrumentation overhead and no access overhead.
The UBSan+ASan combo is great! Rust's miri is another good one.
This post is already fairly long (~10min read) and I didn't want to make it any longer. It's also already getting flamed in predictable ways ("nonsense! UB has to be reachable to be a problem!") so I didn't want to add more ways to get flamed for it (:
(I'm the post's author.)
> Here's the list of guarantees compilers make about the outcomes of undefined behavior:
> That's the whole list. No, I didn't forget any items. Yes, seriously.
I’m a little disappointed there isn’t an empty <ol> or <ul> in the middle there. I imagine screen readers would announce something like “list, zero items”.
Post author here. I'm very curious about the screen reader idea. If you or someone else is able to confirm what a good screen reader might do currently / what it might do with an empty list element, I'll happily update the post to whatever is the better behavior.
I wish there were a best practices doc + a matching linter for accessibility things like this.
I just tested "<ol></ol>" in VoiceOver on Mac and nothing gets read. Different screen readers are notoriously inconsistent though.
> I wish there were a best practices doc + a matching linter for accessibility things like this.
This sounds like it was meant to be a fun tweak/joke for screen readers but it's accessible without.
So it's not practical to automate a lot of accessibility checks like this one because it depends on context and there's subjectivity to it, similar to visual design and text readability. The best way to get more of a feel for this is to try a popular screen reader for your OS, do the tutorial, and try it on the page yourself.
> Different screen readers are notoriously inconsistent though.
In other words, UB!
Implementation defined
> "If my program contains UB, and the compiler produced a binary that does X, is that a compiler bug?" It's not a compiler bug.
True, if the language spec is the only thing in the universe you care about. But there are better and worse things compilers can do in the presence of UB, and the big one is warning you about it when it possibly can.
There's another big discussion on the front page about signed integer overflow in particular, and gcc screwing the developer over because it was UB when they thought it was just implementation defined:
https://news.ycombinator.com/item?id=33770277
Exactly, some things are probably UB at compile time, these IMHO should by default be a compile time error.
Integer overflow is harder ofc.
> gcc screwing the developer over because it was UB when they thought it was just implementation defined
No, the developer thought "UB" produced an integer with an undefined (but still valid integral?) value. They misunderstood what UB is, not whether or not something was UB.
That's from the spec POV. On x86 for example, signed int overflow is well defined.
Both "UB" and "IB" are terms defined in the language standards, and it's obvious that's what we're talking about here.
x86 defines what specific instructions do, nothing so crass as "signed integer overflow" - that's part of why the language standard has to define it as such.
Yeah, except that they effectively aren't part of the language standard. Possible behavior is to ignore UB, behave in a documented manner, or issue a compiler error. Not remove other code or any other nonsense optimizing compilers do.
I'm not gonna wage a holy war against optimizing compilers, but let's not pretend that this is a language standard issue.
> x86 defines what specific instructions do, nothing so crass as "signed integer overflow"
I don't understand what you mean. The x86 standard defines what happens if an `add` instruction is performed and the result overflows, possibly affecting the sign bit (what happens is that the OF flag is set, and the target register is set to the two's complement representation of the sum modulo 2^bitlength).
That's irrelevant. A C compiler targetting x86 has no obligation at all to handle signed integer overflow in the same way the processor natively does. Because it is not an assembler - it is compiling against the theoretical C virtual machine.
This is for the same sort of reason that unsigned overflow does have a well defined meaning in C, and will always do exactly that even for targets that do not natively do that (if necessary it would emulate that behaviour in software).
> A C compiler targetting x86 has no obligation
It kinda has an implied obligation to not screw with its users. And doing the same optimizations to fold bloated Cpp templates properly messes with my C code.
unfortunately for you, gcc and clang disagree with you.
Compiler developers may not think they have that obligation, but they should
Should loop-invariant code motion be an allowed optimization only when the compiler can statically prove that a for loop's increment won't overflow?
If it would result in different behavior in the event of an overflow (which I think is generally not the case), then no. A typical for loop (using <) can easily been seen to not overflow, so this is a rare case anyway. Loops using signed comparison with <= where the compiler can't rule overflow out should result in a compiler or linter warning.
The inequality argument would seem to apply only to for loops with constant iteration counts, yeah? The compiler can't know that "i < n; i++" won't overflow if "i" could start out higher than "n". (wrong)
I am not an expert on compiler optimization, but having reasoned through a few cases where a compiler is _unable_ to correctly make an obvious-seeming optimization, I tend to think that compiler writers have good reasons for the things they want to be able to assume.
If "i" started out higher than "n", the loop would exit immediately and there would be no increment. More generally, if "n" is an int, then even if n is modified each iteration, the "i < n; i++" still can't overflow because it will always exit at INT_MAX if not before. If "n" is something else like a long then that's likely a mistake that falls under "the compiler should warn if you do this".
If there were good reason for wanting to assume this, I would expect someone to have presented it; responses I hear just take the form of a single (usually contrived) example where more performant code is generated, then insisting that all other factors (e.g. being predictable/unsurprising, acting like the underlying architecture, mitigating bugs) are irrelevant. Just as GP did.
Ah good point re the inequality, over thought it.
If there weren't good reasons for making the assumptions that modern compilers do, I'd have expected users of C and C++ to have converged on a more conservative compiler, at least in some significant number.
x86 also doesn't support 64 bit integers, so would those be entirely undefined behavior or simply compile time errors?
It’s well-defined on x86, but the compiler is allowed to insert INTO instructions (which trigger an interrupt on overflow), which is effectively UB (as the interrupt routine is outside the scope of the C language).
> But there are better and worse things compilers can do in the presence of UB, and the big one is warning you about it when it possibly can.
The problem is that this is really hard to do without producing either too much or too little warnings, as undefined behaviour is usually input-triggered; and the compiler usually doesn't know all constraints on the arguments that the programmer knows about.
For example, the function f() in the article of the other discussion is perfectly well-defined for any argument x <= 0x402010, as in that case no overflow happens. The compiler doesn't know whether all callers ensure the argument is valid (e.g. this might be a library function, or the invariants might not be expressable in a way the compiler can understand). This is a lose-lose situation for the compiler: it can warn, but people will find the unnecessary warnings annoying and turn them off, while if it doesn't warn people will complain about perceivedly bad optimizations.
Of course there's a point to be made that the real problem here is that the C standard makes signed integer overflow undefined behaviour, but that's not gcc's fault and can be easily worked around with -fno-strict-overflow or -Wstrict-overflow.
> This is a lose-lose situation for the compiler: it can warn, but people will find the unnecessary warnings annoying and turn them off, while if it doesn't warn people will complain about perceivedly bad optimizations.
This could still be greatly alleaviated with the use of asserts: A compiler could default to the most conservative set of assumptions for an input value - e.g. that a parameter of a global function assumes the full value range of its type - unless the programmer provides more assumptions through an assert statement.
In debug builds, you could compile those asserts to actual runtime checks, so the program can be appropriately tested. Then, in production builds, the asserts would simply be skipped.
The second improvement would be warnings about "nonensical" statements (from the compiler's POV) that will be optimised away. E.g., many of the most egregious examples of UB are from code where the programmer inserted a bounds check to protect against UB at another location - and then the compiler optimized away that same bounds check as a "nasal demon" action, caused by the UB it was supposed to protect against. In those cases, it would be useful to warn "hey, this bounds check can never fail because that other statement implies that either we're in bounds or there is UB. You might want to check for UB here."
We're re-litigating that other thread... but the function in the other thread had a conditional (trying to check for overflow after the fact) which would never be true, regardless of x. And it could statically prove this.
And that proof, the compiler then decided to use to remove the code, ignoring the red flag that the programmer undoubtedly had tried (and failed!) to achieve something with it. I would have preferred a warning here. I don't turn off "condition is always true"-type checks.
While I agree that in general a warning about "condition is always true/false"-style checks is useful, it's also a bit tricky to do without false positives. With macros and inlining (and in C++, templates) you can easily end up with conditionals of which any given instance is always true or false, but that aren't superfluous. Portability is another example. Depending on your environment and code style hitting these might be exceptional and warning-worthy, or something that happens all over the place.
I also see this come up a lot in code generation and simd code. On the code generation it's way easier to just push all optimizations to the compiler, including constant propagation (i.e. easier to just mark an upstream boolean as constexpr true, instead of adding your own constant propagation). For simd it's common in my experience to do lots of (if size > X), where size and X are both known at compile time but might vary by target.
> there are better and worse things compilers can do in the presence of UB, and the big one is warning you about it when it possibly can.
GCC should add -Wundefined-behaviour flag that will unconditionally print "warning: this program might or not have UB" as the first diagnostic.
60% of the time, it's right every time!
It should also warn if there are infinite loops, so the dev can know whether it will ever halt.
"This program may or may not loop forever"
Seems pretty viable.
Infinite loops are undefined behavior in C++.
No, they aren’t. If they don’t make changes that are observable outside the loop, they can be assumed to terminate. ThTs a compile-time thing. https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1509.pdf (not guaranteed to be equal to the final wording of the standard):
“An iteration statement that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression-3, may be assumed by the implementation to terminate.”
Rationale for including that: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1528.htm (basically, without it compilers can’t optimize loops that terminate as well as with it)
With some exceptions. In C++ infinite loops doing IO, accessing volatile variables or atomics are not undefined.
Hence for(;;) std::this_thread::sleep_for(1s) is not UB.
C is even more lenient.
The way to think about UB is that it’s like a false premise. Anything and everything can be derived from a false premise. Compilers can end up with arbitrarily false conclusions if the program contains a false premise.
The other important thing to realize is that UB is generally a runtime condition and cannot be detected statically, in the general case. That’s usually why it’s been made UB in the first place.
This post is very good. One thing to add, is that if you code has not entered a state of UB yet, but will at some point, then all bets are off. The phenomenon is generally known as time traveling UB.
Compilers assume your code doesnt contain UB. Example:
If(x == 0) printf(“hello”); y /= x;
The compiler can assume x is not zero because it assumes the user will not cause UB, therfore the if statement and printf can be removed.
13. But if the line with UB isn't executed, then the program will work normally as if the UB wasn't there.
14. Okay, but if the line with UB is unreachable (dead) code, then it's as if the UB wasn't there.
15. If the line with UB is unreachable code, then the program won't crash because of the UB.
16. If the line with UB is unreachable code, then the program will at least stop running somehow and at some point.</i>
This is nonsense. The line with UB has to be reached for the execution to be undefined.
No it doesn't. Modern processors do out-of-order execution, so your program might be doing UB before you expect.
No, no, no. UB is a global property of your program. "All bets are off" even at compile time.
"Anything at all can happen; the Standard imposes no requirements. The program may fail to compile, or it may execute incorrectly (either crashing or silently generating incorrect results), or it may fortuitously do exactly what the programmer intended."
https://blog.regehr.org/archives/213
UB is typically a property of the execution of a program. A program may bump into UB for one set of inputs, but not for an other set.
UB is a property of the source code, or more specifically a property of how source code should be compiled into machine code. It means that there is no constraint on the machine code which will be produced if your source code has certain properties.
That is a misconception.
"If any step in a program’s execution has undefined behavior, then the entire execution is without meaning. This is important: it’s not that evaluating (1<<32) has an unpredictable result, but rather that the entire execution of a program that evaluates this expression is meaningless. Also, it’s not that the execution is meaningful up to the point where undefined behavior happens: the bad effects can actually precede the undefined operation."
I agree, but this doesn't refute what I wrote.
"If any step in a program’s execution has undefined behavior, then the entire execution is without meaning..."
Yes. So UB is a property of a given execition, not a static property of the program itself.
I don't argue that you can reason about the behavior of any part of the execution, once it evaluates operations with UB.
once and before.
If a given execution would eventually lead to UB, then it retroactively it cannot be reasoned about.
This time traveling behavior is what is confusing people in thinking that if any execution is UB then all possible executions are UB.
I think this hits the nail on the head.
Many people in this thread are confused and think that unreachable UB (either statically or dynamically unreachable) compromises the entire program. This is not true, but your comment helps me better understand how people reached this conclusion.
Reachable UB (either provably reached at compile time or dynamically reached at runtime) can retroactively invalidate the correctness of previous statements. But this does not mean that unreachable UB compromises the correctness of well-defined executions.
You are indeed correct. I would add that there are some instances of UB that actually UB at translation time: for example ODR violations and some preprocessor dark corners. At least the latter is getting stamped out.
The C++ standard calls these "ill-formed, no dioagnosrics required" instead of UB, although it's not always consistent. It is indeed more or less "translation-time UB". The difference is exactly that affected lines don't need to be in the execution path to make all executions undefined, and that the compiler/linker may fail the translation.
A program like
contains signed integer overflow in some executions but not others. Can you explain what effect the counterfactual signed integer overflow is allowed to have in executions that do not contain signed integer overflow?
Out-of-order execution isn't the reason for this. The C standard assumes an abstract machine that allows OoOE, of course, but even with strict in-order hardware UB can hit you at any time, even before you'd think it could. That is because the C standard doesn't limit UB to any constraints like "following lines" or "subsequently executed instructions". Independent of the hardware, the compiler is allowed quite a bit of reordering of instructions. The standard just requires that (some) effects of those executions are ordered as written, but that doesn't include UB.
So if you have UB in your future path of execution, the compiler might just do whatever _right now_.
It still has to be executed by the abstract machine for the given set inputs for the whole execution to be undefined.
You are right that once the execution is undefined, you can't reason about the behavior. This includes ordering of side effects, or side effects od operations occurring before the line with UB.
But uttering UB in dead code making execution UB is utter nonsense. Otherwise any execution of any program that just utters __builtin_unreachable() would be undefined.
That's sounds plausible, but isn't. First of all, out-of-order execution has nothing to do with unreachable core - it just means that reachable code sometimes gets executed in a different order than the binary shows. It is speculative execution which can lead to unreachable code actually being executed.
However, even when processors execute code speculatively, they guarantee that no observable effects of executing that code will happen. So, if you have something like `if false {*NULL=7/0}`, the processor might attempt to execute that line speculatively, but it will revert any change it made, and it will not trap regardless of any other flags.
> This is nonsense. The line with UB has to be reached for the execution to be undefined.
Check the footnote at item 14, it links to a page which shows one example of how a line with UB can be reached (basically: the optimizer can reorder the code so that parts of the line with UB will run much earlier than you'd expect).
UB "making dead code live again" is not an example of UB in dead code making the execution undefined.
Yes UB in live code can have surprising effects, including executing dead code.
Unless your code is actually commented out (actually dropped from the program), it can still (maybe) execute.
I think the main reason for this is an if(testSomeState()) { //code with UB } block being optimized to just //code with UB.
Functions which are unreachable, maybe I agree, they can't be executed (without UB somewhere in reachable code). I'm not sure that's true though - if you have UB in your hardware (not your code), you can still execute that code. This is why (at least I was taught) never to leave dead code around. There's this better thing for that called source control you may have heard of...
IF you have perfect hardware, and IF you have perfect logic/compiler and IF you have no UB in your program, then maybe you can guarantee that dead code is really dead, but I'm not really sure you want to heavily define dead code like that.
Just remove it. Especially if it has UB in it to cause more fun bugs...
If UB happens¹, anything can happen. So if the check can only happen after UB, or leads to UB, it can be removed. That isn't the case in your example – testSomeState() can return false (and if it can't, it getting optimized out is unrelated do the code it guards). Points 14-16 from the article are just nonsense, UB is a runtime concept and dead code doesn't run unless you already triggered UB¹.
¹or is guaranteed to happen in the future
I checked the footnote. It describes a case where invoking UB (transmuting "3" to a bool) can cause dead code to become live. But transmuting "3" to a bool is itself UB. So this fails to prove that execution becomes undefined without reaching a line containing UB.
That example specifically states:
> Right now, we have the fundamental principle that dead code cannot affect program behavior.
"The line with UB" becomes a very vague notion well before the program even executes. You can see this by dumping the disassembly of your program. The compiler can determine when values are defined and used and will emit code that satisfies the semantics of your program (according to the language specification). This often means that expressions within the same statement get evaluated at significantly different times, hoisted outside of loops, etc.
The assembly dump does not matter for this analysis (except for finding buggy compilers!) - the C abstract machine itself applies all side-effects before each sequence point.
You have to understand that in the modern understanding of UB as applied by compiler writers, the compiler has every right to assume the program never triggers UB. This means that lines which would be UB if a variable had certain values can be used as proof that the variable doesn't have those values, thereby eliminating checks.
For example, if a program looks like this:
It will very likely be optimized to this:
And even if bar() itself had a line checking if p was not NULL, that will get ellided as well if bar() gets inlined.
This is a good example of how UB on a later line can have an impact on earlier lines, or even in other places in the code.
Note that it's not executions which trigger UB, it is source code programs that do.
Edit: fixed const pointer syntax.
Can you just check, I don't think the line *p=1 will even compile given the definition of p
Oops, I got the const pointer syntax wrong, fixed now.
I don't think this proves the point, because foo() will not invoke UB unless it is called with a NULL argument.
It is true that when foo() is called with a NULL argument, the UB will be realized before bar() has had a chance to run (EDIT: this doesn't actually appear to be true for the given code sample, neither Clang nor GCC elide this NULL check: https://godbolt.org/z/v9v74c3jd, which makes sense as bar() could call exit()).
> Note that it's not executions which trigger UB, it is source code programs that do.
For C, I think this interpretation is directly contradicted by DR #109: https://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_109.html
> Furthermore, if every possible execution of a given program would result in undefined behavior, the given program is not strictly conforming. A conforming implementation must not fail to translate a strictly conforming program simply because some possible execution of that program would result in undefined behavior. Because foo might never be called, the example given must be successfully translated by a conforming implementation.
> It is true that when foo() is called with a NULL argument, the UB will be realized before bar() has had a chance to run. But the line with UB has already been proven reachable prior to the actual dereference, so the UB has in a sense already been invoked.
The point of those numbered lines was to refer to how people normally think about UB. Someone seeing that function may be tempted to say that it's not UB to call with a NULL value as long as the dereference doesn't happen (say, bar() could call exit() or it could run an infinite loop). What you're saying is exactly what the article is saying: the fact that a line with UB is unreachable in practice doesn't mean that it can't affect your program.
> For C, I think this interpretation is directly contradicted by DR #109: https://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_109.html
The standard talks about executions in the sense of theoretical executions by the C abstract machine, not actual executions of the compiled binary. I thought the other comment was rendering to the latter notion of execution, but perhaps I was wrong and it was also talking about the former.
> Someone seeing that function may be tempted to say that it's not UB to call with a NULL value as long as the dereference doesn't happen (say, bar() could call exit() or it could run an infinite loop).
An infinite loop is UB for precisely this reason: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1528.htm So an infinite loop cannot prevent the invocation of UB, because an infinite loop is itself UB.
I would expect that a call to exit() inside bar() would prevent eliding of the NULL check. If the program calls exit() before the dereference, it does not invoke UB and therefore should run correctly. I tried your example and Clang and GCC do not actually elide the NULL check in this case, probably for this reason: https://godbolt.org/z/cnWPz3o59
> The standard talks about executions in the sense of theoretical executions by the C abstract machine, not actual executions of the compiled binary. I thought the other comment was rendering to the latter notion of execution, but perhaps I was wrong and it was also talking about the former.
I don't see how this would change the analysis. Running the binary triggers actual executions of both the abstract machine and the physical binary.
The fact that bar could exit or not return doesn't matter (although it was indeed my initial line of reasoning) as if the value is null bar won't be called and UB happens; hence the value can't be null and bar can be called unconditionally. The value is still dereferenced only after calling bar.
Still gcc doesn't optimize the check away so it is possible there is some other line of reasoning that prevents the transformation.
Btw this is a nice example how this stuff is subtle.
My first answer was going to be that the transformation is not allowed, but then I thought about it more.
I couldn't get gcc to break it either though. Either it misses the optimization or tries to be conservative with reordering UB with side effects.
Now look at a large code base, with lots of checks for inputs. With O0 it's usually straight-forward to read, but with heavy optimizations applied it can become a real nightmare. And in that scenario all it takes is some reordering of the UB before the DCE pass can prove the code to be unreachable (which might not happen on the first DCE pass, because the value analysis/constant propagation didn't work its magic, yet). Also static program analyses can be both forward and backward.
I'm afraid it's not nonsense, that's why it's a common misunderstanding ;)
Looks like you believe in some falsehoods most programmers believe about UB. Well, you are not alone.
No, the commenter is right, at least for Rust (can't say for other languages). UB is a property of a particular execution of a program, it happens when code is executed on the abstract machine. As the blog post linked from the article (point 14, footnote 6) states:
> Right now, we have the fundamental principle that dead code cannot affect program behavior.
On C UB is a property of any code, reachable or not, just like the article's references explain.
Why are people fighting this?
And yes, Rust is much more reasonable about it.
How do you decide if i+1 is UB or not, unless the abstract machine reaches that line with a given value of i?
If you write i+1 anywhere on your code, the compiler is free to assume i is different from the maxumim value and rewrite your code with that in mind.
If you are not satisfied, you can read the article's references (there are 2 for unreacheable code). Or any article about UB here for the last 20 years.
Can someone with more knowledge chip in? If I define
it might get optimized via loop-invariant code motion to
so it could still be UB when called with p=NULL and n=0, right?
Edit: Oops, in the previous version I used a write instead of a read on *p, making the optimization invalid.
No, that transformation is straight-up buggy unless the compiler can also prove that n > 0 (ie by inlining the bar() function into a context where n is always positive).
If you had *p == 0 before calling bar(p, 0) then *p had better not be 1 after!
The first form has no undefined behavior. Since p may be NULL, in general the compiler is not allowed to make that transformation unless it can prove bar never gets called with p=NULL and n=0 (which it probably can't).
If the compiler has some special knowledge of the target architecture which makes the second form behave as if the first form was executed (such as if NULL is a valid address on the target architecture), it may make the transformation, but that still wouldn't cause undefined behavior, because it must behave as if it had the first form.
Thank you for clarifying! It's definitely relieving, after reading the post, to know that there is some limit for undefined behavior.
Yes, thank you. It's clear these points are incorrect if you consider the purpose of undefined behavior. Compilers are allowed to assume undefined behavior is never invoked (which may be useful for optimization and other reasons); it is a promise from the programmer that they will not invoke undefined behavior specifically so that the compiler may assume they don't. Things are often made undefined behavior because the assumption is useful, but hard or impossible to prove in general with static analysis, so it requires a programmer who can consider the context of the code to assure the assumption always holds. If undefined behavior is invoked, some of the compiler's assumptions are wrong, so anything could happen (see the "principle of explosion"[1]). If no undefined behavior is invoked, all of the compiler's assumptions are correct, so the program must behave correctly.
For example, imagine a C programmer writes a function that adds two `int` values. Hypothetically, with some inputs, the addition may overflow. That would be undefined behavior, so the compiler assumes the function is never called with such inputs, and optimizes accordingly. The programmer knows this, and assures the function is never called with those inputs. Since the only assumption the compiler made was that the function wasn't called with input which would cause the addition to overflow, and the programmer assured that assumption is correct, the function always behaves correctly.
Consider if the same function's inputs were provided at runtime from stdin. The behavior of the whole program is undefined if invalid inputs are given, but if only valid inputs are given, the program must behave correctly.
[1]: https://en.wikipedia.org/wiki/Principle_of_explosion
Everyone's getting confused about the word "reach". If the execution of a program triggers UB at any point then its behavior is undefined at all points, even before the UB was triggered. But the compiler does have to respect and preserve the behavior of any completely UB-free execution of your program, even if other executions (say with different input) could trigger UB.
So we could say, unreachable UB is fine, but if you do reach it, then it doesn't matter when you reached it.
Point 14 explicitly says UB in dead code.
The implicit assumption is no UB in live code, otherwise the UB in dead code is a red herring.
So the UB is never reached, by the definition of dead code.
Unreachable UB is fine.
If this wasn't the intended message of point 14, then it's phrased wrong.
Yeah the format of the article makes this mistake extra confusing. The statement in point 14 is in fact true, but the whole premise is that all these statements are supposed to be false.
> 14. Okay, but if the line with UB is unreachable (dead) code, then it's as if the UB wasn't there.
This one came as a surprise to me. Is this really (unconditionally) true? And if so, why? Seems trivial to me to ensure that UB that will never under any circumstance execute will not affect the functioning of an otherwise correct program.
No, it isn't. The line with UB has to be reached in the execution for the execution to be undefined.
Yes, but there is more to it: The compiler may assume that UB will never happen intentionally and optimize out the branch containing UB.
The compiler isn't required to know if code is unreachable. For example, `if (fermats_last_theorem_is_wrong()) {*p == 1} ` will imply to a modern compiler that p is not NULL in the scope where it is defined, even if we now know that this code will never be reached (assuming that function is correctly written).
Or, here is a better example:
Here the compiler will assume that the loop terminates, since non-terminating loops without side-effects are UB in C++.
Then, since the loop terminates, p will be dereferenced, so it can be assumed that p is never NULL, so any NULL checks for p in this context can be ellided.
This remains valid even if you call this function with an n such that *p=1 is unreachable.
If a some translation unit defines int fermats_last_theorem_is_wrong(void) { return 0; } then a well-defined program can execute that sequence even with p equal to NULL.
The compiler is required to successfully translate a program that contains unreachable UB, and such a program is conforming if undefined behavior is never reached. This is explicitly spelled out in https://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_109.html
> Furthermore, if every possible execution of a given program would result in undefined behavior, the given program is not strictly conforming. A conforming implementation must not fail to translate a strictly conforming program simply because some possible execution of that program would result in undefined behavior. Because foo might never be called, the example given must be successfully translated by a conforming implementation.
Sad to see this downvoted here.
To hammer it in, the consider the following:
if(p) *p = 0;
*p is undefined if p is NULL. But as long as that statement is never reached when p is NULL that is not a problem. That is trivially true in this case, but the same applies even for more complex relations between the conditional and the undefinedness of the statement.
Or put another way, in
the compiler is not allowed to remove the if(p) checks and unconditionally print that fermats last theorem is wrong unless it can prove that fermats_last_theorem_is_wrong() is true.
In fact, something having undefined behavior and the compiler being allowed to assume that it is unreachable are effectively the same thing. Which is why the spec for std::unreachable() and the compiler-specific intrinsics that preceded it is that it invokes undefined behavior and not more complex wording.
Yes, my example with Fermat's last theorem was completely wrong, not sure why it seemed plausible to me when I wrote it, it's obviously just a guard, and it obviously is not UB to have a pointer write behind an If...
Presumably (and really don't take my word for this - I'm a Java bod) the p dereference needs to come before the p NULL check for the p NULL check to be ignored. (Otherwise, how can a NULL check protect a dereference.) And therefore, the NULL check would be unreachable as well, and therefore never executed and not relevant.
Please correct my misunderstanding...
Nope, the C spec is just really bad. The NULL check is allowed to be removed.
So the pseudocode:
is UB? That's crazy, but not actually unbelievable.
I am not certain of the language lawyer side of it, but no reasonable (or even moderately unreasonable) compiler would omit that check because of the guarded behavior; way too much code would break, probably including that compiler's standard library.
The poster is just completely wrong. A compiler is required to emit the branch in this case.
There's no misunderstanding on your part, just some muddled thinking on my part for the first example.
> For example, `if (fermats_last_theorem_is_wrong()) {*p == 1} ` will imply to a modern compiler that p is not NULL in the scope where it is defined, even if we now know that this code will never be reached (assuming that function is correctly written).
No it won't. Otherwise all precondition checks would be meaningless. The compiler can only assume that it isn't null when fermats_last_theorem_is_wrong() is true. If the compiler cannot reason about fermats_last_theorem_is_wrong() then it can't just ignore it.
> for(int i=0; i > n;) { } *p=1;
> Here the compiler will assume that the loop terminates, since non-terminating loops without side-effects are UB in C++.
Infinite loops are UB precisely because otherwise the compiler would NOT be able to reason about this. So yes, here the comiler can assume that p is a valid pointer - but only because it can also assume that *p=1 is NOT dead. This falls entirely under the umbrella of behavior not being defined at all after you execute undefined behavior (the infinite loop).
> This remains valid even if you call this function with an n such that *p=1 is unreachable.
Only because the loop itself is undefined behavior in that case.
That is not an infinite loop, it has an exit condition on i > n.
An infinite loop is not UB. It is well defined. Just the definition of a terminating loop is a bit funny.
> An iteration statement whose controlling expression is not a constant expression, that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression, may be assumed by the implementation to terminate.
Plus:
> An omitted controlling expression is replaced by a nonzero constant, which is a constant expression.
The compiler may only use as-if rule here if it can deduce the value is constant or limited. It can (but does not have to) assume the loop will end due to the conditional and lack of the other features. Which is still defined behavior. (since C11 at least, probably earlier)
> An infinite loop is not UB. It is well defined.
From C11:
> An iteration statement whose controlling expression is not a constant expression,156) that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression-3, may be assumed by the implementation to terminate.157)
> 157)This is intended to allow compiler transformations such as removal of empty loops even when termination cannot be proven.
This makes infinite loops UB unless they fall under one of these exceptions (constant controlling expression, performs I/O, accesses volatile objects or atomics).
Not particularly. It does not allow the compiler to assume true infinite loops can terminate.
It only allows the compiler to assume loops with non-constant expressions can terminate. (And a bunch of other exceptions where it cannot.) A true infinite loop has a constant expression as the conditional and cannot be optimized away based on this rule.
What this rule allows the compiler to do is to optimize away loops where number of iterations are known ahead of time if it has a non-constant expression condition.
> It only allows the compiler to assume loops with non-constant expressions can terminate. (And a bunch of other exceptions where it cannot.)
That is basically what I said: loops can be assumed to terminate, except for an enumerated set of exceptions where infinite loops are allowed.
You seem to be defining "true infinite loop" to mean "while(1)" or "for(;;)" exclusively. But "for (int i = 0; i < 1;)" is also an infinite loop, and that one is UB since the controlling expression is not constant.
> What this rule allows the compiler to do is to optimize away loops where number of iterations are known ahead of time if it has a non-constant expression condition.
If the number of iterations are known ahead of time, then it is not an infinite loop so I don't see how the rule would apply.
The rationale for making infinite loops UB is described here: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1528.htm
Specifically, it allows for transformations like turning:
into:
Infinite loop with a constant expression is defined behavior.
for (;;) { }; cannot be assumed to have finite number of executions. Therefore, it cannot be optimized away to finite execution.
You're showing potentially infinite loops with non-constant expression as condition.
for (; i != 0; ) {} can be assumed to have finite number of executions and can be optimized unless i is constant expression.
> Infinite loop with a constant expression is defined behavior.
Yes, because a constant controlling expression is one of the enumerated exceptions to the rule against infinite loops.
> You're showing potentially infinite loops with non-constant expression as condition.
"for (int i = 0; i < 1;)" is an infinite loop, full stop. It is not potentially infinite, it is infinite under all possible executions. It is UB because it does not fall under one of the enumerated exceptions where an infinite loop is allowed.
I think we are saying the same thing, the only difference is that you seem to be defining "infinite loop" to only include loops with constant controlling expressions, whereas I am defining it to mean "any loop that does not terminate."
Yes, the part about Fermat's last theorem I wrote is actually obviously wrong, I don't know what exactly I was thinking when I wrote it.
The example with the loop though is more to point out that our native understanding of "unreachable" may be different than the compiler's. The same problem could occur without invoking UB if we un-conditionally generate a SIGKILL to our own process and then dereference a NULL pointer - the compiler doesn't know what SIGKILL does, so it doesn't know the UB is unreachable in practice, so it may reorder etc.
> the compiler doesn't know what SIGKILL does, so it doesn't know the UB is unreachable in practice, so it may reorder etc
It is the reverse. If the compiler doesn't know what raise(SIGKILL) (or really, any other function it cannot see through) does, it has to assume that it can potentially never terminate, abort or otherwise change the control flow, so it can't safely reorder across it. And even if it does return, it could still inspect memory and observer the violation of the as-if rule.
That's why for example pthread_mutex_lock/unlock worked correctly for the most part even before C++ got a proper concurrent memory model.
That (or more general 13-16) also does not make sense to me. Most UB (although perhaps not all, so it may be true for some obscure UB, or ones with constant arguments) are undefined for specific dynamic conditions, so it is not 'line with UB' but more 'line with UB under specific program state', e.g. expression 'a / b;' has UB if b == 0, 'a->item' has UB if b == NULL, or 'a * 4' has UB if a is int and large enough to cause overflow. If the expression is not executed, the dynamic conditions are not there, and therefore there is no UB.
FYI, an ODR violation (https://en.wikipedia.org/wiki/One_Definition_Rule) is an example of non-dynamic UB.
The compiler may make optimization decisions based on the presence of UB so the code only needs to be present.
The compiler can't know that an arbitrary part of a function is unreachable if, for instance, that code path is controlled by a parameter or global state since that equates to solving the halting problem.
https://blog.llvm.org/2011/05/what-every-c-programmer-should...
I don't think this is correct. If UB is never invoked, the program is conforming and must be translated/executed correctly.
DR #109 specifically states that a compiler may not fail to translate a strictly conforming program just because some possible executions could trigger UB: https://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_109.html
Edit: downvoters, please explain your disagreement. I cited a primary document in support of my position.
The undefined behavior can still affect code generation even though the program gets translated.
It's just that the generated code may not do exactly what you expect it to do because the presence of undefined behavior allowed the compiler to make assumptions which may surprise you.
edit: It seems that I am incorrect
DR 109 allows that a program may be strictly conforming even if some possible executions of the program invoke UB:
> A conforming implementation must not fail to translate a strictly conforming program simply because some possible execution of that program would result in undefined behavior.
This text specifically allows for the case that a program is strictly conforming even if there is a possible execution that invokes UB. If a program is strictly conforming, it must produce the correct behavior.
I would challenge you to show an example where GCC or Clang will break the correctness of a program on account of UB that is not reached during program execution.
Here is an example where GCC and Clang specifically respects the correctness of a program as long as it does not reach the UB: https://godbolt.org/z/befWah77W
It seems that you may be correct.
At the very least I am unable to construct a counter-example so I concede this point
This [0] is the linked article for that point, I don’t do systems programming, so I can’t say I understand if this is trivial or not ;)
[0]: https://www.ralfj.de/blog/2020/07/15/unused-data.html
The linked article argues that making storing invalid values in the bool undefined (rather than just using such bools) is important because otherwise the compiler is not allowed to move the code (without adding appropriate checks that it would have been reached). The compiler can only make that transformation if it has no effect on any cases where the code wasn't reached at all, which includes not introducing any new UB in those cases. Predrag conclusion that this means having UB in dead code matters is simply wrong.
The blog post does not support the assertion that unreachable UB which is not executed is a problem. It starts by saying
> The Rust compiler has a few assumptions that it makes about the behavior of all code. Violations of those assumptions are referred to as Undefined Behavior.
Then it says
> In other words, even just constructing, for example, an invalid bool, is Undefined Behavior—no matter whether that bool is ever actually “used” by the program.
The blog post is talking about how just creating an invalid/trap/niche representation for a value is UB. The act of creating the value is the UB here, not the usage. So, in this case the UB is most definetely reachable and executing (The act of creating the invalid value).
Code that represents UB doesn't have to be executed for the compiler to react to it.
More specifically, the linker, not the compiler.
The compiler has to respect DR 106. The linker, however, is allowed to write garbage into code.
the standard does not distinguish the linker from the compiler.
I have another falsehood: most programmers believe that people working with C/C++ are constantly battling against UB all day long, five days a week, and that they are irresponsable and careless. That they should live in fear and terror, because "unsafe" is around the corner, and that their programs are not mathematically proven and that they might containt an explotaible bug because, you know, every other C/C++ program opens a socket to the internet or is a "sudo" kind of tool.
An automobile recall is a good analogy here.
Most people driving cars with an active recall are unaffected by the problem the recall is supposed to address. They can drive on blissfully unaware that something could go wrong. They might even enthusiastically recommend that others purchase the car they drive.
For a few people, they'll experience a failure, possibly a dangerous one.
And this is the root of the problem. The fundamental concept of using a language prone to UB exposes all programs to a small risk. In isolation, the risk is small. In aggregate, the problem is real.
The same applies to vaccines and medicine drugs. Should we stop giving them to people? "For a few people, they'll experience a failure, possibly a dangerous one.".
Should we try our best to minimize the risks? Absolutely. But we are talking programs here. We shouldn't measure every program with the same ruler. Not all programs need to be MISRA compliant when they don't need to.
My Reddit app crashes several times a day, and I guess there is no immediate danger.
There are no viable alternatives to vaccines and medicine in general (ones that actually work, that is). There are alternatives to C -- for most cases.
> My Reddit app crashes several times a day, and I guess there is no immediate danger.
That's because your app is heavily sandboxed and probably (on Android, at least) running in a VM which enforces memory safety and isolation from other apps.
If each crash had, say, a 1% likelihood of eating a large portion of your data (or leaking pictures or whatever) you'd probably care a lot more about it.
The problem highlighted by the parent poster is that all these little risks add up and often end up being catastrophic (e.g. heartbleed) because so much of what we run is interconnected via the network (or just data in files).
It can be true that most programmers working in C or C++ are spending most of their time solving real problems that are still most effectively solved by C or C++, and at the same time, that even the absolute top C and C++ programmers regularly misapprehend UB itself or whether their own code contains UB in a way that causes a serious issue for the program.
It does seem odd to just "lump unspecified behavior and implementation-defined behavior together" in a side-note under the heading of "implementation-defined behaviour", when that term has a precise technical meaning in the C and C++ specifications.
I think that note just means "this post isn't about unspecified or implementation defined behavior". I'm sure you could write articles about each of those.
Post author here, and yes this ^ is precisely what it's intended to mean. There are also other more exotic flavors of behavior, and the post isn't about those either.
I tried to cover as much ground in the post as possible, but the post is already a 10min read and covering unspecified / impl-defined behavior would have made it a 20min read instead :)
I did get that eventually, but I missed the side-note in my first read through, and it was lumping two concepts together under the name of one of them that I found confusing.
If, in your description of the three buckets, you called the middle one something else like "Platform-dependent behaviour" or whatever, I think that would have been less weird. You'd still only need one short paragraph to say it's compiler/OS/hardware dependent, covers implementation-defined and unspecified, and is not the focus of the article.
I have a suggestion - it may be completely off the wall, but hear me out.
C needs a mechanism to declare in a source file that certain behaviour must be defined, where that behaviour is undefined in the C standard but the target architecture behaves the defined way. Then, when the source is compiled on the target architecture it works as expected, but when compiled on something else it refuses to compile.
For example, unsigned ints. It'd be great to be able to declare "I define int to be 32 bits and to overflow like a sensible 32-bit int - treat this as defined behaviour, and refuse to compile on a system where this isn't true." The number of platforms where this would fail to compile would be extremely small, and at least they would give a sensible compile error on systems that are weird.
I have almost no idea about C compilation but would it be possible with macros or somehow redefining int?
I don't think it'd be possible to get the compiler to treat unsigned integer overflow as defined using macros. But I'm not a subject matter expert.
Not cleanly, but if you assume 2s complement and wrap all signed addition in a macro function you could cast to unsigned before the add and cast the result back to signed.
Suggestions like this are unfortunately quite infeasible (they fall under the hat of "define all the behaviors"), and the talk I linked in the post goes into why: https://www.youtube.com/watch?v=yG1OZ69H_-o
(I'm the post's author.)
He's not trying to define all behaviours, he wants a static assertion that the compiler will compile certain behaviours in a specific way. Seems very different.
That's why I said "falls under the hat of" :) I promise the talk is relevant.
`-fwrapv` is an attempt toward doing such a thing for signed integer overflow specifically. I bet it's a pain to maintain because it significantly increases the surface area of the compiler that needs to be tested.
I don't believe doing this for more cases is feasible at the level of the compiler. Things like CHERI where the hardware and OS help are much more viable, in my book: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/
Signed integer overflow flag is relatively painless to maintain, because compiler needs to be able to track it for unsigned integers anyway. That is, signed integer was modeled with a pair of flags (interpret bits as signed, can't overflow) and unsigned integer (interpret bits as unsigned, wrap on overflow), so signed integer under -fwrapv is just (interpret bits as signed, wrap on overflow).
I think it's pretty clearly feasible in the specific example GP gave, since -fwrapv does this for GCC and clang. In principle it should be do-able from code level with e.g. 'pragma gcc optimize("frapv")' (but the gcc docs for this say that this feature is for debugging only and doesn't always work).
"goes into". https://youtu.be/yG1OZ69H_-o?t=1039 You mean this slide of the Joker with title "Do I really look like the kind of language that would define its behavior?"
As far as I can tell there is no reason LLVM couldn't compile a document like "On x86 division by zero throws a #DE exception. On ARM division by zero returns zero. On x86 shifting by more than 32 bits shifts by (cnt mod 32) bits. On ARM shifting by more than 32 bits clears all bits. [...]" that specifies all undefined behavior. The current "nasal demons" behavior is preferred in the name of performance on some SPEC benchmarks that nobody was asking for, screwing OS writers and anyone who expected sane language semantics.
Could you elaborate on 13-16? I have no idea what you could possibly mean, and the linked blog post doesn't help. As far as I know UB needs to execute for it to apply and make a program invalid. The linked rust blog post is about how in rust, creating some values in and of itself is UB (It's not that there is unreachable UB later on that uses the invalid value, it's that there was already UB executed earlier in the program in the creation of the invalid value). I know there are some "global" UBs that do not need to "execute" at runtime, but those can simply be thought of as executing at compile time.
That mechanism exists: inline asm.
As for your example, unsigned overflow is already defined in C and C++ and you can force the compiler error for non-32-bits using a static_assert.
Ada already does something like this. You can choose whether to enable runtime checks, fail to compile, or accept the risk.
The purpose of C is to be easy to implement on new systems. You're never going to get a lot of traction adding anything to C proper.
> Ada already does something like this. You can choose whether to enable runtime checks, fail to compile, or accept the risk.
It is ok, the accelerometer readings won't go outside the range of an int.
I think it's silly to tell people what to do when you don't know what they're doing. For all you know, they could be generating code from a design tool that already does that static analysis.
What's nice about Ada is you have choices, and the ecosystem facilitates creating and using such tooling.
I wasn't trying to say anything deep. It was just a reference to the Arianne 5 disaster.
I know what you're referencing, but it's not obvious to people who don't already know about it.
And in a way, I think you did say something deep. Web developers often have no idea that problems like this were solved a long time ago, and it lowers their expectations for what should be possible with the tools of their industry.
If the program knew the intention of the programming we wouldn't need programmers to begin with. Undefined behavior may be obvious to the architect but not to the architecture.
Another falsehood programmers believe about UB:
- Compilers and language designs could not improve on this situation without sacrificing performance or portability.
security, it's important
UB is not allowed in a constant expression in c++, which is a good reason to put constexpr in front of as many things as you can.
You need to actually invoke at compile time your constexpr function with the specific arguments that would cause UB to actually get an error.
It is not significantly better than running your test cases with with UBsan but it is a nice feature.
Good point. Also UBsan and Asan are great additions to any C/C++ workflow. But if it's actually part of the code, I'm forced to use it even if I'm lazy, versus an external tool that I can forget to run (or forget to add to my build system).
Leveraging what the compiler gives you is like that saying "the best camera is the one you have on you".
Good point.
I love constexpr because it allows me to write compile time unit tests inline with the library code, with static_assert. The implied UB check is gravy.
While theoretically true, there are a whole bunch of undefined behaviors that are not caught in constexpr context in practice. In particular unsequenced operations on the same object are typically not caught.
It's still super useful though, and more common language UBs are caught.
That's a compiler bug though, right? Doesn't the standard disallow UB in constexpr?
It is. Or if it is unfeasible to implement then it can be considered to be a defect in the standard.
Consider that something like (++*ptr1) + (++*ptr2) may or may not be UB, depending where the pointer points to. The required additional bookkeeping might make compile time evaluation unpractically slow.
Alternatively the sequencing could be always defined between any two subexpressions within constexpr context.
My least favorite thing about "falsehoods X believe about Y" articles is that they make a point out of not backing up or demonstrating any of their claims.
Fortunately, this post includes a link to an excellent 45min talk on undefined behavior which does go into a fair bit of the detail you were looking for. I encourage you to click it -- it's right in the post's introduction.
The purpose of this post and other "falsehoods" posts is to shine a light on the existence of a misconception or set of misconceptions. By that point, there are usually plenty of other high-quality posts about "why / how / when" etc. but for a variety of social dynamics reasons those posts by themselves did not have the wide distribution and readership they deserved. After shining a light on the misconception, interested readers are generally able to continue the learning process both through the sparked discussion and through doing their own searching.
Here's a specific example of such a post with even more details: https://blog.llvm.org/2011/05/what-every-c-programmer-should...
It is not explained why points 31 to 36 are valid, unlike HDL synthesis tools, C compilers are pretty deterministic, same input same output, except timestamps, etc.
If an UB adopt X behavior one time, a reproducible build will take same X behavior the next time.
Of course I am not negating the fact that this undefined in the first instance nor condoning the UB usage.
That is the observed behavior of the compiler not the defined behavior. (and while there’s some logical reason to assume that will likely continue to hold, the list is falsehoods programmers believe about undefined behavior, not observed behavior.)
"but it worked fine before" talks about the observed behavior
Yes. That is the sub-category of “among all the falsehoods about UB, these are the falsehoods that can be categorized as [improper reliance on] OB”.
(Post author here.) Agreed with the parent comment. To restate the same thing in another way in the hope of avoiding confusion:
It isn't a bug in the compiler if the compiler is non-deterministic in the presence of UB. Such behavior does not violate any guarantee provided by the compiler or language spec.
In fact, it's still not a bug if the compiler is non-deterministic, full stop. Many (most?) compilers have non-deterministic behavior due to a variety of factors. To get a sense of what's involved in getting determinism, look up some articles on why it's difficult to get reproducible builds working. It's a lot harder than it might seem at first.
Determinism is a QoI issue. It is an important QoI feature though and I think most compilers meant for production use at least strive to implement it. I guess things like parallel, distributed compilation make it quite hard.
> C compilers are pretty deterministic, same input same output, except timestamps, etc.
"Sometimes gcc will opt to use a randomized model to guess branch probabilities, when none are available from either profiling feedback (-fprofile-arcs) or __builtin_expect. This means that different runs of the compiler on the same program may produce different object code." (from the gcc manual)
Looked into this and it appears it was not the case; the docs were confusingly written, and the docs appear to have been rewritten some time ago
https://marc.info/?l=gcc&m=102068997020453&w=2
I was curious about that as well. If it is just an extension of 37-40, then that makes sense - since UB may or may not effect different runs of the same executable, then certainly the same is true of different runs on different builds of the same executable. But that is a confusing way to put it - it sounds like it is saying that reproducible builds are not possible, which I am skeptical of.
And if each of those outcomes had a percentage associated with them, I suspect that most undefined behavior would cluster around less than 5 outcomes matching the expectations of even the most junior C programmer.
The bunch of amateur bards trying to mythicize undefined behavior really need to find something better to do with their and our time…
Divide by zero was one that I always believed would give segmentation fault…but that is totally not the case; I’ve gotten Infinity as a return once… I forget what compiler gives what…
Javascript?
Ah, I did not specify; with GCC… I believe it was compiling as C vs C++
My experience is that you get SIGFPE for division by zero errors.
I thought this was going to be an actually useful post about the details of UB, when it happens, and what it generally looks like, but instead it's just more of the same often repeated "if you write a program with UB your compiler will literally summon dragons and wipe your hard drive and kill you in your sleep."
The truth is that practice diverges from theory, and, in practice, it is actually useful to talk about UB as a thing that happens and not as some mystical aura the compiler can sense or not. Leading to, in practice, almost all of these points being, if not flat-out wrong, at least unhelpful or misleading to the average programmer. For example:
> 14. Okay, but if the line with UB is unreachable (dead) code, then it's as if the UB wasn't there.[1]
The linked post ([1]) doesn't even showcase this phenomenon, it shows something completely different, where ignoring a 3 transmuted into a bool causes more UB. The problem was that you already had UB to begin with when transmuting a 3 into a bool. I want to see a godbolt output where the claimed:
actually causes problems.
> 28. At least it won't completely wipe the drive.[2]
This is just unhelpful for anyone trying to learn more about UB. UB can call parts of your code that would otherwise be dead, which is a useful piece of information to know. And if you literally write "rm -rf /" in these dead sections, then yeah, I guess UB is technically wiping your drive, but stating it like that just gets in the way of anyone trying to learn more about UB and just perpetrates this unhelpful theory that UB can literally do anything it wants, such as...
> 30. At least it won't start playing Doom if the program didn't already have the Doom source code in it.
Really?
> 31-32. If a UB-containing program "worked fine" previously, recompiling the program without any code changes will still produce a binary that "works fine." Recompiling without code changes and with the same compiler and flags will produce a binary that still "works fine." Recompiling as above + on the same machine will produce a binary that still "works fine."
In practice, compilers are generally deterministic. I see no reason this should be the case. Don't link me a sadistic compiler that randomizes code output.
Finally,
> The moment your program contains UB, all bets are off.
is blatantly untrue. All guarantees are off, but I can bet what most programs with UB do, and it is still useful to be able to reason about what programs with UB do. For example, I can tell you what:
does on O0 on my machine, and most linux machines, despite the fact that it is untrue for "Any kind of reasonable or unreasonable behavior happening with any consistency or any guarantee of any sort."
You guys are engineers right? Software engineers? Why don't you do what all engineers are supposed to do and that is use experience and intuition to know when theory applies in practice, and when it doesn't?
[1]: https://www.ralfj.de/blog/2020/07/15/unused-data.html
[2]: https://kristerw.blogspot.com/2017/09/why-undefined-behavior...
UB can easily wipe "the drive" on my microcontroller by sending a wrong SPI command to the flash chip, or by messing with write unprotected memory mapped built in flash.
Welcome to kernel code. I believe one could make it play DOOM by clever malice in this case.
Yeah, I never said it couldn't, I said the more helpful thing to say is that UB can cause what looks like dead code to not be dead. If that dead code includes, an rm -rf, or, say, a function to enable/disable arbitrary pins on your chip, then yes, that dead code might be called, and with undefined parameters too. But saying UB will wipe your drive is just misleading and almost wrong.
We need a doomcc that will replace all undefined behavior with doom. A more fun version of ubsan if you will.
Thanks for inadvertently realizing my prediction.
> When a C or C++ program triggers undefined behavior, anything is allowed to happen in the program execution. And by anything, I really mean anything: The program can crash with an error message, it can silently corrupt data, it can morph into a colorful video game, or it can even give the right result.
-- https://www.nayuki.io/page/undefined-behavior-in-c-and-cplus...
>> Falsehood #28. At least it won't completely wipe the drive.
> This is just unhelpful for anyone trying to learn more about UB. [...] And if you literally write "rm -rf /" in these dead sections, then yeah, I guess UB is technically wiping your drive, but stating it like that just gets in the way of anyone trying to learn more about UB and just perpetrates this unhelpful theory that UB can literally do anything it wants
The example is contrived, yes. But even if the source code doesn't literally contain "rm -rf /", if the program contains a buffer overflow vulnerability, then someone can craft an input to inject arbitrary code/commands into the program execution and still effect a deletion anyway. So yes, UB really does mean anything can happen.
> Undefined behavior: Anything is allowed to happen, and you might no longer have a computer left after it all happens.
and
> The moment your program contains UB, all bets are off.
Does this include that a compiler, when it can prove that a program it compiles contains UB, erase your disk as part of the compilation process? i.e. without you ever running the compiled program.
> Does this include that a compiler, when it can prove that a program it compiles contains UB, erase your disk as part of the compilation process? i.e. without you ever running the compiled program.
No, but technically compiler would not violate specification by compiling program that on running will scan your disk for secret data, publish it online, apply ransomware to your disk and deliberately destroy your SSD.
Also known as "nasal demons"
What about the C programs? I guess the program
is allowed to row hammer your RAM and inspect and modify the states of other programs, but is it allowed e.g. to delete your file system through a system call?
Is it program with UB?
If yes, then compiler can compile program doing literally anything* and be in compliance with standards as defined.
*including hacking Pentagon to order drone strike on location where it was compiled.
Or summoning demons.
Or deleting your file system through a system call.
The C specification doesn't say anything about what a compiler may or may not do during compilation. It only specifies how the compiled program should behave.
So, yes, the C specification allows a standard-conforming compiler to erase your disk during compilation. It even allows it to do that if your code doesn't contain UB.
What if the UB is in constexpr function used during the compilation?
UB during constexpr evaluation at actual compile time is promoted to ill defined behaviour requiring a diagnostic.
The Haskell people would say that the compiler may decide to launch the missiles.
In practice, it means all bets are off, and yes, depending on your code, you may lose some data.
I'm halfway through watching that Chandler Carruth video the article linked to, and I feel like I'm getting lectured at. It's making me not want to ever write c++ ever again (Unfortunately I'll have to be writing c++ for a while longer)
> If my program contains UB, and the compiler produced a binary that does X, is that a compiler bug?
UB is (usually) input dependent as well. So shouldn't this be phrased as
> If the compiler produced a binary that on some input leads to UB and does X, is that a compiler bug?
> usually
I think this is their point. :)
I guess the thing about dragons was true; it's not mentioned in this list.
Still no warnings when the compiler does something unexpected on an UB. Major compiler bug in most implementations.
What? Undefined behavior means the compiler guarantees nothing, per definition.
I am always amazed by articles like these. This guy thoroughly covers this topic.
I cannot say I have ever written anything that ran into or caused undefined behavior. And I have programmed for a long time.
Is this just an interesting topic or do people actively push the compiler to see what it will do? Maybe I am a boring/simple programmer. :)
> I cannot say I have ever written anything that ran into or caused undefined behavior.
But how would you know? :)
These days it is not quite unknowable, if you always runs your programs with the address sanitizer and UB sanitizer on. They aren't perfect but they would provide a pretty good indicator of whether there was any UB or not.
You’ve never had an integer overflow? You’ve never dereferenced a null pointer?
I suspect every C programmer invokes undefined behavior in their first 50 hours of writing C.
It's quite safe to say that every hand-off between code written by different developers can exercise UB in a few different ways.
I wouldn't call some cases of UB like overflowing a signed int or running into a dangling reference "actively pushing the compiler". I'm having a hard time believing you didn't ever run into these or similar issues having "programmed for a long time".
For other cases (e.g. aliasing, alignment), I'd agree that one is rather safe as long as the dangerous tools (e.g. reinterpret_cast) are not used.
It's the same for me. Well, I've hit segfaults when dereferencing invalid pointers (including NULL). I've had some surprises with shifting and signed integers as well. But I'm not sure that I've hit strange behaviours late in development. All that stuff is just, oh, I got that wrong, let's fix it. At most - oh I hadn't known that detail about signed arithmetic.
If the compiler removes an important check or branch like in the post, and it isn't noticed in development chances are the code is dead, or not that important. And I'm not saying this to downplay the risks. A big risk is that a particular instance of UB is exploited only by a newer version of the compiler, after the code is already finished. But in general, the language remains a remarkably productive one that lets me get very close to where I want to be, very quickly.
Is it possible to know for sure that it's all correct and will continue to be correct with newer compilers? Not 100%, but maybe it's still a good tradeoff in some environments.
> I cannot say I have ever written anything that ran into or caused undefined behavior. And I have programmed for a long time.
Assuming you're working with C or C++, you've never:
- Dereferenced a pointer? - written a loop with a signed index? - Accessed an index in an array or a vector? - Used a variable? - Used an optimising compiler?
Compilers use undefined behaviour in all of the above scenarios as much as possible, and to do what you likely intended. That's not to say _all_ UB is as reasonable, but a significant amount of what you write relies on some implicit assumptions about UB for a compiler to output reasonable code.
Compilers use reasoning based on UB to optimize and simplify code. But that's not a complete argument why it's highly likely that a given code exhibits UB at run time.
I think they meant "never written UB"? Which would still be a bold claim, much like "I never write bugs" (only features :P).
In a similar vein to "dereferenced a pointer", "null check on a pointer" is one of those things that often gets optimized out without the developer realizing it.
> I cannot say I have ever written anything that ran into or caused undefined behavior. And I have programmed for a long time.
If during that long time you were writing C or C++, it's very likely that you did but you may not have attributed the bugs you were fixing to Undefined Behavior. You may have considered them either to be your bugs or "compiler bugs that you worked around."
I have written in C/C++ and gotten my fair share of null pointers or broken things but all related to logic or my own programming errors. But to me those are just that logic errors or my own crappy code errors.
Back when I worked on VAX/Alpha (I am old) I absolutely had to get down and read assembly/binary to figure out the compiler was doing what I thought. And in some cases had to work around compiler bugs.
But for most of the more recent things I have written I just have not run into these types of errors. I am really boring or I write code that is boring ha.
It is fascinating that people study this stuff and interesting to read about, I just do not run into it.
> I cannot say I have ever written anything that ran into or caused undefined behavior. And I have programmed for a long time.
Perhaps this is true, or perhaps you are just unaware of the undefined behavior you have relied on.
What I can say for sure, is that all of the non-trivial C++ programs written by a large team which I have worked on in my career have invoked some undefined behavior. Usually, the compiler does do "the right thing" despite the error, but sometimes it blows up in our face, and causes incredibly hard to debug bugs, such as a pointer appearing to simultaneously point to two different addresses, because an array was declared as two different sizes in two different translation units.
Even when I spot the instances of undefined behavior, it's often hard to convince other developers that, although this seems to behave correctly now, it could break in unexpected, hard to debug ways at any time in the future.
The biggest problem with UB, as interpreted by C/C++ compilers is the unfounded belief that the spec says that you can assume any path that leads to UB is defacto incorrect and can be ignored. That's the bogus logic required for steps 13-16 to be true.
That interpretation of the specification language is clearly bogus as that means more or less all C and C++ programs can be compiled to a single return instruction - when compiler devs introduced this new new interpretation of UB, it turned out they had to then go through and add a bunch of exceptions for basics like memset, memcpy, and memmove could work without falling afoul of their invented nonsense.
This is an absurd reduction of the current situation.
> That interpretation of the specification language is clearly bogus as that means more or less all C and C++ programs can be compiled to a single return instruction - when compiler devs introduced this new new interpretation of UB, it turned out they had to then go through and add a bunch of exceptions for basics like memset, memcpy, and memmove could work without falling afoul of their invented nonsense.
I've never heard of anything like this, have you any sources for this happening?
Read the gcc mailing list. There is alot of user hostile "fix your broken program" from language lawyers.
I think aliasing rules messes up memcpy if you are dogmatic?
To be fair it is extremely easy to write broken C/C++ program
Theres a big difference between "the GCC mailing list is full of language lawyers" and "memcpy doesn't work because UB and the compiler Devs fixed it by adding an exception for it".
> I think aliasing rules messes up memcpy if you are dogmatic?
I'm _not_ dogmatic or a language lawyer and aliasing is a total pain in the ass, but that's a whole different topic to compilers getting fundamentals wrong because of language lawyering.
I think memcpy is fine because it operates on a special type that is allowed to alias anything. Strict aliasing has other fun issues like the impossibility of implementing malloc-like functions or the lack of any requirement for access to non-char members of structs to work at all[0].
[0]: https://stackoverflow.com/questions/49298704/how-reason-abou...
memcpy is UB because it reads padding bytes in structs.
memmove fails as it requires pointer comparisons between separate allocations, which is UB.
Sounds like we shouldn't use memcpy on structs containing padding!
No, memcpy is fine because it's a language level function. What you _can't_ do is implement your own memcpy alternative safely.
memcpy() and memmove() are language-level functions. They are allowed to do things that cannot be implemented in user code. The C standard library's implementation of these functions have tight integration with the compiler to ensure that it can implement all the required semantics without triggering undefined behavior.
reading padding bytes is not UB.
Comparing unrelated pointers is unspecified, not undefined. It's also an implementation detail of memmove. It can use a magical total order comparison of pointers that is otherwise unavailable to users of the language.
memset, memcpy, and memmove, etc all depend on UB.
memset and memcpy read and write padding bytes, which is UB. If you try to say memset/cpy is a "primitive operation", then you're still stuck with the "correct" struct initialization (memset, bzero) being UB if you did something like:
The ludicrously broad definition of what UB apparent allows, this can be "optimized" to
Because bzero() writes to padding, and that is UB, which means the condition==true branch "can't happen" \o/
memmove falls to UB much requires pointer comparisons between pointers to different objects, which is UB.
Wouldn't really be a problem if the design of those languages wasn't horribly ridden with footguns.
Compiler designers just chose the notion of "obviously user wouldn't invoke the footgun, so any code that would be causes by footgun can be removed"
This is a necessary consequence (in the formal logic sense) of the "principle of explosion"[1] for optimizing compilers. If the compiler assumes some undefined behavior does not occur, but in fact that undefined behavior does occur, all of conclusions it makes which depend, directly or indirectly, on the contradicted assumption are invalid.
Since, in the process of optimization, optimizing compilers make conclusions about parts of a program based on other parts of the program which may be executed later, undefined behavior in one part of a program causes invalid conclusions about other parts which may be executed earlier. Therefore undefined behavior cannot be contained to just parts of a program executed after a particular part, if the compiler tries to optimize parts of the program based on parts that may be executed later.
[1]: https://en.wikipedia.org/wiki/Principle_of_explosion