Facebook open-sources RacerD: Fast static race detection at scale
code.facebook.com/posts/293371094514305Posted Oct 19, 2017, 4:09 PM UTC
14027
code.facebook.com
Well, I sure hope they've got a magic-bullet solution to race conditions in the data-consumption sense.
With that out of the way, maybe someone at Facebook would like to tl;dr the rest of us here: how does this detect the condition of data being passed between transactions, and its equivalent in non-transactional systems? And the corollary: how does it permit revalidation of such data in cases where it's not actually a programming error? And then the kicker: how does it detect the former posing as the latter, without false positives?
8
What does static mean in this context? Just that the detector can scan for race conditions without actually running the program?
7
Yes.
10
Is the performance any better than actually running the program? It seems you would run into the halting problem.
1†
-
In order to discover races while running the program without any false positives or false negatives (i.e., in a sound and complete way), you'd need to run it many, many, many many times, with precise control over scheduling.
-
It does run into the halting problem, which is why it's either unsound or incomplete, i.e., it must either have false positives or false negatives. I assume it is sound, so that means it has false positives -- it reports problems that may not be real.
5
static analysis
4
At first I thought they were referring to a different racerd: Rust Auto-Complete-ER. Has a daemon, racerd, that runs in the background and autocompletes Rust snippets for IDEs.
6
The name insipired by Initial D?
6
Huge miss on calling it 'Speed Racer'
Edit: I'll be honest I also thought it was a skin color detection algorithm for photos at first too. I was curious of why it was so important to know the race of the person in photos but I figured Facebook would have a good reason.
6
Like to imagine they started with that name, then moved to what seems to be an InitialD refernce after HR/legal got back to them about the name.
My mind went to race conditinos first, but skin color race wasn't too far behind. Wouldn't be surprised if they have an internal tool for detecting skin color (for various reasons), but I would be surprised on why they would open source it. Seems like a tool that can cause some unethical actions to arise in the wrong hands if it was accurate enough.
2
Oh good, another FB project with the PATENTS poison pill. Removing it from React clearly got enough heat off them that they're happy to keep it on their other projects.
6
Looking at the name, I thought that it's written in the D language... Apparently not.
5
Alternative solution: prevent the vast majority of these problems from ever entering your program in the first place (Rust).
1†
something something productivity something something maturity
24
Vast majority of problems occur due to faulty reasoning/algorithms specially in lock free areas. Rust does helps there but only in situations where the problem arose because of ownership of shared data, not because of logic error.
10
Rust prevents accidental data-races. Writing performant concurrent data-structures in Rust (and almost any type of data-structure) needs unsafe code at one point or another, meaning that all static guarantees are off. However, you can offer an API that promises to maintain those guarantees, and then test and fuzz the hell out of the data structure, e.g., using run-time race detection tools like ThreadSanitizer, to make sure that the data-structure upholds those guarantees.
The difference is mainly that "low-level" data-structure writers need to care about data-races, but data-structure users do not need to care.
Writing these low-level data-structures is extremely error prone even for experts (which repeatedly fail to uphold the guarantees). However, the safety guarantees are a great blame assigning mechanism: if you don't have unsafe code and got a data-race, "it's not your fault".
Today because of this many concurrent data-structures are built on top of safe lower level primitives (e.g. epoch-based garbage collection), so depending on the kind of data-structures, not even data-structure writers need to care about data-races.
2
meaning that all static guarantees are off.
Just to be clear about unsafe it doesn't "turn off" checks, it adds new capabilities that are unchecked. Even within unsafe, many things will end up being checked.
This doesn't detract from your point, and I know you know this, but it's a common misconception, and since we're on r/programming...
3
Thanks for adding this, it's exactly what I meant.
Expanding a bit on this: unsafe doesn't turn off any checks (AFAIK). There is a lot of data-race-like code that the compiler can prove to be safe. However, there is some data-race-like code that is safe, but the compiler thinks its unsafe. What unsafe allows is to, in these cases, tell the compiler "trust me, this code is correct". This doesn't happen by turning off checks on safe tools, but by allowing the use of "sharper" unsafe tools, where the compiler expects the user to manually maintain some invariants.
When proving any code, the compiler assumes that the user restored those invariants in those unsafe blocks. So with "all static guarantees are off" I meant that, once unsafe appears somewhere, safety becomes conditional on the unsafe code being correct. Without any unsafe code the compiler guarantees unconditional safety.
1
meaning that all static guarantees are off. However, you can offer an API that promises to maintain those guarantees
Absolutely and this is how we should approach it in every language, although I've to admit rust does make it easier.
1
You're right, Rust does not solve every problem (yet).
-1
Which is fine to say for certain pieces of NEW software/components.
1
You mean racial profiling at scale? :)
1
I thought it was going to detect black people... Im dumb.
-7
wut? usually bad joke is a bad joke but this one i can't even understand in the slightest?!
2
"Race detection". The black race
7
ohh wow ;o
1
Always add the licence.
https://www.infoworld.com/article/3234328/java/racerd-detects-hard-to-find-race-conditions-in-java-code.html
"Facebook is evaluating whether to change RacerD’s license as well."
So, what is the licence now?