Cheap tricks for high-performance Rust
deterministic.space/high-performance-rust.htmlPosted Mar 4, 2020, 11:43 AM UTC
34567
deterministic.space
Nice to have this in one place. However, those unnecessary clone calls that apparently need a separate article are usually caught by clippy. I highly recommend using it!
In Visual Studio Code, the RLS extension even allows you to run it every time you save the file.
44
Thanks! I'm a big clippy fan myself :)
That article will start with telling people to use clippy but then goes down into how to actually use referenced data instead if possible -- something that sadly is a bit of a tall order for a linter to do automatically :)
28
Same for the rust-analyzer you can define which command is run at each change. Just set it to clippy ;)
2
I'm not sure whether the optimisation potential is actually realised, but isn't another point in using panic = "abort" lie in the fact that it enables some inline optimizations and code movement not normally allowed with unwinding, since it effectively removes many hidden control flows that might block them...?
18
Nice read!
Now I really wanna know some strategies to optimize all those clone() calls
14
It's probably gonna take me quite a while to get to write that post… How about https://deterministic.space/secret-life-of-cows.html for now? :)
24
Added some more links at the end of the post now :)
2
Awesome, all in one place, very useful, thanks
13
And as always, benchmark. :) You may find that some of these provide significant performance boosts, and some of them make no difference besides slowing down your compile times, depending on your program.
13
Yep. Against all expectation, for all projects I've gotten far enough to care about optimizing, lto=fat has been a performance regression compared to lto=thin.
The sweet spot for the one I've been working on lately seems to be O3, lto=thin, codegen-units=1, panic=unwind. target-cpu=native made no difference, at least on a Whiskey Lake i7.
This is an actix-web REST API using MySQL, so the bottleneck isn't going to be computation anyway, but the difference between worst case compile options and best case compile options was about 16% average throughput with best-case concurrency, so far from nothing. Not that any of it matters, since I'll never be serving 20k requests /sec anyway, but hey, cheap optimization is fun ;)
2
On rav1e, we do see about 7% improvement from fat lto and about 9% from cpu=native on i7 6700 CPU. This is on top of hand-written ASM for some functions. So it does help in some circumstances! But a video encoder has a very different performance profile than a web server.
12
Absolutely. Especially with all the instructions that have been added to x86-64 over the years specifically with video encoding/decoding in mind, I'd expect a huge improvement like that from target-cpu=native. That it didn't help me and that lto=fat was actually negative were huge surprises, and still not things I would expect for most projects ;)
EDIT: And I haven't yet had a chance to benchmark non-default allocators with this project. I would expect huge differences one way or the other with those, since this does a ton of heap allocation and syscalls, but we'll see.
3
Small typo
Rust can use multiple inker flavors
Missing an L?
12
Tattoo typo fixed
31
Current Rust binaries us the default system allocator by default, previously they included their own with the standard library.
Also missing an e.
Great article by the way. I’m gonna learn awk or sed snippet to turn these on and off.
First I gotta learn those tools though.
8
Thanks! You can also use conditional compilation and feature flags for switching allocators :)
4
Couldn't you just create two different release profiles with different settings? You aren't limited to just debug and release, are you?
4
Setting the target-cpu to native might work out for toy programs that you run on your development machine but it's a very bad idea if you need to deploy or distribute the executable.
8
I've always wondered this. How do you create this whole matrix of binary releases for each architecture and/or OS using cargo? What should I Google?
8
you can see options with rustc --print target-cpus
10
If you're looking to remove some of those clone calls, I gave a workshop a couple of years ago that some other people did again last year, focussing on precisely that subject. It's got examples, tips, and exercises, so maybe it'd be a good place to start http://troubles.md/posts/rustfest-2018-workshop/
EDIT: Rereading the workshop materials, it's funny that I mention the hashbrown crate as young and experimental, but possibly worthwhile. At the time it had only been released a couple of weeks beforehand, but now it's actually been integrated as the default hashmap implementation in the standard library.
8
Question about the CPU target stuff. I know for your machine when you get the list of CPU targets from Rustc it tells you what the machine you are on is (so you'll know what native is setting it to). Is there anyway that lists what CPUs each of the profiles in the list targets? On the off chance I ever get to build rust targeting a server I'd like to know how to figure out the right target for that machine w/o having to put rustc on it and check.
5
if you have gcc installed you can do:
gcc -march=native -Q --help=target|grep march
if not and it's an intel chip you can do:
cat /proc/cpuinfo | grep 'model name'
then look the architecture up on https://ark.intel.com
3
Nice summary!
But I feel like people should measure perf even before turning on such optimizations. E.g. it could be that panic=abort does not improve performance at all, while increasing unsafety greatly. Maybe the article should include at least a link to another site that describes a basic perf measuring setup (I don't have such a link at hand, sorry).
Anyhow, it's cool to have all these options conveniently bundled in one place. Thank you!
5
`panic=abort` doesn't increase unsafety though. A program that's not running anymore cannot corrupt any memory! However it does make debugging harder and could break some libraries that expected to be able to catch panics.
7
thank you for the clarification :)
1
Thanks!
I don't have such a link at hand, sorry
Let me know when you find one and I'll add it! :)
1
I think the Criterion library is very useful to do this.
3
Why does stack unwinding have better "cache usage"? Memory use I understand -- the code that does the unwinding has to live somewhere. But it should be possible to put all the unwinding code together in a cold region, and then not affect cache usage at all shouldn't it? (Assuming that stack unwinding actually being triggered is rare)
4
Even if all the unwinding code is pulled out of the real code, the real code needs all kinds of instructions inserted to support jumping into and out of the unwinding.
6
Does rust not use the zero cost exceptions model that all modern C++ implementations do? In that model those instructions are not in the main path at all. When an exception is thrown cold metadata is referred to that it can use to infer how to do the unwinding.
2
Rust uses the same system as C++ exceptions internaly, but those are only zero-cost in the sense that they don't impact performance during execution of the function. Both Rust and C++ can see performance improvements with disabled exceptions due to secondary effects, like for example less instruction cache pressure (that is, code that is never executed but still takes up space in the cache)
5
(that is, code that is never executed but still takes up space in the cache)
Yes but my point is that code should be able to entirely lie on a cold region separate from the main code in which case it should take up absolutely no space in cache. It's only a problem if the compiler/linker are mixing non-exception path and exception path code together.
1
The post says otherwise, disabling unwinding has better cache usage.
2
Well that's just it it asserts it but does not justify it. I'm saying the claim doesn't make sense.
2
jemalloc does not create smaller binaries universally. It increases my mips binary size by 15%.
3
I meant the change to not use jemalloc by default lead to smaller binaries!
15
Ah I see, the sentence order confused me.
3
I changed it to be a bit clearer :)
3
Any ideas why adding target-cpu = "native" produces the following when building?:
warning: unused manifest key: profile.release.target-cpu
3
Oh yeah sorry that's a RUSTFLAGS thing. Updated the post.
4
For others, an example of using it in ~/.cargo/config, alongside a faster linker lld:
[target.x86_64-unknown-linux-gnu]
rustflags = ["-Clink-arg=-fuse-ld=lld", "-Ctarget-cpu=native"]
7
If you're trying out different memory allocators, it might be worth trying tcmalloc. To be precise, the new tcmalloc released separately last month, not the ancient one included in google-perftools.
One thing that makes tcmalloc unique (so far) is that it supports per-CPU caches based on restartable sequences, which jemalloc doesn't (open issue). This can mean better performance (than per-thread caches) on programs which follow a thread-per-request model. Probably doesn't make much difference in ones which do essentially everything async and have a thread per CPU.
Another is huge page awareness, which can decrease time spent stalled due to dTLB misses. This can be a big deal in memory-hungry programs. jemalloc doesn't seem to have this either (open issue).
3
Looks like there's an old crate for the past version. Would be great if someone could package up the new one.
1
i get warning: unused manifest key: profile.release.target-cpu
when i include this to my cargo.toml:
[profile.release]
lto = "fat"
codegen-units = 1
target-cpu = "native"
2
Unfortunately it has to be a rustflag and isn't supported in Cargo.toml. See this comment
5
Thanks, good to know
1
I've been doing all those things, and I wanted to try rpmalloc and mimalloc, but both aren't working for me (on Windows):
rpmalloc issue, mimalloc issue
(Mimalloc also fails to build on wasm.) Jemalloc doesn't build with rustc-msvc, so right now I'm stuck with the default allocator.
Btw, what's the fastest allocator for wasm? (I don't care about binary-size, it's for a desktop app's UI). I'm using Yew, and profiled my frontend: memcpy (cloning) and PartialEq are my bottlenecks (because Yew always clones properties on every re-render).
2
Actual profiling and optimizing your code
Now this is where you need to actually adjust your code and fix all those clone() calls.
I highly recommend using [cargo-]flamegraph for finding the parts of your program that will benefit the most from optimizations.
2
thanks!
Current Rust binaries us the default
small typo here
1
It's very sad that Actual profiling and optimizing your code is a mere footnote. It must be the first entry, bold, red, large font.
1
See second paragraph and the title being "cheap tricks" 😅
3
Right, but it kinda fades into background ¯\(ツ)/¯. Red, bold, large font.
-1
Here's the deal: you give a link to a post that goes into detail about profiling and then optimizing rust code in 2020 and I'll give you red text. (All links are red :P)
And yes I'm very much saying this to trick someone into writing that post so I don't have to do it myself!
2
Found it: http://likebike.com/posts/How_To_Write_Fast_Rust_Code.html
2020, heavy emphasis on profiling and optimizing critical paths. Not as detailed as I would like to see, but good enough.
I'm so picky here because I keep seeing juicy "Try this magic option!" posts pop up instead of "Avoid this kind of code, employ that kind of code, benchmark, profile, PROFILE!". Those options do help, but many of those come with trade-offs.
For example, panic = "abort" is a great way to say goodbye to tracebacks for hypothetical 1% improvement. (Yes there are corner cases where this really matters; certain sorts of heavy computational code, like renders, tracers, etc..., but "doesn't panic" execution path should be heavily optimized in 99% cases). You don't mention that.
Another example: lto can easily result in 2x and more compilation time increase. Have you met a case where enabling it results in 20min => 50min compilation time bump? Lucky you.
Thanks for the post, anyway.
2
Note that CPU targets break occasionally, even in stable: #63959, #63791. Ensure you have at least a basic test before using that. (Or at least manually check output is identical if you can)
If you only care about yourself you can add the following to your
~/.cargo/configso that any locally built binaries will take advantage of it without worrying about it breaking for anyone else.[deleted]
If they have a cpu that doesn't support the instructions your cpu uses, then maybe.
I don't have a solid answer, all I can say is I set it and forgot I had it for the longest time. So if that's any promise of it not breaking things. I'm on a Mac so I can't speak to it on other platforms.
[deleted]
Also, see: #64609
-Ctarget-cpu=nativemay also break tools like Callgrind, which are not ready to newer instructions.So when you start trying to do "actual profiling" you get weird segfaults reported instead.