Recent and future pattern matching improvements | Inside Rust Blog
blog.rust-lang.org/inside-rust/2020/03/04/rec...Posted Mar 4, 2020, 12:11 AM UTC
Posted Mar 4, 2020, 12:11 AM UTC
Awesome to see them polishing existing features so much. That's something I really like about Rust. Making the existing day to day stuff a little bit better again and again over time.
polishing existing features
Another thing that would fall into the same category is destructuring assignment (e.g. let a; let b; (a, b) = foo_returns_tuple(); for assigning multiple return values in one line). I'm really looking forward to this.
let ref x @ ref y = a;
This looks really bad for me. Yes of course it is not like this valid C++17 program:
void *$(){[&](...){$:[](){({$:;_:[](){},""[+-(&&$<&&_)];});;}();}([=](){});}
but it is getting closer.
The following is a valid Rust program:
fn main() {
return return return return;
}
But why would you ever write such a thing?
To me, it is a sign of strength in the design of a language that compositions can lead to silly (but harmless) things. let ref x @ ref y is not useful, but rejecting it would be more complicated (and ad-hoc) than simply allowing any pattern to the right of @ as long as the borrow checker is happy.
But why would you ever write such a thing?
I suppose the answer is "because you can". From my experience, If some strange and funny thing possible in language X, you definitely find it in old and big enough codebase. I have no idea why people write such strange things in production code, but they definitely do it.
If people insist on code golfing and writing useless code, then have at it I guess. There are many ways to write useless code in Rust, and it's not a good idea to introduce ad-hoc restrictions in the language to prevent it, making the specification more complicated in the process.
After reading Perl Best Practices I became fully convinced that most of Perl's reputation for illegibility was cultural and up to developer choice.
Just like it's easy for content creators to fill a page with meaningless drivel, it's easy for developers to lazily string together meaningless code.
This is why code review exists.
You're using a bunch of non-standard extensions and you've stripped away all the names and formatting. It won't compile with msvc, and GCC will reject it if you compile in pedantic mode. The code doesn't even do anything. It's basically just a bunch of nested lambdas, pointless brackets and random labels.
Yeah, last time this discussion came up here i think the Scala familiar people liked getting a syntax they were used to and a large amount of others were pretty wary of it.
The bigger issue, unfortunately, is that this is a case where we had a special case not work for an already allowed syntax. I think the right decision would be revisiting the @ syntax overall, but that is a bit of a big ask and outside of the scope of match improvements.
This could also be good for macro authors as they wouldn't have to make a special case for avoiding generating this pattern. (Maybe someone more experienced than me could provide an example?)
I read it as an example to illustrate the borrow checker working in the next example with ref mut y
Wow I didn’t realize how insane the pattern matching for rust is.
So very excited for subslice patterns in particular, but the others look pretty good too. Thanks to everyone who has made them possible :D
They all look very nice!
In nightly `or_patterns` and `move_ref_patterns` will simplify lot of things!
feelin like Haskell!
Yay; this is what I'm going for! =P
This feature of or-patterns can not work with different types?
All bindings in an or-pattern must have the same types so that no matter what alternative gets selected, the binding ends up with the same type. If that weren't the case, and the compiler for example picked the type of the first binding, then you would have a soundness hole.
OK thanks.
What would the type of x be?
I was thinking something like.
Right, but in order for that to work, x has to have a type. What type would you want it to be?
To be fair, the type system could infer the type of x to be the existential type impl Display or the anonymous sum type String|i32 but Rust isn't quite there yet.
subslice patterns are great. Hope something similar can happen to tuples.
It's already possible to ignore parts of a tuple or tuple struct in a similar fashion to slices, e.g. let (x, ..) = (1, 2, 3);. However, it is not possible to bind subtuples as that would necessarily imply restricting the layout of tuples such that you couldn't reorder (assuming we allow let (head, ref tail @ ..) = (1, 2, 3); and not just by-move). This would have performance implications.
I think it's not necessary to have the concept of "subtuple"s. Index-range within tuples can be represented with a triplet of <tuple_type, start_index, length>, i feel it should be represented by reference to some unsized type in analogy to the concept of "slice" but for tuples. In this way, reordering no longer matters.
Can you elaborate on how that triplet would avoid giving up the ability to reorder layout of tuples? Another way to avoid it is to have tail in (head, ref tail @ ..) result in a tuple of references.
I'm thinking about something in the line of:
let v = (1usize, 2usize, 3usize);
let s: TupleSliceRef<(usize, usize, usize), {1..3}> = v.get(1..3).unwrap();
where TupleSliceRef works as a proxy (just like shared slice but with a little more context, since it knows about the whole tuple), and it knows how to retrieve a specific item from the reordered layout of tuples.
So with these changes can I match on a slice of unknown length where one of my match arms is longer than the length of the slice? For example, I want to match on a slices with four elements, but at runtime it happens to have two. I just want that to be caught by an arm with 2 bindings or the _ arm. I think right now if I do this it gets an out of bounds panic.
Finally, now I can shorten this thing in my personal project
This is somewhat tangential but is there a meta feature gate that can be used to enable all relatively stable nightly features?
That sounds like it'd be a lot of effort on the part of the Rust maintainers classifying how stable each unstable feature is, while, at the same time, potentially encouraging the kind of de facto stabilization that forced browsers to gate CSS vendor prefixes on special builds of browsers that end users couldn't be expected to install.
If we had the staged RFC process, we would already have this information with not really any extra work.
Good point... though it's still questionable whether "enable everything more stable than X" is a good direction to go for how people should be encouraged to think about nightly features.
Agreed.
The de facto stabilization is already occurring. For that not to happen would require dropping support for nightly-only crates. And anyways, Rust only has one relevant implementation and no formal standard. If Rust maintainers believe a feature is at stability level X we can be reasonably sure that that's representative.
If it's a lot more work then I can see why it hasn't been done, but it would certainly be convenient to have something more fine grained than "going into next stable release" but more coarse grained than "individually enable each of hundreds of features."
My point is that it's a balancing act to encourage that such features get testing but, at the same time, not make implicit promises about keeping nightly-only APIs working after a rustup update.
For which definition of "relatively stable"? You could use the beta release channel though.
That would probably lead to *less* feedback on unstable features, since you'd be using some of them without realizing it, and would be less likely to report issues ?
I tend to find out about unstable features by bumping into a language limitation and digging a bit further. At that stage, I have an incentive to try out the feature and help with stabilization if I can.
Or you could also browse https://doc.rust-lang.org/unstable-book/ and pick something that looks interesting.
Interesting! I have a newbie question if I may:
Is it possible to bind a constant in a pattern like so?
let c = "bar";
/* ... */
Foo {x: c :y _} => /* ... */
Sorry, I should have read the links in the article first. Just found out about 'match guards', which cover this use case.
It's nice that existing features are polished, in the sense that you can combine them in natural ways. The nested or patterns definitely fall into this category and they make sense intuitively. I even tried to do this at some point.
This binding syntax is hard to read and understand, though. Why was the @ symbol used? It doesn't convey any meaning. And it just seems to replace one let expression, eg let x = array[1 .. len-1], which is a lot easier to understand.
When learning Rust, pattern matching was one of the weirdest things to learn. I had never encountered it before (in C, Java or Lua). Everything seemed backwards and inside-out. Also, where and why do I use ref instead of &? Once you know it, pattern matching is easy and powerful, but it's a lot to learn.
Please don't over-complicate pattern matching for the sake of writing one line less.
Also, where and why do I use ref instead of &?
They are opposites. & will match a &T and give you a T, and ref T will match a T and give you a &T.
You almost never need ref these days, thanks to https://doc.rust-lang.org/stable/edition-guide/rust-2018/ownership-and-lifetimes/default-match-bindings.html
They come from ML.
The @ symbol was already used, this just made it usable in additional places - a case of, as you said, combining existing features. An example where @ would have been used previously was something like:
match chars.next() {
Some(digit @ '0' ... '9') => read_number(digit, chars),
...
}
Another example are cases where you need matching, but not dereferencing. For example (like the previous one, stolen from the O'Reilly "Programming Rust" book):
match self.get_selection() {
Shape::Rect(top_left, bottom_right) =>
optimized_paint(&Shape::Rect(top_left, bottom_right)),
other_shape => paint_outline(other_shape.get_outline()),
}
In the first match arm we destructure a Rect only to rebuild an identical one. Using @ capture this can be expressed as:
rect @ Shape::Rect(..) => optimized_paint(&rect),
Note that we can't just replace this with a non-match expression because we still need the matching (and possibly destructuring) in other match arms.
C, Java or Lua
Because patterns come from a world of ML, Haskell, etc.
Everything seemed backwards and inside-out
This is correct, a pattern describes how something was assembled to explained what parts you want to remove.
Think of it as showing a car and circling a wheel and saying "this wheel must be removed" (maybe it's flat). You had to draw the whole car though. Patterns are like that. They work backwards (because they are used as guides to disassembly, though they look like assembly) and work inside out, because they get the innards out, instead of hiding them inside!
why do I use ref instead of &
Remember patterns work in the opposite way. When building an Option<T> I write Some(x) and takes the value of x and hides it inside the Option, patterns do the opposite so when I write Some(x) in patterns it takes out the stuff inside the Option and puts it in x.
So when I write &T it does the opposite, instead of borrowing the value, it will remove the value out of the borrow! Basically if I want to match a borrowed value I can remove it by using &x. But how do I do the opposite? Borrow an owned match? Well all I need to do is use the opposite, the pattern that converts &T into T. And there's none! So we need to make up a new thing, a way to convert owned values into borrowed ones in the backwards world of patterns. This is why we use ref, it's named like that because you can't use it outside patterns, it's illegal to turned a borrowed value into an owned one (that would be stealing).
The basic example is I have an &Option<T> and I want to get what's inside with a pattern, well I can get what's inside by using Some(x) but that's not how you build an &Option<T> you have to borrow it, so I do &Some(x). The problem is that the only way to build a value given there is if x is owned. But you can't take an owned value from a borrowed Option, the borrow checker would never let you! So we need to convert that owned x into a borrowed on instead &Some(ref x) which basically says borrow (instead of moving) the value that was used to build this borrowed &Optional.
Why was the @ symbol used
Similar to above. We have a thing were we can get the innards, but what if we want to match on the insides, but get the whole thing? We need the opposite operation. In our car analogy above I circle the whole car and say (needs a new car). Now we can build a car with a car, by not doing anything, but it's kind of silly to say that thing and most people would roll their eyes if you said that. So there really isn't a way to say that in Rust, which means you can't say the opposite either. So we need a special way to "circle the whole thing".
Why is it needed? Imagine that we have a function that can give a series of errors and we can handle some of them. Imagine I have a certain Result<T, Err> I can get from a function, the Err it returns can have an issue Fixed. So when I get the Error I want to either fix it, or log the error otherwise. If the value can't be fixed (because it's fine or non fixable) then I should get the original value.
// This happens inside an impl ErrFix for Result<T, Err>
fn fix_err(&mut self: Result<T, Err>)->&mut Result<T, Err> {
match(self) {
&Ok(ref x) => &mut Ok(x), //won't work it'll give us `Result<&mut T, _>`!
&mut Err(FixableErr(_)) => self.fix(), // Again fails, self is being used by the match!
&mut Err(ref err) => {
log(err);
Err(err) // See above
},
}
}
The problem is that we can only borrow the inside, but we can't repack a borrowed value because it would be a different thing. Of course we could just add a return afterwards, but this won't fix the part were we want to call fix() only when the fixable error happens. We could store some other variable but that's complicated. What if I could say: I want the whole thing?
What if we could bind to the pattern the whole thing?
// This happens inside an impl ErrFix for Result<T, Err>
fn fix_err(&mut self: Result<T, Err>)->&mut Result<T, Err> {
match(self) {
s @ &mut Ok(_) => s, // works, s is the whole thing without unpacking
s @ &mut Err(FixableErr(_)) => s.fix(), // Again works now
s @ &mut Err(ref err) => { // Note that we need the inside but also will deal with the whole part. This is possible thanks to NLL.
log(err);
s
},
}
}
Sorry about how abstract this concepts look, but it's hard to keep it easy to show just what matters without going into the details of a real case in a way that doesn't feel too silly.
Arguably since the operation &T -> T is spelled *x in Rust, then ref binding should have been *binding.
As for @, in hindsight I might have chosen * as "intersection" and + as "union" as they are analogous to products and sums.
I miss function pattern matching i get in Erlang, really useful in writing clean code, something along the lines of (excuse the syntax):
fn f(u8: 3) -> print("its a 3"); f(u8: 4) -> print("its a 4"); f(u8: n) -> print(n)
It's not really that different in Rust...
fn f(x: u8) {
match x {
3 => println!("It's a 3"),
4 => println!("It's a 4"),
n => println!("{}", n)
}
}
which I find a lot more readable
The only thing Rust doesn't have is the case functions. Manually desugar to a match (case) and you get the same result.
I feel there's always something missing to easily implement hierarchical state machines, when some event applies to several states.
enum State {
A,
B,
C,
D,
}
impl State {
// this is a hierarchical state
fn is_ab (&self) {
self == State::A || self == State::B
}
}
match (state, event) {
(s if s.is_ab() | s @ C, e @ Event::Foo) => transition(s, e),
}
This doesn't compile because the ifcan only apply to the whole pattern.
It’s truly amazing that this can be done as part of a systems programming language.
The subslice pattern matching seems incredibly useful. Out of curiosity, is there anything similar planned for substring matching?
As others have pointed out, this might be tricky to get working with Unicode strings. That said, it would be great if byte-string literals worked this way (i.e:
b".json")The stabilization PR even includes an example string prefix matching:
This is okay, but if the prefix/postfix was any longer than a few chars, things would start to get unreadable.
It would be nice if this new pattern matching syntax played nicely with the existing byte-string pattern matches.
I wonder if this would be linear, or instead panic. Seems like if you want what you're describing it may be better to instead pattern match on the underlying bytes?
But this can be mechanically transformed into a pattern against bytes (every string literal has a bytes representation and every captured string binding also capture a byte slice)
json @ .. ".json", for example, doesn't need to scan the string from the beginning; it just need to look at the last 5 bytes.I also don't see how this could panic
Well, strings are Unicode, and Unicode equivalence is much harder than just matching individual bytes. You could probably lower string patterns to the use of the
PatternAPI, but that one's still unstable…The same way that string indexing panics.
I suppose if the matches enforce anchors maybe this would still be constant time and not panic...
I don't think we have anything planned for substring matching at the moment.
After recently starting to learn Rust, that was one of the features that really pulled me in. Having to work primarily in Python recently for my job, I absolutely love the idea of having high level zero cost abstractions available in rust. Especially after having mainly been working with C++ before that.