Can someone please explain this cool but creepy feature
I have a Google pixel 2. Recently my brother forwarded me an email from a travel company containing a pdf of his itinerary, i also took photos of the itinerary.
The next morning I woke up with a notification stating that the flight was due to take off in x amount of hours. Later that day i received a notification stating that the flight had landed.
My question is how did my phone process the information? I never searched for the flights so it either read the photo or PDF?
Posted Mar 17, 2018, 10:55 AM UTC
gmail parses email data to find relevant info inside for you. if it find bills to pay it will create google now cards to remind you to pay. if it finds purchases from stores, like Amazon, it will give you shipping info on those packets. if it finds plane tickets it will give you flight info (exactly like you had received). and many many more
I would totally understand if it was typed into an email but the fact that it could decipher an attachment was creepy
They only need a reservation number or a flight number, it can be in the subject of the email and somewhere in the body of the email. It will figure out what it is on it's own and give you the info you saw with just that. I don't think it looks in attachments though.
I'll add that you don't have to have to do anything on your phone concerning it, if your phone is linked to your Google account and you got the information via Gmail it'll show up in your feed regardless.
I've found that the location of hotels I'm booked at showing up automatically is also very helpful.
[deleted]
Ok cool, ill have to go back and scour the email :)
That would be my guess too, that the intinerary is in the list of e-mails and it was included when your brother forwarded the e-mails.
PDFs are text searchable and indexable. It was from the PDF attachment.
There's a markup format that can be attached to emails that provide this information. It's not visible to you, but Gmail uses it to do things like this (flight reservations, bills, etc). It doesn't necessarily need to parse anything in the email other than this extra markup.
It's likely the travel company's email included flight reservation markup (https://developers.google.com/gmail/markup/reference/flight-reservation).
Damn, this is a great answer. This guy brought sources.
Also a PDF is easy to parse.
I do doubt they did ocr on the picture of the itinerary you took. Maybe one day they will, but I suspect not just yet.
Optical Character Recognition(OCR) is used all the time to discover text in photos. That's want this is. I have due dates for invoices added as reminders all the time.
Why? Optical Character Recognition has been around for decades. I am not at all surprised that Google, who make money from mining your data, use it to mine your data.
Some would think it reading an email text is creepy and some would think it reading text out of an image and notifying unl of the flight as very useful.....it's all subjective I guess.
They check every attachment already to scan for viruses, so it's not that hard to imagine they added a step to that to scan for keywords.
Huh, if you think reading marketing attachments is creepy wait till you get round to Google Photos ;-)
PDFs aren't just "images" to computers. They can contain actual text and formatting. They can also just be images, it depends on what the PDF is formatted as.
Heck, just go to Google and search for something and add filetype:pdf to the end of the search - you'll see that Google has indexed a TON of PDFs.
In any event, OCR is great at recognizing text from images, so even if it were an image, it's trivial to do an OCR and pull some relevant data out.
Evernote is a product that scans and indexes any jpgs or PDFs I store. I'm sure Google uses the same technology to scrape info out of your attachments.
[deleted]
[deleted]
My Google Now stopped doing that on my Pixel XL after the new UI came out. It still flights to my calendar though.
It still does that, just that the new UI only left news when you swipe to the left. If you press the upcoming button (the weird looking button next to the settings on the top right) you'll get the old interface with the exact content as before.
Oh shit. Thanks! Even though they've ruined it
You can also get there by opening the Google app in the application dock