Can someone please explain this cool but creepy feature

I have a Google pixel 2. Recently my brother forwarded me an email from a travel company containing a pdf of his itinerary, i also took photos of the itinerary.

The next morning I woke up with a notification stating that the flight was due to take off in x amount of hours. Later that day i received a notification stating that the flight had landed.

My question is how did my phone process the information? I never searched for the flights so it either read the photo or PDF?

Posted Mar 17, 2018, 10:55 AM UTC

10554
snapilica2003Quite Black•

gmail parses email data to find relevant info inside for you. if it find bills to pay it will create google now cards to remind you to pay. if it finds purchases from stores, like Amazon, it will give you shipping info on those packets. if it finds plane tickets it will give you flight info (exactly like you had received). and many many more

117
CheckrecheckOP•

I would totally understand if it was typed into an email but the fact that it could decipher an attachment was creepy

9
snapilica2003Quite Black•

They only need a reservation number or a flight number, it can be in the subject of the email and somewhere in the body of the email. It will figure out what it is on it's own and give you the info you saw with just that. I don't think it looks in attachments though.

52
HohlraumHe3•

I'll add that you don't have to have to do anything on your phone concerning it, if your phone is linked to your Google account and you got the information via Gmail it'll show up in your feed regardless.

I've found that the location of hotels I'm booked at showing up automatically is also very helpful.

7
[deleted]•

[deleted]

-3
CheckrecheckOP•

Ok cool, ill have to go back and scour the email :)

5
MudHollandJust Black•

That would be my guess too, that the intinerary is in the list of e-mails and it was included when your brother forwarded the e-mails.

4
stacecomClearly White•

PDFs are text searchable and indexable. It was from the PDF attachment.

1
telemachuszero•

There's a markup format that can be attached to emails that provide this information. It's not visible to you, but Gmail uses it to do things like this (flight reservations, bills, etc). It doesn't necessarily need to parse anything in the email other than this extra markup.

It's likely the travel company's email included flight reservation markup (https://developers.google.com/gmail/markup/reference/flight-reservation).

30
No_MannersVery Silver•

Damn, this is a great answer. This guy brought sources.

5
hawkinsst7•

Also a PDF is easy to parse.

I do doubt they did ocr on the picture of the itinerary you took. Maybe one day they will, but I suspect not just yet.

10
b1ack1323Straight Talk•

Optical Character Recognition(OCR) is used all the time to discover text in photos. That's want this is. I have due dates for invoices added as reminders all the time.

4
getmoneygetpaid•

Why? Optical Character Recognition has been around for decades. I am not at all surprised that Google, who make money from mining your data, use it to mine your data.

1
cheeto0Black & White•

Some would think it reading an email text is creepy and some would think it reading text out of an image and notifying unl of the flight as very useful.....it's all subjective I guess.

1
aheadwarp9•

They check every attachment already to scan for viruses, so it's not that hard to imagine they added a step to that to scan for keywords.

1

Huh, if you think reading marketing attachments is creepy wait till you get round to Google Photos ;-)

1
binaryvisions•

PDFs aren't just "images" to computers. They can contain actual text and formatting. They can also just be images, it depends on what the PDF is formatted as.

Heck, just go to Google and search for something and add filetype:pdf to the end of the search - you'll see that Google has indexed a TON of PDFs.

In any event, OCR is great at recognizing text from images, so even if it were an image, it's trivial to do an OCR and pull some relevant data out.

1
TubaKid44Quite Black•

Evernote is a product that scans and indexes any jpgs or PDFs I store. I'm sure Google uses the same technology to scrape info out of your attachments.

0
[deleted]•

[deleted]

1
[deleted]•

[deleted]

1
hyphygreekQuite Black•

My Google Now stopped doing that on my Pixel XL after the new UI came out. It still flights to my calendar though.

1
snapilica2003Quite Black•

It still does that, just that the new UI only left news when you swipe to the left. If you press the upcoming button (the weird looking button next to the settings on the top right) you'll get the old interface with the exact content as before.

1
hyphygreekQuite Black•

Oh shit. Thanks! Even though they've ruined it

1
snapilica2003Quite Black•

You can also get there by opening the Google app in the application dock

1
alienwpopcorns•

Google does collect data and uses it to give you better services. I guess that's this scenario.

84
[deleted]•

[deleted]

0
bootski•

If you see something auto entered on your calendar and don't double check it, to the point of missing your flight completely, that's 100% on you.

If you know you have a flight coming up and the only reminder you have is the Google calendar that you haven't entered yourself or bothered to double check, that's just dumb.

In fact, it's so much of a stretch to me that I question the truthfulness of this comment, especially since it looks like you've copied it all over this thread.

1
dcdevitoJust Black•

It was in your email. Google's been doing this since the glory days of Google Now

37
hawkinsst7•

... Sigh... Those were the days

20
nkh85•

Google Past: Always one step ahead

5
DJ-Salinger•

"YOU WON'T BELIEVE WHAT DONALD TRUMP JUST DID IN WASHINGTON"

"MELANIA TRUMP FIRES BACK AT CRITICISMS"

"IVANKA TRUMP VISITS ENGLAND FOR NEW BUSINESS VENTURE"

Summary of my Google Now feed these days.

And before anyone asks, yes, I've said I'm not interested in Trump stuff dozens of times.

2
Achandab•

Calender -> settings -> events from Gmail

9
CheckrecheckOP•

Brilliant, thanks ill check it out

1
Wasp9277•

I'm new to Android and was wondering how you turn off the "Leave a review or take photos of Walmart" for example, that appear after every place I leave. Google said to switch off location services, which I did but they're still appearing.

1
Achandab•

Settings -> apps -> notifications -> rate and review places

You can leave location services on. Not sure why they said to turn that off.

Also a tip. Whenever you get a notification you can swipe halfway right and click the settings cog and have the option to turn off notification for that particular type of notification without affecting other notifications from that app

3
Wasp9277•

Thank you, appreciate it.

1
[deleted]•

[deleted]

-2
misspiggiePixel 1 XL 128GB :pixel1xlsilver:•

Once my cousin emailed me her flight information. I began receiving all notifications and alerts pertaining to her flight to the point that it was even easier for her to use to my phone to check in for her return flight than it was for her to try to do it on her own (also, my phone reminded her to check in!)

8
[deleted]•

[deleted]

0
metarugia•

If the service is free, you're the product.

Google data mines EVERYTHING.

6
SpiderStratagemPixel 32GB Quite Black•

I understand the privacy implications, but I have to say that -- as a frequent traveler -- the fact that itineraries sent to gmail results in flights automatically loading to calendar and google now/feed, with accompanying flight status updates, etc., is an absolute godsend.

It's frankly something I would pay for, and I get it for free. Yes, at some cost to privacy, but my travel is hardly a state secret anyway.

7
[deleted]•

[deleted]

-2†
SpiderStratagemPixel 32GB Quite Black•

True, it could be smarter in that regard.

1
qalsip1977Black & White•

You had to be using Gmail...yes? So Google gathers information in your Gmail (like flight information, movie times, appointments in your calender, etc.) and serves up reminders in your google info feed. Personally I love the feature! Other Google apps use information from Gmail and Calender as well (such as the Google Trips app).

5
[deleted]•

[deleted]

-4
lantaarnappelJust Black•

Travel companies have the option to include specific tags in their emails, which contain information about the trip. Gmail checks for those tags.

You can read more about them here: https://developers.google.com/gmail/markup/reference/flight-reservation

4
tenbre•

Gmail. Not photos.

3
Seathaskymusic•

Don't look behind you.

3
_casshern_Panda was sold out!•

Do you use gmail?

2
RazielKainly•

Not creepy. Smart.

2
BenR31415Very Silver•

If the flight number was in the subject line of the email, Google registers that and gives you notifications accordingly.

1
[deleted]•

[deleted]

-2
programmerqQuite Black•

PDF itinerary is what it looks at in gmail. I forward itineraries from work to my personal Gmail account specifically to activate this feature.

1
[deleted]•

[deleted]

-1
KilroyTwitchQuite Black XL•

Did you seriously just reply the same rare senario to nearly every reply here? 😳

My partner and I travel together frequently. It always shows both our information on my phone, which is incredibly helpful. Sounds like either you fucked up, or it was a one off bug.

4
[deleted]•

[deleted]

0
idrive2fast•

This is a feature - it's one of the main reasons to choose Android.

1
Rancid_Peanut•

You can turn this feature on/off. Press and hold your homescreen and select home settings. From the go into at a glance and you can see a flight information tab to turn on or off.

1
Le_VaStorm Trooper•

It's due part to the Google assistant. I had helped my mom set up a flight via JetBlue and i got notifications just like this.

Hopefully it'll never give me notifications about incognito searches.

1
spooksmagee•

The feature that threw me at first was the phone reading a Fandango receipt in my email and then alerting me to leave home by X time to make the movie on time.

So that means the phone had to know I was home, know the theater and its address, look up traffic along the shortest route, perfrom the calculation and then notify me.

Cool, but a bit creepy.

1
5hakeharQuite Black•

If you have Google Trips app installed it will curate your tickets and give you some info about the city you are going to. Some of that content can be downloaded for offline use as well

1
y2julioBlack & White•

Gmail. You might find it creepy but I find it useful. Download Google travel for even more creepiness.

0