What are embeddings?

What the hell are embeddings in deep learning? They're used all the time and I haven't been able to get a proper understanding of the meaning.

Posted Dec 13, 2018, 10:26 AM UTC

25
shakalen96•

Embeddings are the mapping of your features. If you want to learn colours you can map each colour to a vector using whatever algorithm. The scalar or the vector does not have meaning in itself, but the pattern and relative distance to other vectors do.

The algorithm you train learns using the 'pattern' of vectors, this is only possible to the specific mapping you made.

​

TensorFlow also has a great article covering this: https://www.tensorflow.org/guide/embedding

3
Hari_a_s•

Nice. Would like to add this : https://www.youtube.com/watch?v=186HUTBQnpY

1
UnfazedButDazed•

Think of an embedding as a way to represent data that normally does not have a direct translation to what a machine can read.

For example, word embeddings are very popular. If I asked you, are a cat and dog similar things? You'd say yes. But if I have a computer a string with the word cat and one with the word dog, it would think they are two very different things because they have completely different letters.

So what word embeddings do is basically converting a word like dog into a vector that would have similar values in it as the word cat's vector. So now you can say that cat and dog are the similar. Or boat and yacht. Or tree and plant.

How does it do this? Well you train a neural network to learn about the context of the words. It basically learns what other words are around the words you're inputting. For example, the sentence "The dog ate from its bowl" and "The cat ate from its bowl" are both valid sentences. The network learns that eating from a bowl is things that a cat does and a dog does so they must be similar.

1
toastedbread22OP•

This is the simplest explanation I have come across!

1
UnfazedButDazed•

Thanks! I'm not %100 confident on the validity of this info so I hope someone else that's more educated than me can come in and chime in!

1
[deleted]•

[deleted]

-1