Embeddings are a fascinating topic.
Like our brains, they take input data and compress it to keep only that which is valuable.
Or, moreover, they take input data and convert it into a meaning representation.
But how do you define meaning?
How do you define importance?
It’s funny how we let these huge automated data machines decide for themselves what details are importance and what value means in human terms.
This is different from normal compression where you might compress the entire file so that you can extract all the information back out again.
You can retrieve what you put in.
But embedding are different in the sense that they compress and create the vectors according to a specific meaning space they find important.
In this way let’s say for simplicity your evil data versus ethical data is highly skewed. Your embedding space could also be skewed towards valuing evil meaning.
We have to be very careful with Embeddings, we have to be very careful with the meaning things are assigned.
