LLMs harness human knowledge

November 30, 2023 (1y ago)

One fascinating thing about AI today - at least for now - is that all the training data has actually been created by humans. Synthetic data learning is coming, but this round of models have all learned from human knowledge.

The information is all there - spread across people’s minds. Collected, collated, analysed, documented and put on the web by humans.

But when an LLM looks at this as a whole at an industrial scale ( a very simplified view I know), it finds connections that humans have missed.

AI isn’t doing anything original, but it’s relentlessly effective at finding the needle of logic in the haystack of documented human experience.

It’s a bit like the periodic table - where simply putting the known elements in a certain order unlocked new patterns and insights.

Also a lesson for humans. The truth isn’t always out there. Sometimes it’s right in front of us but we have to look at it more methodically.