As part of Meta’s Llama series of open-generative AI models, the third entry has just been released: Llama 3. The company has introduced two models in its Llama 3 family, with the rest to follow at an unspecified time. Let’s take a closer look at these new models and what Meta claims about them.
The two new models are called Llama 3 8B and Llama 3 70B. The numbers refer to how many parameters they each have: 8 billion and 70 billion, respectively. According to Meta, these models are much better than the previous Llama models, thanks to the larger number of parameters. Meta says they tested the new models on some common AI tasks and benchmarks. While these tests have limitations, Meta claims Llama 3 8B outperforms other 7 billion parameter models. They also say the Llama 3 70B can keep up with the biggest-name models from other companies.
Meta used a ton of data to train the new Llama models—a staggering 750 billion words! This is over 7 times more data than the previous Llama models trained on. Meta won’t say exactly where this data came from, but it included code, non-English texts, and some computer-generated data too. All this data helps the models understand different topics and languages and do things like suggest code better.
An important part of new AI models is making sure they avoid toxic, harmful, or biased responses. Meta has developed tools to filter their training data better and introduce safety features like Llama Guard. Only real-world testing will show how well the new models perform, though. Meta is also releasing the models through various cloud services so many developers can use them.
In the future, Meta aims to improve the Llama models further with capabilities like understanding images, multi-language conversations, and more advanced reasoning skills. Stay tuned for more updates from Meta on their new Llama 3 models; it seems they have a lot more in development for generating help from AI!
