Meta has unveiled Code Llama, an advanced Large Language Model (LLM) that possesses the ability to develop code as well as natural language descriptions related to code in response to prompts given in either code or natural language.
Currently, Code Llama can be used for both research and commercial purposes without any cost. It is built on top of Llama 2 and is offered in three versions: Code Llama – the core code model, Core Llama – Python, designed specifically for Python language, and Code Llama – Instruct, which is fine-tuned to understand natural language instructions.
There are three variants of Code Llama, featuring 7B, 13 B, and 34B parameters, respectively. Each model is trained with a dataset of 500B tokens containing code and code-related information. The Code Llama – Python is fine-tuned using a dataset containing 100B tokens of Python code.
Read More: Cohere Introduces LLM University to Teach about LLMs and NLP for Free
In addition to its code completion features, it can also be used for debugging purposes. It supports popular programming languages, including Java, Python, Typescript (Javascript), Bash, C++, Bash, and PHP.
Beyond its necessity for generating extended programs, having a longer input sequence brings new possibilities for a code language model. For example, users can give the model more information from their existing code or pass the entire code to make its generated code more relevant. These features can be helpful to developers while debugging when trying to fix issues in large codebases.
Developers can request access to Llama from the Meta Webpage, and their training methodologies are also accessible through the GitHub repository.
Overall, Code Llama is designed to cater to the needs of software engineers across diverse sectors. This includes professionals working in research, industry open-source projects, and businesses.
