Meta has unveiled Code Llama, an advanced Large Language Model (LLM) that possesses the ability to develop code as well as natural language descriptions related to code in response to prompts given in either code or natural language.
Currently, Code Llama can be used for both research and commercial purposes without any cost. It is built on top of Llama 2 and is offered in three versions: Code Llama – the core code model, Core Llama – Python, designed specifically for Python language, and Code Llama – Instruct, which is fine-tuned to understand natural language instructions.
There are three variants of Code Llama, featuring 7B, 13 B, and 34B parameters, respectively. Each model is trained with a dataset of 500B tokens containing code and code-related information. The Code Llama – Python is fine-tuned using a dataset containing 100B tokens of Python code.
Beyond its necessity for generating extended programs, having a longer input sequence brings new possibilities for a code language model. For example, users can give the model more information from their existing code or pass the entire code to make its generated code more relevant. These features can be helpful to developers while debugging when trying to fix issues in large codebases.
Overall, Code Llama is designed to cater to the needs of software engineers across diverse sectors. This includes professionals working in research, industry open-source projects, and businesses.