Meta’s free Code Llama AI programming tool closes the gap with GPT-4?

Community
1

Anyone have a chance to play with it yet?

Meta’s latest update to its code generation AI model, Code Llama 70B, is “the largest and best-performing model” yet. Code Llama tools launched in August and are free for both research and commercial use. According to a post on Meta’s AI blog, Code Llama 70B can handle more queries than previous versions, which means developers can feed it more prompts while programming, and it can be more accurate.

Code Llama 70B scored 53 percent in accuracy on the HumanEval benchmark, performing better than GPT-3.5’s 48.1 percent and closer to the 67 percent mark an OpenAI paper (PDF) reported for GPT-4.

2 Likes
2

Wes Roth with a great review as always, one of my favorite AI youtubers. :slight_smile:

I am writing a blog post about code llama 70b and will be testing it later today.

Have we heard much from OpenAI recently about improving coding capabilities?

3 Likes
3

:crossed_fingers: A new codex model is on my 2024 wishlist…

2 Likes
4

GPT-5 talk has mostly been around multimodal but I am sure there will be improvements across the board.

I threw together a quick blog about the Code Llama 70B news.

The Meta blog is definitely worth reading, I had some trouble interpreting the benchmarking.

Looking forward to testing it out.

1 Like