GPT-4 Did Not Get Even 30% Success Rate on Simpler Tasks: Meta's Researchers

GPT-4 Did Not Get Even 30% Success Rate on Simpler Tasks: Meta's Researchers"

www.analyticsdrift.com Image source: Analytics Drift

Yann LeCun & Meta present GAIA

[{"selector":"#anim-8b92f5f1-005a-4255-abd2-790e751d5890","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ed33eb48-7712-4f98-8fed-38b89819f1a9","keyframes":{"transform":["translate3d(0px, 199.12619%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-c288ab8d-7775-4081-b429-f534da7737a4","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-65d16343-2851-4611-904d-5d5820685903","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] Under LeCun's guidance, Meta introduces GAIA, revolutionizing AI's approach to everyday challenges. Image source: AD

A benchmark for General AI Assistants

[{"selector":"#anim-7eaad5c4-5890-4d32-a9cb-dbaf438669d1","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-cdfd8511-15b9-4f69-962e-693940eedc21","keyframes":{"transform":["translate3d(0px, 200.24977%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-af420615-aba7-4c47-bbe4-c5fe577fd86f","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-1f703bd4-9d44-4825-87f0-e3f93df9468d","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] GAIA emerges as a gold standard, pushing AI towards unprecedented versatility and adaptability. Image source: Canva

Challenging AI with real-world tasks

[{"selector":"#anim-965e0527-b1a7-4a6f-9905-a2db61ded42a","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-5885a583-c339-4b50-b5f1-1b8ed7f91a40","keyframes":{"transform":["translate3d(0px, 222.06116%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-4a91792f-69b6-495f-9a31-68500123aa0f","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-176a1ca6-9ad3-41a4-9e31-fd9bd6b9e83a","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] Designed to mimic daily tasks, GAIA tests AI's ability to navigate and solve complex problems. Image source: Canva

GAIA: Bridging human-AI performance gap

[{"selector":"#anim-9ce755e7-5dd5-4cb7-bfa6-e4bdfd02403e","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-8cc1bb97-fe01-4225-a9b0-6febc7241c4f","keyframes":{"transform":["translate3d(0px, 226.40902%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-1d674153-3662-4fa0-b231-871f9733df21","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ee6b00c8-6341-436c-9d1a-e8162e6b7fac","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] GAIA aims to narrow the divide, enhancing AI's understanding and execution to human levels. Image source: Canva

Involving reasoning, web browsing, and tool use

[{"selector":"#anim-a7a561df-586f-4c5e-a94c-870df3bff467","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-3b722fe0-21d2-4eb0-89eb-87885507beab","keyframes":{"transform":["translate3d(0px, 195.75538%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-f89fcfde-7e58-4ded-8247-6703f5874faa","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-fbb7f91a-5684-41ba-8ca8-9c2b1e82f4fd","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] Through reasoning, web exploration, and tool utilization, GAIA assesses AI's practical intelligence. Image source: Meta

466 questions designed to test AI

[{"selector":"#anim-6fda7ec5-794e-4017-8e2c-3e032adfd760","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ca5bdbec-4199-43b6-95e1-723e9356c04e","keyframes":{"transform":["translate3d(0px, 198.02467%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-135683b4-6f3b-4845-8fc9-63ae35db0e04","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-9f7bca56-39ed-4c8a-a123-1e70aa89bae5","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] With 466 diverse questions, GAIA challenges AI across multiple domains, testing depth and breadth. Image source: Canva

Human vs AI: A performance showdown

[{"selector":"#anim-741592d9-7bba-4c40-a43c-00cc7e07aa3e","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-9941ed26-0e89-425c-8a75-d62b0f1b5986","keyframes":{"transform":["translate3d(0px, 194.63180%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-a70a82c2-61a4-4f43-bc50-fc50723e7fb1","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-06140e2a-4b80-4ed5-a591-7b2fad1036fb","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] GAIA sets the stage for a direct comparison, showcasing how closely AI can emulate human reasoning. Image source: Meta

GAIA's goal: Simulating average human robustness

[{"selector":"#anim-b01e74d7-af6f-4539-b238-71f816ca2b82","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-30a27232-4537-49b2-91b3-61b725b98b28","keyframes":{"transform":["translate3d(0px, 233.65538%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-cfa41d2b-fead-4e40-ba25-cd505fcec8ff","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-8437ee40-132b-41ab-bd3f-8ab034840b3d","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] Striving for AI that mirrors human resilience, GAIA evaluates AI's adaptability to change. Image source: Canva

A milestone in AI research and development

[{"selector":"#anim-687fa091-8174-491f-b934-2b37916c96b3","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-2f69ca7f-aebb-4025-8287-38a6bb0f82d4","keyframes":{"transform":["translate3d(0px, 227.85827%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-7456077b-3ba1-4b48-8a55-27cfd77a4362","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-bafc2117-a86f-4040-b772-42952b308488","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] GAIA represents a significant milestone, marking a new era in AI's evolution and capability. Image source: Canva

Joining forces for the next AI breakthrough

[{"selector":"#anim-be454749-c572-4559-ac8b-3ca52da73586","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-d45bc4dd-8a1e-4509-84ba-a046c267a8c4","keyframes":{"transform":["translate3d(0px, 232.20613%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-aba24a03-a126-4964-98fd-97f5b6499749","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-29519693-de43-4d06-a89b-a8bf929a46ba","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] Together, Meta and LeCun are at the forefront, steering AI towards the next major breakthrough. Image source: Canva

GAIA's Challenging Reality for AI

[{"selector":"#anim-7f3e0f5d-90d1-4a41-ae2d-a93708fe8395","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-837b8246-434a-40e2-a23e-2176afbe255e","keyframes":{"transform":["translate3d(0px, 200.24977%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-e4026a5b-e1a3-4301-8f89-24a0b94a563d","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-0301ea8a-cad5-47d2-a929-913435fdb25f","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] Even the most capable AIs struggle on GAIA, with GPT-4 not exceeding a 30% success rate on simpler tasks. Image source: Meta

Human vs. AI: The Gap Remains

[{"selector":"#anim-c9db1769-e9af-4279-8985-c243da9cda82","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-dd71ed6a-ef42-4d28-b4ac-89589cb8e624","keyframes":{"transform":["translate3d(0px, 200.24977%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-e0e7418f-7469-4d88-a474-5556e1115bf5","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-a93455ba-5612-4896-841a-d9b9723ae37e","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] Compared to AI's performance, humans show a robust 92% success rate, highlighting the intelligence gap. Image source: Canva

The Path to True General AI

[{"selector":"#anim-079e1c7b-db4c-482a-ad76-dea3d48c4fa7","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-a85cadda-4da4-4672-a586-3670364e0e47","keyframes":{"transform":["translate3d(0px, 200.24977%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-74f41716-dbb8-4dce-89dc-65dcb791ae99","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-3fe82356-b325-4861-969d-7de2af47aef7","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] GAIA's results underscore the journey towards achieving a General AI, marking significant milestones ahead. Image source: Canva

GAIA: Beyond Benchmarking

[{"selector":"#anim-6d20a05a-6d52-4e46-98f8-a42b05f386c1","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-2b3fa6fb-c2bd-4630-a2e7-111be90aa123","keyframes":{"transform":["translate3d(0px, 199.12619%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b35e969e-5a15-496f-95e0-2729af6416c0","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-251a1dc6-bad3-4dbf-8f53-2378a04f06df","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] GAIA not only tests AI but also invites the global community to contribute towards safer, more capable AI systems. Image source: Canva

The Future of AI, Post-GAIA

[{"selector":"#anim-e8bbb20a-223c-4e69-9009-d2d122e74d0d","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b006a48a-c79b-4e57-8bbd-2fdfb5dc016b","keyframes":{"transform":["translate3d(0px, 203.62051%, 0)","translate3d(0px, 0px, 0)"]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-6f1b1d8b-30a1-4c69-9267-4546fc3c441f","keyframes":{"opacity":[0,1]},"delay":120,"duration":1300,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-480cd874-442a-4e07-b890-cff4d3cf5d57","keyframes":{"opacity":[0,1]},"delay":120,"duration":1200,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] Success in GAIA will herald a new era in AI, pushing beyond current capabilities towards more generalized intelligence. Image source: Canva Read more

Get the latest updates on AI developments

[{"selector":"#anim-cb50e7c9-7631-48c7-85b0-cb6f91d58c05","keyframes":{"opacity":[0,1]},"delay":200,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-be76cd8c-d8dc-4e18-922a-8cbbef479679","keyframes":{"transform":["translate3d(-103.35917%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-5c1c2de0-e552-4166-84f7-0a1af4a2570b","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-f7d80847-868a-4566-9f1a-5729472f2545","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}] [{"selector":"#anim-5fd04b20-235d-4087-bd44-606a637ed76a","keyframes":{"transform":["translate3d(134.00810%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-6f32500a-96d7-4fed-b0f2-38758a11ce05","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-1b20bb0f-befb-4056-9f27-ca916b977a4a","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}] [{"selector":"#anim-2726157c-9d0c-45ae-a1bf-57471bcbf67e","keyframes":{"transform":["translate3d(129.34363%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-926c2f05-7841-4abe-9715-92c744cb569b","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-4c8a7f25-0ca0-468a-ab43-4f1c0ff7a0f2","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}] Produced by: Analytics Drift Designed by: Prathamesh Join Now