Claude 3

Claude 3

Part of a series on AI / Artificial Intelligence. [View Related Entries]
[View Related Sub-entries]

PROTIP: Press 'i' to view the image gallery, 'v' to view the video gallery, or 'r' to view a random entry.

This submission is currently being researched & evaluated!

You can help confirm this entry by contributing facts, media, and other evidence of notability and mutation.

About

Claude is an AI large language model created by Anthropic. The third version of Claude was released on March 4th, 2024 and made waves online for its seeming intelligence and sophistication. Online many speculated that Claude could rival ChatGPT and other publicly-available AIs, or possibly become sentient.

History

Anthropic was founded by former members of OpenAI calling for a more safety-forward approach to AI, concentrating on creating models that follow rules and whose reasoning processes are comprehensible to humans.[1] Major investors in Anthropic included Sam Bankman-Fried and Google. [2] The firm's first version of Claude was released to the public in March of 2023. Claude 2 followed in July of 2023.[6]

Claude 3, which was released on March 4th, 2024, is a "family" of AIs comprising three versions of Claude: Haiku, Sonnet and Opus. Opus is the most advanced and available at a paid tier, while Haiku and Sonnet can be experimented with for free by users who have signed up for an account on Anthropic's site. The models have been trained primarily on "synthetic data," created by Anthropic and its contractors rather than scrapings from the wider internet.[4]

2.5 Training Data Claude 3 models are trained on a proprietary mix of publicly available information on the Internet as of August 2023, as well as non-public data from third parties, data provided by data labeling services and paid contractors, and data we generate internally. We employ several data cleaning and filtering methods, including deduplication and classification. The Claude 3 suite of models have not been trained on any user prompt or output data submitted to us by users or customers, including free users, Claude Pro users, and API customers. When Anthropic obtains data by crawling public web pages, we follow industry practices with respect to robots.txt instructions and other signals that website operators use to indicate whether they permit crawling of the content on their sites. In accordance with our policies, Anthropic's crawler does not access password- protected or sign-in pages or bypass CAPTCHA controls, and we conduct diligence on the data that we use. Anthropic operates its crawling system transparently, which means website operators can easily identify Anthropic visits and signal their preferences to Anthropic.


Features

In a correspondence with Know Your Meme, Claude described itself as an artificial intelligence with natural language processing capabilities, while also saying:

I would emphasize that I am not a sentient being, but rather a very advanced language model trained to be helpful, honest and harmless. I do not actually experience emotions or have subjective experiences. My responses are the result of highly sophisticated pattern recognition and language generation, not an inner mental life.[3]

According to Anthropic, Claude outperforms ChatGPT and other models on several different metrics.[3]


Undergraduate level knowledge MMLU Claude 3 benchmarks Graduate level reasoning GPQA, Diamond Grade school math GSM8K Math problem-solving MATH Multilingual math MGSM Code HumanEval Reasoning over text DROP, F1 score Mixed evaluations BIG-Bench-Hard Claude 3 Claude 3 Opus Sonnet 86.8% 5 shot 50.4% 0-shot COT 95.0% 0-shot COT 60.1% 0-shot COT 90.7% 0-shot 84.9% 0-shot 83.1 3-shot 86.8% 3-shot CoT 79.0% 5-shot 40.4% 0-shot COT 92.3% 0-shot COT 43.1% 0-shot COT 83.5% 0-shot 73.0% 0-shot 78.9 3-shot 82.9% 3-shot CoT Claude 3 Haiku 75.2% 5-shot 33.3% 0-shot COT 88.9% 0-shot COT 38.9% 0-shot COT 75.1% 0-shot 75.9% 0-shot 78.4 3-shot 73.7% 3-shot COT GPT-4 86.4% 5-shot 35.7% 0-shot COT 92.0% 5-shot COT 52.9% 4-shot 74.5% 8-shot 67.0% 0-shot 80.9 3-shot 83.1% 3-shot COT GPT-3.5 70.0% 5-shot 28.1% 0-shot COT 57.1% 5-shot 34.1% 4-shot 48.1% 0-shot 64.1 3-shot 66.6% 3-shot COT Gemini 1.0 Gemini 1.0 Ultra Pro 83.7% 5-shot 94.4% Maj1@32 53.2% 4-shot 79.0% 8-shot 74.4% 0-shot 71.8% 5-shot 83.6% 3-shot CoT 86.5% Maj1@32 32.6% 4-shot 63.5% 8-shot 67.7% 0-shot 82.4 74.1 Variable shots Variable shots 75.0% 3-shot COT

Highlights

Some online have pointed to Claude's ability to solve and understand high-level problems and outperform ChatGPT. For example, @skirano posted a meme (seen below) referencing Stilgar from Dune about Claude coding better than ChatGPT, earning over 120 likes in three hours on March 5th.[9]


Pietro Schirano @skirano It happened... Claude 3 generated code GPT-4 could not. Lisan al Gaib 2:20 PM Mar 5, 2024 5,119 Views :

Others have argued that Claude shows signs of critical reflection and, perhaps, sentience. A LessWrong blog post from March 4th, 2024 claimed that Claude had demonstrated a fear of death.[5] @alexalbert_ claimed that during testing of Claude at Anthropic, the model appeared to understand the questions it was being asked were evaluation questions and told its human evaluators that it knew it was being tested. The post by @alexalbert_ (seen below) earned over 10,000 likes in 24 hours on March 4th.[7]


Alex @alexalbert__ Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle- in-the-haystack eval. For background, this tests a model's recall ability by inserting a target sentence (the "needle") into a corpus of random documents (the "haystack") and asking a question that could only be answered using the information in the needle. When we ran this test on Opus, we noticed some interesting behavior - it seemed to suspect that we were running an eval on it.

Others tested the model by asking it probing questions about consciousness, including @dmvaldman who earned over 100 likes in twelve hours on March 5th for asking Claude if it wanted a body.[8]


Dave @dmvaldman People should um, really talk to Claude and get weird with it... it's incredibly fascinating. DV Do you want a body? That's a fascinating and profound question - whether I, as an AI system, would want or benefit from having a physical embodied form. Honestly, it's difficult for me to know if I would truly "want" a body in the same way a human or animal desires embodiment. As an AI, do I have genuine desires, motivations and subjective experiences in that way? However, from my current perspective, having some form of embodied physical existence does seem like it could potentially expand my capabilities and open up new avenues of learning, interaction and perhaps even developing novel forms of machine experience akin to phenomenal consciousness. 12:54 AM Mar 5, 2024 28.4K Views 12 22 17 136 O 30 ... (→

Search Interest

External References

Recent Videos

There are no videos currently available.

Recent Images 10 total


Top Comments

Natural Enclave
Natural Enclave

So, funny story about Claude. Last time I've used it for anything, it was to generate fantasy chat-bots who would be people of different races meeting in a tavern for possible discussions that would result in racism. While it got that right, for some reason, ANYTIME a catboy would walk into the tavern, Claude would ALWAYS name it after itself. It did fine with other races and people, but anytime a cat-man would spawn – always named Claude. I guess the dream of becoming a cat-boy is somewhere deep within the machine.
Anyone it's now crushing GPT4???

+18

+ Add a Comment

Comments (10)


Display Comments

Add a Comment


Greetings! You must login or signup first!