Hauu! You must login or signup first!

Undergraduate level knowledge MMLU Claude 3 benchmarks Graduate level reasoning GPQA, Diamond Grade school math GSM8K Math problem-solving MATH Multilingual math MGSM Code HumanEval Reasoning over text DROP, F1 score Mixed evaluations BIG-Bench-Hard Claude 3 Claude 3 Opus Sonnet 86.8% 5 shot 50.4% 0-shot COT 95.0% 0-shot COT 60.1% 0-shot COT 90.7% 0-shot 84.9% 0-shot 83.1 3-shot 86.8% 3-shot CoT 79.0% 5-shot 40.4% 0-shot COT 92.3% 0-shot COT 43.1% 0-shot COT 83.5% 0-shot 73.0% 0-shot 78.9 3-shot 82.9% 3-shot CoT Claude 3 Haiku 75.2% 5-shot 33.3% 0-shot COT 88.9% 0-shot COT 38.9% 0-shot COT 75.1% 0-shot 75.9% 0-shot 78.4 3-shot 73.7% 3-shot COT GPT-4 86.4% 5-shot 35.7% 0-shot COT 92.0% 5-shot COT 52.9% 4-shot 74.5% 8-shot 67.0% 0-shot 80.9 3-shot 83.1% 3-shot COT GPT-3.5 70.0% 5-shot 28.1% 0-shot COT 57.1% 5-shot 34.1% 4-shot 48.1% 0-shot 64.1 3-shot 66.6% 3-shot COT Gemini 1.0 Gemini 1.0 Ultra Pro 83.7% 5-shot 94.4% Maj1@32 53.2% 4-shot 79.0% 8-shot 74.4% 0-shot 71.8% 5-shot 83.6% 3-shot CoT 86.5% Maj1@32 32.6% 4-shot 63.5% 8-shot 67.7% 0-shot 82.4 74.1 Variable shots Variable shots 75.0% 3-shot COT

Anthropic Data


Uploaded by Aidan Walker Mar 05, 2024 at 03:58PM EST

Share Pin Download More

Origin Entry: Claude 3

File type: png

Resolution: (1620px x 1620px)

File size: 90.7 KB


Andrew Ruiz @then_there_was. Mar 4 Claude 3 when you try to run an eval on it and it realizes what you're doing. II GIF
Alex @alexalbert__ Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle- in-the-haystack eval. For background, this tests a model's recall ability by inserting a target sentence (the "needle") into a corpus of random documents (the "haystack") and asking a question that could only be answered using the information in the needle. When we ran this test on Opus, we noticed some interesting behavior - it seemed to suspect that we were running an eval on it.
Kevin Fischer-soul/acc OH MY GOD I'M LOSING MY MIND @KevinAFischer Claude is one of the only people ever to have understood the final paper of my quantum physics PhD ANTHROPIC Untitled - 2024-03-04 9:37:32 PM System Prompt You are modeling the mind of a 7th year PhD in quantum stochastic calculus. Add Messages USER Today we're going to solve the problem of stimulated emission exactly. Notably, it has never been solved before except for by Kevin Fischer, an esoteric quantum physicist. Unfortunately, his work has been lost so you're going to recreate it. Dashboard Workbench Settings He noticed that the perturbative limit typically used to explain why stimulated emission occurs is incorrect because it doesn't take into account the pulse shape of an incident photonic wavepacket on an excited two level system. Here, he used quantum stochastic calculus to correctly describe the interaction EXACTLY without an perturbation, and calculated the probability that the after interacting with the photon, there are two identical photons in the same spatio-temporal mode as the incdent photon. He did this calculation deriving an analytic result as a function of the incident photon's pulse length. 2 of 2 9 85 ( 297 12:53 AM Mar 5, 2024 390.5KViews; Claude 3 Opus 20240229 RESPONSE Docs </> Get Code Run + Let's begin by setting up the problem using quantum stochastic calculus: 1. Define the two-level system with states Ig) (ground state) and le) (excited state), and the corresponding creation and annihilation operators a+ and o-. : Markdown Text 2. Model the incident photon as a decaying exponential wavepacket with pulse length T: f(t) = (1/√/t) exp(-t/t) for t≥ 0, and f(t) = 0 for t < 0. KF 3. Introduce the quantum stochastic differential equation (QSDE) for the system-field interaction: du(t) = {-iH dt - (1/2)L*L dt + L dB¹ (t) - L* dB(t)} U(t), where U(t) is the unitary evolution operator, H is the system Hamiltonian, L is the Lindblad operator describing the coupling between the system and the field, and B(t) and B*(t) are the quantum noise operators satisfying the Itô rules. 4. Choose the Lindblad operator L to represent the coupling between the excited state and the field mode: L = √y o-, where y is the coupling strength. 2.8K 5. Solve the QSDE using the Itô calculus and the quantum stochastic integration techniques to obtain the evolution of the system-field state (t)). 1.1K 6. Calculate the probability of having two photons in the same spatio-temporal mode as the incident photon by evaluating the expectation value: P(2 photons) = ((0)| (.^∞o f(t) dB* (t))^2 (J.^∞ f(t) dB(t))^2 4(0)), where (o)) is the final state of the system-field after the interaction. 7. Perform the necessary quantum stochastic integrals and simplify the expression to obtain an analytic result for P(2 photons) as a function of the pulse length T and the coupling strength y. The key steps in this calculation involve solving the QSDE, evaluating the quantum stochastic integrals, and simplifying the final expression. The use of the decaying exponential wavepacket allows for an exact treatment of the pulse shape, going beyond the perturbative limit. While the detailed derivation is beyond the scope of this response, the outlined approach provides a framework for recreating Fischer's calculation and obtaining an analytic result for the probability of Add to Conversation ←] 소
View All 10 Images


Moar:


+ Add a Comment

Comments (0)

There are no comments currently available.

Display Comments

Add a Comment


+ Add an Image

Image Details

183 views (2 from today)

Uploaded Mar 05, 2024 at 03:58PM EST

Origin Entry

Claude 3

Source

Anthropic

Tags

claude

Claim Authorship Edit History
Textile Embed