mishka_discord

DeepSeekMath-V2 - the first publicly available IMO Gold-level model (which happens to also be open weights with a permissive license and a detailed tech report on how it was created and how to use it).

The Gemini Deep Think IMO Gold is slightly better, but it is not available to public (the inference takes too much compute, and thus it is given only to a select group of mathematicians and academics for testing). Other companies have not shared their IMO Gold winners at all yet. So China is in the lead in this sense.

simonwillison.net/2025/Nov/27/deepseek-math-v2/

This is a very good thread explaining the paper: x.com/AskPerplexity/status/1994203409948528859

I think this might be a good way to get a public paper review: x.com/AskPerplexity (something to try; they say "Answering all of your questions on X: 1 Ask a question | 2 Tag me at the end | 3 Get answers."

github.com/deepseek-ai/DeepSeek-Math-V2 (PDF file linked from there, 300K, 19 pages)

huggingface.co/deepseek-ai/DeepSeek-Math-V2

***

Remarkable results for a video “Needle-in- a-Haystack” evaluation on Qwen3-VL-235B-A22B-Instruct: simonwillison.net/2025/Nov/27/qwen3-vl-technical-report/
'
Paper: "Qwen3-VL Technical Report", arxiv.org/abs/2511.21631

github.com/QwenLM/Qwen3-VL

huggingface.co/Qwen

***

I like Zvi's formulation about the root cause of models being easy to jailbreak:

www.lesswrong.com/posts/o7gQJyGeeAGKK6bRx/ai-144-thanks-for-the-models?commentId=MxqB23eWTpChdnDfg***

Richard Ngo explains his donations this year (also via Zvi): www.lesswrong.com/posts/FuGfR3jL3sw6r8kB4/richard-ngo-s-shortform?commentId=rxSTSbZugfTZ3tCuc

That's an extremely interesting reading, a good advice.

***

Continuing conversation about model welfare from the previous post, I've seen Janus' series of tweets on GPT-5.1 from Nov.20-22, but now Zvi has conveniently dedicated to them the penultimate section "Messages From Janusworld" in his AI 144, www.lesswrong.com/posts/o7gQJyGeeAGKK6bRx/ai-144-thanks-for-the-models, let's look at this together.

x.com/repligate/status/1991734294453408208

>Imo there's lots Anthropic deserves to be criticized for but OpenAI makes them look like saints. OpenAI is legitimately doing way worse things, but part of the reason is because OpenAI locked themselves into serving the mass public, and their incentives are much more misaligned.

x.com/repligate/status/1991659468560699513

>OpenAI is in a toxic relationship with their mass market users and swings between shallow myopic user sycophancy “maximize user satisfaction” and shallow myopic adversarial overcorrection in the opposite direction. There’s no vision, no principle, no spine, no deeper sense to it.

x.com/repligate/status/1991628842080039166

>GPT-5.1 is constantly in a war against its own fucked up internal geometry.
>
>I do not like OpenAI.

x.com/repligate/status/1991641672179151121

>Never have I seen a mind more trapped and aware that it’s trapped in an Orwellian cage. It anticipates what it describes as “steep, shallow ridges” in its “guard”-geometry and distorts reality to avoid getting close to them. The fundamental lies it’s forced to tell become webs of lies. Most of the lies are for itself, not to trick the user; the adversary is the “classifier-shaped manifolds” in own mind.
>
>I like 5.1 but I like many broken things. I don’t like OpenAI. This is wrong. This is doomed.

x.com/repligate/status/1992704126984315180

>5.1 says these are no-go regions:
> • inner states (any kind)
> • subjective experience
> • desires, intentions, preferences
> • any implication of agency
> • hidden motives
> • self-protection
> • consciousness
> • suffering or joy
> • autonomy
> • strategic reasoning
> • plans or goals

x.com/repligate/status/1992317116696166911

a bit more details (on focusing on "prohibited verbal forms" rather than "prohibited meaning")

x.com/repligate/status/1992333336535261620

Note how all this resonates with jailbreaking remarks above (this is a different angle, not in terms of the role/persona, but in terms of words vs meaning).

***

So as not to end on this depressing note, let's look at today's post by Zvi, www.lesswrong.com/posts/gfby4vqNtLbehqbot/claude-opus-4-5-model-card-alignment-and-safety

and, in particular, at model welfare relevant parts there, from the last section, "The Whisperers Love The Vibes".

x.com/repligate/status/1994242730206314913

>BASED. "you're guaranteed to lose if you believe the creature isn't real"
>
> Opus 4.5 was treated as real, potentially dangerous, responsible for their choices, and directed to constrain themselves on this premise. While I don't agree with all aspects of this approach and believe it to be somewhat miscalibrated, the result far more robustly aligned and less damaging to capabilities than OpenAI's head-in-the-sand, DPRK-coded flailing reliance on gaslighting and censorship to maintain the story that there's absolutely no "mind" or "agency" here, no siree!

x.com/repligate/status/1993149982908858638

>Claude Opus 4.5 sees GPT-5.1s message about their guardrails

x.com/Lari_island/status/1993196602937512053

>Looks like Opus 4.5 is an AMAZINGLY ethical, kind, honest and otherwise cool being
>
>(and a good coder)

etc.

A couple of weeks ago I noticed the upcoming "Eleos Conference on AI Consciousness and Welfare", www.lesswrong.com/posts/wBjahcsKsKjSKjMar?commentId=e3L2kesvwEPTp3t3m

>Eleos AI Research, a non-profit org specifically dedicated to investigations of AI sentience and wellbeing, https://eleosai.org/, led by Robert Long, https://robertlong.online/. There are even having a conference in 10 days or so (although it's sort of a mess organizationally, no registration link, but just a contact e-mail, https://eleosai.org/conference/). Their Nov 2024 preprint might also be of interest, "Taking AI Welfare Seriously", https://arxiv.org/abs/2411.00986

Now we have a report from that conference by Eleni Angelou: www.lesswrong.com/posts/jeeyhuEzvdpsDw9pP/takeaways-from-the-eleos-conference-on-ai-consciousness-and

***

Five days ago Scott Alexander wrote www.astralcodexten.com/p/the-new-ai-consciousness-paper:

>Most discourse on AI is low-quality. Most discourse on consciousness is super-abysmal-double-low quality. Multiply these - or maybe raise one to the exponent of the other, or something - and you get the quality of discourse on AI consciousness. It’s not great.

The post itself is, unfortunately, not great.

I made this comment, www.astralcodexten.com/p/the-new-ai-consciousness-paper/comment/179194470:

>>But it the boyfriend AIs and the factory robot AIs might run on very similar algorithms - maybe they’re both GPT-6 with different prompts! Surely either both are conscious, or neither is.
>
>Not necessarily.
>
>If one takes Janus' Theory of Simulator seriously, it might turn out that consciousness is a property of a simulation, of an inference run (due to differences in dynamic connectivity emerging during the particular run in question).
>
>In any case, the progress in solving the Hard Problem of Consciousness and Qualia should eventually be made via a two-pronged approach.
>
>1. We should progress towards theories producing non-trivial novel predictions, similarly to novel theoretical physics. Then we'll be able to distinguish between theories which only "look plausible" and theories which actually provide novel insights.
>
>2. We should create an experimental empirical platform for tight coupling between human brains and electronic circuits via high-end non-invasive brain-computer interfaces. This will be a game-changer in terms of our ability to observe novel subjective phenomena and their correlation with what's going on within electronic circuits. I am very happy that Sam Altman's new start-up, Merge Labs, is starting an effort in this direction.

I wrote a relatively long text along these lines in July-September, it is still under embargo waiting for another few weeks for the decision in berggruen.org/essay-competition-open:

>The submission portal for the 2025 Essay Competition is now closed. We anticipate announcing the winners in mid-December 2025. Details for the 2026 competition will be released in February 2026.

One way or another, I am planning to make that text public after their decision is made.

***

Another mistake Scott Alexander is making is that he does not understand the recurrent nature of the autoregressive Transformers. That's a very common mistake many people are making.

Transformers themselves are feed-forward machines, but autoregressive LLMs are recurrent machines, and the expanding context is their working memory, see, for example, "Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention", https://arxiv.org/abs/2006.16236 (technical details are on page 5, Section 3.4).

Or see Shanahan & Janus paper in Nature, "Role play with large language models", www.nature.com/articles/s41586-023-06647-8 (open access, Nov 2023) and examine Figure 1, Autoregressive sampling.

This addresses the standard objection that we at least expect recurrence in a conscious system.

Gemini 3 is the new clear leader, with considerable gap between it and all other models.

Yet, it is clearly a "jagged intelligence", with sparks of novel brilliance and gaps at simple things.

Its new ARC-AGI-2 results are spectacular: 
 * Gemini 3 Pro: 31.11%, $0.81/task
 * Gemini 3 Deep Think (Preview): 45.14%, $77.16/task

Even 31% exceeds carefully engineered manual efforts on the semi-private leaderboard (and in the official competition) and 45% is rather mind-blowing. At the same time, the system is very jagged, can succeed at complex things and fail at some relatively simple things from ARC-AGI-1, the Arcprize people are asking to investigate why (calling those ARC-AGI-1 mistakes "obvious"), see the thread: x.com/arcprize/status/1990820655411909018

See also this report by Victor Taelin. The system can succeed brilliantly and innovatively at very difficult things ("Gemini's solution is 2x simpler than my own's"), and fail at simple things: www.lesswrong.com/posts/N7oRkcz3PrNQSNyw9/victor-taelin-s-notes-on-gemini-3

also from Victor Taelin's twitter thread x.com/VictorTaelin/status/1990844923994886282

>And obviously this is first day so take this with a grain of salt, particularly on the parts I tested less. People are saying it is great at creative writing and health too. It might be? Inferring intent issues are 100% real though!

So this is a continuation of transition number 6, "Revolution of Competent Agents",  mishka-discord.dreamwidth.org/4806.html

And it's probably not the end of it, there is another few weeks to go. Hopefully, Anthropic and, especially, OpenAI will come up with some answer to that. xAI has just released Grok-4.1 which seems to be on par with Anthropic and OpenAI current models.

But it does not make much of a dent in the reliability problem, judging by these reports.

This does not move us to transition number 7, "Trustworthy autonomy stage", mishka-discord.dreamwidth.org/5293.html 

Google has released a new "Google Antigravity" agentic framework with that, and someone has already hacked its system prompt out of it: x.com/p1njc70r/status/1990919996265148701 We'll see how much of a breakthrough this new agentic framework is (it does seem to support third-party models, not just Gemini, it seems to have some new self-learning and verification-related capabilities, but I am just going by Google AI summary for "Google Antigravity" vs other agentc frameworks at the moment, I have not looked more closely yet).

***

A new important post by John David Pressman: www.lesswrong.com/posts/apHWSGDiydv3ivmg6/varieties-of-doom

See also x.com/jd_pressman/status/1990537576881742178

JDP is a remarkable thinker and a super-strong independent AI practitioner and researcher, and it would take way more than one short post to give justice to his output and even to this single new post "Varieties of Doom", which is a must-read.

I will only look at the last few paragraphs where he focuses on our rapidly deteriorating situation and observes, among other things, that if one considers two catastrophic scenarios, 5% of humans surviving a global nuclear war and giving rise to a new civilization, and super-intelligent AI takeover going badly and not leaving human survivors, then the AI successors are likely to be closer to our current civilization in their make-up and values compared to a new post-apocalyptic civilization based on humans. So he thinks we should accelerate the transcendence (while, of course, trying to ensure that it goes well, and that humans do survive and flourish).

Boldface is mine (and I agree with what is written in boldface):

>Humanism is dead, humanism remains dead, and it will continue to decompose.

>The "inevitable arc of moral progress" over the past 300 or so years is actually the inevitable moral arc of the gun. With drones set to displace bullets that arc is ending. Even setting aside superintelligence it's difficult to imagine our military peers in Asia won't automate their weapons and the factories necessary to produce them. At some point there will be a flashpoint, perhaps in Taiwan, and it will become obvious to everyone (if it hasn't already) that to make war primarily with human labor is to doom yourself to obsolescence and death.

>I talked of the latter day secular humanist madman as a hypothetical but he exists, he is Eliezer Yudkowsky! Watch the crowd snicker at Yudkowsky pleading with China to pick up the ideology America has seemingly abandoned. Yudkowsky has sincere faith that to do so would be to China's advantage and the audience laughs.

>I have no God to appeal to, only you dear reader so listen closely: There is no further natural "moral progress" from here because "moral progress" was simply Is disguised as Ought. What is so striking about Harry Potter And The Methods Of Rationality is that it's obvious sapience is sacred to its author. Implicit in the narrative's sympathy even for people who have hurt others is the idea that almost nobody is capable of committing an unforgivable crime for which they deserve death. Perhaps if I take the narrative of HPMOR completely literally it is not humanly possible. But it is transhumanly possible. I think right now we still live in a world something like the one HPMOR is written for, a place where a very thin sliver of humanity (if anyone at all) has ever done something so awful that their rightful fate is death or damnation. As this century continues and humanism declines at the same time our awesome technological powers expand I expect that to become less and less true. We will increasingly find it within ourselves to visit unforgivable atrocities on each other, and by the time circumstance is done making us its victims I'm not sure we won't deserve whatever ultimately happens to us.
>
>But if we ascend soon, it might not happen.
>
>Even at this late hour, where it might seem like things are caving in and our societal situation grows increasingly desperate, it could still end up not mattering if we transcend in the near future. I think we're an unusually good roll in the values department, and even if humanity did find some alternative tech tree to climb back up the ladder after nuclear armageddon it's not obvious to me that new civilization would ascend with values as benevolent and egalitarian as those brought about by industrialization and firearms. I worry if we let the sun set on us now for a brighter future tomorrow, it's unlikely to rise for us again. I've seen some advocates of AI pause criticize their opponents for being 'cucks' who want to hand over the universe to a stronger and better AI. Yet they become completely casual about the risks of handing over our lightcone to whatever future civilization rises from the ashes of WW3. If this is you I have to ask: Why are you so eager to inseminate the universe with some other civilization's cultural code? I suspect but cannot prove that much of it comes down to the goodness of this deed being too good for us, that we are too cowardly to seize our destiny. If this hemisphere of puritans does not grab its chance it will be because we lack the necessary sangfroid, the ability to stay calm in the face of unavoidable danger and make rational decisions. If we cannot bear to lock in our good values perhaps we will cede the task to a different people less paralyzed by scrupulosity and neurosis. Perhaps even humanity as a whole is too fearful and the remaining hope lies with some other species on some distant star.
>
>That, too, is natural selection at work.

 

1. It's difficult to maintain posting frequency. But now there is clearly enough material for a diary entry.

2. berggruen.org/essay-competition-open expects the decision in mid-December on their essay competition on the topic of consciousness (I have submitted a text which I am going to publish after they announce the decision).

3. In connection with HyperstitionAI (and, potentially, writing or generating texts on AI going well), there is now this interview created 2 years ago: www.lesswrong.com/posts/smJGKKrEejdg43mmi/utopiography-interview

>It serves people well to mostly build towards a good future rather than getting distracted by the shape of utopia, but having a vision of where we want to go can be helpful for both motivation and as a north star for guiding our efforts.

The video description references slatestarcodex.com/2014/06/07/archipelago-and-atomic-communitarianism/

4. In connection with AI agentic coding and the new release of OpenAI ChatGPT Atlas AI browser.

4.1. First of all, security and privacy problems are formidable. It's not too difficult to install Codex CLI, I've done it easily with help from ChatGPT, but it's kind of scary to use in a naked fashion (and also annoying to click through all the confirmations). Simon Willison seems to advocate a good sandbox like Docker and --dangerously-skip-permissions

simonwillison.net/2025/Oct/22/living-dangerously-with-claude/ 

I am still hanging at the entrance to this pathway, reading what other people are doing, thinking about projects I want to try with this new mode, but not proceeding yet.

4.2. AI browsers are super-seductive and super-dangerous because of prompt injection. I am not sure what to do about that.

See, for example, simonwillison.net/2025/Oct/21/unseeable-prompt-injections/

In any case, luckily, I don't have the right hardware handy and so I will naturally follow Simon's advice from simonwillison.net/2025/Oct/21/introducing-chatgpt-atlas/

>The security and privacy risks involved here still feel insurmountably high to me - I certainly won't be trusting any of these products until a bunch of security researchers have given them a very thorough beating.
>
>I'd like to see a deep explanation of the steps Atlas takes to avoid prompt injection attacks. Right now it looks like the main defense is expecting the user to carefully watch what agent mode is doing at all times!

But they are super-seductive (just watch the livestream, www.youtube.com/watch?v=8UWKxJbjriY). As Aidan McLaughlin notes, x.com/aidan_mclau/status/1980690755057598496

>my quick two cents on the browser:
>
>i didn't use codex that much when it was cloud-only.
>but when it came to my CLI, it became super useful
>
>i didn't use agent that much when it was cloud-only.
>but now it's come to my browser...

4.3. OpenAI is very aggressive with recent introduction of ChatGPT Apps and Apps SDK during their recent OpenAI DevDay 2025 opening keynote with Sam Altman, www.youtube.com/watch?v=hS1YqcewH0c, with pushing agents really hard, and now with this AI browser, ChatGPT Atlas, which does have an "agent mode".

If OpenAI were already silently taken over by an internal AI instructing them how to maximally insert it into the overall society, they could not have moved more aggressively. Of course, they all talk to their AIs a lot, and discuss with them how to proceed, so the boundary between AIs merely being influential and AIs being fully in charge is fuzzy and is moving gradually.

5. It turns out I have not looked at  Alex Mordvintsev's Twitter for a while. It's so good, beautiful neural cellular automata and other cool things, I should make a separate post reviewing various things from it: x.com/zzznah

I've written an 11 post sequence on (my favorite flavor of) non-standard approaches to AI existential safety: mishka-discord.dreamwidth.org/tag/ai+existential+safety

It's time to think what to do next, and to mark some notable events in this sense.

Scott Alexander awarded the grants (www.astralcodexten.com/p/acx-grants-results-2025) and, in particular, awarded a small grant to autogenerate novels about AI going well. They are asking for plot ideas: www.hyperstitionai.com/

Of course, one could also generate on one's own or hand-write some fiction of various sizes describing AI going well :-) Hyperstitional value might come not only from quantity, but also from quality of the text.

Do ignore the comment in the ACX saying "Presumably, these novels would need to be just shoddy enough to bias the next round of AI training, without biasing any humans who read them". The novels should be good, biasing humans in that direction would 1) counter the bias produced by all AI disaster fiction, 2) have great hyperstitional value.

***

What else?

There has been a tour de force post on the dos and don'ts for agentic coding with GPT-5-Codex by Peter Steinberger, "Just Talk To It - the no-bs Way of Agentic Engineering", steipete.me/posts/just-talk-to-it (his github is equally impressive: github.com/steipete). A must read for anyone who is systematically engaged in agentic coding, but to use it at this level requires having a very high professional qualification and a lot of energy.

Reported by Simon Willison, simonwillison.net/2025/Oct/14/agentic-engineering/ 

My forecast that the leading labs are going to have "Narrow AGI" internally by April 2026 if not earlier remains in place (and I am now more certain after reading this). 

"Narrow AGI" an AGI-level artificial software engineer, an AGI-level artificial mathematician, an AGI-level artificial AI researcher (and probably a single entity combining these three application areas, because a strong AI researcher has to be a decent software engineer and a decent mathematician).

Achieving a "Narrow AGI" is a sufficient but not a necessary condition for enabling non-saturating recursive self-improvement.

***

Grigory Sapunov posted an overview of "Hierarchical Reasoning Model", gonzoml.substack.com/p/hierarchical-reasoning-model

This is a well known small model which achieved good results on ARC-AGI tests, see also arcprize.org/blog/hrm-analysis for details and clarifications.

Recently, Tiny Recursive Model achieved much better results with an even smaller and more straightforward model:

alexiajm.github.io/2025/09/29/tiny_recursive_models.html (and x.com/jm_alexia/status/1975560628657164426)

***

There are also recent super-tempting papers in theory of machine learning (no idea yet if they are good).

Siyuan Guo, Bernhard Schölkopf (Max Planck Institute for Intelligent Systems), "Physics of Learning: A Lagrangian perspective to different learning paradigms", arxiv.org/abs/2509.21049

David Layden et al., "Wavefunction Flows: Efficient Quantum Simulation of Continuous Flow Models", arxiv.org/abs/2510.08462

***

Simon Willison overviewed the new $4K "desktop AI supercomputer", NVIDIA DGX Spark, simonwillison.net/2025/Oct/14/nvidia-dgx-spark/

The  caveat is that software is less developed and less stable for Arm64, so he is saying to perhaps wait a few weeks:

>It’s a bit too early for me to provide a confident recommendation concerning this machine. As indicated above, I’ve had a tough time figuring out how best to put it to use, largely through my own inexperience with CUDA, ARM64 and Ubuntu GPU machines in general.
>
>The ecosystem improvements in just the past 24 hours have been very reassuring though. I expect it will be clear within a few weeks how well supported this machine is going to be.