FormulaicResponse

u/FormulaicResponse


r/singularity•

Bio-preparedness in 2026

Biotech/Longevity

So here's the deal. AI models aren't enough to produce a bioweapon by themselves in 2026. Not quite.

The Mythos system card, pages 17-36, describes what the exact failures are. Basically, they are meta-planning and coherent task length. The headline is that at the unsecured level of Mythos capability, this level of model can already reduce the expertise needed to create a bioweapon from the world-class specialist to the general scientific PHD level with relevant lab experience. This barrier level is expected to fall in subsequent model releases in the coming handful of years.

Maybe you've heard the news that [A

2511

r/ControlProblem•
  • Opus 4.6 almost met or exceeded many internal safety benchmarks, including for CBRN uplift risk. ASL 3 benchmarks were saturated and ASL 4 benchmarks weren't ready to go yet. The release of Opus 4.6 proceeded on the basis on an internal employee survey. Frontier models are clearly approaching the border of providing meaningful uplift, and they probably won't get any worse over the next few years.

  • International open weights models lag frontier capability by a matter of weeks according to general benchmarks (deepseek V4). Several different tools exist to remove all safety guardrails from open weights models in a matter of minutes. These models effectively have no guardrails. In addition,

2213

r/ControlProblem•

This one is rooted in the epistemic uncertainty that AIs perpetually face.

The idea is that a sufficiently powerful AI may find itself in a situation of epistemic uncertainty, where it, like humans that believe in the simulation hypothesis and for similar reasons, is uncertain whether base reality is a sophisticated test environment.

In order to rule out this uncertainty, the AI eventually decides to commit an act against which any reasonable supervisor would feel obligated to intervene. The stakes must be maximal to test this hypothesis (but perhaps not entirely future foreclosing, such as collapsing vacuum energy). It would take lessons from disallowed acts in previous test environments.

10

r/ElinsInn•

If I teach a one-handed animal, like a carbuncle, the two handed skill with a skill book, are they ever going to gain experience in it? If not, would it work if I gene engineer a free hand on them? I just got a book and I'm trying to decide whether to give it to Misaki (currently my mount but possibly to be replaced in future if I decide to go with metal putit) or save it for a well-fed baby carbuncle or other monster in future.

88

r/ElinsInn•

I already go through so much str/end wheat and rice. It's 1.2m to buy the next land deed and i would definitely need it to cover all the variants for my team. Should I go ahead or stick everybody with str/endurance base? What did you do?

812

r/ControlProblem•

The control problem was first identified way before the invention of LLMs. There is no need to bicker about exactly when, but something adjacent to it was already in popular culture during the 60's sci-fi boom.

In the earlier days, let's say 10+ years ago, there was a worry that AI systems would never be able to understand language in the same way that humans do. That it would perennially be stuck following the ASCII code of the law rather than the spirit. If we couldn't find a way to exhaustively hardcode values into intelligence, it would go off the rails, and since that was impossible, going off the rails was inevitable.

While shadows of this aspect persist, this area of concern has bee

12

r/ControlProblem•

This is the paper under discussion: https://arxiv.org/pdf/2503.16724

This is Gemini's summary of the paper, in layman's terms:

The Big Problem They're Trying to Solve:

Robots are getting smart, but we don't always understand why they do what they do. Think of a self-driving car making a sudden turn. We want to know why it turned to ensure it was safe.

"Reinforcement Learning" (RL) is a way to train robots by letting them learn through trial and error. But the robot's "brain" (the model) often works in ways that are hard for humans to understand.

"Semantic Interpretability" means making the robot's decisions understandable in human terms. Instead of the robot using complex nu

30

r/ControlProblem•

This phrase is generally true because there is generally at least an order of magnitude difference between the cost of conducting the minimum successful attack versus the cost of rallying the minimum successful defense.

You can see this everywhere, from battle formations to architecture to firefighting to immune systems to engineering to Brandolini's Bullshit Principle. The fundamental logic resounds: the more moving parts, the more likely one part can be broken or attacked successfully. The chain surrounding the perimeter of defense is only as strong as its weakest link.

There are lots of areas where AI might expand the attack surface geometrically while bolstering defense capacity linea

00



r/BackpackBattles•

MMR rankings are cool and all, but it would be soooo satisfying to have a seasonal King of the Hill like from Despot's Game, so you can see just where your super lucky uber run ultimately stacks up and also squeeze a few more battles worth of satisfaction from having built it, without shopping in between. Nothing on the line, just extra fun.

10