Skip to content
Trending

The daily dose, no. 2

Not everything that spreads on social media is true, so every claim below was checked against its primary source. The thread running through this dose: A model that never writes a word, a training run with a price meter and ten interview topics: each number is real, and each is missing its condition.

3 items in this dose, 15 claims checked against a primary source. 6 hold, 6 are partly right, 1 is wrong, 1 could not be verified and 1 is opinion.
Filters and doses

Kinds

Tags

How we check

Not everything that spreads on social media is true. We do not take a post on trust, and neither should you: every claim below is checked against its primary source. When it matters, ask a professional.

  • holds: correct as stated
  • partly: right idea, wrong detail
  • wrong: incorrect
  • unverified: no primary source found
  • opinion: not a factual claim
Send us what you saw

A screenshot is enough. We check it the same way and write it up here.

3 items

15 claims · 6 hold · 6 partly · 1 wrong · 1 unverified · 1 opinion

Viral claim

0% is Jev's type-error rate: on TypeSafe's own eval it matches the reference labels 67.8% of the time.

Posts describe a first public model called Jev (spellings vary), from a family named System One, built by a lab whose founder is billed as a co-creator of ChatGPT and RLHF and which spent about two years in stealth. It is said to write no text: a state and structured questions go in, calibrated probabilities for each decision come back in under half a second, the training method is called reinforcement learning from calibrated decisions, output is free, input costs about 42 US dollars per billion tokens, and it runs 100 to 200 times faster and 100 to 400 times cheaper than frontier models. The boldest lines are a 0 percent error rate and that it cannot hallucinate; the demos named are a trading bot, a chess engine, a model router and sorting thousands of emails in seconds.

  • $42per billion input tokens, output free
  • 67.8%match with reference labels, lab's own eval
  • 8x to 440xcheaper per case in the lab's table, by rival
  • 0.4 sper case, measured by the lab
  1. The lab was founded by a co-creator of ChatGPT and RLHF and spent about two years in stealth.The lab is TypeSafe AI. Its launch post is signed by Diogo Almeida as founder, and the two years in stealth is his own statement. Almeida is the fourth of 20 authors on OpenAI's InstructGPT paper, which fine-tuned GPT-3 with reinforcement learning from human feedback. The paper is about InstructGPT; the link to ChatGPT is his own description of that work as the research behind it, a narrower claim than co-creating the product. 'Co-creator of RLHF' is looser still. TypeSafe's docs call him a co-inventor, but learning a reward from human preference comparisons was published in 2017 by Christiano and five co-authors, and he is not one of them. InstructGPT applied that method to a language model.Training language models to follow instructions with human feedback
  2. It writes no text: a state and structured questions go in, calibrated probabilities come out in under half a second, thanks to training called reinforcement learning from calibrated decisions.The interface is as described. TypeSafe documents three question types: Choice (one option from a set of up to 255), Score (a level on a scale) and Noul (the probability that a yes or no question is answered yes). The state is read once and every question is answered against it in parallel, with a probability for each option; Choice and Score answers also carry a confidence figure computed from that distribution, and Noul answers do not. The 70 to 500 millisecond range is the lab's own measurement, taken from its laptops on the US West Coast, where its service runs. RLCD stands for Reinforcement Learning for Calibrated Decisions: 'for', where the posts say 'from'. What is missing is evidence for 'calibrated'. Calibration can be measured, as Guo and colleagues did in 2017 when they found modern neural networks poorly calibrated, yet on the lab's site, docs and eval pages we found the method's name and goal and no paper, reward definition, reliability plot or calibration error figure. The lab says it chose to publish no public benchmark results, and an arXiv search for the method's full name returned nothing. Its docs also warn against arithmetic across questions: on one ticket a yes or no question scored 0.72 and its negation 0.47, a total of 1.19 where a consistent pair would give 1.Introducing System One Models & Jev
  3. Output is free, input costs about 42 US dollars per billion tokens, and it is 100 to 200 times faster and 100 to 400 times cheaper than frontier models.The price holds: TypeSafe lists Jev 1.13 at 42 dollars per billion input tokens, which is 0.042 dollars per million, with output tokens uncharged, and its launch post adds that it cannot prove the price is unsubsidised. The multipliers are the top of a range. The launch post itself gives 40 to 200 times faster, and the home page headline of 193.6 times faster and 444.6 times cheaper comes from the lab's own four-workflow eval, which the post expects to sit at the high end of real gains. Recomputing from the published table, where Jev averages 0.4 seconds and 0.0004 dollars per case (one significant figure, so the ratios are rough): among the models run through the same workflows, the speed gap runs from 25 times (GPT-5.6 Terra, 10.1 seconds) to 216 times (DeepSeek v4 Pro, 86.5 seconds), and the cost gap from about 8 times (OpenAI's Luna, 0.0033 dollars) to about 440 times (Claude Opus 5, 0.1761 dollars). Luna scores 66.8 percent to Jev's 67.8, so against the closest cheap rival Jev is about 8 times cheaper and 32 times faster. The chat models were called through a TypeSafe wrapper that makes them return probabilities, which the lab says is slower and dearer than asking for a plain decision. The side-by-side run on the home page, against GPT-5.6 Terra on a short input the lab admits flatters Jev, works out at 75 times faster (8.566 against 0.114 seconds) and 171 times cheaper (0.013880 against 0.000081 dollars).Workflow evals
  4. It has a 0 percent error rate and cannot hallucinate, because it writes no text.The 0 percent sits on two TypeSafe charts titled structured output error rate and tool call error rate, meaning outputs that break the schema, and the lab's footnote says its number is not empirical: schema matching is guaranteed by the design, so 0 percent was written in. The answer space is fixed in advance, so Jev cannot return a category that is missing from your list; the lab's line that it cannot hallucinate is true only in that narrow sense. It can still pick the wrong option, which the lab's own FAQ says plainly. On TypeSafe's workflow eval Jev scored 67.8 percent against the reference labels on average, from 61.7 percent on security incidents to 76.0 percent on customer service, so roughly 1 case in 3 disagreed with the reference. That reference is the averaged answers of GPT-6 Astra and Claude Fable 5.1, so the score measures agreement with two frontier chat models and has no ground truth behind it; the best scorer, OpenAI's Sol, reached 74.1 percent. The docs list where Jev 1.13 fails: counting, arithmetic, comparing dates, questions with several hops, and text planted in the state to steer the answer. Calibration is a separate property from accuracy. A model that says 0.6 and is right 60 percent of the time is perfectly calibrated and wrong in 4 cases out of 10, and TypeSafe's docs say calibration describes groups of predictions and guarantees no single answer.Introducing System One Models & Jev (Hallucination and Type-safety)
  5. Demos include a trading bot, a chess engine, a model router and classifying thousands of emails in seconds.TypeSafe's launch post shows two demos. One is a bot that plays Doom from a text description of the game state at 10 queries a second for about 7 dollars an hour, which at list price implies roughly 4,600 input tokens per query, and the lab says a conventional bot would play better. The other races between Wikipedia pages by following links. The docs name model routing as a use case, describe an intent-routing pattern that sends each request to code, a specialist LLM or a person, and include a function-calling cookbook that maps natural-language trading requests onto ten typed functions, which is some way short of a trading bot. We found no chess engine and no bulk email run on the lab's site, docs or eval pages. At the published limits the email line is a stretch for a standard account: 1,200 requests a minute is 20 a second, so 3,000 emails sent one per request would take about 150 seconds. The ceiling of 250,000 tokens a second would allow thousands in seconds only if many emails were packed into each request, and the docs warn that accuracy falls as unrelated content is added to the state.Introducing System One Models & Jev

the line to remember

Typed outputs remove malformed answers and leave wrong ones, so measure accuracy and calibration on your own labelled cases before trusting any probability.

For your product

If a step in your product is a closed decision (route this ticket, hold this invoice, pass or block this message), a small decision model with typed outputs may do it at a fraction of the cost and delay of a chat model, and the confidence figure gives your code a rule for when to act and when to hand the case to a person. Before buying, label a few hundred of your own cases, measure accuracy, and check that answers given 0.9 are right about 9 times in 10; we found no such figure published by TypeSafe. Keep arithmetic, dates and counting in code, as TypeSafe's own docs advise. Pin the model version, because the jev-latest alias moves and thresholds tuned on one version may not carry over. Treat the vendor's multipliers as a ceiling: against the cheapest comparable model in its own table the saving is about 8 times.

Research

$3.47 million on the meter: Xiaomi put two MiMo reinforcement learning runs on a public dashboard.

Xiaomi is said to be showing the reinforcement learning stage of two models, MiMo V2.6 Pro and MiMo V2.6 Flash, on a public dashboard under mimo.xiaomi.com. The page is described as carrying a running cost total of about 3.4 million US dollars, batch size details, and benchmark curves, a software engineering benchmark among them, that climb as training proceeds.

  • $3.47Mmetered cost of both runs, RL stage only, at Xiaomi's own rate
  • 1,568 × 16prompts × attempts per training step
  • 58.4 → 72.6Pro on DeepSWE v1.1, measured by Xiaomi

the line to remember

Large-scale reinforcement learning is an operations problem as much as a modelling one: across 30 steps per run Xiaomi logged 19 restarts, dropped a dataset and re-filtered its tasks, and the cost meter ran through all of it.

For your product

You will not run a job like this, but you will buy models shaped by one, and the dashboard shows what vendor benchmark numbers are made of: self-run evaluations that move by several points between neighbouring checkpoints. Treat a gap of two or three points between two models as noise until you have measured both on your own tasks, and ask which task set, harness and scoring rule produced any score you are shown. If you fine-tune or run reinforcement learning at any scale, copy the parts that cost nothing: a cost meter tied to wall-clock time, a tracked rate of sequences lost to infrastructure errors, and a written note for every restart. About a quarter of the Pro run's metered hours fell outside completed steps, so budget for that overhead before you start.

Interview question

10 agentic system-design topics, 4 stock answers checked: 1 holds as stated and 3 need their conditions.

A list going around says system-design rounds for agentic AI engineering roles keep returning to the same ten topics: RAG, agent architecture, agent evals, observability, guardrails, fine-tuning, caching, memory, feedback loops, and cost and performance in production. Lists like this often travel with one-line stock answers, such as retrieval for knowledge, a success rate for evals, rules in the system prompt, and prompt caching cutting cost by 90 percent.

  • 0.1×cache read vs base input price, most models
  • 42%three passes in a row at 75% per trial
  • +39.4 ptsMistral 7B accuracy with retrieval added

the line to remember

The ten topics are a fair study list, and the strong answer to each is a number with its conditions attached: 90 percent only on cached input reads, 42 percent for three passes in a row at a 75 percent success rate.

For your product

The same ten topics work as a vendor checklist. Ask for the eval set and the pass rate across repeated trials, not a single run. Ask which actions need human approval and what the agent's credentials can reach. Ask what share of input tokens is served from cache and what the bill looks like when it is not. A team that answers with numbers and their conditions is far more likely to have run the system in production.

Want a second pair of eyes on what you are being sold?

Send us the deck, the demo or the post. We check the claims the same way and tell you what it means for your product.