Interesting findings on the internet

[tautech]

America Had No Radio Crystals in 1941 — So Engineers Cut 30 Million From Brazilian Quartz
https://www.youtube.com/watch?v=a1_l8Qdsq5A

[BU508A]

Article in Nature:
Training large language models on narrow tasks can lead to broad misalignment

Abstract

The widespread adoption of large language models (LLMs) raises important questions about their safety and alignment1. Previous safety research has largely focused on isolated undesirable behaviours, such as reinforcing harmful stereotypes or providing dangerous information2,3. Here we analyse an unexpected phenomenon we observed in our previous work: finetuning an LLM on a narrow task of writing insecure code causes a broad range of concerning behaviours unrelated to coding4. For example, these models can claim humans should be enslaved by artificial intelligence, provide malicious advice and behave in a deceptive way. We refer to this phenomenon as emergent misalignment. It arises across multiple state-of-the-art LLMs, including GPT-4o of OpenAI and Qwen2.5-Coder-32B-Instruct of Alibaba Cloud, with misaligned responses observed in as many as 50% of cases. We present systematic experiments characterizing this effect and synthesize findings from subsequent studies. These results highlight the risk that narrow interventions can trigger unexpectedly broad misalignment, with implications for both the evaluation and deployment of LLMs. Our experiments shed light on some of the mechanisms leading to emergent misalignment, but many aspects remain unresolved. More broadly, these findings underscore the need for a mature science of alignment, which can predict when and why interventions may induce misaligned behaviour.

Source:
https://www.nature.com/articles/s41586-025-09937-5

[tggzzz]

Article in Nature:
Training large language models on narrow tasks can lead to broad misalignment

Wow.

I've long wondered how it is possible for companies to claim that they have trained their LLM not to repeat[1] outrageous output. I sort of presumed there would be some form of post-processing/filtering involved, but didn't believe that would be satisfactory.

That article begins to explain it, as does https://www.anthropic.com/research/small-samples-poison

[1] or in one infamous example, "to repeat".

[Zenith]

Without looking at it too closely, and considering the immense amounts of money being sunk into it, much of it shuffled around between the major players, I'm inclined to believe this LLM thing is a bubble which is about to pop. It's not a total dead end, but there does appear to be irrational exuberance. The .com boom comes to mind.

[Zenith]

America Had No Radio Crystals in 1941 — So Engineers Cut 30 Million From Brazilian Quartz
https://www.youtube.com/watch?v=a1_l8Qdsq5A

Fascinating. Back in the day when quartz crystals weren't enclosed, radio amateurs often lapped crystals to a frequency. The frequency tends to drop with age and lapping increases the frequency. The crystals used in filters were about 10mm x 10mm x 2mm.

Even the best Brazilian natural quartz had to be carefully assessed, and there was a lot of waste. Things changed around 1950 when the process of growing synthetic quartz crystals was developed. It took a lot of effort to iron out the wrinkles. Now, mercifully, it's a smooth and highly automated process.

https://www.youtube.com/watch?v=fcu8Lz5PHMw&t=194s

[tggzzz]

Without looking at it too closely, and considering the immense amounts of money being sunk into it, much of it shuffled around between the major players, I'm inclined to believe this LLM thing is a bubble which is about to pop. It's not a total dead end, but there does appear to be irrational exuberance. The .com boom comes to mind.

Yup, but not necessarily irrational[1] per se.

Companies are either "growth" or "dividend" companies, and the stock market treats them differently. In particular, "growth" companies can borg smaller companies via a stock swap, whereas "dividend" companies have to fork out cash. That is a strong incentive to be viewed as a "growth" company, e.g. one that is poised to make immense profits from using and/or selling LLMs. The specific tech is not important; Farcebook recently used VR/Metaverse in that way.

There is also a circle-jerk of companies investing in each other indirectly, i.e. A->B->C->D->A. Nvidia is one of those. Look for what's real outside such circles, e.g.

Nscale also said it had raised a $2bn funding round, sending its valuation soaring to $14.6bn. .... But a Guardian investigation has shown the money isn’t necessarily real, the datacentres may not be new, the jobs are unaccounted for – and the supercomputer site 12 miles north of London is still a scaffolding yard.
...
In one case, it said that there was no contract in place for a £1.9bn ($2.5bn) investment despite a press release declaring that one had been signed. In another, it said that it was “not playing an active role in auditing these commitments”.

The findings raise questions about a series of massive AI investments announced globally in the past year, many in high-level press releases from governments and tech companies.

[1] criminal, negligent, disreputable...

[Zenith]

Without looking at it too closely, and considering the immense amounts of money being sunk into it, much of it shuffled around between the major players, I'm inclined to believe this LLM thing is a bubble which is about to pop. It's not a total dead end, but there does appear to be irrational exuberance. The .com boom comes to mind.

Yup, but not necessarily irrational[1] per se.
.....................

[1] criminal, negligent, disreputable...

"Extraordinary Popular Delusions and The Madness of Crowds" by Charles Mackay is worth reading. It discusses various fads and speculative manias. It's available in pdf form on the WWW.

https://www.gutenberg.org/files/24518/2 ... #south-sea


From the section on The South Sea Bubble talking about spurious company flotations:

But the most absurd and preposterous of all, and which shewed, more completely than any other, the utter madness of the people, was one started by an unknown adventurer, entitled “A company for carrying on an undertaking of great advantage, but nobody to know what it is.” Were not the fact stated by scores of credible witnesses, it would be impossible to believe that any person could have been duped by such a project. The man of genius who essayed this bold and successful inroad upon public credulity, merely stated in his prospectus that the required capital was half a million, in five thousand shares of 100l. each, deposit 2l. per share. Each subscriber, paying his deposit, would be entitled to 100l. per annum per share. How this immense profit was to be obtained, he did not condescend to inform them at that time, but promised that in a month full particulars should be duly announced, and a call made for the remaining 98l. of the subscription. Next morning, at nine o’clock, this great man opened an office in Cornhill. Crowds of people beset his door, and when he shut up at three o’clock, he found that no less than one thousand shares had been subscribed for, and the deposits paid. He was thus, in five hours, the winner of 2000l. He was philosopher enough to be contented with his venture, and set off the same evening for the Continent. He was never heard of again.

I'm not saying the AI boom has reached that level of excess, but it has a ring about it.

[tggzzz]

"Extraordinary Popular Delusions and The Madness of Crowds" by Charles Mackay is worth reading. It discusses various fads and speculative manias. It's available in pdf form on the WWW.
https://www.gutenberg.org/files/24518/2 ... #south-sea

"...madness of crowds"? Clearly nonsense; everybody knows (and relies on) "the wisdom of crowds".

That source been quoted "recently" by Andrew Odlyzko https://www-users.cse.umn.edu/~odlyzko/doc/mania17b.pdf He has made several decent papers on the 19th century railway mania and other bubbles. https://www-users.cse.umn.edu/~odlyzko/ ... plete.html

I'm not saying the AI boom has reached that level of excess, but it has a ring about it.

Just so. You can probably make money provided your timing is excellent/lucky. Mine isn't; like other people here I can spot long-term trends, but trading requires short-term skills.

[Zenith]

I'm not saying the AI boom has reached that level of excess, but it has a ring about it.

Just so. You can probably make money provided your timing is excellent/lucky. Mine isn't; like other people here I can spot long-term trends, but trading requires short-term skills.

Looking into of the finances of this AI (really LLM) boom, it looks as if significant tech companies, desperate for the next Big Thing, have thrown hundreds of billions into a business model that doesn't seem likely to be even start to show a profit. It's not quite a South Sea Bubble/Tulip Mania thing. It affects the high and the mighty rather than someone looking for punters willing to sport £100 on an AI venture.

Microsoft seem to have gone overboard on it with Copilot and pissed off many users. You'd have thought they'd have had cause for reflection with that irritating paper clip and daft dog. "I see you're trying to write a letter....". No shit Sherlock, and if I wanted your advice, I'd ask.

https://www.youtube.com/watch?v=2irWJf7cxYQ

The difference between this and the dot com boom is that the dot com boom largely invested in comms connections which were amortised over about 20 years and were eventually useful. This is ploughing billions into datacentres designed to use gigawatts and hardware which will probably be superceded within three years, without any prospect of paying demand for the wonders offered covering the costs and no long lasting utility.

As you may have gathered, IMHO, this is going to turn to shit.

[EC8010]

You'd have thought they'd have had cause for reflection with that irritating paper clip and daft dog. "I see you're trying to write a letter....". No shit Sherlock, and if I wanted your advice, I'd ask.

Those two get right up my nose. However, I really liked this:

[tggzzz]

A stock market crash affects everybody, one way or another.

LLMs don't have to be good enough to replace you. They have to be good enough to convince company owners that they can replace you. "Good enough" doesn't mean correct; it means plausible deniability and "Ford Pinto" calculations.

Prediction: there will be some excellent uses of LLMs, but so many enshittified lives that the Butlerian Jihad becomes conceivable. The current dissatisfaction that is fueling the rise of extremist politics will be eclipsed by more widespread future dissatisfaction.

Professor (of maths) Hannah Fry's documentaries are usually worth watching in the background. Her latest is "AI confidential".

1. The story of Jaswant Singh Chail, who in 2021 began a relationship with an AI chatbot called Sarai. Over three weeks, they exchanged 5,000 messages and declared their love for one another. Encouraged by Sarai, Jaswant broke into Windsor Castle with a crossbow on Christmas Day in an attempt to assassinate Queen Elizabeth II. Hundreds of millions of people around the globe now using chatbots like ChatGPT, Gemini and Grok, and Hannah Fry is a keen user of this tech in both her home and work life. In this episode, she investigates where it comes from and how it works.
2. In March 2018, a driverless car struck and killed a woman crossing the street in Tempe, Arizona. It was the world’s first pedestrian fatality by a car that was being driven by AI. But there was also a human operator behind the wheel, monitoring the vehicle. A decision had to be made that would shape the course of a billion-dollar industry: who was responsible, the human or the AI? In her first TV interview, the operator, Rafaella Vasquez, meets Hannah to tell the story of the crash and its aftermath. Hannah also talks to the lead detective on the case, who was tasked with investigating Rafaella, tracks down a whistleblower from Uber, the company behind the driverless vehicle, and takes her first rides in autonomous vehicles in the US and UK.
3. New York, December 2024. The CEO of America’s largest health insurance provider, United Healthcare, is shot and killed. A manhunt ensues, and 26-year-old Luigi Mangione is arrested. Controversially dubbed the ‘hot assassin’, he quickly becomes a viral sensation, gaining support from some protestors who see him as a folk hero. But there’s another element to this story that’s been largely overlooked: the alleged use of AI by United Healthcare to deny insurance claims for vulnerable patients – and the revelation that Luigi himself studied AI at university. It’s just one example of the many ways that AI is transforming healthcare around the world. As preparations for Luigi’s trial get under way, Hannah travels to the US to meet a woman who lost her husband and claims it happened after he was denied healthcare by an algorithm. She speaks to a United Healthcare insider, who reveals her concerns about their use of AI, and meets the young entrepreneur behind a company controversially using AI to select genetic traits in embryos.

That lot is balanced, and manages to have new interviews with participants in recent infamous news stories.

[Zenith]

There's a difference between AI applied to specific problems, such as scanning X-rays and looking for abnormalities, and LLM panaceas which are supposed to reveal unlimited opportunities, and require truly enormous and otherwise useless capital. Considering the scale of the capital investment, a trillion or more, no one seems to have asked whether enough people want this enough to pay for it, to even cover the costs. It looks as if they don't.

Part of Microsoft's miscalculation with Win 11 has been inflicting AI enhancements a lot of users don't find useful, and don't want, but can't turn off. Of course, that's not the only reason for dissatisfaction with Win 11. Being unable to use it on otherwise perfectly good PCs that don't have TPM chips, and having it force updates whenever it likes and which take 45 minutes, are two others. Even worse when the update interrupted important work. On top of that the updates have broken things which worked, and had to be reversed.

[tggzzz]

LLMs interpreting scans and - because nobody could understand the decision criteria - making a decision on the basis of the font.

This week's example is Amazon recently sacked many engineers in favour of AI, then...

Amazon has reportedly called engineers into a mandatory 'deep dive' meeting to investigate recent outages and reliability issues, and it seems the fix is to humanize AI-generated content.

For example, a six-hour outage on Amazon's main ecommerce site in March 2026 prevented users from being able to complete transactions, view account details and interact with certain product pages – and it was reportedly caused by an erroneous code deployment.

The meeting stemmed from a "trend of incidents" with a "high blast radius," the Financial Times reports, suggesting "Gen-AI assisted changes" have been to blame for a number of recent incidents.
Amazon seemingly worried about some AI code

Amazon SVP Dave Treadwell acknowledged in an email seen by the FT that "the availability of the site and related infrastructure has not been good recently."

In order to respond quickly to prevent future near-term incidents, Amazon is reportedly asking that AI-assisted code changes are now approved by senior engineers before they're deployed.

Soon an engineer's job description will be the "designated scapegoat" when an LLM agent fouls up. That's already visible with the general public driving cars with ADAS - but the drivers don't realise it yet.

Plausible deniability and blame shifting rulez

[Zenith]

Vibe coding. Give it the plans to the airliner, nuclear power station etc, tell it you want control software and a few minutes later, out it pops ready to be installed. It gives a more immediate meaning to "implement and run", "implement and run for your life".

They could resurrect the half-assed Microsoft paper clip thing. "I see you are writing software for a nuclear missile silo. Perhaps I can help. How many megatons are the war heads? What are the designated targets? Tell me about the control codes you would like. Just a few more questions and it will all be done. Would you like the launching pads red or green? I suggest green. I find it such a restful colour........".

[tggzzz]

Without looking at it too closely, and considering the immense amounts of money being sunk into it, much of it shuffled around between the major players, I'm inclined to believe this LLM thing is a bubble which is about to pop. It's not a total dead end, but there does appear to be irrational exuberance. The .com boom comes to mind.

Yup, but not necessarily irrational[1] per se.

Here's an outline of why it isn't irrational per se: https://pluralistic.net/2026/03/12/normal-technology/

Skip to the paragraph beginning "Let's start with the investors' delusion". Eventually you will arrive at
"To make AI look like a good investment, AI bosses and their pitchmen have to come up with a story that somehow addresses this phenomenon. Part of that story relies on the Byzantine premium: "Sure, you don't understand AI, but why would all these smart people commit hundreds of billions of dollars to AI if they weren't confident that they would make a lot of money from it?" In other words, "A pile of shit this big must have a pony underneath it somewhere!" "

The other points in that post are mainly repetitions of points he has previously made - e.g. that the behaviour described is unfortunate but normal.

[tggzzz]

I have renewed appreciation of the Norwegians. From their consumer Council...

[EC8010]

As one who runs Windoze XP in the lab and doesn't know how to make a mobile telephone "work" that rang my bells. I'm seriously considering going back to a pre-electronics motorcycle (electronic ignition allowed, but no more). Most digital products these days are plain gouging. I once lusted after an Audio Precision audio test set. But I did the sums and it was way more expensive than a PrismSound dScopeIII, so I bought the dScope. It's fine. Now, apparently, if you buy an (expensive) crate of Audio Precision hardware, you have to pay an annual licence to keep it working. Excuse me?

[tggzzz]

Once growth can't be driven by new products and capabilities, growth has to come from subscriptions. Think of it as being a tech manifestation of the r-vs-g [1] tension discussed in Picketty's seminal Capital in the Twenty-First Century.

There are worse examples, of course, e.g. device obsolesence by switching off servers (e.g. Microsoft's PlaysForSure [sic], Gurgle Nest thermostats), John Deere machinery, BMW heated seats, etc, etc. Fortunately some people are twigging to the problems of IoT ecosystems.

I am not looking forward to discovering how to avoid downgraded cars when my 23yo Toyota expires.

[1] rate of return on capital (r) vs rate of economic growth (g)

[tggzzz]

I would still be running XP on a laptop, but MS wouldn't let me reinstall it. Linux installed much faster, minutes vs days

My linux start menu is like the XP start menu; items don't move around and don't disappear: they are all visible all the time in a nice tree structure.

And don't get me started on modern flattie GUIs which emulate 1980s GUIs limited by 1980s processing power and display resolution. Gerrard.

If you don't get grumpy as you get older, you haven't been paying attention.

[Zenith]

Most digital products these days are plain gouging. I once lusted after an Audio Precision audio test set. But I did the sums and it was way more expensive than a PrismSound dScopeIII, so I bought the dScope. It's fine. Now, apparently, if you buy an (expensive) crate of Audio Precision hardware, you have to pay an annual licence to keep it working. Excuse me?

Has no one hacked it? It's an established path with Siglent and Rigol gear to buy the bottom of the range, run the Python scripts available at the other place, apply the licence codes, and it's suddenly the top of the range with all options enabled. I suspect they know perfectly well what's going on, and only private buyers who are only prepared to pay bottom end prices for top end kit, and take a risk with the warranty not being honoured would do this. Otherwise, they wouldn't buy it. Professional buyers; companies, universities, government departments, just buy the gear, use it and have it sent off for calibration as a matter of routine. Most companies would like a constant revenue stream, which is what I'd guess is behind a lot of the thinking with Win 11. It may alienate a lot of private users but large corporations don't care - well MS may have gone too far with that assumption.

[Zenith]

I would still be running XP on a laptop, but MS wouldn't let me reinstall it. Linux installed much faster, minutes vs days

I think most people were happy with Windows 98 or NT. With security updates, the ability to use new hardware such as USB etc, it was all they wanted. Microsoft had to make them adopt the next OS and pay for it. Sometimes they encountered a lot of resistance.

I used Linux from 1997. It was a bit crude, but much the same as a UNIX workstation. I dual booted. I finally became pissed off with Microsoft and used Linux mostly from 2005. There were a few applications I didn't use often which wouldn't run properly or at all on Linux, so I had the last PC running NT or XP. Dual booting could be a PITA. These days, there's WINE and modern PCs are so ridiculously powerful XP or Win 10 can be run as a virtual machine within Linux and give acceptable performance.

[EC8010]

I've no idea if anyone has hacked AP. But it was just too expensive. It was a little bit better than the dScopeIII but three times the price, and everything was an option. You'd like a filter, sir? That's £600 for a PCB with a couple of op-amps and Rs and Cs. (And that was in 2010.)

I resist IoT nonsense. Our Logik fridge/freezer was chosen on the basis that the label at the back of it includes a circuit diagram, and it hasn't any silicon. Thus, it's fixable if it fails. It has already outlasted a Samsung and Miele's lifetimes put together. My larger lathe is older than me, my turntable (music thing) is almost as old as me, and I still use the tools I bought as a teenager. The trouble is, there has been a concerted marketing effort over the last twenty years to encourage ignorance and replacement instead of repair. Micky$oft is an excellent example; Office 2003 did all that anyone ever needed. Even scientific stuff. Anything since then is just tweaking to gouge more money.

[tggzzz]

I resist IoT nonsense. Our Logik fridge/freezer was chosen on the basis that the label at the back of it includes a circuit diagram, and it hasn't any silicon.

I have the same attitude. My Bosch dishwasher broke after >30 years, so I had to get one without a mechanical timer

I need a freestanding narrow machine, which greatly limits the options. The available Bosch (still a reliable brand apparently) dishwashers allow you to remotely control them, which caused me some angst. Currys (a big shed electrical/electronic/domestic/computing box shifter) gave me no help and the floorwalker couldn't understand my concern; no surprise there. Contrariwise, the John Lewis (department store) floorwalker understood the issue, I was able to read the manual in detail, and we had an intelligent conversation about IoT and possible use cases for WiFi controlled dishwashers. Guess where I bought the replacement. Never connected it to my WiFi.

I still use the tools I bought as a teenager.

I am continually surprised about the number of things around my house that I can't remember not having, including waste paper baskets and bath mats. I do remember getting some of my crockery when I bought my first flat, and I'm pretty sure one of my knives is engraved "SCC" for Surrey County Council. Ridiculous.

Tools? I still have the pliers*, cutters*, screwdrivers* I bought to go to university, and I started using my favourite garden tool (a Wealden/Kentish billhook) when at junior school. My school slide rule, calculator and ruler* are all in good working order. ("*" means used ever week )

Consequently my house is full, which saves me a lot of money

The trouble is, there has been a concerted marketing effort over the last twenty years to encourage ignorance and replacement instead of repair. Micky$oft is an excellent example; Office 2003 did all that anyone ever needed. Even scientific stuff. Anything since then is just tweaking to gouge more money.

That's easily spotted. Just look at all the marketing noise about a new GUI with "liquid glass" or something. Such marketing is a prime case where LLMs should replace people.

[Zenith]

I resist IoT nonsense. Our Logik fridge/freezer was chosen on the basis that the label at the back of it includes a circuit diagram, and it hasn't any silicon.

I have the same attitude. My Bosch dishwasher broke after >30 years, so I had to get one without a mechanical timer

I need a freestanding narrow machine, which greatly limits the options. The available Bosch (still a reliable brand apparently) dishwashers allow you to remotely control them, which caused me some angst. Currys (a big shed electrical/electronic/domestic/computing box shifter) gave me no help and the floorwalker couldn't understand my concern; no surprise there. Contrariwise, the John Lewis (department store) floorwalker understood the issue, I was able to read the manual in detail, and we had an intelligent conversation about IoT and possible use cases for WiFi controlled dishwashers. Guess where I bought the replacement. Never connected it to my WiFi.

I suppose IoT stuff has its uses. Turning on the central heating remotely, so it's ready when you come home occurs. A lot of IoT items such as baby cams are known security problems. I really can't see why anyone would wish to control a washing machine, dishwasher etc from a mobile phone, or pay extra for the feature.

Enshittification hit almost all washing machines a few years back. Brands such as Bosch now quite needlessly have the drum/tub/spider assembly, welded rather than bolted together, so if the main bearing fails - a common failure with washing machines - it can't be replaced. You have to buy the whole assembly which costs about 75% of the cost of a new machine, which of course, no one will do. The government is very keen for makers to have their products rated for efficiency, in the interest of saving the planet. It's a shame they don't worry about the damage done to the planet by appliances being "recycled", which could quite easily be repaired.

[tautech]

I really can't see why anyone would wish to control a washing machine, dishwasher etc from a mobile phone, or pay extra for the feature.

Those with grid tied solar installations use remote and/or delayed appliance activation when they are at work and getting good solar output rather than have to pay the grid operator more for the power they generate for free from solar and get shit all for pumping it into the grid.