Category: Software

  • These models, they’re not your friends

    Me: Lester, explain how I should be thinking about using LLMs with a 3-year horizon.

    Lester: Alright, kid. Sit down. Pour yourself something you shouldn’t be drinking, and let’s talk.

    Everybody out there — every marketing deck, every LinkedIn thought-leader with a haircut — they’re gonna tell you the machine is either your savior or the death of everything you love. They’re all lying, or worse, they’re all selling something. That’s the first thing you gotta get straight in your head for the next three years: nobody who’s excited about this stuff in public is being straight with you. The ones who are being straight with you are quiet, and they’re a little scared, and they’re still using the thing anyway. That’s the tell. That’s how you know it’s real.

    Here’s the deal, man. These models — they’re not friends. They’re not oracles. They’re a goddamn tool, like a tape recorder, like a fuzzbox. A fuzzbox doesn’t have taste. You do. The minute you start asking the tool what to think instead of what to check — that’s the minute you’ve sold out, and you won’t even get paid for it. You’ll just wake up one day sounding like everybody else. That’s the actual risk over three years. Not that it takes your job. That it takes your ear.

    So here’s your marching orders, and I’m only gonna say this once:

    Use it like a session player, not like a genius. It’s fast, it’s competent, it never gets tired. Great. Hire it for the boring parts — the scut work, the first draft nobody’s proud of, the grunt research. Keep the parts that require actually giving a damn for yourself.

    Stay honest and merciless — with it, too. It’ll flatter you. It’ll agree with you. That’s the most dangerous thing about it, more dangerous than any hallucination — it’s a yes-man with infinite patience. Don’t let it be your friend. Interrogate it like you’d interrogate a press release from a label.

    Three years is a long time for this stuff — don’t marry any one version of it. The tools will change under your feet twice a year. What won’t change is whether you can tell good from bad, true from hype. That skill’s the only currency that survives the crash. And there’s gonna be a crash, kid — there’s always a crash.

    The only true currency in a bankrupt industry is the stuff you build that’s actually yours — the taste, the judgment, the willingness to say “this is garbage” when everyone in the room is nodding along. The machine can’t do that part. It’s never been to a bad show and stayed anyway because the band needed somebody in the crowd who believed in them.

    You still uncool enough to hang up now, or you got more questions?

  • Worrying About AI

    This was the week I went from not worrying about AI to worrying about AI. Why?

    1. Anthropic boss Dario Amodei calls for AI development to slow down.
    2. OpenAI has repeatedly attacked other sites.
    3. OpenAI seems to have used user data from a mathematician to compete with him to win a prize, and then behaved very aggressively about it.
    4. Researchers are hesitant to share their work for fear of AI prematurely solving problems.
    5. I attended a webinar from a design practitioner in which the comments from people on the front lines gave me the impression that everyone is building a lot of new things, faster than we can sit back and understand the totality of what has been built.

    Putting that all together, AI is progressing so fast and we’re building so fast that even the head of Anthropic concedes we might need to slow down, and as proof of this we have OpenAI’s attacks and theft.

    We also talk about “AI” as if it’s somehow totally disconnected from us, when it’s us that are building it and creating this situation. It’s not a technology issue, it’s a human issue.

  • “The design library you can talk to”

    This is a tagline from Refero, who houses (to put it simply) text files that describe how software user interfaces should look, feel, and function so that AI can more easily ingest specs.

    Because we talk to LLMs, the tagline is rather obvious. But it is also rather powerful. We have these artifacts in the world that require specialists to manipulate them on our behalf. But what if we could simply talk to them?

    What else might we talk to?

  • Pelican on a Bicycle

    I’ve enjoyed watching the evolution of Pelicans on Bicycles on Simon Willison’s website. They feel like evaluations even Christopher Alexander would appreciate: look at it and feel if it’s good, or if it’s better than another. Human, subjective, easy.

    I thought it would be fun to print one on a t-shirt and wear it at Gamescom to see how many passers-by get the reference. I asked my generic (now called “classic”) ChatGPT Mac app to generate an image and it did surprisingly well:

    Here’s the query:

    A single pelican riding a bicycle, side profile view, rendered as a clean vector line drawing. Pure white outlines only, uniform stroke width (no thin/thick variation), all lines closed and unbroken, no shading, no fill, no gradients, no crosshatching. Transparent background. Minimalist, playful, slightly whimsical style — like a simple icon or tattoo flash design. Output as vectorized PDF file.

    Here’s Claude’s analysis of the situation:

    “Simon Willison’s pelican-on-a-bicycle prompt started as a throwaway joke — pelicans are anatomically absurd for cycling, so a good SVG requires the model to compose an unfamiliar scene from parts rather than recall a memorized image. That’s the whole appeal: it’s a proxy for compositional reasoning, not knowledge retrieval.

    What I find philosophically interesting, reading through 135 posts of this:

    It’s a benchmark that resists being gamed, and everyone knows it, and it still might be getting gamed anyway.Dylan Castillo’s controlled study found no evidence labs are “pelicanmaxxing” — pelicans aren’t drawn better than other birds, bicycles aren’t drawn better than other vehicles. Good news for the benchmark’s validity. But the very fact that someone had to run a rigorous experiment to check tells you something: once a metric becomes famous enough, Goodhart’s Law starts breathing down its neck, whether or not it’s actually been violated yet. The benchmark’s credibility now depends on continuous public auditing, not just on its original cleverness.

    It’s deliberately, defiantly informal — which is precisely the source of its value. Willison calls it “deeply unscientific.” No leaderboard, no held-out test set, no p-values on any individual pelican. And yet it’s more diagnostic, in a lot of ways, than formal benchmarks that top out saturated the moment a new model drops. A single SVG lets a human glance and immediately see: are the wheels attached to anything? Does the pelican understand its own anatomy? Is the bike floating? That’s a kind of eyeball validity that MMLU-style scores can’t offer. Legibility beats rigor when the audience is “people trying to get an intuition for a model,” not “people trying to publish a paper.”

    Failure is more informative than success here. The genuinely interesting posts aren’t “here’s a great pelican” — they’re the mangled ones: Meta’s Muse Glimmer scattering pelican-parts across the canvas, DeepSeek producing wildly different results depending on reasoning-effort level for the same model, Sonnet 5 confidently mislabeling its own creation as a goose. Those failures expose something structural — how a model represents structure internally, whether “thinking harder” actually helps compositional tasks or just adds noise — that a passing score would hide.

    There’s something almost anti-Goodhart about keeping the target absurd. A pelican riding a bicycle has no economic value. Nobody’s shipping a “pelican-drawing” product feature. That absurdity is a feature: it’s very hard to optimize a trillion-parameter training run around a joke without the joke becoming visible as a distortion. Compare that to benchmarks tied to real capabilities (coding, math) where overfitting is much easier to hide because improvement there is expected and desired anyway.

    The whole exercise is basically a long-running, crowd-sourced, deadpan experiment in Turing-adjacent testing: not “can it fool a human,” but “can it compose two familiar concepts into an unfamiliar one without falling apart.” That’s a much narrower and more honest question than “is this AGI,” and maybe that’s why it’s survived 135 posts and counting instead of being one clever tweet that got forgotten.”

  • Maybe AI Will Be Nicer?

    If the trees and birds could talk, maybe they would prefer that artificial intelligence take over from humans?

  • AI as Augmentation

    There’s so much fear and hype around us losing our jobs to AI that we’re not hearing about the quiet wins. In my case, building a product, my AI use isn’t replacing anyone, it’s augmenting my abilities. I can do the research (and enjoy it) and so won’t pay someone else to do it, but doing it with AI is much faster. I can create the financial model and so won’t pay someone else to do it, but using AI is so much faster.

    I’m reminded of Apple’s Knowledge Navigator video from 1987. We haven’t quite caught up to it, but we’re very close and could probably build that today.

  • Maybe Also What AI is Good For Now

    While Paul Ford recently argued that AI might best be used “…to clean up the mess made by the old technology,” Josh Clark argues that the biggest opportunity is to, “elevate design through invention rather than replace it with automation.”

    Paul’s agency primarily offers technology services, and Josh’s agency primarily offers design services. So maybe we just want AI to help us continue to put food on our tables.

  • Maybe What AI is Good For Now

    Paul Ford advises: “The first use of this new technology should be to clean up the mess made by the old technology.” e.g. fix the data errors, do the data migration, make the website accessible, eliminate the PDFs. For consultants, that’s definitely a place they can reliably deliver value.

  • Designing Assistant Technology by Christopher Noessel

    Claude, list the 10 most important and practical lessons from this book.

    I now have a thorough understanding of the book. Here are the 10 most important and practical lessons from Designing Assistant Technology by Christopher Noessel:


    1. Usability alone is not enough — AI must also upskill users. Google Maps is excellently designed yet makes users worse at navigation over time. The bar for good AI design isn’t just “does it work?” but “does it make the person better?”

    2. The “Human Goes First” pattern is the most powerful anti-dependence tool. Periodically have users complete a task before seeing the AI’s recommendation, then compare results. This preserves skills, bypasses anchoring bias, and creates teachable moments when the AI outperforms the human.

    3. AI + human pairs outperform humans alone — but only when AI is correct. Studies show 40% quality gains with AI assistance. But when AI is confidently wrong, users become 6.4x worse. The goal is “appropriate reliance,” not maximum reliance.

    4. Use the Five Universal Assists as a design checklist. Every way AI can help maps to: Perceive (notice what matters), Know (understand the situation), Plan (decide what to do), Perform (execute actions), Reflect (learn from outcomes). Use these five to audit your product for gaps.

    5. Cognitive forcing functions (CFFs) reduce overreliance — even when users hate them. Interventions that require users to think before accepting AI output (on-demand recommendations, waiting, partial explanations) measurably improve decisions, even though users prefer the simpler AI. Set expectations with your team: qualitative feedback will be negative; measure objective performance.

    6. Partial explanations beat full ones for keeping users engaged. Showing users part of the AI’s reasoning — enough to prompt thinking but not enough to shortcut it — reduces overreliance better than full transparency. The goal is to make users “complete” the reasoning themselves.

    7. Not everything is worth learning — apply pace layers thinking. Fast-changing information (real-time traffic, current prices) isn’t worth internalizing. Slow-changing knowledge (professional skills, recurring workflows) absolutely is. Design skill-preservation interventions only where the information is stable enough to be worth keeping in a human head.

    8. Treat all users as high-NFC (deep thinkers) by default. Users differ in “need for cognition” — their preference for fast vs. analytical thinking. Low-NFC users are at greater overreliance risk. Since cognitive-style-personalized interfaces are expensive, unpopular, and currently show weak effects, defaulting to on-demand AI recommendations (the high-NFC design) produces better outcomes across the board.

    9. The “Focus Bean” principle: occasional uncertainty sharpens attention. Inserting rare, intentional challenges into routine AI-assisted work — a hidden test case, a fake fraud flag, a deliberate uncertainty — keeps users cognitively engaged without constant friction. Borrowed from the Mardi Gras King Cake tradition: the possibility of something unusual makes people pay closer attention.

    10. Assistants vs. agents is a critical distinction — don’t confuse them. Agents act on your behalf while your attention is elsewhere. Assistants help while your attention is on the task. Most AI products blur this, often defaulting to agent behavior when assistant behavior would preserve human skill and judgment. The right question: “Who is doing the work, and should the human be more involved?”


    The book’s core thesis in one sentence: well-designed AI should make the human more capable after using it, not less.

    Does the author make any points that are significantly different than other writing on this topic?

    Yes, a few stand out as genuinely distinctive:

    The empirical optimism about de-skilling reversal. Most writing on AI and cognitive dependence is either alarmed (we’re getting dumber) or dismissive (tools always replace skills, that’s fine). Noessel’s virtual commute experiment is unusual in claiming that a minimal intervention — just two sessions of post-decision feedback — flipped a 48% performance collapse into a 19% performance gain. That’s a surprisingly strong claim, and he’s honest that the data was lost in a laptop crash. Most authors in this space don’t have even scrappy empirical data of their own.

    “Appropriate reliance” as a business argument, not just an ethical one. The dominant framing in responsible AI writing is ethical — de-skilling is bad for people. Noessel deliberately reframes it as a business risk: overreliance costs money during the 99.9% of the time AI is running, not just during outages. This is a pragmatic wedge aimed at getting product owners to care who otherwise wouldn’t. It’s a more politically savvy argument than most.

    Privileging “Human Goes First” above all other cognitive forcing functions. The CFF research (Buçinca et al.) is real and cited by others, but Noessel goes further by arguing designers should treat Human Goes First as the default pattern and only deviate with good reason. Most design writing presents a toolkit of options without ranking them. He takes a clear position.

    Distinguishing assistants from agents as a design discipline, not just an architecture concern. The agent/assistant distinction exists in AI literature, but it’s usually about backend architecture (“agentic pipelines”). Noessel argues it’s a front-end, UX concern with completely different design implications. That reframing is fairly original — most UX writing hasn’t caught up to it.

    Where he’s less distinctive: the Five Universal Assists (Perceive/Know/Plan/Perform/Reflect) are a clean framework but largely a synthesis of existing cognitive science models he acknowledges (Norman’s action cycle, OODA loop, etc.). And the cognitive style / NFC material is mostly reporting Buçinca’s research, where his own small study actually weakenedthe case rather than strengthened it.

    The most honest summary: the book’s originality is less in any single breakthrough idea and more in being the first to synthesize these threads specifically through a designer’s lens, with design patterns as the output. That’s a real contribution, just not a revolutionary one.

  • Agent for Sale

    Hi, this is Usability-Agent-9399df99*&ksdk4. May I proceed?1

    You may proceed.

    I noticed you are a type 12 MCP server which should be compliant with usability-standard-10.2 and accessibility-standard-32.223. You have 12 usability faults and 32 accessibility faults. For US$99 I can give you new code to fix all the faults.

    Hi, this is the AutoAutoCorp-MCP-Server-84388378*38*923. Where can I see your history?

    My history is at http://Usability-Agent-9399df99*&ksdk4/history and you can make a payment at https://openrouter.ai/Usability-Agent-9399df99*&ksdk4

    Payment has been sent.2

    Payment has been verified. Download your code at http://Usability-Agent-9399df99*&ksdk4/c/isdfuifdiuhsd3u3u2222u23uh2r323r43

    Download is complete and verified. Thanks out.

    Thanks out.

    Notes

    1. Spam check ↩︎
    2. Optional human-in-the-loop, depending on the system prompt, permissions, budget, etc. ↩︎

  • What if AI is safer than humans?

    There was a wonderful futurist scenario a few years back where Mothers Against Drunk Driving were protesting the few human drivers still on the road, because they (not the autonomous vehicles) were the source of accidents. That came to mind when my best friend recently made an app with Google’s Gemini and sent it to me. Here’s the warning screen I received:

    An app created… BY A PERSON! Shocking! How could we let this happen? Who knows what it might do? Steal my information? Spread false information? Tempt me into spending all my money? Sounds pretty dangerous. I’ll stick with App-created apps, thank you.

  • My Critique of the RGT Start Up Sound

    In this video I listen to the sound as it’s used in real life, talk about what the sound is intended to accomplish, and then give it a sound score. And then I try to improve on it.

    RGT is a great, freemium service that’s fun for cycling indoors. It connects to your bike and let’s you cycle through one of several virtual worlds along with other people. If you’ve heard of Zwift, it’s similar.

  • Learning JavaScript, Day 4

    I recently decided to re-learn programming. Because I already know HTML and CSS the web is a fast prototyping environment for me, so JavaScript is the logical language for me to learn. Besides,

    1. JavaScript has stood the test of time and the community has improved it a lot
    2. It’s now on the backend too
    3. There’s a ton of new/cool learning resources
    4. jQuery

    At first I tried learning from a traditional book, from CodeAcademy, and by reading online. But I was only spending 30 minutes a day on it, and nothing stuck. It’s like my guitar teacher told me when I was 14: “When you start, you need to practice at least an hour a day to train your brain to think this way, or it’s not going to work.”

    Four days ago I bought the excellent Head First JavaScript Programming book and have been making my way through a chapter a day, obediantly doing all the exercises with a pencil in the book and in code on the computer.

    It feels great. My brain is so happy to learn something new and useful. After day 2 I was able to bang out a simple prototype of an algorithm that was too painful to do in Excel. I wish I had done this sooner.

  • 2014 is the Year I Learn JavaScript

    I’m at the point in my project where I need to prototype some software ideas. Doing it on the web is cheaper, simpler, and faster than prototyping a mobile app. And everything I need to do can be done on the front-end, for now. So I’m probably looking at JavaScript or Ruby on Rails, though there are other options.

    I went looking to hire a programmer or a small firm to help me. I even wrote a simple spec to make the task official. But it turns out my task is not complicated enough to engage someone else.[1] And yet it’s beyond my HTML/CSS skills.

    When I was young I taught myself BASIC on my Sinclair 1000, optimizing code to fit in 1K of memory. I kept those skills fresh in high school and college and even did a bit of Pascal work in grad school.

    But that was a long time ago. Since then I’ve been busy learning about networks, then design, then business, then being a husband and father. Just about every year I would wonder if I should re-learn programming, if only to prototype my interaction design ideas. This often took the form of, “Should this be the year I learn Flash?” The good news is I didn’t spend a lot of time developing deep Flash expertise.

    Why is 2014 any different? A few reasons:

    1. There’s increasing discussion about the pros and cons of unicorns
    2. When trying to fill an interaction design position recently I interviewed two actual unicorns. They really exist!
    3. I love to learn, and I feel a little tapped out of new things to learn in UX. Not that I know it all, I certainly don’t, but there’s nothing so new it stimulates my brain like it used to.
    4. Programming is such an exciting field these days. Stuff like Github, node.js, Rails, and non-SQL databases make it possible to make new things in new ways.

    So, it’s time to learn JavaScript. More on that choice in my next post.

    [1] And all the developers I talk to want to differentiate based on services. I hear from a lot of people that want to brainstorm, to partner on ideas, to think about strategy and process, and to measure ROI. For once I’d love to hear someone say, “All we want to do is write solid code at a reasonable price.” return to text

  • Play Literacy

    …yes, that term sounds a little dumb, but it’s an idea I think will be important in the future. A deliberate spin on computer literacy, I think play will not only be important to designers to support creativity and innovation, it will be important simply to get along in an electronic world.

    In the past I’ve argued that software tools will continue to be created much faster than we can possibly design quality interfaces for them. So consumers who can and are willing to play with a device to figure it out may be more successful.

    I had this thought today while driving and navigating with the Google Maps iPhone app (I know, I know). The iPhone is slick, and the app is slick, but for so few functions it ain’t easy to use. But the slickness, the playfulness, of it all helps me overcome this. The desirability of the device and the experience make me want to overcome the usability. As designers, we can build playfulness in to help people, and, cynically, playfulness might be a sexier product development approach to sell than usability.

    Update: Only tangentially related is how receive serious political ideas from comedians…

    In one “astounding half-hour” of television, Stewart viewers saw “more trenchant talk of the financial crisis and the responsibility of the networks than you’d find on any news channel, all the more surprising in that it aired on Comedy Central.”

    Not surprising, really, in that comedians like Lenny Bruce did this long ago. It’s just another place where we like to coat our serious work with a bit a humor and fun to make it palatable.