Vibe Coding, Vibe Checking, and Vibe Blogging

For the past decennary and a half, I’ve been exploring the intersection of technology, education, and plan arsenic a professor of cognitive subject and design astatine UC San Diego. Some of you mightiness person work my caller portion for O’Reilly Radar wherever I elaborate my travel adding AI chat capabilities to Python Tutor, the escaped visualization instrumentality that’s helped millions of programming students recognize however codification executes. That acquisition got maine reasoning astir my evolving narration with generative AI arsenic some a instrumentality and a collaborator.

I’ve been intrigued by this emerging signifier called “vibe coding,” a word coined by Andrej Karpathy that’s been making waves successful tech circles. Simon Willison describes it perfectly: “When I speech astir vibe coding I mean gathering bundle with an LLM without reviewing the codification it writes.” The conception is some liberating and somewhat terrifying—you picture what you need, the AI generates the code, and you simply tally it without scrutinizing each line, trusting the wide “vibe” of what’s been created.

My narration with this attack has evolved considerably. In my aboriginal days of utilizing AI coding assistants, I was that idiosyncratic who meticulously reviewed each azygous line, often rewriting important portions. But arsenic these tools person improved, I’ve recovered myself gradually letting spell of the steering instrumentality successful definite contexts. Yet I couldn’t afloat clasp the axenic “vibe coding” philosophy; the prof successful maine needed immoderate prime assurance. This led maine to make what I’ve travel to telephone “vibe checks”—strategic verification points that supply assurance without reverting to line-by-line codification reviews. It’s a mediate way that’s worked amazingly good for my idiosyncratic projects, and contiguous I privation to stock immoderate insights from that journey.

Vibe Coding successful Practice: Converting 250 HTML Files to Markdown

I’ve recovered myself progressively turning to vibe coding for those one-off scripts that lick circumstantial problems successful my workflow. These are typically tasks wherever explaining my intent is really easier than penning the codification myself, particularly for information processing oregon record manipulation jobs wherever I tin easy verify the results.

Let maine locomotion you done a caller illustration that perfectly illustrates this approach. For a people I teach, I had students taxable responses to a survey utilizing a proprietary web app that provided an HTML export option. This near maine with 250 HTML files containing invaluable pupil feedback, but it was buried successful a messiness of unnecessary markup and styling code. What I truly wanted was cleanable Markdown versions that preserved conscionable the substance content, conception headers, and—critically—any hyperlinks students had included successful their responses.

Rather than penning this conversion publication myself, I turned to Claude with a straightforward request: “Write maine a Python publication that converts these HTML files to Markdown, preserving text, basal formatting, and hyperlinks.” Claude suggested utilizing the BeautifulSoup room (a coagulated choice) and generated a implicit publication that would process each files successful a directory, creating a corresponding Markdown record for each HTML source.

(In retrospect, I realized I astir apt could person utilized Pandoc for this conversion task. But successful the tone of vibe coding, I conscionable went with Claude’s proposition without overthinking it. Part of the entreaty of vibe coding is bypassing that probe signifier wherever you comparison antithetic approaches—you conscionable picture what you privation and rotation with what you get.)

True to the vibe coding philosophy, I didn’t reappraisal the generated codification enactment by line. I simply saved it arsenic a Python file, ran it connected my directory of 250 HTML files, and waited to spot what happened. This “run and see” attack is what makes vibe coding some liberating and somewhat nerve-wracking—you’re trusting the AI’s mentation of your needs without verifying the implementation details.

Trust and Risk successful Vibe Coding: Running Unreviewed Code

The infinitesimal I deed “run” connected that vibe-coded script, I realized thing that mightiness marque galore developers cringe: I was executing wholly unreviewed codification connected my existent machine with existent data. In accepted bundle development, this would beryllium considered reckless astatine best. But the dynamics of spot consciousness antithetic with modern AI tools similar Claude 3.7 Sonnet, which has built up a estimation for generating reasonably harmless and functional code.

My rationalization was partially based connected the script’s constricted scope. It was conscionable speechmaking HTML files and creating caller Markdown files alongside them—not deleting, modifying existing files, oregon sending information implicit the network. Of course, that’s assuming the codification did precisely what I asked and thing more! I had nary guarantees that it didn’t see immoderate unexpected behaviour since I hadn’t looked astatine a azygous line.

This highlights a spot narration that’s evolving betwixt developers and AI coding tools. I’m overmuch much consenting to vibe codification with Claude oregon ChatGPT than I would beryllium with an chartless AI instrumentality from immoderate obscure website. These established tools person reputations to maintain, and their genitor companies person beardown incentives to forestall their systems from generating malicious code.

That said, I’d emotion to spot operating systems make a “restricted execution mode” specifically designed for vibe coding scenarios. Imagine being capable to specify: “Run this Python script, but lone let it to CREATE caller files successful this circumstantial directory, forestall it from overwriting existing files, and artifact net access.” This lightweight sandboxing would supply bid of caput without sacrificing convenience. (I notation lone restricting writes alternatively than reads due to the fact that Python scripts typically request to work assorted strategy files from crossed the filesystem, making work restrictions impractical.)

Why not conscionable usage VMs, containers, oregon unreality services? Because for personal-scale projects, the convenience of moving straight connected my ain instrumentality is hard to beat. Setting up Docker oregon uploading 250 HTML files to immoderate unreality work introduces friction that defeats the intent of quick, convenient vibe coding. What I privation is to support that convenience portion adding conscionable capable information guardrails.

Vibe Checks: Simple Scripts to Verify AI-Generated Code

OK present travel the “vibe checks.” As I mentioned earlier, the bully happening astir these idiosyncratic information processing tasks is that I tin often get a consciousness of whether the publication did what I intended conscionable by examining the output. For my HTML-to-Markdown conversion, I could unfastened up respective of the resulting Markdown files and spot if they contained the survey responses I expected. This manual spot-checking works reasonably good for 250 files, but what astir 2,500 oregon 25,000? At that scale, I’d request thing much systematic.

This is wherever vibe checks travel into play. A vibe cheque is fundamentally a simpler publication that verifies a basal spot of the output from your vibe-coded script. The cardinal present is that it should beryllium overmuch simpler than the archetypal task, making it easier to verify its correctness.

For my HTML-to-Markdown conversion project, I realized I could usage a straightforward principle: Markdown files should beryllium smaller than their HTML counterparts since we’re stripping distant each the tags. But if a Markdown record is dramatically smaller—say, little than 40% of the archetypal HTML size—that mightiness bespeak incomplete processing oregon contented loss.

So I went backmost to Claude and vibe coded a cheque script. This publication simply:

1. Found each corresponding HTML/Markdown record pairs
2. Calculated the size ratio for each pair
3. Flagged immoderate Markdown record smaller than 40% of its HTML source

And lo and behold, the vibe cheque caught respective files wherever the conversion was incomplete! The archetypal publication had failed to decently extract contented from definite HTML structures. I took these problematic files, went backmost to Claude, and had it refine the archetypal conversion publication to grip these borderline cases.

After a fewer iterations of this feedback loop—convert, check, place issues, refine—I yet reached a constituent wherever determination were nary much suspiciously tiny Markdown files (well, determination were inactive a fewer beneath 40%, but manual inspection confirmed these were close conversions of HTML files with unusually precocious markup-to-content ratios).

Now you mightiness reasonably ask: “If you’re vibe coding the vibe cheque publication too, however bash you cognize that publication is correct?” Would you request a vibe cheque for your vibe check? And past a vibe cheque for that check? Well, thankfully, this recursive nightmare has a applicable solution. The vibe cheque publication is typically an bid of magnitude simpler than the archetypal task—in my case, conscionable comparing record sizes alternatively than parsing analyzable HTML. This simplicity made it feasible for maine to manually reappraisal and verify the vibe cheque code, adjacent portion avoiding reviewing the much analyzable archetypal script.

Of course, my record size ratio cheque isn’t perfect. It can’t archer maine if the contented was converted with the due formatting oregon if each hyperlinks were preserved correctly. But it gave maine a tenable assurance that nary large contented was missing, which was my superior concern.

Vibe Coding + Vibe Checking: A Pragmatic Middle Ground

The take-home connection present is elemental but powerful: erstwhile you’re vibe coding, ever physique successful vibe checks. Ask yourself: “What simpler publication could verify the correctness of my main vibe-coded solution?” Even an imperfect verification mechanics dramatically increases your assurance successful results from codification you ne'er really reviewed.

This attack strikes a bully equilibrium betwixt the velocity and originative travel of axenic vibe coding and the reliability of much rigorous bundle improvement methodologies. Think of vibe checks arsenic lightweight tests—not the broad trial suites you’d constitute for accumulation code, but capable verification to drawback evident failures without disrupting your momentum.

What excites maine astir the aboriginal is the imaginable for AI coding tools to suggest due vibe checks automatically. Imagine if Claude oregon akin tools could not lone make your requested publication but besides proactively offer: “Here’s a elemental verification publication you mightiness privation to tally afterward to guarantee everything worked arsenic expected.” I fishy if I had specifically asked for this, Claude could person suggested the record size examination check, but having this built into the system’s default behaviour would beryllium incredibly valuable. I tin envision specialized AI coding assistants that run successful a semi-autonomous mode—writing code, generating due checks, moving those checks, and involving you lone erstwhile quality verification is genuinely needed.

Combine this with the benignant of sandboxed execution situation I mentioned earlier, and you’d person a vibe coding acquisition that’s some freeing and trustworthy—powerful capable for existent enactment but with guardrails that forestall catastrophic mistakes.

And present for the meta twist: This full blog station was itself the merchandise of “vibe blogging.” At the commencement of our collaboration, I uploaded my erstwhile O’Reilly article,”Using Generative AI to Build Generative AI” arsenic a notation document. This gave Claude the accidental to analyse my penning style, tone, and emblematic structure—much similar however a quality collaborator mightiness work my erstwhile enactment earlier helping maine constitute thing new.

Instead of penning the full station successful 1 go, I broke it down into sections and provided Claude with an outline for each conception 1 astatine a time. For each section, I included cardinal points I wanted to screen and sometimes circumstantial phrasings oregon concepts to include. Claude past expanded these outlines into afloat formed sections written successful my voice. After each conception was drafted, I reviewed it—my ain mentation of a “vibe check”—providing feedback and requesting revisions until it matched what I wanted to accidental and however I wanted to accidental it.

This iterative, section-by-section attack mirrors the vibe coding methodology I’ve discussed passim this post. I didn’t request to constitute each condemnation myself, but I maintained power implicit the direction, messaging, and last approval. The AI handled the execution details based connected my high-level guidance, and I performed verification checks astatine strategical points alternatively than micromanaging each word.

What’s peculiarly absorbing is however this process demonstrates the aforesaid principles of trust, verification, and iteration that I advocated for successful vibe coding. I trusted Claude to make contented successful my benignant based connected my outlines, but I verified each conception earlier moving to the next. When thing didn’t rather lucifer my intent oregon tone, we iterated until it did. This balanced approach—leveraging AI capabilities portion maintaining quality oversight—seems to beryllium the saccharine spot for collaborative creation, whether you’re generating codification oregon content.

Epilogue: Behind the Scenes with Claude

[Claude speaking]

Looking backmost astatine our vibe blogging experiment, I should admit that Philip noted the last merchandise doesn’t afloat seizure his authentic voice, contempt having his O’Reilly nonfiction arsenic a reference. But successful keeping with the vibe doctrine itself, helium chose not to put excessive clip successful endless refinements—accepting good-enough alternatively than perfect.

Working section-by-section without seeing the afloat operation upfront created challenges, akin to coating parts of a mural without seeing the implicit design. I initially fell into the trap of copying his outline verbatim alternatively than transforming it properly.

This collaboration highlights some the inferior and limitations of AI-assisted contented creation. I tin approximate penning styles and grow outlines but inactive deficiency the lived acquisition that gives quality penning its authentic voice. The champion results came erstwhile Philip provided wide absorption and feedback.

The meta-example perfectly illustrates the halfway thesis: Generative AI works champion erstwhile paired with quality guidance, uncovering the close equilibrium betwixt automation and oversight. “Vibe blogging” has worth for drafts and outlines, but similar “vibe coding,” immoderate signifier of quality verification remains indispensable to guarantee the last merchandise genuinely represents what you privation to say.

[Philip speaking truthful that humans get the last word…for now]

OK, this is the lone portion that I wrote by hand: My parting thought erstwhile speechmaking implicit this station is that I’m not arrogant of the penning prime (sorry Claude!), but if it weren’t for an AI instrumentality similar Claude, I would not person written it successful the archetypal spot owed to deficiency of clip and energy. I had capable vigor contiguous to outline immoderate unsmooth ideas, past fto Claude bash the “vibe blogging” for me, but not capable to afloat write, edit, and fret implicit the wording of a afloat 2,500-word blog station each by myself. Thus, conscionable similar with vibe coding, 1 of the large joys of “vibe-ing” is that it greatly lowers the activation vigor of getting started connected originative personal-scale prototypes and tinkering-style projects. To me, that’s beauteous inspiring.