Geek Mode: I Gave Gemini 3.8 Flash and GPT-5.6 Sol the Same Real-World Jobs

I didn’t set out to spend the first week of September running an AI benchmark.

Most of these tests started because I was already trying to get something done.

I needed a stopwatch. I was editing a WordPress interface. I wanted a second opinion on some PHP. I was building a personal application. I needed information from Gmail. I bought a charger a year ago and wanted to document the model and serial number before recycling the box. At one point I needed a deliberately nasty 18-page PDF to test another system’s ability to read complicated documents.

Since I already pay for and regularly use multiple AI systems, I started giving the same jobs to both Gemini and ChatGPT whenever it made sense.

The results surprised me enough that I started saving them.

This is not a comparison based primarily on feature charts, benchmark scores, marketing pages or what either company says its models are supposed to be able to do. I’ll reference those things where they matter, but what interested me was much simpler: what happened when I asked the retail products to do normal work?

Most of the Gemini tests here were run with Gemini 3.8 Flash using the Flash Extended setting in the Gemini interface. Gemini 3.8 Flash had only been released a few days earlier, on September 2, 2026. The ChatGPT tests used GPT-5.6 Sol with the highest retail reasoning setting available to me, High.

Those are the models each app gives you by default on its $20-a-month plan, with each one’s thinking turned up to its highest setting. Most subscribers never change the model, so this is what most people are actually using.

This is therefore a snapshot of early September 2026, not a prediction about which product will be better six months from now. Possibly not even three months from now.

That’s worth stressing because my own preference has changed several times already. Last summer I was using Gemini much more heavily and generally preferred it to ChatGPT. Earlier this year Claude Opus became the model that impressed me most on difficult work, although it could get expensive. Right now I find ChatGPT the strongest overall combination of capability, tools and price for the way I work.

That ranking can change again.

I’m also not claiming that every comparison below is a laboratory-perfect A/B test. These were real jobs. I used the same prompt and source material wherever practical. When one system failed or lost the task, I sometimes had to rephrase, remind it what we were doing or explicitly tell it to continue. That happened much more often with Gemini in these examples, which is itself part of the experience I’m documenting.

If you’re looking for a permanent declaration that one company has won AI, this isn’t it.

If you want to know which one wasted less of my time this week, keep reading.

A quick look at the results

I ended up with eight comparisons substantial enough to document separately, plus a few smaller field notes.

Each test has a separate evidence page with the prompts, screenshots and relevant output. I don’t want to turn the main article into 40,000 pixels of chat screenshots.

Full test index

• Test 1: Stopwatch
• Test 2: Identical-prompt image generation
• Test 3: Iterative WordPress UI editing
• Test 4: ZIP code ingestion and audit
• Test 5: WordPress plugin audit and repair
• Test 6: App development estimates
• Test 7: Anker charger documentation
• Test 8: 18-page PDF project
• Field note: Tool routing

Test 1: I needed a stopwatch, not instructions for making one

This might be the simplest example in the entire comparison, which is why I like it.

I was using my phone to send a prompt to another AI model and wanted to time how long it took. I don’t have seconds displayed on my laptop clock. Yes, I could have searched Google for a stopwatch and found some website covered in advertisements, consent banners and six other things I didn’t need.

Make me a stopwatch so I can click start and stop on in this window with working buttons.

Gemini gave me a perfectly reasonable block of HTML, CSS and JavaScript, followed by instructions telling me to save it as an HTML file and open it in a web browser.

Technically, the code was fine.

It also wasn’t what I asked for.

ChatGPT initially told me it couldn’t guarantee that arbitrary live controls could run directly inside the conversation, but then proceeded to give me a working stopwatch anyway. I could press Start, Stop, Reset and Lap directly in the interface. It also provided the HTML file as a backup.

I didn’t have to create a file, paste code, choose a filename or open another program. I asked for a stopwatch and got a stopwatch.

That sounds trivial, but it gets at one of the biggest differences I’ve noticed in AI products recently. Knowing how to make something is not the same as actually completing the job.

Gemini answered the programming question.

ChatGPT solved the problem I had.

View the full stopwatch test →

Test 2: Both can make good images

Before this starts sounding like a long list of complaints about Gemini, here’s a test where the difference was much smaller.

I gave Gemini and ChatGPT the same detailed prompt for a Hawaiian beach scene. It included a cat in the foreground, people, signs, food stands, a dog doing a specified action and quite a few environmental details.

Both generated good images.

I slightly preferred the ChatGPT version, but that’s subjective. The Gemini image was attractive, coherent and absolutely usable.

That matters because I don’t think the useful conclusion here is “Gemini can’t make images.” It clearly can.

I had also seen a much worse Gemini image earlier in a complicated PDF workflow, where it produced an extremely simple cartoon cat despite the broader context calling for something much richer and more photographic. But that wasn’t a controlled comparison. The beach test was.

When the task was isolated and clear, both systems performed well.

View the identical-prompt image test →

Test 3: Repeated edits were a different story

Generating a new image from scratch and modifying an existing image several times are different jobs.

I was mocking up changes to the WordPress Media Library interface on my site. I supplied screenshots and asked the systems to add new admin-only fields while preserving the existing interface. Then I gave a second screenshot and asked for another field, sample EXIF data and location information.

Both systems received the same source images and essentially the same instructions.

This is much closer to the kind of image work I actually do: don’t redesign everything, don’t get creative, just change these specific things while preserving the rest.

ChatGPT did this very well.

The edited interface remained recognizably the original interface. The requested fields landed where I asked for them, and successive edits generally preserved previous work.

Gemini’s output was weaker. In my experience this is a recurring problem with Gemini image editing. It can produce a good first image, but once I start asking for precise corrections or successive edits, it is more likely to drift, fail to preserve something or simply refuse to fix a detail no matter how explicitly I describe it.

I’m not going to pretend two screenshots prove a universal property of an image model. They don’t.

They do match what I’ve repeatedly encountered in normal use.

View the complete iterative UI-editing sequence →

Test 4: Gemini could audit the code, once I got the code into a format it liked

This one gets more technical.

I had a small WordPress plugin in a ZIP file. We’re not talking about Linux here. It was nine files.

I handed the archive to Gemini and asked it to review the code.

Gemini could apparently inspect enough of the ZIP structure to identify the filenames and infer the architecture of the plugin, but it then told me that the contents were compressed binary data that couldn’t be reviewed line by line.

It proceeded to give me a list of potential WordPress security and architecture problems anyway.

Some of them were sensible things to check. The problem was that most of the response read like advice inferred from the filenames and common WordPress failure modes, rather than findings from the actual implementation.

In other words, it could tell that there was a file called something like class-apmi-gemini-client.php, and from that it could tell me what often goes wrong in a Gemini API client.

That’s not the same as auditing mine.

I then supplied the actual source in a plain-text format.

The experience changed completely.

Gemini clearly read the code and produced a detailed, implementation-specific audit. It referenced actual functions, variables and code paths and found several legitimate things worth examining.

It also confidently got some things wrong.

One of its biggest “critical” findings was that Google’s Gemini Interactions API endpoint used by the plugin didn’t exist and should be replaced by the older generateContent contract. That was incorrect. The current Google API does support the Interactions endpoint and the request/response structures the plugin was using.

It also claimed WordPress automatically EXIF-rotated the image simply by constructing the image editor, leading it to diagnose a double-rotation bug that wasn’t actually there.

Still, once Gemini could read the source properly, it was doing real code review.

That’s why I don’t describe this as “Gemini is bad at coding.”

The coding ability was there.

The retail file-ingestion workflow got between me and the coding ability.

My workaround has become absurdly simple: let another AI unpack the codebase, combine it into one or more clearly labeled plain-text files, then hand those to Gemini.

That works.

It is also exactly the sort of workflow tax that doesn’t appear in a feature matrix.

“Supports file uploads” gets a checkmark either way.

View the ZIP-ingestion and raw-code audit test →

Test 5: Twenty minutes versus twenty seconds

This became the geekiest comparison of the bunch.

I had two external audits of the same WordPress plugin. Some findings agreed. Others directly contradicted each other. I wanted the AI to determine which findings were actually valid against the current WordPress and Google APIs, correct the code, and give me the revised package.

Gemini returned its analysis very quickly, on the order of seconds.

ChatGPT’s interface eventually reported:

Worked for 20m 37s.

A long wait is not automatically a good thing.

If an AI can correctly solve a problem in 20 seconds, spending 20 minutes thinking about it doesn’t make the answer 60 times better.

What made this interesting was that the resulting code really was different.

Gemini produced a plausible implementation that looked capable of handling the ordinary happy path. It found useful issues and corrected a number of things.

But it also accepted some incorrect recommendations from the external audits.

For example, the faulty EXIF double-rotation finding made its way into the code. Another recommendation concerning WordPress’s sideload hooks was implemented even though the actual WordPress lifecycle worked differently.

ChatGPT challenged those assumptions before changing the code.

It checked current upstream behavior, rejected findings that didn’t survive verification, implemented the ones that did, changed the upload-state handling to be more robust, strengthened validation and diagnostics, ran syntax checks, packaged the revised plugin and documented both the changes it accepted and the audit findings it rejected.

It also explicitly left some things unresolved because they required a real staging environment, such as very large image handling, HEIC behavior and concurrent uploads.

The important result here isn’t that Gemini’s code was garbage. It wasn’t. My static review was that it would probably work for ordinary uploads.

The difference was defensive engineering.

Gemini produced something closer to a solid scaffold.

The ChatGPT version was the one I would rather take into staging.

This is where I’ve started thinking of inference budget itself as a product feature. Sometimes I want the answer now. Sometimes I’m giving a system a difficult job where I would much rather it spend 20 minutes checking its assumptions than give me a confident answer in 22 seconds.

In this case, the extra time left fingerprints in the resulting artifact.

View the plugin audit, important code differences and resulting packages →

Test 6: Both AIs were hilariously pessimistic about AI coding

I gave both systems an 80-section specification for a personal application I’ve been building.

Calling it a task manager would be underselling it considerably. It handles tasks, recurrence, calendar items, reminders, push notifications, food and nutrition records, medication data, sleep, long-form logs, incoming email, AI capture, rules, audit history, attachments, reports, shipment information, expected future events and a pile of related automation.

A major architectural rule is that AI interprets, while deterministic server-side PHP owns authorization, validation, calculations, deduplication and database mutations.

I asked how much work a conventional implementation would require.

Then I asked how much AI coding tools would reduce that.

Gemini’s AI-assisted estimate still landed around 300 to 450 engineering hours, roughly ten to fourteen weeks at the schedule it proposed.

ChatGPT was even more conservative in calendar terms. It thought a daily-driver version might take months and the full specification could take roughly a year or more for one person directing AI heavily.

The funny part is that I was already building it.

At the time of the comparison I was roughly ten or eleven days into the project. Around 95 percent of the code was written by my rough estimate, perhaps 60 to 70 percent was already deployed, and the application was functioning and useful.

By around the two-week point I expected most of what mattered to be in production.

The biggest bottleneck wasn’t my ability to write PHP.

It was running out of AI.

I kept hitting five-hour usage windows and weekly limits on inexpensive consumer subscriptions. If I’d been willing to spend substantially more on the highest usage tiers, I believe the project could have moved even faster.

Now, there is an important qualification here. “95 percent written” is not the same as “perfectly hardened forever.” The last few percent of a complex system can expose ugly recurrence bugs, races, strange email formats, migration problems, notification failures and things no design document predicted.

But the estimates were still dramatically disconnected from the rate at which usable software was actually appearing.

I think the models are making an understandable mistake.

They start with conventional software-development estimates and then apply an AI productivity discount.

My actual workflow doesn’t look like conventional software development with faster typing.

There is no product manager writing a ticket for a developer, no meeting where the developer asks what the ticket meant, no QA department filing another ticket, no two-week sprint before I see whether a button feels wrong on my phone.

I notice the problem. I describe the intended behavior and the architectural boundaries. A coding agent inspects the existing system, writes the implementation, runs tests and hands the change back. I use it. If it’s annoying, I change it again.

The bottleneck is moving upstream.

Writing code is getting cheap.

Keeping the entire system straight in your head is not.

The valuable human work is increasingly deciding what should happen, understanding how X affects Y, anticipating what a retry or failure means, maintaining architectural boundaries, and giving the AI enough precise context that it doesn’t have to invent the product while it implements it.

Someone who can do that well can keep modern coding agents astonishingly busy.

View the application specification and both estimates →

Test 7: Sometimes both systems simply do the job

One morning I was setting up an Anker charger and two large Anker battery packs.

I asked practical questions while I was standing there with the hardware. Could I feed the battery through two USB-C cables and make it charge faster? Which port should I use? Were all three ports on the newer charger equivalent for my use? Was the faint warmth I noticed after leaving the packs connected overnight normal?

Both systems handled the central questions well enough for what I needed.

Later I was about to recycle the charger box and wanted a permanent record of the model and serial number. I photographed the product label and the serial label and asked both systems to crop and straighten the important areas, combine them into one image, look up the first day of Prime Day 2025 and add a small record showing the purchase date and the date I first used the charger.

Both produced the right information.

Both preserved the important product identifiers correctly.

The difference was largely presentation.

Gemini produced a functional composite. The labels were arranged vertically and the result looked like a quick collage.

ChatGPT produced something that looked much more like the reference sheet I had imagined: straight horizontal labels, clean whitespace and a large readable information block.

Neither system failed this test.

One simply produced the artifact I would rather keep.

For the public copies shown here, I’ve deliberately obscured part of the charger serial number. The original AI outputs both reproduced it correctly. The redaction is mine and was added only because there is no reason to publish a unique serial number to prove that an image editor read it properly.

View the complete Anker conversation and source images →

Test 8: Make me an 18-page PDF

The final comparison was deliberately harder, but it still came from a real need.

I wanted a complicated PDF to test another AI system’s ability to read mixed document types.

I asked for exactly 18 pages.

The document was supposed to contain a creative story called “The Secret Life of Animals,” five photorealistic animal photographs, three realistic but completely fictional pet-store receipts rendered as images, a wild-versus-captivity longevity table, a graph, normal embedded PDF text, and two complete pages deliberately rasterized as static images so that a parser couldn’t simply extract their text.

I also invited the AI to add other common document elements that might be troublesome for OCR.

The first interesting part of this comparison is that Gemini understood the assignment extremely well.

After an initial generic error, it produced an excellent plan. It mapped all 18 pages, proposed multi-column layouts, watermarks, stamps, barcodes, footnotes, a survival curve and several other OCR stressors.

If I were grading the planning response alone, Gemini did great.

Then I said:

do it please

Gemini responded:

What would you like me to work on?

That was not encouraging.

I reminded it to continue the project it had just planned.

It generated three images: a cat, a raccoon and what was essentially a blue-green blurred background with no animal in it.

Then it asked:

Which project are we picking back up today?

I pointed out that it had just generated images for the project and asked where the writing, receipts and PDF were.

Gemini then told me that my message had apparently come through in a separate or reset conversation thread, and that the previous session wasn’t visible to it anymore.

From my side, I was still looking at the same short conversation.

I don’t know what happened internally, and I’m not going to pretend I do.

From the user side, it had completely lost the plot.

It subsequently generated a stylized horse and a cartoon-like owl, neither of which really satisfied the original request for photorealistic animal photographs. Of the five Gemini image assets produced during the attempt, the cat and raccoon were the only two I would describe as straightforward photographic animal images. One image contained no animal at all.

At that point I stopped.

There is a limit to how many times a normal user should have to explain to an AI what project the AI itself just proposed.

ChatGPT received the same core assignment.

It generated five genuinely photorealistic animals: a snow leopard, scarlet macaw, elephant, fox and sea turtle. It generated three detailed fictional thermal receipts with internally plausible totals. It generated the two complete rasterized text pages.

Then it assembled the entire thing into the requested PDF.

The finished file was exactly 18 pages and around 38 megabytes. It included the images, receipts, lifespan table, graph, narrative, mixed formatting and two pages where the visible text existed only as an image.

The result wasn’t flawless. A couple of layout details could have been cleaner, and the intentionally rasterized page images retained their original printed page numbers, which made the internal pagination slightly odd.

But it was an 18-page PDF.

That distinction is doing quite a lot of work here.

The Gemini animal files were also dramatically smaller than the corresponding ChatGPT photographic assets. In my downloaded set, Gemini’s files ranged from about 7.7 KB for the blue-green non-animal image to about 144 KB for the raccoon. The ChatGPT animal photographs were generally around two megabytes or more apiece.

File size by itself proves almost nothing about image quality, so I wouldn’t use that as a score. Different formats and compression settings can change file size enormously.

What matters is that the larger ChatGPT assets were also visibly higher-resolution photographic images and actually satisfied the requested subjects.

More importantly, this was the second time in a matter of days that I had tried to get Gemini’s retail interface to make me a PDF and failed.

In the earlier attempt, Gemini told me it couldn’t directly produce the requested downloadable PDF and instead offered HTML and instructions for opening it in a browser and using Print > Save as PDF.

This time it didn’t even get that far. It planned the project, started generating partial assets and then apparently lost access to the job it was doing.

Meanwhile, I have used ChatGPT to create much larger finished PDF projects, including a private roughly 20-page document with generated graphics, extensive formatting, research links, multiple revisions and a mobile-friendly second layout.

Whatever the feature matrix says, that is the real-world difference I’m currently seeing.

View the complete 18-page PDF test, source prompts and generated assets →

Field note: Sometimes Gemini forgets which Google tools it has

Not every frustrating example fit neatly into a controlled head-to-head test.

One afternoon I told Gemini to use Workspace to search my Gmail and Google Drive for files with a particular extension.

It replied that it did not have access to my Google Workspace, Gmail or Drive and instructed me how to search them myself.

I opened a fresh conversation and asked it to search just Gmail.

It did.

Then I asked it to search Drive.

It did that too.

The file I was looking for apparently wasn’t there, which isn’t Gemini’s fault. The interesting part is that the product had just successfully used the two integrations it had told me it couldn’t access.

That’s another version of the same problem.

The capability exists.

The product doesn’t always route the request to it.

I have also had Gemini successfully locate specific Gmail messages for me but give me links that led to dead pages. I gave ChatGPT essentially the same retrieval job and the resulting message links worked.

These are boring failures.

That’s precisely why they matter.

Nobody puts “sometimes gives you a link that doesn’t open” on a benchmark chart.

But if you’re using AI all day, small failures like these determine whether the product saves time or quietly creates more work.

View the tool-routing field notes →

This is not really a Gemini hate article

After all of those examples, I should probably say that explicitly.

A year ago I wasn’t particularly happy with the answers I was getting from ChatGPT. I also disliked the tone it often took. Gemini was the product I generally preferred.

Then I started using Claude Opus heavily for difficult work and found it extremely comprehensive, although the usage could get expensive.

Now ChatGPT has moved into the position Gemini occupied for me last year.

It isn’t because GPT-5.6 Sol wins every individual task.

It doesn’t.

Gemini produced an excellent result in the controlled beach-image test. Once I got raw source code into a format it could reliably consume, it performed a substantive code audit. Its planning for the 18-page PDF was arguably more ambitious than ChatGPT’s initial plan.

The frustrating part is the number of times I encounter a good underlying capability hidden behind an unreliable last mile.

Gemini can apparently read the code, but I sometimes have to flatten the ZIP first.

It can search Gmail and Drive, but sometimes it initially tells me it can’t.

It can understand a complex PDF assignment beautifully, but then lose track of the assignment while trying to execute it.

It can make excellent images, but successive precise edits have been less reliable for me.

That isn’t the same thing as saying Gemini is a bad model.

It means that, in early September 2026, the complete retail experience is creating more workflow friction for me than ChatGPT.

And “early September 2026” matters.

The players don’t just improve. They trade places.

Gemini 3.8 Flash itself had been out for less than a week when I ran these tests.

That sentence alone should make anybody cautious about treating this article as a permanent ranking.

These systems are moving at a pace where a comparison can begin aging while you’re still writing it.

My preferred AI has already changed several times.

The larger change I’ve noticed is that I’m increasingly judging these products less by the quality of an isolated answer and more by how much of a job they can carry from beginning to end.

Can it read the files? Can it keep track of the project? Can it use the right tool without being begged? Can it question a bad assumption? Can it make the artifact? Can it revise the artifact without destroying what was already right? Can it give me a link that works? Can it keep working for 20 minutes when the problem deserves 20 minutes?

Those are not traditional chatbot questions.

They’re questions you ask about a coworker, a software tool or a general-purpose computing environment.

That’s why the little failures matter, but it’s also why the successful examples are so remarkable.

A couple of years ago, asking a chatbot to build a working tool, audit and revise a WordPress plugin, generate a set of production-quality images, or assemble a custom 18-page mixed-content PDF would have sounded like stunt prompting.

This week those things were errands.

And right now, for the particular mix of work I do, GPT-5.6 Sol is completing more of those errands with fewer workarounds.

Ask me again in December.