👋 Hey, this is Ben with a 🔒 subscriber-only issue🔒 of Ben’s Bites Pro free version of Ben’s Bites Pro⁵. A weekly newsletter covering AI trends, ideas, business breakdowns and how companies are using it internally. I wrote a series of posts like this as an experiment. Now I’ve launched Ben’s Bites Pro⁵. If you find these posts valuable, consider subscribing.

GPT-4V (or more simply, GPT Vision) search volume is starting to take off, and don’t expect it to slow down as it becomes more widely known. Let’s explore some examples and opportunities from this trend.

What is it?

It’s AI that can see. Using GPT4-Vision API or uploading an image to ChatGPT, you can get the model to interpret what is in an image or video. “What’s this thing on my bike?” is great and all, but how about these examples:

Screenshot to code. It’s what you think it is. Take a screenshot of something, and it’ll turn it into actual code. Clone the YouTube, Instagram, Hacker News websites etc. (github repo here¹⁰⁹, no-code version here¹²²)

Cursor¹⁵⁹, the popular AI coding tool, lets you copy components with a screenshot and add it to your code, modify it etc.

Try out GPT-V in Cursor!

It's pretty good for building/modifying components!

— Aman Sanger (@amanrsanger)
7:22 PM • Nov 29, 2023

Tldraw²⁰⁴ has been everywhere on my Twitter (X) feed recently. And for good reason. An unsuspecting whiteboard app that came alive with the new AI model. They added GPT4-V into its ‘Make Real’ feature - so you could draw boxes of a web application (let's say a calculator) and it would actually create a functional calculator.

Yeah, this is nuts.

Just made a working tip calculator app in 2 minutes.

Sketch > working prototype.

The future of software will be heavily streamlined with AI.

Kudos @tldraw

— Roberto Nickson (@rpnickson)
4:14 PM • Nov 17, 2023

So drawing code is now real. It’s only a matter of time before more applications are made this way. The true no-code (I’ve been harping on about this for years!!)

Also, its ‘drawing’ capabilities are insane too from such a basic starting point.

How to draw an owl with @tldraw

— Taz Singh (@tazsingh)
7:09 PM • Nov 28, 2023

Be My Eyes²⁵² is an app for the visually impaired to let volunteers essentially FaceTime the impaired to help them with daily tasks. Now, powered by OpenAI - AI can be the helper.

Taking control of a user’s computer¹⁸⁰. This guy asked it to find his youtube channel and you see the AI literally go to Google Chrome, type in the address bar and click on a search result. WITHOUT HIM TOUCHING ANYTHING. While this is a basic task, you can imagine what this kind of thing unlocks. 

And, if you can’t, I did:

Opportunities

You can generate AI voiceovers for your product demos like this guy just built^267†. So instead of going through the process of scriptwriting, GPT4-V will help do that for you.

Make Pokemon Go, but for real life, like this demo²³⁴.

Use Vision to count cards at an online casino. Ok, this isn’t recommended, but technically possible?

Get a breakdown of how much you spend waste on social media each week.

‘Sketch your dream’ app.

Virtual time-travel experiences.

Personal stylist assistant.

Analyse my weightlifting technique, my work posture (like this demo¹⁷⁶), my tennis swing etc.

Set up a productised service to turn real estate listings into more enhanced virtual viewing experiences.

Create a ton of infographics and interesting reports on topics that you can sell access to. 

A user feedback tool where I just upload a video (a loom?) of me using your site and it interprets where I’m getting stuck, where I’m spending too much time, where I’m clicking vs where I should be. Forget heatmaps.

How about an automated system that signs up for every AI tool, goes through onboarding and posts the recording on a site so others can check it out? Works for PageFlows²¹² (which I believe, has a human behind it).

The opportunities are endless. Send me¹² what you’re working with Vision. And let me know⁸ what you think of these types of posts - keep ‘em going // don’t bother