I made a YouTube video in French and soon in English that I encourage you to watch.

ChatGPT vision is a new feature of ChatGPT that allows it to "read" images, and thus be able to discuss with us, about our plans, sections, 3D and other architectural sketches.

To benefit from it, you currently need to have the paid version of ChatGTP and then an upload button will appear on the left of the prompt invitation.

You can share different types of files with it such as PDFs, CSVs or images.

In this video, I tested the tool on various concrete examples from the world of architecture:

Comment on a plan
Recognize elements in an image
Read text labels
Make the connection between a plan and external data (construction standards, for example)

The partial conclusion I can draw from it is that it's useful, but Google Vision has limits for now.

For example, I don't think that Chat-GPT can yet do complex geometric calculations. Which would be useful for example to assess whether the plan complies with disability standards.

I don't think it can understand the scale of a drawing, even if a scale is shown on the plan.

Similarly, to have third-party info on an urban plan would require it to query external databases / APIs. Which it does not do yet.

However, it is possible to program an application that does this, using the Open API's API.

On the other hand, it sees the plan, can describe what it contains.

In the case of a construction site photo, by zooming in enough, it can certainly answer general questions. Not yet capable of judging the quality of a coating!

To read texts included in images, a technique that has existed for a long time under the name of OCR, it copes very well.

And so it is entirely possible to take a photo that contains a lot of text, the title on the books of a library for example. And it will identify and decipher it without problem and list all the books, in table form if you ask it.

You can try to take the books from your library for it to make a list in order to put your architecture library in a database. It will probably do a good job!

.