Thread:
I've conducted several informal experiments over the last few weeks about alt text for #photos as described by humans and as provided by #AI systems.
#LLMS, despite providing plethora of details when describing images, still miss the nuances of what the photos contain. Human #descriptions certainly continue to be better at conveying context.
@ppatel did you explore the hybrid approach?
Something I’ve been doing for alt text is pasting the image into Claude, getting its first draft of alt text and then prompting for follow-up improvements: “shorter”, “more details about the lighthouse” etc
(To be honest I usually then manually edit the text as well before using it)
@simon Oh absolutely. I often queery for details and try to sus out the context whenever feasible. In one of the tools I use, my prompt tells OpenAI to give me details while being concise or it would go on forever. I also ask it to give me the text rather than summerizing it if there's text in the image.
@simon @ppatel
One problem when AI tech companies train their models using publicly available data without asking permission first, is that they've very likely scraped up family photo albums and Instagram feeds. Their AI can thus identify people by their full name in any photos a would-be stalker were to upload. To avoid the bad publicity, AI companies blur out all the faces.
As a result the AI will describe the presence of "gray blocks" in the image
https://masto.ai/@bornach/112987668496258631
