Most SEO teams obsess over alt text and file names. Google sees far more than that. It reads the pixels.
While watching Google I/O 2018 online, I went down a rabbit hole into Cloud Vision, the same machine-learning stack Google uses to understand images at scale. If Google can classify what is actually inside a photo, then the stock images on your site are not neutral. They carry signals, and those signals either reinforce your content or quietly contradict it.
So I tested it. I took a stock photo, renamed the file in Albanian to strip any obvious hint, and ran it through the API to see what Google would infer from the image alone.
Here at #googleio2018! We’ll start live-blogging in about an hour so be sure to follow @WIRED for all the news
Here is what it returned:
Labels:
Spice 64%
Harissa 60%
Superfood 59%
Flavor 58%
Bell Peppers And Chili Peppers 54%
Condiment 54%
Ketchup 50%
Web Entities:
Capsicum annuum var. acuminatum 0.95238
Jalapeño 0.76681
Chili pepper 0.7125
Spice 0.6517
Mexican cuisine 0.6093
Health 0.6032
Crushed red pepper 0.5827
Food 0.5632
Capsaicin 0.5345
Scoville Unit 0.513
Paprika 0.4668
Powder 0.4304
Eating 0.4297
Capsicum 0.14192
Capsicum annuum 0.0309
Pages with Matched Images
https://www.healthline.com/nutrition/8-benefits-of-cayenne-pepper
https://www.health.com/food/health-benefits-of-cayenne-pepper
https://www.drweil.com/diet-nutrition/cooking-cookware/cooking-with-spices-cayenne-pepper/
https://www.lneonline.com/hot-spicy-skin-care/
https://www.drweil.com/videos-features/videos/spices-with-dr-weil-cayenne/
https://www.shutterstock.com/search/cayenne+pepper
https://phlabs.com/cayenne-pepper-a-healthy-food-thats-worth-the-kick
https://www.ecodiva.co.za/healthy-benefits-of-cayenne-pepper/
Fully Matched Images
https://articles.mercola.com/sites/articles/archive/2017/07/22/why-spicy-foods-trigger-hiccups.aspx
https://www.mypettingzoo.com/can-dogs-eat-red-peppers/
https://www.healthline.com/nutrition/jalapeno-health-benefits
https://www.stylecraze.com/articles/effective-home-remedies-to-treat-dry-mouth/https://eat-brands.com/blogs/blog/benefits-of-cayenne
https://www.dotemirates.com/en/details/3398381?from=dot
https://www.istockphoto.com/photos/cayenne-powder
https://www.stylecraze.com/articles/home-remedies-for-colon-cleansing/
Partially Matched Images
https://www.istockphoto.com/photos/cayenne-pepper
https://www.thriftyfun.com/Using-Cayenne-Pepper-for-Termites.html
https://www.shutterstock.com/search/cayenne+pepper+powder
Safe Search:
Now sit with what this means. Google did not just guess “pepper.” It resolved the exact species, mapped the image to health and recipe entities, and surfaced the pages already ranking for that photo across the web. If any part of the ranking system weighs image understanding the way this API does, then a page about one topic illustrated with stock photos about another is sending mixed signals to the crawler, no matter how clean the visible text is.
The practical takeaway: treat images as content, not decoration. Use photos that match the subject of the page, prefer original imagery over stock where you can, and assume Google understands the picture as well as it understands the paragraph next to it.
Test your own images: https://cloud.google.com/vision/
Google Cloud Vision API enables developers to understand the content of an image by encapsulating powerful machine learning models in an easy to use REST API. It quickly classifies images into thousands of categories (e.g., “sailboat”, “lion”, “Eiffel Tower”), detects individual objects and faces within images, and finds and reads printed words contained within images. You can build metadata on your image catalog, moderate offensive content, or enable new marketing scenarios through image sentiment analysis. Analyze images uploaded in the request or integrate with your image storage on Google Cloud Storage.