Google Images is one of the more difficult sites to scrape on the web. They don’t explicitly block scrapers, but they really make you work for the data… You’ve got to want it!
From dynamic CSS selectors to Base64 encoding, scraping Google Images is a lot more like solving a puzzle than scraping regular HTML.
Prerequisites
To scrape Google Images with us, you should have a basic understanding of Python and Selenium. You’ll need to make sure you’ve got Selenium installed. We suggest you learn more about web scraping with Python and Selenium if needed.
First, make sure you’ve got ChromeDriver and Chrome installed. You can download the most recent one here.
When downloading ChromeDriver, make sure you’re getting a version that matches your version of Chrome.
You can check your Chrome version with the following command.
google-chrome --version
The output should be similar to what you see below.
Google Chrome 131.0.6778.139
Once you’ve got these, you can install Selenium with pip.
pip install selenium
What To Scrape
We can’t just plunge head first into code. We need to get a better idea of what we’re scraping and how we’ll extract it. Like we said earlier, scraping Google Images is like solving a puzzle.
Let’s examine one of the images from Google. This image is actually embedded in a custom HTML tag called, g-img. We’ll need to find all of these g-img elements.
Once we’ve found all the g-img tags, we need to extract their img elements. You can see one of those below.
If you looked at the img closely, you should’ve noticed something extremely strange. The src is a bizarre string of seemingly random characters.
data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkGBxAREBUQExAVFhUVFxISEBYXEhISFRAXFRUWFhgWGBUYHSggGBolGxUVITIhJikrLi4uGCAzODMtQygtLysBCgoKDg0OGhAQGzcmHyU1Li8tLzc1LS0tMzAvLSsrLTAwMS0tLS0rLS8tLjAvLy83LS0xLy0tNy0vLS0tLS0yLf/AABEIAPQAzgMBIgACEQEDEQH/xAAcAAEAAgIDAQAAAAAAAAAAAAAABgcFCAEDBAL/xABHEAABAwICBwQFCQUHBAMAAAABAAIDBBEFIQYHEjFBUWETInGRIzJCgaEIFFJicoKSscEzQ6Ky0RUkZJPC4fBjc7PxJTSj/8QAGwEBAAMBAQEBAAAAAAAAAAAAAAEEBQYCAwf/xAAvEQEAAgEDAwICCgMBAAAAAAAAAQIDBAUREiExE0Gh0SIjMjNCUWGBkbFSceEk/9oADAMBAAIRAxEAPwC8UREBERAREQEREBEVI69NYDml2FUzyDYfPXtNjZwuIQeoILrcw36QQZfTvXRT0rnU9ExtRKMnSE+gjPIFpvIR0sM95zCp7GtYmLVRJkrpWjOzIndgwA8LR2uPG5UWRSO2epkedp73OPNzi4+ZXsoserIf2VXPHy2J5GfkVjkQWVo5rpxSnIE5ZVR8Q8COQDkJGD4uDld2hWntDijfQv2ZQLvgfZsrQMiQAbObuzF94vbctbsEwRs0ZPFY2aKejmbLG9zHsO0x7SWuaRxBCgbmIoBqo1hNxSExS2bVxAGQDITNyHatHDMgEcCRzCn6AiIgIiICIiAiIgIiICIiAiIgIiIMXpRjDaKjnq3W9FG54BNtp1rMb73Fo9603qql8sj5ZHFz3uc+Rx3uc4lzifEkrY/5QdcY8IEY/fTxRu+y0Pl/ONq1rQERFIIiIJTguKtgi35rG4ji3bXuFiwxxF7Ej8l8IMjgGMTUVVHVQmz4nBw5OG5zT9UgkHoVuBo/jEVbSxVcR7krA8C4JadzmG2W01wLT1BWlyv75OONF9NUUTj+xe2WL7Mtw5o6Bzb/AH1AuNERAREQEREBERAREQEREBERAREQVB8pJ39yphzncfKN39VSej+j09Y4iMANbbae69h0Ft56K6vlJj+6Uv8A3n/+NeXQinpMMwNmJ1QLw7vRxtteRz3ENB5nLwAbfPcgrTGNEamiaZCGyxW9IACC0c7cLcwclFngXy3cFs9oZpXQYreGShbDKWGRsb2xyNlZexLJABci4uCAc+OdqF1j6PigxOemYD2e0HwD6kgD2tHO1y37qCNsYXENAJJIAAzJJyACn2jugDHgOqHuufYYQA3oXWNz4W9679R2j8dViLzMy7YYi8NNx33Oa0EjlYu+CtPWJp9S4MY6eGlZJM8bewNmNkTLkBziASSSDYDkSSMrhAsR1WFsZmoZXdo0bQikLS2W3sh1hYnrcdQqtxGxeTsbDrkSMsRsOGRyO7Pgtj9C9OosQkFLUUwp5pGudAWPDo6gNF3bLrAhwGeyb5DfwVT67tHjSYiJMtmoZ2gIyu5p2XXHA22fO6CvVaHyd6gtxV7L5PppRbqHxuH5HzVXqxNQjrY0zrFOP4b/AKKRs6iIoBERAREQEREBERAREQEREBERBUPykW/3GmP+II84n/0UVx5r59DqN7RcQVHpvqtD6iJpPvewfeUx+Ue3/wCMgPKqYPOGf+iiOpTSmJsU+F1LC+KQPkaC0vaQ4BskbhwByIvlcnO5F4taKxzPhMRMzxCLarsSkbidEwH1Z/wsexzX+7Zc5THWHo5XYriT6mCNkUbGtgjfJJsumDC4mTZaCWglxsDY2AWewPRqipnSGCEt2y7Nztt7Wn2A/eB/wk71JYVzGr360W6cEdvzlepo+I5uhervBMSwytY+ZscsTwYZnRybTo2uIIcWuAJAcBuByJUL16QyNxuZzvVeyB8PIs7Jrcum216vCOlZtdoGN2yLF2yNojlfesVpPoxS4gwMnYSW3Eb2u2Xx337J3W6EEdFOHf5rMetXt+cfL3+DxbSx+FRur+eokxOgiY5x2KmJzQPZbtNMv3dhrrjldTz5Sc7TUUkYPebHM4jo5zQP5HeSmmhGiOH4W50sbJHzOBaJZHNeWNO8MDQ0NvbfYnrZUnrRr6qfE5ZKiJ0Rs1sLHFp2YhfZs5tw652iSCRckcFvafWYNR93bn+/4Vr47V8wiasDUS62Nw9WTj/83H9FX6m2peXZxykN8iZmnrtQSAfGytPDatERQCIiAiIgIiICIiAiIgIiICIiCt9f8G1gznW9SaF/hcln+tV7q6oWx0zX2G1L33niRchovyA/Mq0ddse1gVV0+buHuqIr/C6o3Vk6okmfCyQgMidM1pzFw9jbeB29yo7hgvnwzWk/9ffT5K0vzZcNKshTi5UZw/HGjuyMc1wyNml4v7s/gstDjMI3F58Ipf1bZcPkw5K24mGra0THZINwXRPK1jS5zgAMySbALES4+45Mhdfm8ho8bC5PwWNmL5HB0rtojNrQLMZ4N59Tcr1fH1ee0PFaS9VRikr3gsOxGObRtSdTf1R03/kqs1v1QfUQj22xnaP1S87Iv4h3mrMZG53qj38Aqw1sYNJDURzlxc2VmyPqOYc2+BDgfe7kt/aNFkrljLavERHb9VfVZadHRWeZQRZXRTEhS11NUk2EU0T3/YDhtj8N1ltCo4JduCRoJdm08V59KtG/mvfabsJt4Lpmc29BXKheqHSIV2FQuJvJCPm017k7UYGy4k79phYb8yeSmigEREBERAREQEREBERAREQEReDHcYgoqd9TO/ZjjF3HiTwa0cXE5AIILr9xZkOEugJ79TJHGwXF7RvbK53gNho++FWmoqgc+pqphuZA2M+MkjXD4ROUY010oqcYre1LXWJEVJC3vdm0mzWgD1nuNrniTyAA2J1b6GjDcPbA6xmkPa1Lhu2yLBgPJosOpueKCM43hDnHtYsnDeN21/usQ2vmZk5r/eCrDxGhLTcDJY8s6fBZ+p27Fnt1T2lZxam1I48ovBPM/wBWM+JFh8Vl6LDHHN5v0G5ZyloXP4LOUWFtbmVGn2zBhnq45n9TJqr3jjxDH4fhFxmLBYbWLoV89w+WOMXlZ6anHEvYD3B9ppc3xcDwU7a2y5WgrNJqOqdE8SNNiMwsvjukslVG2MtAAzPUqR67NF/mOJOlY20NVtTx8mvv6Vn4iHcgJAOCr9ehP9TGlow/EBHI60FTsxSkmwY6/o5D4Elp4WeTwW0S0eWyepPToV1MKOd/95gaACTc1EQsA+/FzcgfceJtAs5ERAREQEREBERAREQEREHTWVUcMbpZHhjGNL3ucbBrQLkkrVrWjp7JitRZpLaWIn5vHu2juMrxxceA9kZcSTI9eOnpqpjhsDj2ELvTuBynlb7OW9jD5uF+DSqmAQXT8n/Q4Pc7FZm3DCY6QHcXbny+71R12uQV7LE6KYcylooKZm6KNjD1cB3nHqXXPvWWUD4liDhYheP+y2XvZe4my+GyZoEUIbuC7EXBcOaDlF8GVvMLrfVMHFBBNeWCipwp8gF5KYiobz2R3ZB4bLi77gWsS3Jqo21DXwvzZKx8Thza9pafgVp1UQOje6Nws5jnMcORabH4hTA617MIxOalnjqYXlskTg9juvEHmCLgjiCQvGikbc6v9NIMVphKyzZWWbUxXziceI5sNjY+I3ghShaaaLaQz4fVMqoHWc3JzbnZlYfWjeOLTb3EAjMBba6LY/DiFJHVwnuvGbT60bhk5juoPnkRkQoGWREQEREBERAREQFDtbGkxw7DJJWOtLIRBTni17wbu6bLQ5w6gc1MVQPyj8W26qmowTaON0z88i6V2y0EcwIz+NBT8UZcbAXK92j9BJPVwwxt2nukaAOGRuSTyABJ6Bc1TOxjDPbeNp5+i3g33/l4q7NQmhojpziUze/NdtOCPViBzdY/ScPJoPFBZdDBJvuQFlGMtxuvpFA4cF008fE+5d6IC+HRAr7RB5JcPaeJXgnwt4zabrNIgjrHuYcxZVRrp0EbsnFqVlrm9bG0ZAn9+B1PrW4na+kVek9O14sQseaMWdE9odG8OY9pF2va4WLSOIIJCkaewBju647J4O4Hx5eK4qaV8Zs4eB4HqCs1pxo47D8QmpMy1rtqEn2o395hvxNjY9WleWmuxuxILxH3mO/tN/UKRiFZ2onS40lb8zkd6GqIa25yjn3MPTa9Q8zsclXeI0ZieWndvaeBB3ELzMeWkOBIIIIINiCNxB4FBvAij+gOP/2hh0FUbbbmbM1srSM7r8uALgSOhCkCgEREBERAREQcOcALk2AzJ5LUvSPFxiGK1FaT6Lbc5u/9lGNlmR3EtaMuZV6a7dJ/mWGuia601VeCPmGW9K/3NOzfgXtWt0g7OnA4ym5+yy2XmR5IPdo3hUmJ4jFT5gzSd8j93GM3kfZY028AtvaWnZFG2JjQ1jGtZG0ZBrWgBoA5AAKkPk4YJd9TXuHqhtNEerrPk99hH+Iq9EkERFA4Llyul57wXcgIiICIiAuCLrlEFM/KHwhtqWuAFw51NIbZkEGSPPoWy/iVS1lcx0eytiddFF2uCVOWcfZSt6bEjdo/hLlqupgZk+mpL+1CdnrsHNv6j3LDLL4FmydvNgPkf91iCpF3fJuxuzqmgcd9qqIdRaOTP/K8irzWpeqnFfmuMUr72a+TsH8iJh2Yv0DnNPuW2igEREBERAXBK5UA126Quo8Ke1htJUOFM0je1rgTI78LS3xcEFG61NKv7SxF8jXXhi9DTcixpN3/AHnXN+WyOCj2LO/ZjlG34ucV5IIi9waOKymk1EYnRA8Ym+Yc4H9PNSNktTeGfN8GphazpQ6of17RxLT+DYHuU1XhwOkENLBCBYRxRRjpsMa39F7l5BERB0O9cLvXT7a7kBERAREQEREEf1gx7WE1w/w1QfwxuP6LUBbg6futhVcf8LUjzicP1WnymBmcDyinf9VjR7yT+iwyy7z2dG1vGRxefAd0flf3rEKR9wTOY5r2mzmkOaeRabg+YW7FFUCWNko3Pa148HAEfmtJFuJoG8nCqEnf81pb/wCUxQM6iIgIiICp/wCUlA40dLJ7LZnMPi+MkfyOVwKJ6cVEbwylLWvO0yV1wDsbJuwjk6+fu6r4anUVwY5yW9nvHjm9orDWCPRyv2O1bSzbO8ERuv47O+3Wy7p3uqaexzkhubcS0+sLcxYHzWwjGCyg+n+inaA1lMNmdgu8NH7do4EfTHA8d3K2LpN+jJk6MteInxPz+a5l0XTXms8r3CKktVutwEtosQeBubBUE5dGSnh0f581dq6BQEREHQD313rot313oCIiAiIgIiwOPY/2EjYWgFxbtuJuQ0EkDLiTY+XVfLPmphpN7+IeqUm88Q8OtepEeC1jucXZ/wCY9sf+paq0tJtDac8MbzOZPQDitl9ML4hQyUsr9hjtlz3Mb3gI3B+4m29oWsL3X8BkOgXy0muxarn0/bz24esmK2P7TL1vZTEWl2Q0BrRa4sBYZ3XWzAZnC8Za/wAHWPkVilksCqZGSAtdYceSuPm8E8LmOLXNLSN4IstzdHKE09HT053xQwxHxZG1p/Ja46P4f/aWJQNsOzhfHJUvO4Rh4Jb1LrW+PBbPNcCLg3BzBGYK8Res2msT3jzCeJ45coiL0h0VtZHCwySODWjj+gHErAHTOG+UMpHOzBf+JebTtxLoWcO+4jmRsgeQJ81iooQBuXO7nu2TBlnHj9mhp9LS9OqzL1emeVo4DtcC8gAe5u/zCjtOHve6WQkucbuJ4/8AOS93ZDkvprQFharcc2oji8rmPDTH9mHIXzI24X2ioPspLWZo183m+cxt9HKe+AMo3/0dv8b8ws5qv1sPoQ2krC6SmFmxvzdJSjgLb3xjlvA3XsAp/jeGMqIXwvF2vBH/AK6/0WvWMYa+mmfA/e05G1g4cHDxC7XZtd6+L07far8YZGrw9FuqPEtzKOqjmjbLE9r2PAcx7SHNcDxBC7lqfoBrAqsKk7pMlO43lgc47J+sw+w/qMjxBsLbN6M6Q02IUzaqnftMdkQcnxuG9j28HC48bgi4IK2VRky3O6+kRQCIiAiIg65JQ3eoHpZIHVzSOMTPg+RSjFnFQzGf/ss+wP5nLO3WP/NP7LOk+8h94iCYXN+k1zfMWWtUsZa4tcLFpLXDiCDYhbNTsu1QHSTQqCoeZM2PPrObazurmneeossXaNZTBNov4ld1WGckRNfZUK76GmllkbFE0ue82a1u8/7deCndNq2G136gkcmxhp8y4/kp9ozo7T0gtFGAT6zjm93i48OgyWvqN3w46/Q7z8FTHo72n6XaDQnRltBS7BIMr7PneOJG5oP0W3PvJPGysDQ6pc5kkZN9hwI6B98vMOPvWEAyWQ0YnEczmHLtALfabew8ifJYm3am062L3nvbmJ/fx8eFvUY49GYiPCWIiLsGSienbbGB3WRvmGn9CsXEclltYB9FF/3f9Dv9lhqX1QuL32vGpmf9f02NHP1UO5ERYq0IiIOCFXmtXRvtYvnUbe/H69hm5m8+W/z5qxF1zRhwIIyORVjSam2nyxkr7PnlxxesxLV9WVqEx2SDFG0oJMdU17Xt4B8bHSMf491zfvdFzWap6qeolFI+HZB2hG97mPa08R3bFoOW+4y6XsDVZqpfh0/z2qkY+YNc2FkdyyLaFnPLiAS7ZJFgLC533y/QsOauWkXp4lh3rNZ4laiIi9vIiIgIiIMTi7VXc2KxVFU7snbQi9C5w9UvaS52yeIG0BfmCsBrc1oNkLqKhku3Ns9Q05O4FkR5c38eGWZxurNlqdvUvP8AER+izt1njTys6SPrFnNbcLyzUy9cO5fdlxkTw12OjpF7YorLssuUm0yC65WX/MdF2IoQzeC47tWimNnbmv3B/Q8nfms+q/nAsu/CNKexJjlJcy3cO9zTyvxFr+Fl0u3btM8Y838/P5qGfSfip/D602qhJURwA/swXP8AtPtYeIAv95eaJtgsBHXF8z5Hes5znHpc7vdu9yzMdUCsPcsts2abrmCnRSIelF8NkBX1dZz7OUXC5QFwuUQYrEZZIXNniNnxm45EcWkcQdynej+PQVkYdG4bYAMse0C+I8iN9rg2PFQvExdpVU6ZCSIiohkfHJGe69j3MeA7I2c0gjOy6TY9TNZ9OfEqOsxRMdTZ9FrPgmunFoAGyujqG/8AUZsvAtuD47eZBUrpflAN/eYaR1ZUg/wmMfmuq4Za7UVK1PygIx+zw5x+1UBtvcGG6jONa8MTmBbCyGnB3FrTLIPvP7v8KDYDHMcpaKIzVM7ImDcXHNxtezWjN7ugBK191ka2Zq8OpaYOhpjdrze0tQOIdb1WH6I38TnYV7imJz1MhlnmfK8+095ebb7C+4Z7hkvIgK49B4NmKNvJrb+Nrn4qn4WbTg3mQPM2V2aMEZLH3m31cQu6KPpTKaw7l9rriOS+7rkWk5XBK+XyALH1dcBxUxWZS9z5gF55a5o4qOVuL9V78G0crauztnsoz7cgIuObWb3fAdVcw6LJlniIfO+StPMuK7FhbepLodo24g1FSz1haONwzAJB2nDgchYeKzGBaJU1KQ+xkkH7x9iQfqt3N/Pqs+uh0W11xT1X7z+TPzarqjiqI43oOyWR0sMnZucS5zS3aYSd5GYLbnx8FG63Ryvgz7LtGjjGdv8Ahyd8FaSKzn23Bl7zHE/o+dNTkoppuJFp2XXBG8EEEeIK9UeKjmrTq6GKUWkiY8cnMa63msDW6C0Ml9lj4yeMbyP4XXaPJZeXYv8AGVquuj3hFI8SB4r1x1bTxX1XaupRnBVA8myNLf423/lUexDCsQpM5IHFo9tnpGW5nZzaPEBZ2bZ8tO/D711VLe6TB4XDngKJ0uOg8V6H4uLb1nTpbxPh94tDIYlUCxVcaVh8rXRxsc97rBrWNL3HMbmjMrOYrjAsc1OdU+jkkYdXzNLXSt2IGnItjJDi8jhtENt0H1lu7VpLReLSqarLHTwo6h1cYzMLsw+YfbDYfhIWrJxanscIzpWt8aiD/S4raRF1DKauSanMbAypmHoKiDPzcF459VmNs30Dz9mSB/8AK8ra9EGm1ZoriMN+0oKlgG8mnlDfxWssOVvCvBiWC0tSLT00Mo/6kTJLeG0MkGmdCfSx/bZ/MFa2A1lrKxMW1O4PMdpkT4HXvtQyEC4+o/aaB0ACqrGMPnw+pdTygggkxutZsrL917eYPwNxwWbuOGclYWtLfplYVPiAtvXY6v6qCUuL5b16XYt1XN20dolpxkqklViXVYhjpqmUQwtL3u3AcBxJPADmV4sNhqK2YQQN2nH1jubGPpPd7I/PcLq59FdGoaGLYb3pHWM0hFnSHkOTRwH5m5Wlott6p5t4V8+piscR5Y3RfQeGmtLNaWbfci7Iz9Rp3n6xz5WUtRF0NMdaRxWGZa02nmRERe3kREQEREBERBH8a0Moaolz4th53yRns3E8zbJx+0Cqb0ioBT1HYse4tva7iCd/QAfBEVLV46cc8d1jDaeeOVi6GaCUIayqe10r/WaJCHMYRxDAACftXtwU/RFapWKxxEPjaZme4iIvbyIiICIiAvBjOC01ZH2VRC2Ru8XGbTuu1wzaeoIREFO6faF01B3oXy55hrnNcG9B3b28SV1avdFYK93pnyADOzHNaHW4G7SfIhcoqk46+rxw+8Wno5XPhGEU9JH2UETY27zbe483OObj1JK9yIrcRw+AiIgIiIP/2Q==
The beginning of this string holds the key to everything: data:image/jpeg;base64,. jpeg tells us that this is a JPEG file. base64 tells us that it’s encoded using Base64. When we decode this string, we actually get the binary of the image. We’re not actually able to trace the true source of the image since its binary is actually inside the web page. However, we can write this binary to a file and recreate the image.
Scraping Google Images With Python
Now that we know what we want, it’s actually time to start coding our scraper. In the next few sections, we’ll put the scraper together and go through exactly what the code does.
Getting Started
Go ahead and create a new Python file. We’ll start with just our basic imports and structure.
from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep
import base64
from pathlib import Path
options = webdriver.ChromeOptions()
"""
Our actual scraping logic will go here
"""
if __name__ == "__main__":
scrape_images("linux penguin", 100)
- We import
webdriverandByfrom Selenium.webdriveris used to control our browser.Byis used for locating items on the page. - We’ll use
sleepto pause our scraper for a period of time. For example, if we want the scraper to wait for one second, we’d usesleep(1). - As you might have guessed,
base64is going to decode our image binaries. Pathwill be used to write our images to a folder containing our results.options = webdriver.ChromeOptions()allows us to use custom settings with Selenium. Primarily, this is to run Selenium in headless mode. Headless mode allows us to run the scraper without rendering the actual browser on the machine. This saves valuable resources.
Scraping Google Images
Next, we’ll write our scraping function. The code below contains our entire scraper. Pay close attention to scrape_images().
from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep
import base64
from pathlib import Path
options = webdriver.ChromeOptions()
def scrape_images(keyword, batch_size, headless=True):
if headless:
options.add_argument("--headless")
formatted_keyword = keyword.replace(" ", "+")
folder_name = keyword.replace(" ", "-")
output_folder = Path(f"results-{folder_name}")
output_folder.mkdir(parents=True, exist_ok=True)
result_count = 0
driver = webdriver.Chrome(options=options)
driver.get(f"https://www.google.com/search?q={formatted_keyword}")
sleep(1)
list_items = driver.find_elements(By.CSS_SELECTOR, "div[role='listitem']")
list_items[1].click()
while result_count < batch_size:
driver.execute_script("window.scrollBy(0, 300);")
sleep(1)
img_tags = driver.find_elements(By.CSS_SELECTOR, "g-img > img")
for img_tag in img_tags:
src = img_tag.get_attribute("src")
if not src or not src.startswith("data:image/"):
continue
base64_binary = src.split("base64,")[-1]
mime_type = src.split(";")[0].split(":")[1]
file_extension = mime_type.split("/")[-1]
if file_extension == "gif":
continue
alt_text = img_tag.get_attribute("alt") or "image"
filename = f"{alt_text}-{result_count}.{file_extension}"
image_binary = base64.b64decode(base64_binary)
output_path = output_folder.joinpath(filename)
with open(output_path, "wb") as file:
file.write(image_binary)
result_count+=1
print(f"Saved: {filename}")
driver.quit()
if __name__ == "__main__":
scrape_images("linux penguin", 100)
- We set
headlesstoTrueby default. If the user sets it toFalse, this will launch an actual browser that you can see on screen. This is useful for debugging purposes. - We create a
formatted_keywordandfolder_nameby removing spaces from our actualkeyword. This allows us to store the files without any issues. - We launch our browser with
webdriver.Chrome(options=options). driver.get(f"https://www.google.com/search?q={formatted_keyword}")takes us to the Google search results for ourkeyword.- Now we need to click on the images tab. We do this by finding all
divelements with the rolelistitem.list_items[1].click()clicks on the second item, the images tab. - We use a
whileloop to run our scraping code over and over until we’ve found all the images we want. driver.execute_script("window.scrollBy(0, 300);")runs JavaScript to scroll the page down by 300 pixels. After scrolling, wesleep()for one second while the content loads.driver.find_elements(By.CSS_SELECTOR, "g-img > img")is used to find allimgtags that are nested inside ag-img.- Next, we iterate through the
imgitems we found. - If the
imgdoesn’t start withdata:image/, we usecontinueto skip it. Otherwise, we pull itssrcattribute. - We use some basic string splitting to extract the encoded binary and the file extension (JPEG, PNG, etc.). If the extension is a GIF, we skip it. For some reason, GIFs don’t display when we write them to a file.
base64.b64decode(base64_binary)decodes our image into actual machine readable binary.
If you run the code, you’ll see a new folder pop up inside your project folder. It should be full of images.
Consider Using Bright Data
Our SERP API parses the Google Images so you don’t have to. It even finds the image metadata, so our images will have actual names. Of course, the API is fully scalable and can deal with an enormous number of requests.
First, sign up for our SERP API.
When you’re ready, finish creating the zone.
Under Access Details, you’ll see your credentials.
Copy and paste the code below into a Python file. Replace the credentials in proxy_auth with your own and you’re good to go.
import requests
import base64
from pathlib import Path
import json
proxy = "brd.superproxy.io:33335"
proxy_auth = "brd-customer-<your-customer-id>-zone-<your-zone-name>:<your-zone-password>"
proxy_url = f"http://{proxy_auth}@{proxy}"
def scrape_images(keyword):
formatted_keyword = keyword.replace(" ", "+")
folder_name = keyword.replace(" ", "-")
output_folder = Path(f"serp-results-{folder_name}")
output_folder.mkdir(parents=True, exist_ok=True)
url = f"https://www.google.com/search?q={formatted_keyword}&tbm=isch&brd_json=1"
response = requests.get(
url,
proxies={"http": proxy_url, "https": proxy_url},
verify=False
)
images = response.json()["images"]
result_count = 0
for image in images:
image_binary = base64.b64decode(image["source_logo"].split("base64,")[-1])
title = image["title"].replace(" ", "-").replace("/", "").strip(".")
file_extension = image["source_logo"].split(";")[0].split(":")[1].split("/")[-1]
if file_extension == "gif":
continue
filename = f"{title}.{file_extension}"
with open(output_folder.joinpath(filename), "wb") as file:
file.write(image_binary)
print(f"Saved: {filename}")
if __name__ == "__main__":
scrape_images("linux penguin")
if you run the code, you’ll get a bunch of images again, but this time, they all have names.
Conclusion
In conclusion, scraping images from Google is a bit like trying to solve a puzzle without all the pieces. Our Google Images API finds the metadata and cuts out the need for Selenium!
If you need to scrape images from other sources, we also have an Instagram Image API, Shutterstock Scraper, and different structured datasets. Sign up now and find the perfect product for your needs, including a free trial!
No credit card required
