Scrape Google Images with Node JS

Written by darshan12 | Published 2022/07/25
Tech Story Tags: nodejs | webscraping | web-development | software-development | google-image-search | search-engine | search | puppeteer

TLDRThis tutorial will teach us to scrape Google Images with Node JS. The code explanation is simple. We just have made a GET request to our target URL by passing a header to our request. Then we have to load our HTML content in our Cheerio JS variable, which is `$` here. We used this `$' to parse out HTML data. Using this API, you don’t have to worry about creating and maintaining the scraper, and also you can scale the number of requests easily without getting blocked.via the TL;DR App

This tutorial will teach us to scrape Google Images with Node JS.

Target

Install Libraries

Before we begin, we need to install some libraries we may need in this tutorial.

Code

You can copy this code from the following link: https://github.com/Darshan972/GoogleScrapingBlogs/blob/main/GoogleImagesScraping.js

Explanation

The code explanation is simple. We just have made a GET request to our target URL by passing a header to our request - User-Agent

Make sure you use it as a header because Google can block your request.

Then we have to load our HTML content in our Cheerio JS variable, which is $ here. Then we used this $ to parse out HTML data.

You can see all the tags we have used in our code, as shown in the image.

Using Google Image API

Using this API, you don’t have to worry about creating and maintaining the scraper, and also you can scale the number of requests easily without getting blocked.

Example

   const axios = require('axios');

   axios.get('https://api.serpdog.io/images?api_key=APIKEY&q=football&gl=us')
  .then(response => {
    console.log(response.data);
  })
  .catch(error => {
    console.log(error);
  });

You can get your API Key by registering on this link.

Results

      "image_results": [
      {
        "image": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS_Tu78LWxIu_M_sN_kMfj2guqIbu2VcSLyI84CQGbuFRIyTCVR&s",
        "title": "Football - Wikipedia",
        "source": "en.wikipedia.org",
        "link": "https://en.wikipedia.org/wiki/Football",
        "original": "https://upload.wikimedia.org/wikipedia/commons/b/b9/Football_iu_1996.jpg",
        "rank": 1
      },
      {
        "image": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTxvsz_pjLnFyCnYyCxxY5rSHQCHjNJyYGFZqhQUtTm0XOzOWw&s",
        "title": "Soft toy, American football/brown - IKEA",
        "source": "www.ikea.com · In stock",
        "link": "https://www.ikea.com/us/en/p/oenskad-soft-toy-american-football-brown-90506769/",
        "original": "https://www.ikea.com/us/en/images/products/oenskad-soft-toy-american-football-brown__0982285_pe815602_s5.jpg",
        "rank": 2
      },
      {
        "image": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTNJYuYBLrUxLrXkbnP18Y6DEgKf_H4HYGCzecsGRAoFtkiGEM&s",
        "title": "NFL postpones three games due to Covid ...",
        "source": "www.cnbc.com",
        "link": "https://www.cnbc.com/2021/12/17/nfl-will-postpone-some-games-over-covid-surge-source-says.html",
        "original": "https://image.cnbcfm.com/api/v1/image/106991253-1639786378304-GettyImages-1185558312r.jpg?v=1639786403",
        "rank": 3
      },
      {
        "image": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTg4WJ88A83JqXwGXsWtiB5qSoHrU_RukbrfXdkWggEKMsJ5Ro1&s",
        "title": "USFL schedule Week 2: What football ...",
        "source": "www.sportingnews.com",
        "link": "https://www.sportingnews.com/us/nfl/news/usfl-schedule-week-2-football-tv-channels-times-scores/oadvrtsc5vn9l4knu8hvnpo0",
        "original": "https://library.sportingnews.com/2022-04/usfl-football-042122-getty-ftr.jpg",
        "rank": 4
      },
      {
        "image": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTsOPQ2-nzsTrdRK1HHXIqB-x96yuiPg7pwfJsm8mToVlw5-UaM&s",
        "title": "Why is football called 'football'?",
        "source": "www.newsnationnow.com",
        "link": "https://www.newsnationnow.com/us-news/hold-why-is-football-called-football/",
        "original": "https://www.newsnationnow.com/wp-content/uploads/sites/108/2022/02/FootballGettyImages-78457130.jpg?w=1280",
        "rank": 5
      },
      {
        "image": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS0WimlY0Nykjy8k5An6k4wViDvyVZzo_0K1MSZNZcwDFGsugNW&s",
        "title": "The Duke NFL Football | Wilson Sporting ...",
        "source": "www.wilson.com · In stock",
        "link": "https://www.wilson.com/en-gb/product/the-duke-nfl-football-wf10011",
        "original": "https://www.wilson.com/en-gb/media/catalog/product/b/c/bc340309-c2a3-441d-ac36-a26187fd94f0_yceho2py9sgzklxk.png",
        "rank": 6
      },
      {
        "image": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQ6ZuMtA_I4WZZKXPhiqMpIHsdKJX-SVkcN2yo1KAQ9SQuxdCI&s",
        "title": "National Football League",
        "source": "www.nfl.com",
        "link": "https://www.nfl.com/",
        "original": "https://static.www.nfl.com/image/private/t_editorial_landscape_mobile/f_auto/league/iaesayubxpbxmbxfwm3b.jpg",
        "rank": 7
      },

Conclusion

In this tutorial, we learned to scrape Google Search Images Results. If you have any questions, feel free to ask me in the comments. Follow me on Twitter. Thanks for reading!

Additional Resources


Written by darshan12 | Founder at serpdog.io
Published by HackerNoon on 2022/07/25