How Can I Scrape Thousands of WordPress Domains via Google Dorking Without Hitting Rate Limits?

[DD-Worm]

Hey everyone,
I'm kinda new to this field and currently testing out some things. I’m trying to build a list of WordPress sites by using a Google dork like this:

inurl:xmlrpc.php/?rsd

The idea is to scrape domains from the Google search results and use them in my script for testing. The problem is — Google keeps blocking me after a few requests and throws rate limit/captcha issues.

• I’ve tried:
  Using random user-agents
  Adding random time.sleep() intervals
  Using proxies (even rotating ones)
  
  But nothing really worked. I still get blocked quickly and can’t gather enough domains.
  
  
  

• Is there a better or alternative way to get a large list (thousands) of WordPress domains that have xmlrpc.php enabled, or that match that dork or Any other Dork ?

  
  Any help, advice, or tools you’d recommend would be super appreciated
  
  
  Thanks in advance!

 

[DD-Worm]

Partially Solved:
I’m still limited by the daily quota.
I need a list of thousands of URLs from Google Dorking.


Code used:

[/COLOR]
[COLOR=rgb(65, 168, 95)]#!/usr/bin/python3
import requests
from colorama import Fore as F
API_KEY = 'REPLACE THIS WITH GOOGLE API KEY'
CSE_ID = 'REPLACE THIS WITH CUSTOM SEARCH ENGINE ID'
with open('xmlurls.txt', 'r') as file:
    ex_urls = [line.strip() for line in file.readlines()]
def is_url_allowed(url, ex_urls):
    return not any(ext in url for ext in ex_urls)
def fetch_search_results(query, num_results=100):
    search_results = []
    start = 1  # API supports paging, 10 results per page
    while len(search_results) < num_results:
        url = (
            f"https://www.googleapis.com/customsearch/v1?"
            f"key={API_KEY}&cx={CSE_ID}&q={query}&start={start}"
        )
        res = requests.get(url)
        if res.status_code != 200:
            print(f"{F.RED}[-] Error fetching results: {res.text}{F.RESET}")
            break
        data = res.json()
        items = data.get("items", [])
        if not items:
            break
        for item in items:
            link = item.get("link")
            if link and is_url_allowed(link, ex_urls):
                search_results.append(link)
                if len(search_results) >= num_results:
                    break
        start += 10
    return search_results

if __name__ == "__main__":
    query = input(f'{F.YELLOW}[+]{F.RESET} Enter Your Search Query: ')
    results = fetch_search_results(query, num_results=100)
    print(f'{F.YELLOW}[+] Top Search Results For "{query}" Are:')
    for idx, result in enumerate(results, start=1):
        print(f'{F.GREEN}{idx}.{F.RESET} {result}')[/COLOR]
[COLOR=rgb(184, 49, 47)]