How can I make a list of all URLs indexed by Google?
-
I have a large site with over 6000 pages indexed but only 600 actual pages and need to clean up with 301 redirects. Haven't had this need since Google stopped displaying the url's in the results.
-
very depressed of being not indexing URL - https://www.wrightleatherworks.com/collections/gun-holsters-for-women please someone help
-
Open the sitemap in a text editor or XML viewer. Look at the <loc> tags in the sitemap file. These tags contain the URL of each page on the website. You can copy the URLs from the <loc> tags into a spreadsheet or text document.
-
Open the sitemap in a text editor or XML viewer. Look at the <loc> tags in the sitemap file. These tags contain the URL of each page on the website. You can copy the URLs from the <loc> tags into a spreadsheet or text document.
-
@aplusnetsolutions create a seperate sitemap for not indexed pages and upload to google web master tools. i had a website had same issues. Also remember it will take some time also
-
@aplusnetsolutions
Make sure all these things are selected in screaming frog and
-
Going to Google search console it will tell you how many inches are inside Google search index there are a few other ways to do this if you want
#1
By far the easiest is to going to search console and select coverage and all known pages
number 2
For the quick and dirty method, simply perform a simple site search in your Google search bar with “site:yourdomain.com”
#3
You can crawl your site using screaming frog (it will 500 pages for free but every SEO should have it) use it to index your site and connect it to Google search console API as well as Google analytics and you will have a easy-to-read CSV export of your index and non-index pages
more on how this is done
image url)video https://youtu.be/iYeXSdUt_hg
#4
or indexer site with any thing that can export a CSV download the CSV uploaded here and it will tell you which pages are in Google's index
https://www.rankwatch.com/free-tools/google-index-checkerhttps://searchengineland.com/check-urls-indexed-without-upsetting-google-follow-267472
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What Tools Should I Use To Investigate Damage to my website
I would like to know what tools I should use and how to investigate damage to my website in2town.co.uk I hired a person to do some work to my website but they damaged it. That person was on a freelance platform and was removed because of all the complaints made about them. They also put in backdoors on websites including mine and added content. I also had a second problem where my content was being stolen. My site always did well and had lots of keywords in the top five and ten, but now they are not even in the top 200. This happened in January and feb. When I write unique articles, they are not showing in Google and need to find what the problem is and how to fix it. Can anyone please help
Technical SEO | | blogwoman10 -
Is 301 redirects a deal breaker for Migrating content or moving to new software?
I have this forum with about 2 million posts for 16 years on root of the domain. I am looking to switch softwares but the top ones won’t help setup 301 redirects. But I can still migrate all my members and all my content (threads/posts), would Google still reindex all our content or if we don’t setup redirects would it really kill our entire traffic for a long time or maybe just a month or so? I really want to migrate to software that isn’t forum based but rather something that offers courses, chat, live video streaming, subscription based etc. and this is the only way to do so OR to set it up on an entirely new domain OR subdomain but to me that is like starting all over from scratch? I could archive the forum to read only and set it up on subdomain or another root domain - then on the archived forum setup banners and a pop up linking to the new site or new subdomain? . This is such a hard decision for us as the current forum we have had for so many years has lost members posting from 1k a day to just a handful a day, our fb group though gets 1k a day so I’m trying to revive a site into something more modern and has all the training features we can offer as well.
Technical SEO | | vbsk1 -
Unsolved Finding broken urls
Hello, This feels like it should be straightforward but I'm not having much luck. I have a pdf url which is no longer in use and takes the user to a 404 page. I want to find all instances of when this pdf url is used on my site so I can update the copy and remove the link. Please note - a redirect is not the right solution for this issue. Hope you can help. Thanks, Mary
Moz Bar | | rj_dale0 -
301 Redirects from example.com to store.example.com and then removing store.example.com subdomain
Hi I'm trying to wrap my head around the best approach for migrating our website. We're migrating from our example.com (joomla) site to our existing store.example.com (shopify) site... with the plan to finish the redirects/migration then remove the subdomain from shopify and use example.com moving forward. I've never done this and asking here to see if any harm will come from re-directing example.com URLs to store.example.com URL's then changing the store.example.com URL's to example.com. Right now my plan would run like this: redirect example.com URL's to store.example.com remove subdomain on store.example.com use example.com moving forward. wonder what happens next? Is there going to be any issues here, possible harm to the URL's?
Technical SEO | | Minarets0 -
Should we remove a keyword from the base URL?
We have a client that we are rebuilding their website and they are thinking about dropping one word from the current base URL. That particular word is one of the keywords we are trying to rank for. Do you think that because one of our keywords is in the current domain name that it would be a mistake to change it to just the brand name without the extra keyword added on?
SEO Tactics | | grayloon2 -
When to re-write and redirect a blog url?
What are best practices for rewriting (and then redirecting) blog URLs? I refresh old blog posts on our blog every month and many of them have URLs that are too long or could be improved. However, many of them also already get decent organic traffic and I don't want to lose traffic due to a URL redirect. Are there any best practices or "rules" I can follow when deciding whether to re-write and redirect blog URLs?
Content Development | | Emily.R.Monrovia
Thanks!0 -
Plagiarized Site Effecting Google Rankings
Can someone provides insights on a de-indexing example? I have gone through the depths of Google lack of support and requesting duplicate content flags, so no avail. Here's the scenario: Client had a competing SEO provider try to earn his business. In doing so, he copied word for word our blog that we have been producing content on over the last 5 years. He also integrated Google reviews in the structured data on this new URL. Well, fast forward 1-2 months later, our rankings started to drop. We found this 100% plagiarized site is taking away from our keyword rankings on GMB, and is no and Google search, and our site GMB is now only displaying on a branded name search as well as our search traffic has dropped. I have identified the plagiarized, duplicated content, being tied to our GMB as well, as the source of the problem. Well, I finally obtain ed control of the plagarized domain and shut down the hosted, and forwarded the URL to our URL. Well, Google still has the HTTS version of the site indexed. And it is in my professional opinion, that since the site is still indexed and is associated with the physician GMB that was ranking for our target keyword and no longer does, that this is the barrier to ranking again. Since its the HTTPS version, it is not forwarded to our domain. Its a 504 error but is still ranking in the google index. The hosting and SSL was canceled circa December 10th. I have been waiting for Google to de-index this site, therefore allowing our primary site to climb the rankings and GMB rankings once again. But it has been 6 weeks and Google is still indexing this spam site. I am incredibly frustrated with google support (as a google partner) and disappointed that this spam site is still indexed. Again, my conclusion that when this SPAM site is de-indexed, we will return back to #1. But when? and at this point, ever? Highlighted below is the spam site. Any suggestions? Capture.PNG
SEO Tactics | | WebMarkets0 -
301 Redirects from Subdomain to Location Pages
I have a client site that is getting redesigned. Its a multi location service provider. Currently (for whatever reason) the location pages are sub domains. https://<location-name>.site.com/ In the new design the locations will be on the main domain. https://site.com/locations/<location-name> We are considering using 301 redirects from the current sub domains to the new location pages on the main domain. The current sub domains are setup on a multi-site with A records for each one in our GoDaddy account. Would like to get feedback on any unforeseen SEO issues that anyone might have input on.
Technical SEO | | ColeBField12210