ACD-594: Disallow indexing #2823
Open
+5
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
exonian
reviewed
Mar 10, 2025
exonian
previously approved these changes
Mar 10, 2025
patrick-laa
force-pushed
the
ACD-594-disallow-indexing
branch
from
2fa3ffd to
f7ecebd
Compare
patrick-laa
force-pushed
the
ACD-594-disallow-indexing
branch
from
f7ecebd to
0c2088c
Compare
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Instruct Google and other crawlers not to index the site in any environment.
Ticket
Hide VCD from search engines
Why
Because these pages are not for public consumption and their presence on Google may cause confusion.
"VCD’s intended users have other ways of getting the link (from emails, intranet, bookmarks etc) so the downsides of it being on search engines outweigh any benefits."
How
Do this by:
- Adding a global directive in robots.txtmetatag on each individual page(robots.txt tells crawlers whether they can crawl pages. SInce Google already has crawled our pages and believes the content is indexable, we want to allow the crawlers to keep crawling so they at some point crawl back and notice that the pages are now
noindex.)Note that these directives ask Google not to index us. We cannot force Google not to unless we make it impossible for Googlebot to load any pages on the site.
Finally, note that for me to run bundler, given my processor on my laptop, certain adjustments to the Gemfile are necessary. These aren't directly relevant to the ticket, but I hope they're small enough it's OK to include here rather than a separate PR.