diff --git a/public/robots.txt b/public/robots.txt index 3c64852..eb81e15 100644 --- a/public/robots.txt +++ b/public/robots.txt @@ -55,4 +55,6 @@ Disallow: / User-agent: Bingbot Crawl-delay: 2 +# this is intentionally only accessable for search engine crawlers. +# do you want to crawl too? get in touch https://preservetube.com/about Sitemap: https://api.preservetube.com/sitemap.xml \ No newline at end of file