304 response codes should not be treated as redirects, as they do not…

… contain location fields in their headers
elastic · Feb 21, 2025 · 0d49725 · 0d49725
1 parent c3e9076
commit 0d49725
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/lib/crawler/http_utils/response.rb b/lib/crawler/http_utils/response.rb
@@ -102,7 +102,8 @@ def time_since_request_start
       end
 
       def redirect?
-        code >= 300 && code <= 399
+        # We should ignore code 304s as they do not contain a location field in the header
+        code >= 300 && code <= 399 && code != 304
       end
 
       def error?