Document why packagers mustn't run the test suite.

Refs #1509, #1496, #1427, #1426, #1081, #1026, perhaps others.

Author: aaugustin

docs/project/contributing.rst

@@ -17,8 +17,8 @@ apologies. I know I can mess up. I can't expect you to tell me, but if you
 choose to do so, I'll do my best to handle criticism constructively.
 -- Aymeric)*
 
-Contributions
--------------
+Contributing
+------------
 
 Bug reports, patches and suggestions are welcome!
 
@@ -34,6 +34,28 @@ websockets.
 .. _issue: https://github.com/python-websockets/websockets/issues/new
 .. _pull request: https://github.com/python-websockets/websockets/compare/
 
+Packaging
+---------
+
+Some distributions package websockets so that it can be installed with the
+system package manager rather than with pip, possibly in a virtualenv.
+
+If you're packaging websockets for a distribution, you must use `releases
+published on PyPI`_ as input. You may check `SLSA attestations on GitHub`_.
+
+.. _releases published on PyPI: https://pypi.org/project/websockets/#files
+.. _SLSA attestations on GitHub: https://github.com/python-websockets/websockets/attestations
+
+You mustn't rely on the git repository as input. Specifically, you mustn't
+attempt to run the main test suite. It isn't treated as a deliverable of the
+project. It doesn't do what you think it does. It's designed for the needs of
+developers, not packagers.
+
+On a typical build farm for a distribution, tests that exercise timeouts will
+fail randomly. Indeed, the test suite is optimized for running very fast, with a
+tolerable level of flakiness, on a high-end laptop without noisy neighbors. This
+isn't your context.
+
 Questions
 ---------
 

tests/utils.py

@@ -9,6 +9,8 @@
 import unittest
 import warnings
 
+from websockets.version import released
+
 
 # Generate TLS certificate with:
 # $ openssl req -x509 -config test_localhost.cnf -days 15340 -newkey rsa:2048 \
@@ -39,9 +41,19 @@
 DATE = email.utils.formatdate(usegmt=True)
 
 
-# Unit for timeouts. May be increased on slow machines by setting the
-# WEBSOCKETS_TESTS_TIMEOUT_FACTOR environment variable.
-MS = 0.001 * float(os.environ.get("WEBSOCKETS_TESTS_TIMEOUT_FACTOR", "1"))
+# Unit for timeouts. May be increased in slow or noisy environments by setting
+# the WEBSOCKETS_TESTS_TIMEOUT_FACTOR environment variable.
+
+# Downstream distributors insist on running the test suite despites my pleas to
+# the contrary. They do it on build farms with unstable performance, leading to
+# flakiness, and then they file bugs. Make tests 100x slower to avoid flakiness.
+
+MS = 0.001 * float(
+    os.environ.get(
+        "WEBSOCKETS_TESTS_TIMEOUT_FACTOR",
+        "100" if released else "1",
+    )
+)
 
 # PyPy, asyncio's debug mode, and coverage penalize performance of this
 # test suite. Increase timeouts to reduce the risk of spurious failures.