Merge pull request #97 from hjacobs/kubeconfig

Kubeconfig & Backoff

README.rst

@@ -108,11 +108,11 @@ The provided ``Makefile`` will generate a Docker image by default:
 Multiple Clusters
 =================
 
-Multiple clusters are supported by passing a list of API server URLs in the ``CLUSTERS`` environment variable.
-These can either be unprotected ``localhost`` URLs or OAuth 2 protected API endpoints.
-Note that authentication via client-certificates is currently not supported!
+Multiple clusters are supported by passing a list of API servers, reading a kubeconfig file or pointing to an HTTP Cluster Registry endpoint.
 
-The needed OAuth credentials (``Bearer`` access token) must be provided via a file ``${CREDENTIALS_DIR}/read-only-token``.
+See the `documentation on multiple clusters`_ for details.
+
+.. _documentation on multiple clusters: https://kubernetes-operational-view.readthedocs.io/en/latest/multiple-clusters.html
 
 
 Configuration
@@ -132,8 +132,14 @@ The following environment variables are supported:
     Directory to read (OAuth) credentials from --- these credentials are only used for non-localhost cluster URLs.
 ``DEBUG``
     Set to "true" for local development to reload code changes.
+``KUBECONFIG_PATH``
+    Path to kubeconfig file to use for cluster access.
+``KUBECONFIG_CONTEXTS``
+    Comma separated list of contexts to use when reading the kubeconfig file from ``KUBECONFIG_PATH``.
 ``MOCK``
     Set to "true" to mock Kubernetes cluster data.
+``QUERY_INTERVAL``
+    Interval in seconds for querying clusters (default: 5). Each cluster will at most queried once per configured interval.
 ``REDIS_URL``
     Optional Redis server to use for pub/sub events and job locking when running more than one replica. Example: ``redis://my-redis:6379``
 ``SERVER_PORT``

docs/multiple-clusters.rst

@@ -2,8 +2,66 @@
 Multiple Clusters
 =================
 
+Multiple clusters are supported by either passing a static list of API server URLs, using an existing kubeconfig file or pointing to a Cluster Registry HTTP endpoint.
+
+Static List of API Server URLs
+==============================
+
 Set the ``CLUSTERS`` environment variable to a comma separated list of Kubernetes API server URLs.
 
+These can either be unprotected ``localhost`` URLs or OAuth 2 protected API endpoints.
+
+The needed OAuth credentials (``Bearer`` access token) must be provided via a file ``${CREDENTIALS_DIR}/read-only-token-secret``.
+
+
+Kubeconfig File
+===============
+
+The `kubeconfig file`_ allows defining multiple cluster contexts with potential different authentication mechanisms.
+
+Kubernetes Operational View will try to reach all defined contexts when given the ``--kubeconfig-path`` command line option (or ``KUBECONFIG_PATH`` environment variable).
+
+Example:
+
+Assuming ``~/.kube/config`` as the following contents with two defined contexts:
+
+.. code-block:: yaml
+
+    apiVersion: v1
+    kind: Config
+    clusters:
+    - cluster: {server: 'https://kube.foo.example.org'}
+      name: kube_foo_example_org
+    - cluster: {server: 'https://kube.bar.example.org'}
+      name: kube_bar_example_org
+    contexts:
+    - context: {cluster: kube_foo_example_org, user: kube_foo_example_org}
+      name: foo
+    - context: {cluster: kube_bar_example_org, user: kube_bar_example_org}
+      name: bar
+    current-context: kube_foo_example_org
+    users:
+    - name: kube_foo_example_org
+      user: {token: myfootoken123}
+    - name: kube_bar_example_org
+      user: {token: mybartoken456}
+
+Kubernetes Operational View would try to reach both endpoints with the respective token for authentication:
+
+.. code-block:: bash
+
+    $ # note that we need to mount the local ~/.kube/config file into the Docker container
+    $ docker run -it -p 8080:8080 -v ~/.kube/config:/kubeconfig hjacobs/kube-ops-view --kubeconfig-path=/kubeconfig
+
+You can select which clusters should be queried by specifying a list of kubeconfig contexts with the ``--kubeconfig-contexts`` option:
+
+.. code-block:: bash
+
+    $ docker run -it -p 8080:8080 -v ~/.kube/config:/kubeconfig hjacobs/kube-ops-view --kubeconfig-path=/kubeconfig --kubeconfig-contexts=bar
+
+This would only query the Kubernetes cluster defined by the ``bar`` context.
+
+
 Cluster Registry
 ================
 
@@ -22,10 +80,14 @@ Set either the ``CLUSTER_REGISTRY_URL`` environment variable or the ``--cluster-
         ]
     }
 
-The cluster registry will be queryied with an OAuth Bearer token, the token can be statically set via the ``OAUTH2_ACCESS_TOKENS`` environment variable.
+The cluster registry will be queried with an OAuth Bearer token, the token can be statically set via the ``OAUTH2_ACCESS_TOKENS`` environment variable.
 Example:
 
 .. code-block:: bash
 
     $ token=mysecrettoken
     $ docker run -it -p 8080:8080 -e OAUTH2_ACCESS_TOKENS=read-only=$token hjacobs/kube-ops-view --cluster-registry-url=https://cluster-registry.example.org
+
+Otherwise the needed OAuth credentials (``Bearer`` access token) must be provided via a file ``${CREDENTIALS_DIR}/read-only-token-secret``.
+
+.. _kubeconfig file: https://kubernetes.io/docs/user-guide/kubeconfig-file/

kube_ops_view/backoff.py

@@ -0,0 +1,48 @@
+import random
+
+
+def expo(n: int, base=2, factor=1, max_value=None):
+    """Exponential decay.
+
+    Adapted from https://github.com/litl/backoff/blob/master/backoff.py (MIT License)
+
+    Args:
+        base: The mathematical base of the exponentiation operation
+        factor: Factor to multiply the exponentation by.
+        max_value: The maximum value to yield. Once the value in the
+             true exponential sequence exceeds this, the value
+             of max_value will forever after be yielded.
+    """
+    a = factor * base ** n
+    if max_value is None or a < max_value:
+        return a
+    else:
+        return max_value
+
+
+def random_jitter(value, jitter=1):
+    """Jitter the value a random number of milliseconds.
+
+    Copied from https://github.com/litl/backoff/blob/master/backoff.py (MIT License)
+
+    This adds up to 1 second of additional time to the original value.
+    Prior to backoff version 1.2 this was the default jitter behavior.
+    Args:
+        value: The unadulterated backoff value.
+    """
+    return value + random.uniform(0, jitter)
+
+
+def full_jitter(value):
+    """Jitter the value across the full range (0 to value).
+
+    Copied from https://github.com/litl/backoff/blob/master/backoff.py (MIT License)
+
+    This corresponds to the "Full Jitter" algorithm specified in the
+    AWS blog's post on the performance of various jitter algorithms.
+    (http://www.awsarchitectureblog.com/2015/03/backoff.html)
+
+    Args:
+        value: The unadulterated backoff value.
+    """
+    return random.uniform(0, value)

kube_ops_view/cluster_discovery.py

@@ -1,14 +1,16 @@
+import logging
+import re
 import time
+from pathlib import Path
 from urllib.parse import urljoin
 
 import kubernetes.client
 import kubernetes.config
-import logging
-import re
 import requests
 import tokens
 from requests.auth import AuthBase
 
+# default URL points to kubectl proxy
 DEFAULT_CLUSTERS = 'http://localhost:8001/'
 CLUSTER_ID_INVALID_CHARS = re.compile('[^a-z0-9:-]')
 
@@ -25,16 +27,21 @@ def generate_cluster_id(url: str):
     return CLUSTER_ID_INVALID_CHARS.sub('-', url.lower()).strip('-')
 
 
-class StaticTokenAuth(AuthBase):
-    def __init__(self, token):
-        self.token = token
+class StaticAuthorizationHeaderAuth(AuthBase):
+    '''Static authentication with given "Authorization" header'''
+
+    def __init__(self, authorization):
+        self.authorization = authorization
 
     def __call__(self, request):
-        request.headers['Authorization'] = 'Bearer {}'.format(self.token)
+        request.headers['Authorization'] = self.authorization
         return request
 
 
 class OAuthTokenAuth(AuthBase):
+    '''Dynamic authentication using the "tokens" library to load OAuth tokens from file
+    (potentially mounted from a Kubernetes secret)'''
+
     def __init__(self, token_name):
         self.token_name = token_name
         tokens.manage(token_name)
@@ -46,11 +53,13 @@ def __call__(self, request):
 
 
 class Cluster:
-    def __init__(self, id, api_server_url, ssl_ca_cert=None, auth=None):
+    def __init__(self, id, api_server_url, ssl_ca_cert=None, auth=None, cert_file=None, key_file=None):
         self.id = id
         self.api_server_url = api_server_url
         self.ssl_ca_cert = ssl_ca_cert
         self.auth = auth
+        self.cert_file = cert_file
+        self.key_file = key_file
 
 
 class StaticClusterDiscoverer:
@@ -71,7 +80,7 @@ def __init__(self, api_server_urls: list):
                     generate_cluster_id(config.host),
                     config.host,
                     ssl_ca_cert=config.ssl_ca_cert,
-                    auth=StaticTokenAuth(config.api_key['authorization'].split(' ', 1)[-1]))
+                    auth=StaticAuthorizationHeaderAuth(config.api_key['authorization']))
             self._clusters.append(cluster)
         else:
             for api_server_url in api_server_urls:
@@ -116,3 +125,41 @@ def get_clusters(self):
         if now - self._last_cache_refresh > self._cache_lifetime:
             self.refresh()
         return self._clusters
+
+
+class KubeconfigDiscoverer:
+
+    def __init__(self, kubeconfig_path: Path, contexts: set):
+        self._path = kubeconfig_path
+        self._contexts = contexts
+
+    def get_clusters(self):
+        # Kubernetes Python client expects "vintage" string path
+        config_file = str(self._path)
+        contexts, current_context = kubernetes.config.list_kube_config_contexts(config_file)
+        for context in contexts:
+            if self._contexts and context['name'] not in self._contexts:
+                # filter out
+                continue
+            config = kubernetes.client.ConfigurationObject()
+            kubernetes.config.load_kube_config(config_file, context=context['name'], client_configuration=config)
+            authorization = config.api_key.get('authorization')
+            if authorization:
+                auth = StaticAuthorizationHeaderAuth(authorization)
+            else:
+                auth = None
+            cluster = Cluster(
+                context['name'],
+                config.host,
+                ssl_ca_cert=config.ssl_ca_cert,
+                cert_file=config.cert_file,
+                key_file=config.key_file,
+                auth=auth)
+            yield cluster
+
+
+class MockDiscoverer:
+
+    def get_clusters(self):
+        for i in range(3):
+            yield Cluster('mock-cluster-{}'.format(i), api_server_url='https://kube-{}.example.org'.format(i))