Skip to content

Implement combined Geoserver service and Workspace/Layer effective permissions #495

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 37 commits into from
Feb 1, 2022
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
cf794a3
fix lint
fmigneault Jan 6, 2022
4de9fdb
first implementation of geoserver multi-ows services
fmigneault Jan 6, 2022
da2bbf8
adjust UI to improve display within edit service page when many permi…
fmigneault Jan 6, 2022
1e96bb0
Bump version: 3.19.1 → 3.20.0
fmigneault Jan 6, 2022
f656831
allowed scoped resource names using colon character
fmigneault Jan 6, 2022
9b04a1e
add tests to validate proper support of scoped names for service and …
fmigneault Jan 6, 2022
e078ace
support specific nested structure of children resource types
fmigneault Jan 7, 2022
2648904
fixes to reporting applied service configuration from API and setting…
fmigneault Jan 8, 2022
aa23605
add missing WFS permissons
fmigneault Jan 8, 2022
3992207
add tests for new features
fmigneault Jan 8, 2022
d0c45f9
updates to geoserver/ncwms/wms/wfs service definitions based on permi…
fmigneault Jan 8, 2022
7d501a6
provide values defined by user on UI invalid value during add service…
fmigneault Jan 8, 2022
e4f16ac
add workspace/layer hierarchy under geoserverwms + update changelog
fmigneault Jan 8, 2022
a62ea1a
reuse geoserver workspace/layer resource resolver for all geoserver s…
fmigneault Jan 10, 2022
a120adf
add wps test for geoserver(wfs+wms+wps) impl
fmigneault Jan 10, 2022
20761f9
Merge branch 'master' into geoserver
fmigneault Jan 10, 2022
c6b84d5
fix lint
fmigneault Jan 10, 2022
4b3699d
fix typo and add resource descriptions
fmigneault Jan 11, 2022
932b7ab
use function for recuring children-allowed verifications
fmigneault Jan 11, 2022
47feda7
fix css checker & css lint
fmigneault Jan 11, 2022
86bc072
fixes and even more tests to validate Geoserver effective resolution
fmigneault Jan 12, 2022
5d74a43
fix tests
fmigneault Jan 12, 2022
9c298f5
update 'child_structure_allowed' property to employ dict-list resourc…
fmigneault Jan 12, 2022
49cf03d
fix lint
fmigneault Jan 12, 2022
7d51d3b
fix formatting of service with child structured allowed definition
fmigneault Jan 19, 2022
328b769
Merge branch 'master' into geoserver
fmigneault Jan 19, 2022
6441bd7
Merge branch 'master' into geoserver
fmigneault Jan 19, 2022
ba9a566
adjust changelog
fmigneault Jan 19, 2022
974b951
add more tests to valiate service res-type strucutures + add them for…
fmigneault Jan 20, 2022
f81e80d
add support of multiple resources ACL resolution
fmigneault Jan 24, 2022
d210036
update docs
fmigneault Jan 25, 2022
65a1231
document new services and attributes
fmigneault Jan 25, 2022
b0ec6ae
more fixes to docs references
fmigneault Jan 25, 2022
40dab93
fixes and test geoserver multi-layer (comma-separated) ACL resolution
fmigneault Jan 25, 2022
6304983
add large matrix of geoserver workspace/layer test combinations
fmigneault Jan 25, 2022
00b4d0e
fix linting
fmigneault Jan 26, 2022
947f8ab
fix format string backward compatible
fmigneault Jan 28, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ venv

# Node
node_modules
package.json
package-lock.json

# Makefile
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ share

# Node
node_modules
package.json
package-lock.json

# Makefile
Expand Down
34 changes: 18 additions & 16 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,6 @@ Changes
`Unreleased <https://github.com/Ouranosinc/Magpie/tree/master>`_ (latest)
------------------------------------------------------------------------------------

* Nothing new for the moment.

`3.20.1 <https://github.com/Ouranosinc/Magpie/tree/3.20.1>`_ (2022-01-19)
------------------------------------------------------------------------------------

Bug Fixes
~~~~~~~~~~~~~~~~~~~~~
* Fix `Twitcher` ``/verify`` endpoint integrated through ``MagpieAdapter`` to validate synchronized authentication
policies across both services. The endpoint now supports query parameter credentials to facilitate call directly
within a web browser (not assuming JSON contents) and properly parses cookies from both policies to report more
detailed error messages in case of failure.
* Fix ``error`` and ``type`` representation when handling raised exception by ``evaluate_call`` utility function.

`3.20.0 <https://github.com/Ouranosinc/Magpie/tree/3.20.0>`_ (2022-01-06)
------------------------------------------------------------------------------------

Features / Changes
~~~~~~~~~~~~~~~~~~~~~
* Add missing ``ServiceWFS`` permissions according to `OGC WFS standard <https://www.ogc.org/standards/wfs>`_.
Expand Down Expand Up @@ -51,6 +35,13 @@ Features / Changes
* Adjust UI to consider ``child_structure_allowed`` definitions to propose only applicable ``Resource`` types in the
combobox when creating a new ``Resource`` in the tree hierarchy.
* Add UI submission field to provide ``Service`` JSON configuration at creation when supported by the type.
* Add ``child_structure_allowed`` to ``ServiceAPI`` and ``ServiceTHREDDS`` to be more explicit about allowed structure
hierarchies in API responses. Their original behaviour remains unchanged, but is further enforced during validation
of their children resource type creation against explicit structure.
* Add multi-``Resource`` ACL resolution within the same request to support ``Service`` implementations that can refer
to multiple items simultaneously. An example of this is the comma-separated list of ``Layer`` defined by ``typeNames``
of new ``ServiceGeoserverWMS`` implementation. Access is granted if the ``User`` has access to **ALL** ``Resource``
resolved from parsing the request.

Bug Fixes
~~~~~~~~~~~~~~~~~~~~~
Expand All @@ -65,6 +56,17 @@ Bug Fixes
* Fix `Effective Resolution` of ``Permission`` applied for ``ServiceGeoserverWMS`` to consider ``Scope`` modifier
of ``Service`` and ``Workspace`` for access to be resolved at the ``Layer`` level.

`3.20.1 <https://github.com/Ouranosinc/Magpie/tree/3.20.1>`_ (2022-01-19)
------------------------------------------------------------------------------------

Bug Fixes
~~~~~~~~~~~~~~~~~~~~~
* Fix `Twitcher` ``/verify`` endpoint integrated through ``MagpieAdapter`` to validate synchronized authentication
policies across both services. The endpoint now supports query parameter credentials to facilitate call directly
within a web browser (not assuming JSON contents) and properly parses cookies from both policies to report more
detailed error messages in case of failure.
* Fix ``error`` and ``type`` representation when handling raised exception by ``evaluate_call`` utility function.

`3.20.0 <https://github.com/Ouranosinc/Magpie/tree/3.20.0>`_ (2022-01-06)
------------------------------------------------------------------------------------

Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -696,7 +696,7 @@ test-remote-only: ## run only remote tests with the environment Python
.PHONY: test-custom-only
test-custom-only: ## run custom marker tests using SPEC="<marker-specification>"
@echo "Running custom tests..."
@[ "${SPEC}" ] || ( echo ">> 'TESTS' is not set"; exit 1 )
@[ "${SPEC}" ] || ( echo ">> 'SPEC' is not set"; exit 1 )
@bash -c '$(CONDA_CMD) pytest tests $(TEST_VERBOSITY) -m "${SPEC}" --junitxml "$(APP_ROOT)/tests/results.xml"'

.PHONY: test-docker
Expand Down
22 changes: 22 additions & 0 deletions docs/_static/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,25 @@ div.bordered-content {
border: 1px solid #C6C9CB;
padding: 0 1em 1em 0;
}

/* apply the parent list number prefixed to the current list numbers
https://stackoverflow.com/a/13841818
*/
div.parent-list-numbers ol {
counter-reset: item;
}

div.parent-list-numbers ol li {
counter-increment: item;
}

div.parent-list-numbers ol ol > li::before {
content: counters(item, ".") ")";
margin-left: -3em;
}

div.parent-list-numbers > ol > li > ol > li {
list-style: none;
display: inline-flex;
display: -webkit-box;
}
69 changes: 36 additions & 33 deletions docs/authentication.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,10 @@ Query Parameters

This method employs the query string parameters in the URL to provide the credentials. The format is as follows.

.. code-block::
.. code-block:: http

GET {MAGPIE_URL}/signin?user_name=<usr>&password=<pwd>
GET /signin?user_name=<usr>&password=<pwd> HTTP/1.1
Host: {MAGPIE_URL}


The response will contain :ref:`Authentication Headers` detail needed for user identification.
Expand All @@ -89,37 +90,39 @@ Body content requests allow multiple variants, based on the specified ``Content-
All variants employ a similar structure, but indicate the format of the body to be parsed.
By default, ``application/json`` is employed if none was specified.

.. code-block::
.. code-block:: http

POST {MAGPIE_URL}/signin
Headers
Content-Type: multipart/form-data; boundary=<boundary-string>
Body
user_name: "<usr>"
password: "<pwd>"
provider_name: "<provider>" # optional
POST /signin HTTP/1.1
Host: {MAGPIE_URL}
Content-Type: multipart/form-data; boundary=<boundary-string>

--<boundary-string>
user_name: "<usr>"
password: "<pwd>"
provider_name: "<provider>" # optional
--<boundary-string>--

.. code-block::

POST {MAGPIE_URL}/signin
Headers
Content-Type: application/x-www-form-urlencoded
Body
user_name=<usr>&password=<pwd>&provider_name=<provider>
.. code-block:: http

POST /signin HTTP/1.1
Host: {MAGPIE_URL}
Content-Type: application/x-www-form-urlencoded

.. code-block::
user_name=<usr>&password=<pwd>&provider_name=<provider>

POST {MAGPIE_URL}/signin
Headers
Content-Type: application/json
Body
{
"user_name": "<usr>",
"password": "<pwd>",
"provider_name": "<provider>"
}

.. code-block:: http

POST /signin HTTP/1.1
Host: {MAGPIE_URL}
Content-Type: application/json

{
"user_name": "<usr>",
"password": "<pwd>",
"provider_name": "<provider>"
}


The response will contain :ref:`Authentication Headers` detail needed for user identification.
Expand Down Expand Up @@ -196,7 +199,7 @@ Authentication Headers
After execution of an :term:`Authentication` request, a ``Set-Cookie`` header with `Magpie` user identification token
named according to :ref:`config_security` should be set in the response as follows.

.. code-block::
.. code-block:: http

Set-Cookie: {MAGPIE_COOKIE_NAME}=<auth-token>!userid_type:int;
[Domain=<domain>; Path=<path>; HttpOnly; SameSite=Lax; Max-Age=<seconds>; expires=<datetime>]
Expand Down Expand Up @@ -239,8 +242,8 @@ Authorization Headers
Following any successful :term:`Authentication` request as presented in the previous section, the obtained ``Cookie``
defines which :term:`Logged User` attempts to accomplish an operation against a given protected URI. `Magpie` employs
the same ``Cookie`` both for operations provided by its API and for accessing the real :term:`Resource` protected
behind the :term:`Proxy` according to resolution of :term:`Effective Permissions` based on :term:`Applied Permissions`
definitions.
behind the :term:`Proxy` according to resolution of :term:`Effective Permissions <Effective Permission>` based on
:term:`Applied Permission` definitions.

Access to Magpie Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand All @@ -256,14 +259,14 @@ Access to Protected Resources

When sending requests to the :term:`Policy Enforcement Point` (PEP) (e.g.: `Twitcher`_ :term:`Proxy`),
appropriate ``Cookie`` headers must be defined for it to identify the :term:`Logged User` and resolve its
:term:`Effective Permissions` accordingly. Not providing those tokens will default to using
:term:`Effective Permissions <Effective Permission>` accordingly. Not providing those tokens will default to using
:envvar:`MAGPIE_ANONYMOUS_USER`, which will result into either one of HTTP ``Unauthorized [401]`` or
``Forbidden [403]``, depending on how the PEP interprets and returns the response indicated by `Magpie`, unless the
corresponding :term:`Resource` was allowed for :ref:`perm_public_access`.

When appropriately authenticated, access to the targeted :term:`Resource` will be granted or denied depending on the
:term:`Effective Permissions` that :term:`Logged User` has for it. This decision is extensively explained in section
:ref:`perm_resolution`.
:term:`Effective Permissions <Effective Permission>` that :term:`Logged User` has for it. This decision is extensively
explained in section :ref:`perm_resolution`.

Another alternative to obtain :term:`Authorization` (only when using the :ref:`utilities_adapter<Magpie Adapter>`) is
by providing the ``Authorization`` header in the request with appropriate credentials. In this situation, the adapter
Expand All @@ -276,7 +279,7 @@ employed instead to process :term:`Authentication` only once.

The format of the ``Authorization`` header is has follows.

.. code-block::
.. code-block:: http

Authorization: Bearer <access_token>

Expand Down
8 changes: 7 additions & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@
"doc_redirect",
"sphinxcontrib.redoc",
"sphinx.ext.autodoc",
"sphinx.ext.autosectionlabel", # help make cross-references to title/sections
"cloud_sptheme.ext.autodoc_sections", # allow sections in docstrings code
"sphinx.ext.todo",
"sphinx.ext.viewcode",
"sphinx.ext.intersphinx",
Expand Down Expand Up @@ -174,7 +176,11 @@ def doc_redirect_include(file_path):

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = []
exclude_patterns = [
# ignore multiple over-verbose warnings caused
# by file not yet generated when TOC is processed
"api.rst",
]

# The reST default role (used for this markup: `text`) to use for all
# documents.
Expand Down
4 changes: 2 additions & 2 deletions docs/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -686,7 +686,7 @@ remain available as described at the start of the :ref:`Configuration` section.
(Value: ``"admin"``)

Name of the :term:`Permission` used to represent highest administration privilege in the application. It is one of
the special :term:`Access Permissions` known by the application (see also :ref:`Route Access` section).
the special :term:`Access Permission` known by the application (see also :ref:`Route Access` section).

.. envvar:: MAGPIE_LOGGED_PERMISSION

Expand All @@ -695,7 +695,7 @@ remain available as described at the start of the :ref:`Configuration` section.

.. versionadded:: 2.0

Defines a special condition of :term:`Access Permissions` related to the :term:`Logged User` session and the
Defines a special condition of :term:`Access Permission` related to the :term:`Logged User` session and the
targeted :term:`User` by the request. See details in :ref:`Route Access` for when it applies.

.. envvar:: MAGPIE_LOGGED_USER
Expand Down
39 changes: 31 additions & 8 deletions docs/glossary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,13 @@ Glossary
grants or denies :term:`Permission` access to the applicable :term:`User` for the targeted :term:`Resource`.
Formed of multiple :term:`ACE`.

Allowed Permissions
Allowed Permission
Set of applicable :term:`Permission` values onto an element.

.. seealso::
:ref:`Allowed Permissions <allowed-permissions>` section for details.

Applied Permissions
Applied Permission
An active :term:`Permission` for a given :term:`User` or :term:`Group` depending on context.

.. seealso::
Expand Down Expand Up @@ -75,13 +75,20 @@ Glossary
:term:`Group` that has property ``discoverable=True``, making it publicly viewable to any-level user.
Otherwise, groups can be listed or accessed only by administrators.

Effective Permissions
Effective Permission
A :term:`Permission` that has been completely resolved according to all applicable contexts, that indicates
the final granted or denied result.

.. seealso::
:ref:`Effective Permissions <effective_permissions>` section for details.

Effective Resolution
Process of resolving :term:`Effective Permission` over a :term:`Resource` considering any applicable
:ref:`permission_modifiers`.

.. seealso::
:ref:`perm_resolution` section for details.

External Providers
Set of all known user-identity :term:`Provider` defined externally to `Magpie`. Each of these :term:`Provider`
require specific connection methodologies, as configured in :mod:`magpie.security`.
Expand All @@ -97,14 +104,14 @@ Glossary
request to ask for confirmation. The terms and conditions can only be defined upon the :term:`Group` creation
and can never be modified afterwards.

Immediate Permissions
Immediate Permission
Describes a :term:`Permission` that originates directly and only from a :term:`Service`.
This is referenced in only a few use-cases, notably for :ref:`Finding User Permissions`.

.. seealso::
:ref:`Immediate Permissions <immediate_permissions>` section for details.

Inherited Permissions
Inherited Permission
Describes a :term:`Permission` that includes both :term:`User` and :term:`Group` contexts simultaneously.

.. seealso::
Expand All @@ -130,9 +137,9 @@ Glossary
geospatial data and service methodologies in order to improve access to geospatial and location information.

OWS
Acronym that regroups all :term:`OGC` Web Services. This includes `Web Feature Service` (WFS),
`Web Map Service` (WMS) and `Web Processing Service` (WPS) for which `Magpie` offers some specific
:term:`Service` request parser implementations.
Acronym that regroups all :term:`OGC` Web Services. This includes :term:`Web Feature Service <WFS>` (WFS),
:term:`Web Map Service <WMS>` (WMS) and :term:`Web Processing Service <WMS>` (WPS), amongst others, for which
`Magpie` offers some specific :term:`Service` request parser implementations.

Pending User
Account that is pending for validation or approval following self-registration when the application is
Expand Down Expand Up @@ -219,3 +226,19 @@ Glossary

.. seealso::
:ref:`config_webhook` and :ref:`config_file` sections for details.

Web Feature Service
WFS

One of the :term:`OWS` implementation which `Magpie` offers an implementation
for controlling access to layers and their features.

Web Map Service
WMS
One of the :term:`OWS` implementation which `Magpie` offers an implementation
for controlling access to layers and generated maps from them.

Web Processing Service
WPS
One of the :term:`OWS` implementation which `Magpie` offers an implementation
for controlling access to description and execution of processes.
8 changes: 4 additions & 4 deletions docs/performance.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ requests are execute in rapid succession. `PostgreSQL`_ and `SQLAlchemy`_ are us
when more than a couple requests per second are needed, some solutions are possible to
improve the performance of these requests by avoiding unnecessary reload of static data.

We can take advantage of the fact that individual :temr:`Permission` and :term:`Service` definitions are not
We can take advantage of the fact that individual :term:`Permission` and :term:`Service` definitions are not
susceptible to change often and cache the results of these queries.

While not activated by default, it's possible to cache the :term:`Access Control Lists` (ACLs) and :term:`Service`
While not activated by default, it's possible to cache the :term:`Access Control List` (ACL) and :term:`Service`
retrieval operations for all services, and give it an expiration timeout.

.. code-block:: ini
Expand All @@ -26,8 +26,8 @@ retrieval operations for all services, and give it an expiration timeout.
.. warning::
Take into consideration that settings must be applied to `Twitcher`_ INI file such that incoming proxy requests
will be effective in its web application, in turn using the :class:`magpie.adapter.MagpieAdapter`. Caching settings
defined in `Magpie` INI file will be employed only when requesting :term:`Effective Permissions` resolution using
`Magpie`'s API endpoints.
defined in `Magpie` INI file will be employed only when requesting
:term:`Effective Permissions <Effective Permission>` resolution using `Magpie`'s API endpoints.


In the above example, for a particular request that queries a :term:`Logged User`'s ACL for a specific :term:`Service`,
Expand Down
Loading