Uploaded GitLab's contents

INESCTEC · Feb 23, 2024 · 47b83ca · 47b83ca
1 parent a703075
commit 47b83ca
Show file tree

Hide file tree

Showing 20 changed files with 283 additions and 101 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,5 +1,7 @@
 files/*
 examples/*
+!examples/files/artifacts/default.json
+!examples/files/contracts/default.json
 # IDE
 .idea/
 .vscode/

diff --git a/README.md b/README.md
@@ -1,15 +1,15 @@
+<div align="left">
+  <img src="/docs/source/_static/logo.png"  align="middle" width="33%" height="auto">
+</div>
+
+-----------------------------------------------------
 # INESC TEC Client for TNO Security Gateway (TSG) Dataspace Components (TSG-Client)
 
 [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 [![Version](https://img.shields.io/badge/version-0.0.1-blue.svg)]()
 [![Status](https://img.shields.io/badge/status-development-brightgreen.svg)]()
 [![Python](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/release/python-390/)
 
-This document provides detailed instructions for setting up the INESC TEC Client for TNO Security Gateway (TSG-Client) environment. 
-Please follow these steps carefully to ensure successful configuration and deployment.
-
-> **_NOTE:_** This library assumes that you have a running instance of the TSG Core Connector and OpenAPI Data APP. For this, please follow the [Official TSG Documentation](https://tno-tsg.gitlab.io/).
-
 
 ## Overview
 
@@ -24,15 +24,19 @@ It provides a simple and easy-to-use interface for tasks such as:
 - Queries to the dataspace Metadata Broker to list registered connectors and respective self-descriptions
 - Perform requests via OpenAPI Data APP
 
+This document provides detailed instructions for setting up the INESC TEC Client for TNO Security Gateway (TSG-Client) environment. 
+Please follow these steps carefully to ensure successful configuration and deployment.
+
+> **_NOTE:_** This library assumes that you have a running instance of the TSG Core Connector and OpenAPI Data APP. For this, please follow the [Official TSG Documentation](https://tno-tsg.gitlab.io/).
 
 > **_WARNING:_** This library is under active development and is not yet recommended for production use at this time.
 
 > **_IMPORTANT:_** This development is an internal initiative from INESC TEC within [ENERSHARE](https://enershare.eu/), and it is not officially maintained/supported by TNO team.
 
 ## Documentation & Examples
 
-A detailed documentation is available in the [docs](docs/) directory.
-Python examples (and respective description) are available in the [examples](examples/) directory.
+A detailed documentation is available [here](https://cpes-power-and-energy-systems.github.io/tsg-client/).
+Python example scripts (and respective description) are also available in the [examples](examples/) directory of the repository.
 
 
 ## Installation

diff --git a/docs/source/_static/logo.png b/docs/source/_static/logo.png
diff --git a/docs/source/contacts.md b/docs/source/contacts.md
@@ -5,5 +5,6 @@ Contacts
 If you encounter any issues or have questions, please feel free to reach out to the support team:
 
 - **Carolina Catorze:** [carolina.catorze@inesctec.pt](mailto:carolina.catorze@inesctec.pt)
-- **Vasco Guedes:** [vasco.r.maia@inesctec.pt](mailto:vasco.r.maia@inesctec.pt)
+- **Vasco Maia:** [vasco.r.maia@inesctec.pt](mailto:vasco.r.maia@inesctec.pt)
+- **José Luís Rodrigues:** [jose.l.rodrigues@inesctec.pt](mailto:jose.l.rodrigues@inesctec.pt)
 - **José Ricardo Andrade:** [jose.r.andrade@inesctec.pt](mailto:jose.r.andrade@inesctec.pt)
diff --git a/docs/source/getting_started.md b/docs/source/getting_started.md
@@ -3,52 +3,93 @@ Getting Started with the TSG Client
 ====================================
 
 ## Overview
-TSG Client is a Python library for interacting with the TNO Security Gateway (TSG). It provides a simple and easy-to-use interface for tasks such as:
 
-- Connecting to TSG connectors
+TSG Client is a Python library for interacting with the TNO Security Gateway (TSG). 
+It is current version, it is a simple REST API client that interacts with [TSG Core Connector APIs](https://tno-tsg.gitlab.io/docs/core-container/api/) and [TSG OpenAPI Data APP](https://gitlab.com/tno-tsg/data-apps/openapi).
+It provides a simple and easy-to-use interface for tasks such as:
+
+- Connecting to a TSG core container (via API KEY)
 - Retrieving connector self-descriptions
-- Working with catalogs and artifacts
-- Requesting and consuming data artifacts
-- Knowing what connectors are in the dataspace
-- Take advantage of the OpenAPI functionalities
+- Parsing / filtering connector catalogs and artifacts, retrieved from self-descriptions
+- Requesting and consuming data artifacts (via dataspace)
+- Queries to the dataspace Metadata Broker to list registered connectors and respective self-descriptions
+- Perform requests via OpenAPI Data APP
+
+> **_WARNING:_** This library is under active development and is not yet recommended for production use at this time.
+
+> **_IMPORTANT:_** This development is an internal initiative from INESC TEC within [ENERSHARE](https://enershare.eu/), and it is not officially maintained/supported by TNO team.
 
 
 ## Installation steps
 
 ### Install essential tools
 
+- **OS:** Windows, Linux, macOS
 
 - **Install GIT:**
   - Download and install GIT from [Git Download Page](https://git-scm.com/downloads)
 
-- **Python**: Install Python (tested using Python 3.9)
+- **Python**: Install Python (version 3.9+)
 
 - **Install Python requirements:**
   - Open a terminal and execute the following command:
     ```bash
     pip install -r requirements.txt
     ```
 
+
 ### Clone the project and setup the environment
 
-Open a terminal and execute the following commands:
+To install the TSG-Client, please follow the steps below:
+
+  1. Clone the repository.
 
 ```bash
-git clone https://gitlab.inesctec.pt/cpes/european-projects/enershare/tsg-client.git
-cd tsg-client
+$ git clone https://github.com/CPES-Power-and-Energy-Systems/tsg-client.git
 ```
 
-### OR Install the Library
+  2. (optional) Create a virtual environment and activate it.
 
-Open a project and execute the following commands:
+  3. Install the required dependencies:
 
 ```bash
-pip install git+https://gitlab.inesctec.pt/cpes/european-projects/enershare/tsg-client.git
+$ pip install -r requirements.txt
+```
+
+  4. Run the following command to install the python library:
+
+```bash
+$ pip install .
+```
+
+Once you install the library, you'll be able to use the TSG-Client in your Python projects by just importing it (see basic example below):
+
+```python
+from tsg_client import TSGController
+
+# Set up the TSG connector:
+conn = TSGController(
+    api_key="<your-api-key>",
+    connector_id="<your-connector-id>",
+    access_url="<your-access-url>",
+    agent_id="<your-agent-id>",
+    metadata_broker_url="<your-dataspace-metadata-broker-url>"  # this one is optional
+)
+
+# Retrieve the connector self-description:
+self_description = conn.get_connector_self_description()
+print(self_description)
+
+# List self-description of other connectors in the dataspace:
+# Request data from DS Metadata Broker:
+result = conn.query_metadata_broker()
+print(result)
 ```
 
 ### Set up environment variables
 
-Create a `.env` file with the following contents:
+Ideally, you should set up the environment variables to avoid hardcoding the API key and other sensitive information in your code.
+We have a `dotenv` file in the project that may work as an example for you to create your `.env` file. Start by copying that file:
 
 ```bash
 cp dotenv .env
@@ -62,16 +103,10 @@ Open the `.env` file in a text editor and modify the following variables with yo
 - **CONNECTOR_ID**: Replace with the connector ID for your TSG instance.
 - **ACCESS_URL**: Replace with the access URL for your TSG connector.
 - **AGENT_ID**: Replace with the agent ID associated with your TSG connector.
+- **METADATA_BROKER_URL**: Replace with the URL of the dataspace metadata broker (optional, just in case you will need to query the metadata broker)
 
 Make sure to save the changes after updating the values.
 
 ### Examples
 
-Follow the examples provided in the Examples section
-
-
-## Additional Information
-
-The TSG-Client is a REST API that interacts with the connector of the dataspace of TNO, deployed following a specific tutorial. The available requests are currently being tested in `main.py`. The script demonstrates interactions with TSG OpenApi Data APP and inter-connector API.
-
-
+Follow the examples provided in the [Examples](examples) section.
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -3,9 +3,23 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
+.. image:: _static/logo.png
+   :width: 572
+
 Welcome to TSG Client's documentation!
 ======================================
 
+TSG Client is a Python library for interacting with the TNO Security Gateway (TSG). It is current version, it is a simple REST API client that interacts with TSG Core Connector APIs and TSG OpenAPI Data APP. It provides a simple and easy-to-use interface for tasks such as:
+
+ - Connecting to a TSG core container (via API KEY)
+ - Retrieving connector self-descriptions
+ - Parsing / filtering connector catalogs and artifacts, retrieved from self-descriptions
+ - Requesting and consuming data artifacts (via dataspace)
+ - Queries to the dataspace Metadata Broker to list registered connectors and respective self-descriptions
+ - Perform requests via OpenAPI Data APP
+
+This document provides detailed instructions for setting up the INESC TEC Client for TNO Security Gateway (TSG-Client) environment. Please follow these steps carefully to ensure successful configuration and deployment.
+
 .. toctree::
    :maxdepth: 2
    :caption: Contents:

diff --git a/...layground_tsg_connectors_TestConnector_artifacts_4510e3df-df71-4c95-8607-b00f64a11ecd.pdf b/...layground_tsg_connectors_TestConnector_artifacts_4510e3df-df71-4c95-8607-b00f64a11ecd.pdf
diff --git a/examples/delete_artifact.py b/examples/delete_artifact.py
@@ -39,13 +39,6 @@
     # Load environment variables:
     config = dotenv_values('.env')
 
-    # Example of external connector configs (TNO Playground)
-    EXTERNAL_CONNECTOR = {
-        "CONNECTOR_ID": 'urn:playground:tsg:connectors:TestConnector',
-        "ACCESS_URL": 'https://test-connector.playground.dataspac.es/selfdescription',
-        "AGENT_ID": 'urn:playground:tsg:TNO'
-    }
-
     # Connect to our TSG connector:
     conn = TSGController(
         api_key=config['API_KEY'],

diff --git a/examples/edit_artifact.py b/examples/edit_artifact.py
@@ -43,13 +43,6 @@
     # Load environment variables:
     config = dotenv_values('.env')
 
-    # Example of external connector configs (TNO Playground)
-    EXTERNAL_CONNECTOR = {
-        "CONNECTOR_ID": 'urn:playground:tsg:connectors:TestConnector',
-        "ACCESS_URL": 'https://test-connector.playground.dataspac.es/selfdescription',
-        "AGENT_ID": 'urn:playground:tsg:TNO'
-    }
-
     # Connect to our TSG connector:
     conn = TSGController(
         api_key=config['API_KEY'],
@@ -63,7 +56,7 @@
     artifact_path = "<artifact_path>"
     artifact_description = "<description>"
     artifact_title = "<title>"
-    contract_offer_path = "../files/contracts/default.json"
+    contract_offer_path = "./files/contracts/default.json"
 
     # Read the contract offer content from the file:
     with open(contract_offer_path, 'r') as file:

diff --git a/examples/get_connector_self_description.py b/examples/get_connector_self_description.py
@@ -51,3 +51,22 @@
     print("-" * 79)
     print("> Connector Self Self Description:")
     pprint(self_description.to_dict())
+
+    # Get artifacts list of your connector self-description:
+    # Note: you can refine search using the filters below
+    resource_type = "ids:ContractOffer"
+    artifacts = conn.parse_catalog_artifacts(
+        self_description,
+        catalog_id=self_description.catalogs[0].id,
+        resource_type=resource_type,
+        creation_date_gt="2021-08-10T00:00:00.000Z",
+        creation_date_lt="2030-01-01T00:00:00.000Z",
+        return_last_artifact=True,
+        valid_contract_only=False
+    )
+
+    print("-" * 79)
+    print(f"> Connector {config['CONNECTOR_ID']} in "
+          f"catalog {self_description.catalogs[0].id} with "
+          f"resource type {resource_type} has this list artifacts:")
+    pprint(artifacts)
diff --git a/examples/get_external_artifacts.py b/examples/get_external_artifacts.py
@@ -40,7 +40,7 @@
     # Example of external connector configs (TNO Playground)
     EXTERNAL_CONNECTOR = {
         "CONNECTOR_ID": 'urn:playground:tsg:connectors:TestConnector',
-        "ACCESS_URL": 'https://test-connector.playground.dataspac.es/selfdescription',
+        "ACCESS_URL": 'https://test-connector.playground.dataspac.es',
         "AGENT_ID": 'urn:playground:tsg:TNO'
     }
 

diff --git a/examples/get_external_catalogs.py b/examples/get_external_catalogs.py
@@ -40,7 +40,7 @@
     # Example of external connector configs (TNO Playground)
     EXTERNAL_CONNECTOR = {
         "CONNECTOR_ID": 'urn:playground:tsg:connectors:TestConnector',
-        "ACCESS_URL": 'https://test-connector.playground.dataspac.es/selfdescription',
+        "ACCESS_URL": 'https://test-connector.playground.dataspac.es',
         "AGENT_ID": 'urn:playground:tsg:TNO'
     }
 

diff --git a/examples/get_external_download_artifact.py b/examples/get_external_download_artifact.py
@@ -43,7 +43,7 @@
     # Example of external connector configs (TNO Playground)
     EXTERNAL_CONNECTOR = {
         "CONNECTOR_ID": 'urn:playground:tsg:connectors:TestConnector',
-        "ACCESS_URL": 'https://test-connector.playground.dataspac.es/selfdescription',
+        "ACCESS_URL": 'https://test-connector.playground.dataspac.es',
         "AGENT_ID": 'urn:playground:tsg:TNO'
     }
 

diff --git a/examples/get_external_open_api_specs.py b/examples/get_external_open_api_specs.py
@@ -38,7 +38,7 @@
     # Example of external connector configs (TNO Playground)
     EXTERNAL_CONNECTOR = {
         "CONNECTOR_ID": 'urn:playground:tsg:connectors:TestConnector',
-        "ACCESS_URL": 'https://test-connector.playground.dataspac.es/selfdescription',
+        "ACCESS_URL": 'https://test-connector.playground.dataspac.es',
         "AGENT_ID": 'urn:playground:tsg:TNO'
     }
 

diff --git a/examples/get_external_request_openapi_dataapp.py b/examples/get_external_request_openapi_dataapp.py
@@ -42,7 +42,7 @@
     # Example of external connector configs (TNO Playground)
     EXTERNAL_CONNECTOR = {
         "CONNECTOR_ID": 'urn:playground:tsg:connectors:TestConnector',
-        "ACCESS_URL": 'https://test-connector.playground.dataspac.es/router',
+        "ACCESS_URL": 'https://test-connector.playground.dataspac.es',
         "AGENT_ID": 'urn:playground:tsg:TNO'
     }
 
@@ -54,15 +54,35 @@
         agent_id=config['AGENT_ID']
     )
 
-    # Execute external OpenAPI request:
+    # Get external connector info (self-descriptions):
+    description = conn.get_connector_selfdescription(
+        access_url=EXTERNAL_CONNECTOR['ACCESS_URL'],
+        connector_id=EXTERNAL_CONNECTOR['CONNECTOR_ID'],
+        agent_id=EXTERNAL_CONNECTOR['AGENT_ID']
+    )
+
+    # Get external connector OpenAPI specs:
     api_version = "0.9.2"
-    endpoint = "uuid"
+    open_api_specs = conn.get_openapi_specs(description, api_version)
 
+    # Get first API specification:
+    open_api_specs = open_api_specs[0]
+    example_endpoint = open_api_specs['endpoints'][-1]
+    example_agent_id = open_api_specs['agent']
+
+    print(f"""
+    Performing a request to:
+    - Agent ID: {example_agent_id}
+    - API Version: {api_version}
+    - Endpoint: {example_endpoint}
+    """)
+
+    # Execute external OpenAPI request:
     response = conn.openapi_request(
         external_access_url=EXTERNAL_CONNECTOR['ACCESS_URL'],
-        external_connector_id=EXTERNAL_CONNECTOR['CONNECTOR_ID'],
+        data_app_agent_id=example_agent_id,
         api_version=api_version,
-        endpoint=endpoint,
+        endpoint=example_endpoint,
         params="",
         method="get"
     )

diff --git a/examples/get_external_self_description.py b/examples/get_external_self_description.py
@@ -43,7 +43,7 @@
     # Example of external connector configs (TNO Playground)
     EXTERNAL_CONNECTOR = {
         "CONNECTOR_ID": 'urn:playground:tsg:connectors:TestConnector',
-        "ACCESS_URL": 'https://test-connector.playground.dataspac.es/selfdescription',
+        "ACCESS_URL": 'https://test-connector.playground.dataspac.es',
         "AGENT_ID": 'urn:playground:tsg:TNO'
     }
 

diff --git a/examples/post_artifact.py b/examples/post_artifact.py
@@ -44,13 +44,6 @@
     # Load environment variables:
     config = dotenv_values('.env')
 
-    # Example of external connector configs (TNO Playground)
-    EXTERNAL_CONNECTOR = {
-        "CONNECTOR_ID": 'urn:playground:tsg:connectors:TestConnector',
-        "ACCESS_URL": 'https://test-connector.playground.dataspac.es/selfdescription',
-        "AGENT_ID": 'urn:playground:tsg:TNO'
-    }
-
     # Connect to our TSG connector:
     conn = TSGController(
         api_key=config['API_KEY'],
@@ -60,10 +53,10 @@
     )
 
     # Specify the required parameters:
-    artifact_path = "<artifact_path>"
-    artifact_description = "<description>"
-    artifact_title = "<title>"
-    contract_offer_path = "../files/contracts/default.json"
+    artifact_path = ".files/artifacts/default.json"
+    artifact_description = "Example artifact (default) - TSG Client"
+    artifact_title = "TSG-Client Example"
+    contract_offer_path = "./files/contracts/default.json"
 
     # Read the contract offer content from the file:
     with open(contract_offer_path, 'r') as file: