Skip to content

Commit c76f23d

Browse files
PfeilPfeil
authored
Merge pull request #13 from kit-data-manager/development
Cleanup / Typed PID Maker Major Version Preparation
2 parents 4a376ca + 5c6d18e commit c76f23d

15 files changed

+371
-431
lines changed

‎.github/dependabot.yml

-4
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,6 @@ updates:
99
directory: "/fairdoscope" # Location of package manifests
1010
schedule:
1111
interval: "weekly"
12-
- package-ecosystem: "docker"
13-
directory: "/indexer"
14-
schedule:
15-
interval: "weekly"
1612
- package-ecosystem: "docker"
1713
directory: "/pit-service"
1814
schedule:

‎README.md

+2-4
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ The FAIR DO Lab Software Stack is an extendable and adjustable architecture for
88
Current high-level functionality roadmap:
99

1010
- [x] Create, update, retrieve and validate FAIR DOs (status: done)
11-
- [x] Search within PID records (status: prototype)
11+
- [x] Search within PID records (status: done)
1212
- [x] Notification of PID activities for other services (status: done)
1313
- [x] Includes an instance of our collection registry to create and manage collections (status: done)
1414
- [x] browse FAIR DOs (status: done)
@@ -29,9 +29,7 @@ Related Publications:
2929
| component | status |
3030
|-----------|--------|
3131
| FAIRDOscope (a FAIR DO Browser) | production-ready |
32-
| Fairris | deprecated (demo interface) |
3332
| Typed PID Maker | production-ready, 1.0 release soon |
34-
| Indexer | prototype |
3533
| Messaging of FAIR DO activities | production-ready |
3634
| Collection Registry | production-ready |
3735
| Elasticsearch (search index) | production-ready, external development |
@@ -76,7 +74,7 @@ Wait until the containers started. After that, you may explore the possibilities
7674
- [PIT-Service API Documentation: http://localhost:8090/swagger-ui.html](http://localhost:8090/swagger-ui.html)
7775
- Use this API to manage PID records. You can find [details about the PIT-Service here](https://github.com/kit-data-manager/pit-service).
7876
- [Message Broker Management UI](http://localhost:15672/#/)
79-
- The message broker will receive messages from the PIT service when records are created or updated. The broker will persist those messages in the queues of interested (registered) clients. In the FAIR DO Lab is currently only one such client: The indexer, which will receive the messages, resolve the record and finally ingest all information into the seach index. This interface will let you watch into internal statistics and the internal state of the broker.
77+
- The message broker will receive messages from the PIT service when records are created or updated. The broker will persist those messages in the queues of interested (registered) clients.
8078
- [Search Index Management UI: http://localhost:5601/app/kibana](http://localhost:5601/app/kibana)
8179
- Use this interface to create a view on the record index and create queries. There are also APIs available directly to the elasticsearch index, which enables developers to create different kinds of search experiences.
8280
- [Collection Service API Documentation: http://localhost:8091/swagger-ui.html](http://localhost:8091/swagger-ui.html)

‎docker-compose.yml

+22-23
Original file line numberDiff line numberDiff line change
@@ -2,59 +2,58 @@ version: '3'
22
services:
33

44
fairdoscope:
5+
container_name: fairdoscope
56
build:
67
context: ./fairdoscope
78
ports:
89
- "8081:80"
910

1011
collectionapi:
12+
container_name: collectionapi
1113
image: kitdm/collection-api
1214
ports:
1315
- "8091:8080"
1416

15-
indexer:
16-
build:
17-
context: .
18-
dockerfile: ./indexer/Dockerfile
19-
container_name: indexer
20-
depends_on:
21-
- messagebroker
22-
2317
pit-service:
18+
container_name: pit-service
2419
build:
2520
context: . # without this context, the dockerfile can not access the java-release-run.sh and similar files.
2621
dockerfile: ./pit-service/Dockerfile
27-
container_name: pit-service
2822
ports:
2923
- "8090:8080"
3024
depends_on:
31-
- messagebroker
32-
33-
fairris:
34-
build:
35-
context: . # without this context, the dockerfile can not access the java-release-run.sh and similar files.
36-
dockerfile: ./fairris/Dockerfile
37-
container_name: fairris
38-
ports:
39-
- "8080:80"
25+
messagebroker:
26+
condition: service_started
27+
elasticsearch:
28+
condition: service_healthy
4029

4130
messagebroker:
42-
image: rabbitmq:3-management-alpine
4331
container_name: messagebroker
32+
image: rabbitmq:3-management-alpine
4433
ports:
4534
- "5672:5672"
4635
- "15672:15672"
4736

4837
elasticsearch:
49-
image: elasticsearch:7.8.1
38+
container_name: elasticsearch
39+
image: elasticsearch:8.8.1
5040
environment:
5141
- discovery.type=single-node
52-
- logger.org.elasticsearch=ERROR
42+
- xpack.security.enabled=false
5343
ports:
5444
- "9200:9200"
5545
- "9300:9300"
46+
healthcheck:
47+
test: ["CMD", "curl", "-f", "http://localhost:9200"]
48+
interval: 5s
49+
timeout: 2s
50+
retries: 10
51+
start_period: 40s
5652

5753
kibana:
58-
image: kibana:7.8.1
54+
container_name: kibana
55+
image: kibana:8.8.1
5956
ports:
60-
- "5601:5601"
57+
- "5601:5601"
58+
depends_on:
59+
- elasticsearch

‎documentation/examples/create-and-resolve-pid.hurl

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# create/register a pid record
2+
POST http://localhost:8090/api/v1/pit/pid/
3+
Content-Type: application/vnd.datamanager.pid.simple+json
4+
Accept: application/vnd.datamanager.pid.simple+json
5+
{
6+
"record": [
7+
{ "key": "21.T11148/076759916209e5d62bd5", "value": "21.T11148/301c6f04763a16f0f72a" },
8+
{ "key": "21.T11148/397d831aa3a9d18eb52c", "value": "2021-12-21T17:36:09.541+00:00" },
9+
{ "key": "21.T11148/8074aed799118ac263ad", "value": "21.T11148/37d0f4689c6ea3301787" },
10+
{ "key": "21.T11148/92e200311a56800b3e47", "value": "{ \"sha256sum\": \"sha256 c50624fd5ddd2b9652b72e2d2eabcb31a54b777718ab6fb7e44b582c20239a7c\" }" },
11+
{ "key": "21.T11148/aafd5fb4c7222e2d950a", "value": "2021-12-21T17:36:09.541+00:00" },
12+
{ "key": "21.T11148/b8457812905b83046284", "value": "https://test.repo/file001" },
13+
{ "key": "21.T11148/c692273deb2772da307f", "value": "1.0.0" },
14+
{ "key": "21.T11148/c83481d4bf467110e7c9", "value": "21.T11148/ManuscriptPage" }
15+
]
16+
}
17+
18+
# on success, we get a 201
19+
HTTP 201
20+
# in the response body, you'll get the record and the pid
21+
[Captures]
22+
pid: jsonpath "$.pid"
23+
24+
# if we resolve the pid...
25+
GET http://localhost:8090/api/v1/pit/pid/{{pid}}
26+
27+
# we get exactly the same body on success
28+
HTTP 200
29+
[Asserts]
30+
jsonpath "$.pid" == {{pid}}

‎documentation/examples/query-elasticsearch.hurl

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
POST http://localhost:8090/api/v1/search?page=0&size=20
2+
accept: application/hal+json
3+
Content-Type: application/json
4+
{
5+
"query": {
6+
"exists" : {
7+
"field" : "pid"
8+
}
9+
},
10+
"_source": ["pid"]
11+
}

‎documentation/examples/validate-existing-pid.hurl

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Create/register a pid record.
2+
# As we use the Typed PID Maker, it must be valid, otherwise it will not work.
3+
POST http://localhost:8090/api/v1/pit/pid/
4+
Content-Type: application/vnd.datamanager.pid.simple+json
5+
Accept: application/vnd.datamanager.pid.simple+json
6+
{
7+
"record": [
8+
{ "key": "21.T11148/076759916209e5d62bd5", "value": "21.T11148/301c6f04763a16f0f72a" },
9+
{ "key": "21.T11148/397d831aa3a9d18eb52c", "value": "2021-12-21T17:36:09.541+00:00" },
10+
{ "key": "21.T11148/8074aed799118ac263ad", "value": "21.T11148/37d0f4689c6ea3301787" },
11+
{ "key": "21.T11148/92e200311a56800b3e47", "value": "{ \"sha256sum\": \"sha256 c50624fd5ddd2b9652b72e2d2eabcb31a54b777718ab6fb7e44b582c20239a7c\" }" },
12+
{ "key": "21.T11148/aafd5fb4c7222e2d950a", "value": "2021-12-21T17:36:09.541+00:00" },
13+
{ "key": "21.T11148/b8457812905b83046284", "value": "https://test.repo/file001" },
14+
{ "key": "21.T11148/c692273deb2772da307f", "value": "1.0.0" },
15+
{ "key": "21.T11148/c83481d4bf467110e7c9", "value": "21.T11148/ManuscriptPage" }
16+
]
17+
}
18+
19+
# on success, we get a 201
20+
HTTP 201
21+
# in the response body, you'll get the record and the pid
22+
[Captures]
23+
pid: jsonpath "$.pid"
24+
25+
# if we resolve the pid...
26+
GET http://localhost:8090/api/v1/pit/pid/{{pid}}?validation=true
27+
Accept: application/vnd.datamanager.pid.simple+json
28+
29+
# on validation fail, we get 400 instead of 200
30+
# unfortunately we can (by default) not create invalid PIDs using the Typed PID Maker,
31+
# so we can not properly demonstrate it here, as long as it is in sandboxed mode.
32+
# This limitation is being worked on (current version at the time of writing is 2.0.0):
33+
# In future we plan to allow resolving all kinds of real PIDs even in sandboxed mode.
34+
HTTP 200

‎documentation/examples/validate-record-without-registration.hurl

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Sending a pid record with dryrun parameter will skip registration,
2+
# but still include the validation!
3+
POST http://localhost:8090/api/v1/pit/pid/?dryrun=true
4+
Content-Type: application/vnd.datamanager.pid.simple+json
5+
Accept: application/vnd.datamanager.pid.simple+json
6+
{
7+
"record": [
8+
{ "key": "21.T11148/076759916209e5d62bd5", "value": "21.T11148/301c6f04763a16f0f72a" },
9+
{ "key": "21.T11148/397d831aa3a9d18eb52c", "value": "2021-12-21T17:36:09.541+00:00" },
10+
{ "key": "21.T11148/8074aed799118ac263ad", "value": "21.T11148/37d0f4689c6ea3301787" },
11+
{ "key": "21.T11148/92e200311a56800b3e47", "value": "{ \"sha256sum\": \"sha256 c50624fd5ddd2b9652b72e2d2eabcb31a54b777718ab6fb7e44b582c20239a7c\" }" },
12+
{ "key": "21.T11148/aafd5fb4c7222e2d950a", "value": "2021-12-21T17:36:09.541+00:00" },
13+
{ "key": "21.T11148/b8457812905b83046284", "value": "https://test.repo/noregistration321.zip" },
14+
{ "key": "21.T11148/c692273deb2772da307f", "value": "1.0.0" },
15+
{ "key": "21.T11148/c83481d4bf467110e7c9", "value": "21.T11148/ManuscriptPage" }
16+
]
17+
}
18+
19+
# On success, we get a 200 (success). Note it is _not_ 201 (created)!
20+
HTTP 200
21+
# In the response body, you'll get the record and the pid
22+
[Captures]
23+
pid: jsonpath "$.pid"
24+
25+
# If we resolve the pid...
26+
GET http://localhost:8090/api/v1/pit/pid/{{pid}}
27+
28+
# ...we will not be able to get it, because we did not registered it (dry run / validation only)!
29+
HTTP 404

0 commit comments

Comments
 (0)