Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging new changes from fork #104

Open
wants to merge 73 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
d63ebd5
Dummy check-in
nshweta90 Aug 13, 2024
52ea72a
Changes for Docker
nshweta90 Aug 20, 2024
83c5f51
Changes for k8s
nshweta90 Aug 20, 2024
a567026
Changes for upgrade to java 11
nshweta90 Aug 25, 2024
d48d5ed
Changes for logging and temporary landing page
nshweta90 Aug 26, 2024
303e6af
Changes for docker repository name correction
nshweta90 Aug 26, 2024
12a14cf
Changes for k8s app name correction
nshweta90 Aug 26, 2024
e7f8a31
Changes for k8s app name correction
nshweta90 Aug 26, 2024
ce0b2ea
Changes for k8s app name correction
nshweta90 Aug 26, 2024
4a83503
Changes for k8s app name correction
nshweta90 Aug 26, 2024
93bedf6
Changes for response handling and config reading
nshweta90 Aug 28, 2024
9c28cba
Changes for response handling and config reading
nshweta90 Aug 28, 2024
d4bc89f
Changes for response handling and config reading
nshweta90 Aug 28, 2024
10cba03
Changes for response handling and config reading
nshweta90 Aug 29, 2024
0622fd4
Changes for response handling and config reading
nshweta90 Sep 3, 2024
375ffeb
Changes for response handling and config reading
nshweta90 Sep 3, 2024
9f1c575
Changes for response handling and config reading
nshweta90 Sep 3, 2024
a3b3216
Changes for response handling and config reading
nshweta90 Sep 3, 2024
bc2835f
Changes for id matching
nshweta90 Sep 3, 2024
eb7f677
Changes for id matching
nshweta90 Sep 3, 2024
12be446
Changes for id matching
nshweta90 Sep 3, 2024
d205f4a
Changes for id matching
nshweta90 Sep 3, 2024
609ef6e
Changes for id matching
nshweta90 Sep 3, 2024
d69f50a
Changes for id matching
nshweta90 Sep 3, 2024
0278708
Changes for id matching
nshweta90 Sep 3, 2024
ffde511
Merge pull request #1 from europeana/EA-3884-Upgrade-to-docker-and-se…
nshweta90 Sep 19, 2024
a751aeb
Changes for common domain name config
nshweta90 Sep 19, 2024
cdd9528
Changes for overriding properties
nshweta90 Sep 19, 2024
135f58c
Changes for overriding properties
nshweta90 Sep 19, 2024
422e20f
Changes for conf defaults and cleanup
nshweta90 Sep 25, 2024
fdd3f78
Changes for conf defaults and cleanup
nshweta90 Sep 27, 2024
dc702d1
Changes for conf defaults and cleanup
nshweta90 Sep 30, 2024
b622039
Changes for conf defaults and cleanup
nshweta90 Oct 1, 2024
93d333f
Changes for conf defaults and cleanup
nshweta90 Oct 1, 2024
9330fe5
Changes for conf defaults and cleanup
nshweta90 Oct 1, 2024
53bc234
Changes for conf defaults and cleanup
nshweta90 Oct 1, 2024
7cce55c
Merge pull request #2 from europeana/EA-3930-Set-Up-APM-Agent
P-Ehlert Oct 7, 2024
0799d22
Update application.conf
nshweta90 Oct 8, 2024
1133b2e
Update application.conf
nshweta90 Oct 8, 2024
ec2a87a
Changes for using different base image of jdk 17
nshweta90 Oct 10, 2024
9d9a279
Changes for endpoint for liveness readiness probe
nshweta90 Oct 10, 2024
288cfa3
Changes for endpoint for liveness readiness probe
nshweta90 Oct 28, 2024
dd01e9b
Changes for adding charset for data responses
nshweta90 Oct 29, 2024
e28103b
Update messages.properties
hugomanguinhas Oct 31, 2024
bee6575
Update messages_ca.properties
hugomanguinhas Oct 31, 2024
7b6e3bf
Update messages_de.properties
hugomanguinhas Oct 31, 2024
093685c
Update messages_es.properties
hugomanguinhas Oct 31, 2024
cf26d67
Update messages_et.properties
hugomanguinhas Oct 31, 2024
63c08e5
Update messages_fi.properties
hugomanguinhas Oct 31, 2024
70f4700
Update messages_fr.properties
hugomanguinhas Oct 31, 2024
ddfdebf
Update messages_hi.properties
hugomanguinhas Oct 31, 2024
6063735
Update messages_hr.properties
hugomanguinhas Oct 31, 2024
59a47d5
Update messages_it.properties
hugomanguinhas Oct 31, 2024
59b772b
Update messages_lt.properties
hugomanguinhas Oct 31, 2024
2382c6c
Update messages_nl.properties
hugomanguinhas Oct 31, 2024
af16a4c
Update messages_pl.properties
hugomanguinhas Oct 31, 2024
3b77b27
Update messages_sv-FI.properties
hugomanguinhas Oct 31, 2024
3409118
Merge pull request #5 from europeana/add-nofollow
nshweta90 Nov 7, 2024
0e6c885
Changes for probe settings
nshweta90 Nov 8, 2024
41cabe7
Merge pull request #6 from europeana/EA-3980-RightsStatements-Livenes…
P-Ehlert Nov 20, 2024
7b946ff
Merge pull request #3 from europeana/EA-3974--fix-log-format
P-Ehlert Nov 20, 2024
2a5608e
Merge pull request #4 from europeana/EA-3927-Issue-with-char-encoding
P-Ehlert Nov 20, 2024
6986093
Changes for probe settings
nshweta90 Nov 25, 2024
3bcf99a
Changes for param validation
nshweta90 Nov 27, 2024
e06fece
Changes for param validation
nshweta90 Nov 28, 2024
b2f0a5f
Merge pull request #7 from europeana/EA-4000-filter-and-validate-params
P-Ehlert Nov 28, 2024
e55df22
Changes for language param validation
nshweta90 Feb 6, 2025
0f890dd
Changes for language param validation
nshweta90 Feb 10, 2025
1722964
Changes for language param validation
nshweta90 Feb 12, 2025
005eb10
Changes for language param validation
nshweta90 Feb 12, 2025
a59b3e1
Changes for language param validation
nshweta90 Feb 13, 2025
6aa7f1e
Changes for language param validation
nshweta90 Feb 13, 2025
02a4156
Merge pull request #8 from europeana/EA-4052-language-param-validation
P-Ehlert Feb 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
498 changes: 276 additions & 222 deletions app/controllers/Application.java

Large diffs are not rendered by default.

8 changes: 5 additions & 3 deletions app/modules/LayoutProviderModule.java
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
package modules;

import com.google.inject.AbstractModule;
import play.Configuration;
import play.Environment;
import play.api.Configuration;
import play.api.Environment;
import play.api.Play;
import services.LayoutProvider;

/**
Expand All @@ -21,7 +22,8 @@ public LayoutProviderModule(Environment environment, Configuration configuration

protected void configure() {

String vocabVersion = configuration.getString("layout.provider");
String vocabVersion = configuration.underlying().getString("layout.provider");

try {
Class<? extends LayoutProvider> bindingClass = environment.classLoader().loadClass(vocabVersion)
.asSubclass(LayoutProvider.class);
Expand Down
6 changes: 3 additions & 3 deletions app/modules/VocabProviderModule.java
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package modules;

import com.google.inject.AbstractModule;
import play.Configuration;
import play.Environment;
import play.api.Configuration;
import play.api.Environment;
import services.VocabProvider;

/**
Expand All @@ -21,7 +21,7 @@ public VocabProviderModule(Environment environment, Configuration configuration)

protected void configure() {

String vocabVersion = configuration.getString("vocab.provider");
String vocabVersion = configuration.underlying().getString("vocab.provider");
try {
Class<? extends VocabProvider> bindingClass = environment.classLoader().loadClass(vocabVersion)
.asSubclass(VocabProvider.class);
Expand Down
20 changes: 11 additions & 9 deletions app/services/DirectoryVocabProvider.java
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,12 @@
import com.google.inject.Inject;
import com.hp.hpl.jena.rdf.model.Model;
import com.hp.hpl.jena.rdf.model.ModelFactory;
import com.typesafe.config.Config;
import com.typesafe.config.ConfigList;
import com.typesafe.config.ConfigValue;
import java.util.List;
import org.apache.jena.riot.Lang;
import play.Configuration;
import play.api.Configuration;
import play.Logger;

import java.io.File;
Expand All @@ -13,6 +17,7 @@
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import scala.Option;

/**
* Created by fo on 10.03.16.
Expand All @@ -23,17 +28,16 @@ public class DirectoryVocabProvider implements VocabProvider {

@Inject
public DirectoryVocabProvider(Configuration configuration) {

Configuration source = configuration.getConfig("source.data");
Config source = configuration.underlying().getConfig("source.data");
String sourceDir = source.getString("dir");
Configuration formats = source.getConfig("formats");
Config formats = source.getConfig("formats");
Path sourcePath = new File(sourceDir).isAbsolute()
? Paths.get(new File(sourceDir).getPath())
: Paths.get(ClassLoader.getSystemResource(sourceDir).getPath());

for (String format : formats.asMap().keySet()) {
String ext = formats.getConfig(format).getString("ext");
String lang = formats.getConfig(format).getString("lang");
for (String formatName : formats.root().keySet()) {
String ext = formats.getConfig(formatName).getString("ext");
String lang = formats.getConfig(formatName).getString("lang");
try (DirectoryStream<Path> files = Files.newDirectoryStream(sourcePath, "*".concat(ext))) {
for (Path file : files) {
vocab.read(Files.newInputStream(file), null, lang);
Expand All @@ -42,12 +46,10 @@ public DirectoryVocabProvider(Configuration configuration) {
Logger.error(e.toString());
}
}

}

@Override
public Model getVocab() {
return vocab;
}

}
14 changes: 7 additions & 7 deletions app/services/GitLayoutProvider.java
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import org.apache.jena.atlas.RuntimeIOException;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import play.Configuration;
import play.api.Configuration;
import play.Logger;

import java.io.File;
Expand All @@ -34,7 +34,7 @@ public class ResourceTemplateLoader extends URLTemplateLoader {

protected URL getResource(final String location) throws IOException {

File localPath = new File(gitSource.getString("local"), location.substring(1));
File localPath = new File(gitSource.underlying().getString("local"), location.substring(1));
Logger.debug("Fetching " + localPath.toURI().toURL());
return localPath.toURI().toURL();

Expand All @@ -45,14 +45,14 @@ protected URL getResource(final String location) throws IOException {
@Inject
public GitLayoutProvider(Configuration configuration) {

gitSource = configuration.getConfig("source.site.git");
gitSource = (Configuration) configuration.underlying().getConfig("source.site.git");

try {
Logger.debug("Checking out template branch ".concat(gitSource.getString("branch")));
File localPath = new File(gitSource.getString("local"));
Logger.debug("Checking out template branch ".concat(gitSource.underlying().getString("branch")));
File localPath = new File(gitSource.underlying().getString("local"));
FileUtils.deleteDirectory(localPath);
try (Git git = Git.cloneRepository().setURI(gitSource.getString("remote")).setDirectory(localPath).call()) {
git.checkout().setName(gitSource.getString("branch")).call();
try (Git git = Git.cloneRepository().setURI(gitSource.underlying().getString("remote")).setDirectory(localPath).call()) {
git.checkout().setName(gitSource.underlying().getString("branch")).call();
git.getRepository().close();
}
} catch (IOException | GitAPIException e) {
Expand Down
33 changes: 16 additions & 17 deletions app/services/GitVocabProvider.java
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@
import com.google.inject.Singleton;
import com.hp.hpl.jena.rdf.model.Model;
import com.hp.hpl.jena.rdf.model.ModelFactory;

import com.typesafe.config.Config;
import com.typesafe.config.ConfigValue;
import java.util.Map.Entry;
import java.util.Optional;
import java.util.Set;
import org.apache.commons.io.FileUtils;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
Expand All @@ -16,13 +22,14 @@
import org.eclipse.jgit.revwalk.RevWalk;
import org.eclipse.jgit.treewalk.TreeWalk;
import org.eclipse.jgit.treewalk.filter.PathSuffixFilter;
import play.Configuration;
import play.api.Configuration;
import play.Logger;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import scala.Option;


/**
Expand All @@ -37,29 +44,28 @@ public class GitVocabProvider implements VocabProvider {
@Inject
public GitVocabProvider(Configuration configuration) {

Configuration source = configuration.getConfig("source.data");
Configuration gitSource = source.getConfig("git");
Configuration formats = source.getConfig("formats");

Config source = configuration.underlying().getConfig("source.data");
Config gitSource = source.getConfig("git");
Config formats = source.getConfig("formats");
try {

String remoteURL = gitSource.getString("remote");
File localPath = new File(gitSource.getString("local"));
File localPath = new File(String.valueOf(gitSource.getString("local")));
FileUtils.deleteDirectory(localPath);

try (Git git = Git.cloneRepository().setURI(remoteURL).setDirectory(localPath).call()) {

Repository repository = git.getRepository();
ObjectId lastCommitId = repository.resolve(gitSource.getString("rev"));
ObjectId lastCommitId = repository.resolve(String.valueOf(gitSource.getString("rev")));

try (RevWalk revWalk = new RevWalk(repository)) {

RevCommit commit = revWalk.parseCommit(lastCommitId);
RevTree tree = commit.getTree();

for (String format : formats.asMap().keySet()) {
String ext = formats.getConfig(format).getString("ext");
String lang = formats.getConfig(format).getString("lang");
for (String formatName : formats.root().keySet()) {
String ext = formats.getConfig(formatName).getString("ext");
String lang =formats.getConfig(formatName).getString("lang");

try (TreeWalk treeWalk = new TreeWalk(repository)) {
treeWalk.addTree(tree);
Expand All @@ -74,19 +80,12 @@ public GitVocabProvider(Configuration configuration) {
vocab.read(bais, null, lang);
}
}

}

revWalk.dispose();

}

}

} catch (IOException | GitAPIException e) {

Logger.error(e.toString());

}

}
Expand Down
4 changes: 0 additions & 4 deletions app/services/HttpLayoutProvider.java
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,11 @@ public URLTemplateLoader getTemplateLoader() {
}

public class HttpTemplateLoader extends URLTemplateLoader {

protected URL getResource(final String location) throws IOException {

Logger.debug("Fetching " + location);
// TODO: cache templates, see e.g. https://stackoverflow.com/a/45439170
return new URL(location);

}

}

}
2 changes: 0 additions & 2 deletions app/services/LayoutProvider.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,5 @@
* Created by fo on 19.04.16.
*/
public interface LayoutProvider {

URLTemplateLoader getTemplateLoader();

}
2 changes: 0 additions & 2 deletions app/services/VocabProvider.java
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,5 @@
*/

public interface VocabProvider {

public Model getVocab();

}
12 changes: 12 additions & 0 deletions app/views/index.scala.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
@()

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Rights statements</title>
</head>
<body>
<h2>Rights statements</h2>
</body>
</html>
14 changes: 11 additions & 3 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name := "rights-app"

version := "0.1"

packageName in Universal := "rights-app-dist"
Universal / packageName := "rights-app-dist"

lazy val root = (project in file(".")).enablePlugins(PlayJava)

Expand All @@ -12,7 +12,15 @@ libraryDependencies += "org.apache.jena" % "apache-jena-libs" % "2.13.0"
libraryDependencies += "com.github.jknack" % "handlebars" % "2.2.2"
libraryDependencies += "org.eclipse.jgit" % "org.eclipse.jgit" % "4.0.1.201506240215-r"
libraryDependencies += "commons-io" % "commons-io" % "2.4"
libraryDependencies += "com.google.inject" % "guice" % "3+"
libraryDependencies += "co.elastic.apm" % "elastic-apm-agent" % "1.34.1"
libraryDependencies += guice

javaOptions in Test += "-Dconfig.file=conf/test.conf"
Test / javaOptions += "-Dconfig.file=conf/test.conf"
Universal / javaOptions ++= Seq("-Dpidfile.path=/dev/null","-Dconfig.file=conf/environment.conf")
scalaVersion := "2.13.14"

scalaVersion := "2.11.11"
enablePlugins(JavaAppPackaging,DockerPlugin)
dockerExposedPorts := Seq(9000)
dockerBaseImage := "eclipse-temurin:17-jdk"
dockerRepository := Option("docker.io/europeana")
50 changes: 29 additions & 21 deletions conf/application.conf
Original file line number Diff line number Diff line change
@@ -1,34 +1,42 @@
play.modules.enabled += "modules.VocabProviderModule"
play.modules.enabled += "modules.LayoutProviderModule"
vocab.provider = "services.GitVocabProvider"
source.data.formats.turtle.ext = ".ttl"
source.data.formats.turtle.lang = "TURTLE"
source.data.formats.json.ext = ".json"
source.data.formats.json.lang = "JSONLD"
source.data.git.remote = "https://github.com/rightsstatements/data-model.git"

source.data.git.remote = "https://github.com/europeana/rightsstatements-data-model.git"
source.data.git.local = "/tmp/data-model"
source.data.git.rev = "master"
layout.provider = "services.HttpLayoutProvider"
source.site.git.remote = "https://github.com/rightsstatements/rightsstatements.github.io.git"
source.site.git.local = "/tmp/rightsstatements.github.io"
source.site.git.branch = "origin/production-pages"
source.site.git.remote = "https://github.com/europeana/rightsstatements-website.git"
source.site.git.local = "/tmp/rightsstatements-website.git"
source.site.git.branch = "origin/master"
default.mime = "application/json"
default.parser = "JSON-LD"
params.InC-OW-EU = "relatedURL"
params.NoC-CR = "relatedURL"
params.NoC-OKLR = "relatedURL"
params.NoC-NC = "date"
parser.text/turtle = "TURTLE"
parser.application/json = "JSON-LD"
parser.application/ld"+"json = "JSON-LD"
parser."*"/"*" = "JSON-LD"
extension.text/turtle = "ttl"
extension.application/json = "json"
extension.application/ld"+"json = "jsonld"
extension."*"/"*" = "json"
queries.vocab = "CONSTRUCT WHERE {?s <http://www.w3.org/2002/07/owl#versionInfo> \"%1$s\" . ?s ?p ?o}"
queries.statement = "CONSTRUCT WHERE {?s <http://www.w3.org/2002/07/owl#versionInfo> \"%1$s\" . ?s <http://purl.org/dc/elements/1.1/identifier> \"%2$s\" . ?s ?p ?o}"
queries.collection = "CONSTRUCT WHERE {<http://rightsstatements.org/vocab/collection-%1$s/%2$s/> <http://www.w3.org/2002/07/owl#versionInfo> \"%2$s\" . <http://rightsstatements.org/vocab/collection-%1$s/%2$s/> ?p ?o }"
queries.localize = "CONSTRUCT {?s ?p ?o} WHERE {?s ?p ?o . FILTER(!isLiteral(?o) || lang(?o) = \"\" || langMatches(lang(?o), \"%1$s\" ))}"

queries{
vocab = "CONSTRUCT WHERE {?s <http://www.w3.org/2002/07/owl#versionInfo> \"%1$s\" . ?s ?p ?o}"
statement = "CONSTRUCT WHERE {?s <http://www.w3.org/2002/07/owl#versionInfo> \"%1$s\" . ?s <http://purl.org/dc/elements/1.1/identifier> \"%2$s\" . ?s ?p ?o}"
collection = "CONSTRUCT WHERE {<http://rightsstatements.org/vocab/collection-%1$s/%2$s/> <http://www.w3.org/2002/07/owl#versionInfo> \"%2$s\" . <http://rightsstatements.org/vocab/collection-%1$s/%2$s/> ?p ?o }"
localize = "CONSTRUCT {?s ?p ?o} WHERE {?s ?p ?o . FILTER(!isLiteral(?o) || lang(?o) = \"\" || langMatches(lang(?o), \"%1$s\" ))}"
}

languages.available = "en ca de es et fi fr hi hr it lt nl pl"
source.site.http = "https://rightsstatements.org"
source.site.local="/resources"
source.data.formats{
json {
ext =".json"
lang ="JSONLD"
}
turtle {
ext=".ttl"
lang="TURTLE"
}
}
play.server.pidfile.path=/dev/null
play.http.secret.key = "^Z5sUMS0j^inD7xJ4/vt0rFC?F>y5@kkdiGN:ROV5lfaP1:[ag72[S8gLD90PZWS"
play.filters.hosts {
allowed = ["."]
}
1 change: 1 addition & 0 deletions conf/environment.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include "application.conf"
Loading