Levandor

trust_list_proxy — EU LOTL/Trust List downloader + nginx serving proxy

12 min read

trust_list_proxy

Two-part component: (1) a single-shot Kotlin/JVM batch job that downloads and cryptographically validates the EU List of Trusted Lists (LOTL) and all member-state Trust Lists (TLs) via the EU DSS library, writing the validated files into a local cache directory; and (2) a thin nginx Docker image that serves that cache directory (plus a generic forward-proxy) to the FaceKom PDF Service at runtime. It is NOT a long-running service — main() runs to completion and exits (src/main/kotlin/Main.kt:7-15).

No README in this repo

There is no README.md/README.adoc anywhere in the tree (verified by full git ls-tree -r). The authoritative prose docs are the two Hungarian-language AsciiDoc files under documentation/ (DSS-proxy.adoc, LOTLDownload.adoc) plus the PlantUML sequence diagram LOTLDownload.puml. This doc draws the “role in FaceKom” facts from those.

Base branch: master. HEAD at doc time: 0e3f0fe “Merge pull request #7 from TechTeamer/docs/update-proxy” (2024-07-23, agostonromhanyi). Project version 0.1.0 (gradle.properties:2).

1. Purpose & role in FaceKom

From documentation/DSS-proxy.adoc and documentation/LOTLDownload.adoc (verified, translated):

  • eIDAS trust anchoring for PDF signature validation. The EU LOTL (https://ec.europa.eu/tools/lotl/eu-lotl.xml) lists every member state’s Trust List; each TL enumerates the qualified trust-service-provider CA certificates for that country. The PDF Service needs these CA certs to trace a signed PDF back to an eIDAS Trust Anchor; if a TL is invalid, all its CAs are treated as untrusted and the PDF cannot be validated (LOTLDownload.adoc:24).
  • Why a proxy and not direct download. DSS-proxy.adoc:16-22 describes the proxy’s two runtime jobs: (a) serve the tl_cache contents as a JSON directory listing + individual files so the PDF Service can fetch normalized-URL-named TL files, and (b) act as the single controlled internet egress for the PDF Service — every outbound request from pdfservice (incl. OCSP/CRL revocation lookups) is routed through this proxy.
  • DSS URL normalization contract. DSS-proxy.adoc:7 documents the file-naming convention: the DSS library normalizes a TL URL such as https://hun-domain.hu/hun-trustlist.xml to https___hun_domain_hu_hun_trustlist_xml. Both the downloader (this repo) and the PDF Service’s DSS use the identical normalization, so files land under matching names — that is what makes the nginx cache lookup work (e.g. …/tl_cache/https___ec_europa_eu_tools_lotl_eu_lotl_xml).
  • Three integration options are documented (DSS-proxy.adoc:1-7): full tl_cache download + offline run inside PDF Service; collect X509 certs to a base64 file handed to PDF Service; or PDF Service downloads/parses TLs from the proxy. The repo implements the cache-and-serve path.

See sibling architecture-overview for where the PDF Service sits. This repo is a leaf dependency feeding trust material to it.

2. Tech stack & runtime

AspectValueSource
LanguageKotlin (JVM)build.gradle.kts:2 kotlin("jvm") version "1.9.23"
JVM toolchainJava 18build.gradle.kts:38-40 jvmToolchain(18)
Build toolGradle 8.5 (Kotlin DSL)gradle/wrapper/gradle-wrapper.properties:4
Packagingapplication plugin + fat jarbuild.gradle.kts:4,42-50
App name / maintrust-list-proxy / com.facekom.tlproxy.MainKtbuild.gradle.kts:29-32
Groupcom.facekom.trustlistproxybuild.gradle.kts:7
Config formatYAML via SnakeYAMLbuild.gradle.kts:17, ConfigurationBuilder.kt:3
LoggingSLF4J + Logback (classic 1.4.14)build.gradle.kts:16, logback.xml
Serving layernginx:latest Docker imagedocker/nginx_proxy/Dockerfile:1

Core domain dependency — EU DSS (Digital Signature Service) 6.0 (build.gradle.kts:19-26):

  • eu.europa.ec.joinup.sd-dss:dss-tsl-validation:6.0 — the Trust List validation job engine.
  • eu.europa.ec.joinup.sd-dss:dss-service:6.0 — HTTP data loaders (CommonsDataLoader, FileCacheDataLoader).
  • eu.europa.ec.joinup.sd-dss:dss-utils-apache-commons:6.0.
  • BouncyCastle bcprov-jdk18on / bcpkix-jdk18on pinned to 1.78.1, with the transitive BC from dss-tsl-validation explicitly excluded (build.gradle.kts:18-23) — comment cites CVE-2024-34447 and CVE-2024-29857 in the DSS-shipped BC version.

DSS 6.0 requires Java 17+; this project targets Java 18. The release pipeline runs the jar on the openjdk:18 Docker image (jenkins/release.Jenkinsfile:74).

3. Build & run

This is a Gradle project — there is no package.json (it is not a Node service). The closest analogue to “scripts” are the Gradle tasks defined/customized in build.gradle.kts:

Gradle task / commandEffectSource
./gradlew buildStandard Gradle build; check is wired to also run OWASP dependencyCheckAnalyze (build.gradle.kts:54-56). CI uses clean build -x test -x check.jenkins/ci.Jenkinsfile:14
./gradlew checkRuns tests + OWASP dependency-check (build.gradle.kts:55). Fails build on CVSS ≥ 6.6 (build.gradle.kts:67); scans runtimeClasspath; emits XML+HTML reports (build.gradle.kts:64-70).build.gradle.kts
./gradlew testJUnit 5 platform (useJUnitPlatform(), build.gradle.kts:34-36).build.gradle.kts
./gradlew jarBuilds fat jar — bundles every compileClasspath entry via zipTree, Main-Class=com.facekom.tlproxy.MainKt, duplicatesStrategy=INCLUDE (build.gradle.kts:42-50). Output: build/libs/trust-list-proxy-0.1.0.jar.build.gradle.kts, jenkins/release.Jenkinsfile:84
./gradlew runProvided by the application plugin; runs MainKt. (Plugin applied at build.gradle.kts:4,52.)inferred-from-plugin
./gradlew printVersionCustom task: prints project.version to stdout (build.gradle.kts:58-62). Used for CI versioning.build.gradle.kts
./gradlew dependencyCheckAnalyzeOWASP scan task (from org.owasp.dependencycheck plugin v8.2.1, build.gradle.kts:3).build.gradle.kts

Run the batch job (downloader):

java -jar build/libs/trust-list-proxy-0.1.0.jar

main() loads application.yml from the classpath, runs TrustListDownloader(...).downloadLists(), logs “Trust list download complete.”, exits (Main.kt:7-15).

Build & run the serving image (nginx):

  • Dockerfile: docker/nginx_proxy/Dockerfile — FROM nginx:latest AS NGINX; copies tl_cache/ → /workspace/tl_cache/ and default.conf → /etc/nginx/conf.d (so the cache must be populated by the batch job before the image is built).
  • Release pipeline does: docker build ./docker/nginx_proxy/ -t tl-server-nginx (jenkins/release.Jenkinsfile:116).
  • IntelliJ run config tlproxy-server deploys image tlproxy-docker:latest as container tlproxy-server with --network "host" (.run/tlproxy-server.run.xml).

Two Dockerfiles? No — only ONE.

There is exactly one Dockerfile in the repo: docker/nginx_proxy/Dockerfile. There is no Dockerfile that builds/runs the Kotlin jar — the jar is run directly on an openjdk:18 image in the pipeline (jenkins/release.Jenkinsfile:71-84), not packaged into its own image.

4. Top-level structure

PathWhat it isVerified
src/main/kotlin/Kotlin source: Main.kt + com/facekom/tlproxy/{configuration,logging,service}. 5 source files total.listed + read
src/main/resources/application.yml, logback.xml, and lotl-certs/ (8 PEM certs + dss-keystore.p12).listed + read
src/test/kotlin/3 test files: 2 unit (configuration, service) + 1 functional/LotlCertificateTest.kt.listed + read
src/test/resources/config-test.yml (config-loading fixture).read
docker/nginx_proxy/nginx serving image: Dockerfile, default.conf, .dockerignore. Runtime cache goes in tl_cache/ (gitignored, .gitignore:44).read all
documentation/DSS-proxy.adoc, LOTLDownload.adoc (Hungarian prose), LOTLDownload.puml (sequence diagram).read all
jenkins/3 pipelines: ci.Jenkinsfile, periodic-check.Jenkinsfile, release.Jenkinsfile.read all
gradle/wrapper/Gradle 8.5 wrapper jar + properties.read properties
.run/IntelliJ run configs: Debug.run.xml (remote JVM debug, port 5005), tlproxy-server.run.xml (docker deploy).read both
build.gradle.kts, settings.gradle.kts, gradle.properties, gradlew, gradlew.batGradle build + wrapper scripts.read

5. Helper scripts under bin/

No first-party bin/ directory exists.

The only executable helper scripts are the Gradle wrapper scripts at repo root (these are stock Gradle-generated, not first-party logic):

  • gradlew — POSIX shell wrapper that bootstraps Gradle 8.5 (header: “Copyright © 2015-2021 the original authors”, gradlew:1-4).
  • gradlew.bat — Windows batch equivalent.

.gitignore:27-29 explicitly ignores bin/ (except under src/main/src/test), and none is tracked. There are no custom shell/python/node helper scripts in this repo.

6. Key modules / source files

All first-party Kotlin (5 files) — read in full:

src/main/kotlin/Main.kt (entrypoint, 15 lines): loads application.yml, constructs TrustListDownloader, calls downloadLists(), logs progress. Linear, no args, no loop.

src/main/kotlin/com/facekom/tlproxy/service/TrustListDownloader.kt (the core, ~67 lines): builds and runs a DSS TLValidationJob. Key behavior:

  • downloadLists() → createTrustListValidationJob().offlineRefresh() (:22-24). Note it calls offlineRefresh(), but the online data loader is what actually hits the network (DSS semantics: offlineRefresh synchronizes the in-memory model + cache using the configured loaders).
  • createTrustListValidationJob() (:26-34): TrustedListsCertificateSource, the LOTL source, online + offline data loaders, and ExpirationAndSignatureCheckStrategy() as the synchronization strategy (i.e. drop expired/invalid TLs — corroborated by LOTLDownload.adoc:22 and LOTLDownload.puml:51-53).
  • createLotlSource() (:36-46): LOTLSource with url = configuration.listOfTrustListsUrl, isPivotSupport = true, EU LOTL/TL predicates (TLPredicateFactory.createEULOTLPredicate() / createEUTLPredicate()), OfficialJournalSchemeInformationURI(...) for signing-cert announcement, and GrantedTrustService() trust-service predicate (only “granted” services).
  • createLotlCertificateSource() (:48-50): loads KeyStoreCertificateSource(File("src/main/resources/lotl-certs/dss-keystore.p12"), "PKCS12", <password>.toCharArray()). Path is hard-coded relative to CWD (see gotcha #10).
  • createOnlineLoader() (:52-58): FileCacheDataLoader with expiration 0, real CommonsDataLoader (internet-capable, proxy-aware), cache dir = configuration.cacheDirectory.
  • createOfflineLoader() (:60-66): FileCacheDataLoader with expiration -1 and IgnoreDataLoader (never fetches), same cache dir.

src/main/kotlin/com/facekom/tlproxy/configuration/Configuration.kt (12 lines): plain mutable POJO with 4 fields — cacheDirectory, officialJournalSchemeInformationUri, listOfTrustListsUrl, lotlKeystorePassword. KDoc notes the YAML must carry the fully-qualified class tag.

src/main/kotlin/com/facekom/tlproxy/configuration/ConfigurationBuilder.kt (11 lines): createConfigFromFile(classPathResource) — SnakeYAML Yaml().loadAs(inputStream, Configuration::class.java) from a classpath resource.

src/main/kotlin/com/facekom/tlproxy/logging/Logger.kt (5 lines): top-level var logger = SLF4J LoggerFactory.getLogger("trust-list-proxy-log").

Quirky package nesting

Logger.kt declares package com.facekom.tlproxy.com.facekom.tlproxy.logging (doubled segment), and Main.kt:3 imports it as com.facekom.tlproxy.com.facekom.tlproxy.logging.logger. The physical path is .../tlproxy/logging/Logger.kt — the package name does not match the directory. This compiles (Kotlin permits it) but is clearly an unintended copy-paste artifact.

7. Configuration

Primary config: src/main/resources/application.yml (loaded from classpath by createConfigFromFile("application.yml"), Main.kt:9). Fields (verified values):

KeyDefault valueMeaning
cacheDirectory"docker/nginx_proxy/tl_cache/"Where DSS writes downloaded TL files. Relative to CWD (comment: “base will be the project’s root … and the jar’s directory”). Note this default writes straight into the nginx image’s copy source.
officialJournalSchemeInformationUrihttps://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.C_.2019.276.01.0001.01.ENGOJ URL identifying the LOTL signing-cert keystore; builds the keystore from pivots before that URL.
listOfTrustListsUrlhttps://ec.europa.eu/tools/lotl/eu-lotl.xmlThe EU LOTL XML.
lotlKeystorePassword"facekom"Password for dss-keystore.p12. Comment (application.yml:18): the certs are public, used only for signature validation, not signing — so the in-repo password is intentional/non-sensitive.

The config object is not validated. There is no AJV/getconfig/JSON-schema layer (those are Node patterns; absent here). SnakeYAML loadAs requires the YAML to deserialize into Configuration — application.yml has no explicit !!com.facekom... type tag despite the KDoc in Configuration.kt:6 saying one "has to be part of the yaml file". It works because loadAs(..., Configuration::class.java) supplies the root type; the test fixture config-test.yml is likewise untagged.

Logging config: src/main/resources/logback.xml — single STDOUT ConsoleAppender, pattern %d [%thread] %-5level %logger %msg%n, root level warn (so the app’s own INFO “Loading configuration…” lines from Main.kt are suppressed at the root level unless the trust-list-proxy-log logger is raised — they are not configured, so they do not print).

Env vars: none read by application code (no System.getenv in the Kotlin sources). The only env-style values are in CI: NGINX_DOCKER_IMAGE_NAME=tl-server-nginx (release.Jenkinsfile:8).

nginx config: docker/nginx_proxy/default.conf —

  • root /workspace, listens on port 80 (IPv4+IPv6).
  • location /tl_cache/ → autoindex on; autoindex_format json; + sendfile (this is the JSON directory listing the PDF Service consumes, per DSS-proxy.adoc:18).
  • location / → generic forward proxy: resolver 8.8.8.8; and proxy_pass http://$http_host$uri$is_args$args; (this is the “controlled internet egress” path for the PDF Service, DSS-proxy.adoc:22).

8. Tests

  • Framework: JUnit 5 (Jupiter), kotlin("test"). useJUnitPlatform() (build.gradle.kts:34-36). Run with ./gradlew test.
  • Location: src/test/kotlin/. Three files:
TestWhat it checksNotes
configuration/ConfigurationBuilderKtTest.ktcreateConfigFromFile("config-test.yml") loads all 4 fields correctly.Pure unit, fast. Fixture: src/test/resources/config-test.yml.
service/TrustListDownloaderTest.ktFull LOTL download writes ≥1 file into a temp cache.@Disabled (:15) — “downloading the entire lotl takes a lot of time and it will be part of a pipeline anyway.” Cleans temp dir in @AfterEach.
functional/LotlCertificateTest.ktRuns LOTLWithPivotsAnalysis and asserts every cached validation DTO isValid — i.e. the in-repo Official Journal certs/keystore still validate the live LOTL pivot chain.Network-dependent. Class comment (:21-26) documents the manual cert-refresh procedure when it fails (download new certs from eur-lex, rebuild keystore).

CI invocations:

  • ci.Jenkinsfile runs Build → ./gradlew check (vuln scan) → ./gradlew test, publishing GitHub checks.
  • periodic-check.Jenkinsfile runs only ./gradlew :test --tests "LotlCertificateTest" and ./gradlew check; on cert-test failure it emails agoston.romhanyi@facekom.com to update the certs (:17-20); on vuln scan it archives + publishes the dependency-check report. This is the “planned daily check” described in LOTLDownload.adoc:26-31.

9. Dependencies on other FaceKom services

Evidenced relationships (no RabbitMQ / DB / Redis anywhere in this repo — verified absent):

  • PDF Service (consumer, HTTP). Documented in DSS-proxy.adoc / LOTLDownload.adoc. The PDF Service (a) fetches the tl_cache JSON listing + individual TL files over HTTP from this nginx (location /tl_cache/), and (b) routes all its outbound internet traffic (incl. OCSP/CRL) through this nginx as a forward proxy (location /). The PDF Service configures its DSS ProxyConfig host to this proxy’s URL (DSS-proxy.adoc:12). This repo contains no code calling the PDF Service — the dependency is inbound (PDF Service → proxy).
  • External (non-FaceKom) endpoints the batch job calls directly over HTTPS: ec.europa.eu/tools/lotl/eu-lotl.xml (LOTL), the LOTL pivot XMLs, each member-state TL URL listed in the LOTL, and eur-lex.europa.eu (OJ cert reference). See LOTLDownload.puml for the full sequence.

The PDF Service lives in a separate repo — see architecture-overview / vuer_oss for the broader service map. This component is a one-way trust-material supplier.

10. Verified gotchas

Hard-coded keystore path breaks if CWD ≠ repo root

TrustListDownloader.kt:49 (and LotlCertificateTest.kt:58) load the keystore via File("src/main/resources/lotl-certs/dss-keystore.p12") — a path relative to the current working directory, not the classpath. The fat jar bundles the resource, but this code reads from the source tree path, so the jar must be launched from a dir that has src/main/resources/... beside it. The release pipeline mounts the workspace into /usr/project/ and runs the jar from there (release.Jenkinsfile:75,84).

cacheDirectory default writes into the Docker build context

Default cacheDirectory: "docker/nginx_proxy/tl_cache/" (application.yml:5) means the batch job’s output is exactly what the nginx Dockerfile later COPYs. Order matters: run the jar first, then build the nginx image. tl_cache/* is gitignored (.gitignore:44), so the dir may not exist on a fresh clone until the job runs — the pipeline verifies non-empty before building (release.Jenkinsfile:94-108).

App INFO logs are silent by default

logback.xml root level is warn; the trust-list-proxy-log logger is not separately configured, so the logger.info(...) calls in Main.kt do not appear on stdout. Raise the level to see progress.

release.Jenkinsfile is partially stubbed / inconsistent

The Build/Check/Test stages are commented out (release.Jenkinsfile:12-55). The verify stage greps /usr/jar/docker/nginx_proxy/tl_cache/ (:100) while the jar runs against ${WORKSPACE}/…/tl_cache/ mounted at /usr/project/ (:75) — the paths don’t line up. The jar version 0.1.0 is hard-coded (:84, with a // todo make versioning dynamic). The Harbor upload stage is a literal echo Implement me (:134). Treat this pipeline as WIP.

Doubled package name in Logger.kt

Package is com.facekom.tlproxy.com.facekom.tlproxy.logging (see §6). Cosmetic but confusing when searching imports.

No Git-LFS in this repo

There is no .gitattributes and no LFS pointers. src/main/resources/lotl-certs/dss-keystore.p12 is a real 16.8 KB PKCS#12 binary (committed directly), and cert-1.pem…cert-8.pem are plaintext PEM X509 certs (public EU Official Journal LOTL-signing certs). Safe to inspect; the .p12 is binary (do not cat).

Unverified / gaps

  • Did not run any build, test, or Docker command (read-only task) — task entries verbatim are sourced from build.gradle.kts and the Jenkinsfiles, not from execution.
  • gradlew / gradlew.bat read only at the header level (confirmed stock Gradle 8.5 wrapper); did not line-by-line audit the wrapper bootstrap logic (third-party generated).
  • gradle/wrapper/gradle-wrapper.jar is a binary (not read). dss-keystore.p12 and the 8 .pem certs: existence + first cert header confirmed; binary/cert internals not decoded.
  • The two .adoc docs and .puml are in Hungarian; “role in FaceKom” facts are my translation of them — meaning preserved, exact wording is translated.
  • DSS 6.0 library internals (TLValidationJob, LOTLSource, data loaders, predicates) are third-party and not documented here beyond how this repo configures them.
  • Exact DSS offlineRefresh() vs network behavior is asserted from the configured loaders + DSS-proxy/puml docs, not from stepping the DSS source.

Sources

Files read in full unless noted:

  • build.gradle.kts, settings.gradle.kts, gradle.properties, .gitignore
  • gradle/wrapper/gradle-wrapper.properties; gradlew (header only)
  • src/main/kotlin/Main.kt
  • src/main/kotlin/com/facekom/tlproxy/service/TrustListDownloader.kt
  • src/main/kotlin/com/facekom/tlproxy/configuration/Configuration.kt
  • src/main/kotlin/com/facekom/tlproxy/configuration/ConfigurationBuilder.kt
  • src/main/kotlin/com/facekom/tlproxy/logging/Logger.kt
  • src/main/resources/application.yml, src/main/resources/logback.xml
  • src/test/kotlin/com/facekom/tlproxy/configuration/ConfigurationBuilderKtTest.kt
  • src/test/kotlin/com/facekom/tlproxy/service/TrustListDownloaderTest.kt
  • src/test/kotlin/functional/LotlCertificateTest.kt
  • src/test/resources/config-test.yml
  • docker/nginx_proxy/Dockerfile, docker/nginx_proxy/default.conf, docker/nginx_proxy/.dockerignore
  • documentation/DSS-proxy.adoc, documentation/LOTLDownload.adoc, documentation/LOTLDownload.puml
  • jenkins/ci.Jenkinsfile, jenkins/periodic-check.Jenkinsfile, jenkins/release.Jenkinsfile
  • .run/Debug.run.xml, .run/tlproxy-server.run.xml
  • src/main/resources/lotl-certs/ (dir listing; cert-1.pem header; dss-keystore.p12 file-type only)
  • git ls-tree -r HEAD, git log -1, git branch -a (repo metadata)

Graph View