ghga/data-portal repository overview

⁠GHGA Data Portal

This is the front-end application for the GHGA data portal. It lives in the GHGA monorepo⁠ as a self-contained pnpm workspace with its own lockfile, and is not part of the uv workspace that holds the services and libraries.

• Technology stack⁠
• Local development⁠
  • The four modes⁠
  • Per-developer settings⁠
  • API requests & the backend⁠
  • Authentication⁠
• Code scaffolding⁠
• Building⁠
• Package Manager⁠
  • Dependency overrides⁠
• Linter, Commits, and Documentation⁠
  • Ease of use⁠
• Automated tests⁠
  • Test levels⁠
  • Unit-tests⁠
  • End-to-End tests⁠
    • Issues relating to headed execution⁠
• Analytics⁠
• The Architecture Matrix⁠
• AI assisted coding⁠
• References⁠
• License⁠

⁠Technology stack

This project is a single Angular application designed as a modularized frontend monolith. Major building blocks:

• Angular (version 22)
• Angular Material
• Tailwind CSS (version 4)
• Unit testing: Vitest
• E2E testing: Playwright
• API mocking in development: Mock Service Worker (MSW)

⁠Local development

To start a local development server, run this from the repository root:

just fe-dev

Once the server is running, open your browser and navigate to http://localhost:8080/. The application will automatically reload whenever you modify any of the source files.

By default, this will not use a proxy configuration; the API will be provided via the mock service worker, and the authentication will be faked as well.

The MSW handlers in src/mocks intentionally return static responses. We avoid adding complex backend logic to mocks to prevent duplicating server behaviors in the frontend codebase, which can differ or drift over time and produce misleading local test results.

The same mock layer also serves the Playwright tests, so this decision shapes what those tests are able to prove. See Automated tests⁠ for the consequences.

⁠The four modes

The API and the authentication are two independent switches, giving four modes:

Each recipe is a thin wrapper around node run.js --dev with --with-backend and --with-oidc in the corresponding combination, which you can also run directly from this directory. The run.js launcher is what generates public/config.js from the settings before handing over to ng serve, so a bare pnpm start is not equivalent — it serves the application without a fresh runtime configuration.

The settings themselves come from three places, each overriding the previous one: data-portal.default.yaml (the same defaults the production image uses), data-portal.dev.yaml (the development overrides, e.g. the Development ribbon and the ghga-dev-client OIDC client), and finally the environment, including local.env⁠.

⁠Per-developer settings

Settings that differ per developer — above all the credentials that must not be committed — belong in local.env in this directory. Nothing creates it for you and the dev server runs fine without it; copy the template when you first need one:

cp local.env.example local.env

The copy is also how you pick up keys added to local.env.example later, so it is worth a glance at the template when a new setting appears.

Every key of data-portal.default.yaml can be set there as data_portal_<key>, and everything in the file also becomes a plain environment variable, so settings that no YAML key backs (such as data_portal_ignore_cert, which proxy.conf.mjs reads) work too. A typical file looks like this:

data_portal_base_url=https://data.staging.ghga.dev
data_portal_basic_auth=USERNAME:PASSWORD
data_portal_oidc_client_id=THE_OIDC_CLIENT_ID

Variables already present in the environment win over the file, so data_portal_base_url=... just fe-dev-backend overrides it for a single run.

⁠API requests & the backend

If you want to test the application against the backend provided by the staging deployment, then run:

just fe-dev-backend

In this case, a proxy configuration will be used that proxies all API endpoints to the staging environment, while the application itself is still served by the development server. You can change the name of the staging backend via the setting data_portal_base_url; by default it will be data.staging.ghga.dev.

If the staging backend requires an additional Basic authentication, you can set it in data_portal_basic_auth.

⁠Authentication

If you want to test authentication using the real OIDC provider, then run:

just fe-dev-oidc

In this mode, the data_portal_oidc_client_id and the other OIDC settings must be set properly as required by the OIDC provider.

The provider redirects back to a registered URI that carries no port number, so this mode is not free to choose where it listens: the development server switches to HTTPS on port 443, under the hostname of the backend rather than localhost. Three things follow from that, and the just recipe takes care of the first two:

• A certificate. just fe-cert creates a local CA in .certs/ and a certificate for the backend hostname signed by it (both gitignored). Add .certs/ca-cert.pem to the trusted certificates of your web browser or host computer to avoid the warnings when loading the page. Only the CA needs to be trusted, so re-issuing the certificate for a different hostname later does not mean touching the browser again.
• Permission to bind port 443. The dev container shares the host's network namespace, and therefore its rule that only root may bind ports below 1024. Instead of relaxing that on your machine, the recipe grants cap_net_bind_service to the container's node binary once, using sudo. A container rebuild discards it and the next run grants it again.
• A hosts entry on your host computer, mapping the backend hostname to 127.0.0.1, so that the browser reaches the development server rather than the real deployment. With the default backend you then browse the application at https://data.staging.ghga.dev. (Inside the container, run.js adds the opposite entry — the hostname to the deployment's real address — so that the proxy still reaches the actual backend.)

To test against the real backend and with the real OIDC provider at the same time:

just fe-dev-backend-oidc

⁠Code scaffolding

The Angular CLI includes powerful code scaffolding tools. To generate a new component, run:

ng generate component component-name

or

ng g c component-name

for short.

For a complete list of available schematics (such as components, directives, or pipes), run:

ng generate --help

⁠Building

To build the project run:

ng build

This will compile the project and store the build artifacts in the dist/ directory. By default, the production build optimizes the application for performance and speed.

Site verification files that shall be deployed at the root path in production can be specified in the root_files setting, using file names as properties and file contents as values.

⁠Package Manager

This project uses pnpm to install dependencies, which is a replacement for the much slower npm. Run

pnpm install

to install the dependencies, or just fe-install from the repository root to install them exactly as CI does (--frozen-lockfile).

You do not normally need either: the dev container provisions both stacks when it is created, installing these dependencies along with the Chromium build that Playwright uses (.devcontainer/post-create.sh⁠). Only pnpm e2e:all needs the other browsers, which pnpm exec playwright install firefox webkit adds.

⁠Dependency overrides

There are currently no dependency overrides in use.

When a transitive dependency needs to be pinned (for example to pull a security patch that a direct dependency has not yet picked up), add a .pnpmfile.cjs file at the repository root with a readPackage hook, list the override here together with its reason and removal criteria, and run pnpm install to refresh pnpm-lock.yaml. Review such overrides from time to time and remove them once no longer necessary.

NOTE

You should not have a package-lock.json but instead a pnpm-lock.yaml. You can still use npm for running other commands or to install global packages but not to add dependencies or to install all dependencies. Configuration of pnpm overrides should be done in .pnpmfile.cjs rather than in package.json.

⁠Linter, Commits, and Documentation

The repository is set up in such a way to only allow linted commits. That means commits are blocked if they cause linter errors (currently, warnings are accepted). This ensures that code quality standards are maintained without building up technical debt that has to be fixed later on.

Since the move into the monorepo the hooks come from the root .pre-commit-config.yaml rather than from Husky (ADR-0018⁠); ESLint and Prettier run over the files you touched, out of this package's own node_modules, so they are the same versions pnpm lint and pnpm format:check use. Install them once per clone with just hooks from the repo root — the dev container already does.

To ensure deterministic behavior, the pre-commit hook does not attempt to fix linter errors. Most of the time, you will be fine by simply running ng lint --fix, which attempts to automatically fix most of the issues. If we ran that in the hook, however, you would be committing different code than the one you checked. So if you cannot commit your code, run lint fix. If that doesn't resolve all the issues (which you can see by running ng lint), resolve those issues and try again. Formatting is not auto-fixed either: run pnpm format (or just fe-format).

⁠Ease of use

For comfort, we are adding these shorthands: pnpm lint, pnpm lf (for lint --fix), and pnpm run docs (to build and serve the documentation; use pnpm run docs rather than pnpm docs, because pnpm has a built-in docs command that shadows the script). Apart from seeing the linter warnings when you (try to) commit or run the linter manually, your IDE should also show you these warnings in the code, and fixing (the auto-fixable ones) should be offered in the context menu on hover or via Ctrl-..

⁠Automated tests

⁠Test levels

Two levels of automated tests live in this repository, and a third lives elsewhere. They can prove quite different things, so it is worth choosing deliberately:

Unit tests are the default and carry most of the coverage: request shapes, state transitions, cache invalidation, rendering and event wiring all belong there. The two other levels each add something the level above cannot see, and each costs more to run.

⁠Unit-tests

We are using Vitest⁠ for unit testing in this project. If possible, the queries and matchers from the Testing Library⁠ should be used. See the documentation for the Angular Testing Library⁠ and jest-dom⁠. Note that jest-dom also supports Vitest, not just Jest.

The unit tests are not included in the linting process and can be executed separately. The following variants of running the tests are provided:

• pnpm test - run unit tests once
• pnpm test:watch - watch mode
• pnpm test:ui - interactive tests in the browser

Note: the VS Code Vitest extension runs plain vitest directly, which does not work for Angular component tests in this repository (external templateUrl/styleUrl and Angular TestBed setup are handled by the Angular test builder). Use pnpm test / pnpm test:ui instead. See also this issue⁠.

⁠End-to-End tests

We are using Playwright⁠ for end-to-end (e2e) testing in this project. See the documentation for Playwright⁠ for details.

"End-to-end" is admittedly a misnomer here. These tests do not reach the actual backend and database: the API is served by the same MSW mocks⁠ that back the development server, and those return static responses by design. What is covered is the frontend from the browser inwards — everything above the network boundary. Comprehensive end-to-end tests for real backend behavior are maintained in the separate GHGA archive test bed repository.

Within that boundary, however, these tests are considerably more real than unit tests, and catch a class of problems unit tests structurally cannot:

• The application really boots. It starts from app.config.ts with the actual providers, routes lazy-load their real chunks, and guards run. Unit tests assemble a TestBed per spec and never exercise that configuration.
• Requests are really issued. They travel the whole HttpClient chain, including the caching and CSRF interceptors. Unit tests use HttpTestingController, which replaces the backend and only sees the interceptors a spec explicitly provides, so an interceptor misconfiguration is invisible there.
• The real services run. Unit tests replace services with hand-written mocks, which can silently drift from the API of the service they stand in for. Here the actual implementations are used, against each other.
• It is a real browser. Layout, focus handling, Angular Material overlays (dialogs, menus, snack bars) and view transitions all behave as they do for users, none of which jsdom reproduces. pnpm e2e:all additionally runs across the configured browsers.
• Journeys span components and pages. Navigation, shared services and cross-page state are exercised together rather than one unit at a time.
• The expected API contract is checked. Mock handlers are registered per URL and query parameters, so a request the application makes that no handler matches surfaces immediately.

What they cannot show is anything that depends on the backend actually changing state, because the mocks answer identically before and after a mutation. Do not write a test here that performs a change and then asserts that the change is reflected — it could only assert that a request was sent, which a unit test does more precisely and far more cheaply. Such flows belong in the archive test bed, which is more expensive to run.

Treat the tests in this repository as a fast smoke layer: keep them few and cheap, so they stay useful for quick feedback during frontend development.

They need only the development server — the API is mocked in the browser — so the demo platform (just up in the monorepo) is not required for them and only competes for memory. If a run hangs with no output at all, that competition is the first thing to check; just down frees it. playwright.config.ts prints a warning when memory is already low.

• pnpm e2e - run e2e-tests in headless mode on Chromium (fast local default)
• pnpm e2e:all - run e2e-tests in headless mode on all configured browsers using --workers=2
• pnpm e2e:headed - run e2e tests in headed mode
• pnpm e2e:debug - run e2e tests in headed mode with Playwright inspector
• pnpm e2e:report - open HTML report for e2e tests

Worker configuration:

• Default is 1 worker for stability.
• For faster local runs with the same command, do export PLAYWRIGHT_WORKERS=<number of workers>.
• If tests get flaky, lower the value (or return to 1).

Stopping early:

• A local run stops at the first failure, which is usually what you want while fixing something.
• CI stops after 5, so that a single broken test can be reported for each of the three browsers rather than aborting the remaining tests after the first of them. Beyond that the suite is broken enough that stopping saves more than it hides.

Timeouts:

• The per-test timeout is raised to 60s and the expect timeout to 10s, well above the Playwright defaults. CI runners are far slower than a local machine: logging in and reaching an admin page alone costs around 15 seconds there, and even a mocked request can take more than a second, which used to exhaust the default 30s budget mid-test.
• These are upper bounds for tests that are genuinely stuck, not targets. A test that only passes because of them is telling you something — check the Playwright trace before raising them further.

Recommendations for writing stable e2e tests:

• Assert stable end states (final URL, final title, final visible content), not transient intermediate states.
• Use small bounded retries for known flaky UI transitions (menu/dialog open, click-triggered navigation).
• Keep retries minimal (usually 1-2 attempts) and always retain a strict final assertion.

Note: The Playwright HTML reporter is configured to not auto-open at the end of pnpm e2e runs, so test commands terminate cleanly in CI and local terminals.

Reports are still generated in playwright-report/ and can be viewed on demand:

• Open the report file directly: playwright-report/index.html
• Or start the Playwright report server manually: pnpm e2e:report

Like for unit testing, you can also use the VS Code extension for Playwright⁠ to run tests interactively using the test explorer in the side bar. VS Code is able to support different test providers (like Vitest and Playwright) along with each other.

⁠Issues relating to headed execution

Headed execution (pnpm e2e:headed, pnpm e2e:debug, pnpm e2e:ui) needs an X11 server on the host, and a dev container that can reach it. The container does not mount one by default, because the path to mount differs per host operating system and a mount that fits one host makes container creation fail on another — see the comment above mounts in .devcontainer/devcontainer.json⁠, which is where you add it for your own host, together with a matching DISPLAY in containerEnv. Rebuild the container afterwards.

On macOS, you can use XQuartz⁠. On Windows with WSL 2, you can use the built-in WSLg as X11 server. See here⁠ if that is not working properly.

The directory /tmp/.X11-unix should exist in the container and should be mounted on the corresponding host directory, which is /mnt/wslg/.X11-unix for WSLg. If you run ls /tmp/.X11-unix, it should show X0. You may also need to add /tmp/.X11-unix to the virtual file shares in Docker Desktop under Linux, and run xhost +local:docker on the host system.