informatievlaanderen/oslo-specification-generator repository overview

⁠The specification generator

The specification generator is a collection of tools to generate different artifacts based on the output of the OSLO tool EA-to-RDF.

The tools are

• json-ld-generator.js: a tool to generate a complete json-ld context file for the output
• exampletemplate-generator.js: this tool generates a json-ld context and default payload structure for each class in the specification.
• html-generator.js: this tool generates html to be rendered by a static webpage server. The main target is Data.vlaanderen.be. The default templates to be used are found in the directory /views
• shacl-generator.js: this tool generates SHACL templates

In addition some closely related tools are:

• csv-renderer.js: this tool renders the content of a csv file according to a template generated by the exampletemplate-generator.
• pretty-print.js: a tool to sort and pretty print the content of EA-to-RDF to create more stability in the saved artifacts created by the generators.

⁠build

This is a node.js project for node >= version 10.

npm set unsafe-perm true
npm install

The easiest method is the build it as a docker container using as Dockerfile Dockerfile.circleci.

⁠Running the tools

To run any of the tools is done on the commandline as:

> node <tool>.js --help

which will give a short help about the tool.

⁠code style

In the node configuration the javascript code style is implemented in the test script using eslint.

⁠Strucutre of this repository

⁠Specification generator JS version

To minimize the impact of a different implementation language for the specification generator we will leave the templating engine untouched from an external perspective. The python implementation used Jinja2. We will use nunchucks.js for this version.

⁠necessary changes to the templates compared to the python implementation of nunjuncks.

The encountered differences between the python and javascript implementation are:

⁠absence of a variable that is undergoing filtering

When a filter (selectattr, lower, ...) is used on a non existent variable then the template will crash. The solution here is to test the presence of the variable prior to filtering:

{% if object.property %}
  {{ object.property | lower }}
{% endif %}

⁠lower() empty parameter method call not supported

In jinja the following construct is valid

# NOT SUPPORTED IN NUNJUCKS
{{ "TEST".lower()}}

for nunchucks this needs to be written as a 'true' filter such as

{{ "TEST" | lower }}

⁠the complex form of the selectattr filter is not supported

The Jinja2 selectattr filter is not supported in the complex case. The simple case is supported though; In jinja2 the following would filter out all object for which the property is set to 1

# NOT SUPPORTED IN NUNJUCKS
{{ objects | selectattr('property', 'equalto', 1)}}

The simple case of this filter is similar but wil only test for truthy values ( property in object && object.property != false ). This would look like:

{{ objects | selectattr('property')}}

To handle the complex case in nunjucks the following construct, base on groupings will work:

{% for property, objects in objects | groupby(property) %}
  {% if property == 1 %}
    # handle the filtered objects here
  {% endif %}
{% endfor %}

⁠Autoescaping is enabled

For all templates, the given variables (the values of the jsonld) will be autoescaped. This means any characters that might be interpreted (e.g. '<', '>', ...) will be replaced with a neutral string and displayed as such. Autoescaping is currently deactivated for the usage values and can be for any variable if the safe filter is added. Doing so requires all entered values to be html safe and might cause errors if they are not.

{{ entity.usage[language] | safe }}

⁠Comments on Tools

⁠translation-json-generator.js

Creates a file to translate the language aware values. The Json itself contains only the information important for a translator and the EA-Guids to trace the items back its original object in a general jsonld. The goal here is to have an easily readable file that does not need further knowledge to be translated.

⁠generate-translation-report.js

Creates a report file on a given translation Json. Checks if it is fully translated by searching for empty strings or the placeholder.

⁠jsonld-merger.js

Merges the translation Json back into the Jsonld it was created from so all information can be found in one place. Additionally, it overwrites values of the jsonld that are refined for the given language in the config (this includes the title and the template).

⁠linkeddataparser4.js

This parser is an example that adds the possibility to refer to external links in the created html. To try it out you'll need to replace the linkeddataparser3.js in the html generator with linkeddataparser4.js and use the template ap2ext_en.j2 (the example is for english only). The requirement is that the used jsonld has for all classes and properties the language-tagged attribute "externallink" (in case you want to refer to different translated links).

⁠Usage of the mu-config-generator.js

If you have no mu-project yet, you can clone this⁠ repository as a base. To make it easier you should define a port for your resource. As an example, you could add this to your docker-compose:

resource:
    image: semtech/mu-cl-resources:1.18.0
    links:
      - db:database
    volumes:
      - ./config/resources:/config
    ports:
      - "8888:80"

The mu-config-generator.js automatically creates you a mu-semtech-project configuration based on the jsonld of your specification. For that, you have to enter said jsonld, a language that is needed to access the name value and an output directory. That directory you should either set to the config->resources directory in your mu-semtech-project or copied the created files into that. After adding the files, you will have to stop, remove and restart the docker-compose.

Some decisions had been made beforehand. One of which is that, when a class is in the domain-parameter of a property, it will have said property relationship as a "has-one"-value while, if the object is in the range-parameter, it will have an inverse "has-many" connection. Each class has attributes (called "properties" in the lisp file) if there are properties in the jsonld that point to a literal. By default, these are not language tagged. If you want them to be, you can enter true for the option -s causing the attributes to be changed to a :language-string. By default they are :string. For the paths and the has-many relations, the plurals of the names are automatically created. If you come across falsely created plurals, you can add those as a rule to the file by using one of these⁠ functions. It is recommended to check the plurals to make sure they are correct, especially when it is in a non-English language. In the following there is an example on how to create an address object based on this config:

(define-resource Address ()
   :class (s-url "http://www.w3.org/2002/07/owl#Class")
   :properties `((:definition :string ,(s-prefix "sh:definition"))
                 (:name :string ,(s-prefix "sh:name"))
                 (:usage :string ,(s-prefix "sh:usage")))
   :has-one `((Location :via ,(s-url "https://sdg.semic.euAddress.address")
                        :as "Location"))
:resource-base (s-url "https://sdg.semic.euAddress")
:on-path "Addresses")

For this, you can send a post request with the following body to the port of your resource that you set in your docker-compose and the ending /Addresses (you add the on-path value here). As a header you need to define "Content-Type" as "application/vnd.api+json".

{
  "data": {
    "type": "Addresses",
    "attributes": {
      "name": "An Address",
      "definition": "A Description",
      "usage": "A Usage"
    },
    "relationships": {
      "Location": {
        "data": {
          "type": "Locations",
          "id": "5FCFCE2A873EE90008000001"
        }
      }
    }
  }
}

Assuming the ID points to a location object that already exist, this will create an address that has a tripel like this:
<AddressURI> <https://sdg.semic.euAddress.address> <LocationURI>.
If you enter true for all three string objects, causing :string to be changed to :language-string, the following will work:

{
  "data": {
    "type": "Addresses",
    "attributes": {
      "name": {
        "language": "en",
        "content": "An Address"
      },
      "definition": {
        "language": "en",
        "content": "A Description"
      },
      "usage": {
        "language": "en",
        "content": "A Usage"
      },
    "relationships": {
      "Location": {
        "data": {
          "type": "Locations",
          "id": "5FCFCE2A873EE90008000001"
        }
      }
    }
  }
}

In this case, one of the created triples would be:
<AddressURI> sh:name "An Addres"@en .
You can check your results using sparql on http://localhost:{db port}/sparql.