indigodatacloud/alien4cloud-deep repository overview

⁠Alien4Cloud DEEP Docker

Documentation version 3.0.1

Intro⁠
Administration Guide⁠
    Getting Started⁠
    Prerequisites⁠
    Docker Container⁠
        Parameters⁠
        Variables⁠
    Settings Manager⁠
    SSL⁠
    Deployment⁠
    Upgrade⁠
    REST API⁠
    Usage⁠
        Plugin Activation⁠
        Instance Creation⁠
        Users⁠
        Plugin Parameters⁠
        Obtain Certificate⁠
    Testing⁠
        Coverage⁠
        Style⁠
User Guide⁠
    Registration⁠
    Login⁠
    Topology⁠
        Creation⁠
        Editor⁠
            UI Editor⁠
            Text Editor⁠
        Inputs⁠
        Deployment⁠
        Undeployment⁠
Authors⁠
License⁠
Acknowledgments⁠
Resources⁠

⁠Intro ⇑⁠

This repository contains the necessary parts needed to create a Docker image holding the Alien4cloud application, along with the DEEP - Indigo Data Cloud TOSCA types, and the plugin which connects Alien4Cloud (A4C) to the orchestrator used in DEEP. Each time the Docker image is created, the latest version of the normative TOSCA used by IndigoDataCloud and the latest version of the plugin are integrated in this image.

⁠Administration Guide

⁠Getting Started ⇑⁠

Right now, we use a customized version of A4C 2.0. The code for our version is in the a4c directory in the root of this repository. The plugin code is in the indigodc-orchestrator-plugin directory.

The default port for A4C is 8088.

The default username / password are admin/admin. Please change them!

⁠Prerequisites ⇑⁠

Aside of the official requirements found at [Res2]⁠, Alien4Cloud needs at least 3GB of RAM to run and a dual core CPU.

You also need Docker, we tested with the CE version 17.03 and above. We strongly advise you to install the latest version from the official Docker site.

⁠Docker Container ⇑⁠

This repository contains a Dockerfile that allows the creation of a container with A4C ready to be used by the final user. Due to various requirements determined during development, we expose a number of container runtime parameters and environment variables that allow the customization of the container.

⁠Parameters ⇑⁠

We defined a number of constants to allow easy parametrisation of the container.

• user_uid - the UID of the non-priviledged user that runs the A4C server
• user_gid - the group UID of the non-priviledged user that runs the A4C server
• a4c_ver - the version of A4C used in the container, e.g. 2.0.0 or 2.1.0-SNAPSHOT; This is the official version, released by the original developers
• a4c_install_path - the path to the instalation root directory of A4C
• a4c_src_dir - the relative path used to store the source of A4C
• a4c_install_dir - the relative path denoting the root where the A4C war is stored
• a4c_deep_ver - the version of A4C modified and released by the DEEP team; It uses the a4c_ver as prefix to which an suffix is added, e.g. -DEEP-1.0.0
• a4c_user - the user name of the non-priviledged user that executes the A4C server
• a4c_settings_manager_ver - The version of the settings manager that imports various settings from the environment into the A4C conf
• tosca_normative_url - The URL to the TOSCA normative types ram YAML file as implemented by OpenStack; It must be respect the version set by tosca_normative_version
• tosca_normative_version - The version of TOSCA currently supported by this bundle
• tosca_normative_name - The name set in the TOSCA normative types file obtained from tosca_normative_url; This name is set in the metadata->template_name tag
• tosca_indigo_types_branch - The branch of the Indigo custom types
• spring_social_oidc_branch - The branch of the Spring OIDC Social plugin used in this build
• a4c_deep_branch - The branch of the DEEP A4C fork used in this build
• templates_deep_oc_branch - The branch for the templates used in the DEEP project
• a4c_sh_name - The name of the script that contains the A4C init commands and the launch itself

⁠Variables ⇑⁠

The docker file contains environment variables. Some of them can be modified by the user when running a container. These are denoted by RW (read-write). The other group exists just to indicate features that were builtin in the container. This group is marked with RO (read-only). The RO group can be modified by modifying the args detailed in the previous paragraph. If the name of a variable is the same as the name of an argument, but with capital letters, then the variable is initialized with the valued of the argument.

• A4C_SH_NAME (RO) - the name of the script that executes A4C; Script generated by the Settings Manager in the a4c_install_dir folder (it has to be in the same folder with the war)

• A4C_CERTS_ROOT_PATH (RW) - the full path to the directory containing the files refered by A4C_PEM_CERT_FILE, A4C_PEM_KEY_FILE

• A4C_INSTALL_PATH (RO) - the path to the installation root directory of A4C

• A4C_SRC_DIR (RO) - the relative path used to store the source of A4C

• A4C_INSTALL_DIR (RO) - the relative path denoting the root where the A4C war is stored

• A4C_USER (RO) - the user name of the non-privileged user that executes the A4C server

• A4C_SETTINGS_MANAGER_VER (RO) - The version of the settings manager that is used in the Dockerfile

• A4C_PORT_HTTP (RO) - the port on which A4C is exposed

• A4C_PORT_HTTPS (RO) - the port on which A4C is exposed

• A4C_SPRING_OIDC_ISSUER (RW) - the URL of the IAM that performs the authentication of the user; For more details, please check the Spring OIDC A4C plugin at [Res8]⁠

• A4C_SPRING_OIDC_CLIENT_ID (RW) - the ID of the client registered on the IAM; The client normally represents an A4C instance; For more details, please check the Spring OIDC A4C plugin at [Res8]⁠

• A4C_SPRING_OIDC_CLIENT_SECRET (RW) - the secret of the client registered on the IAM; The client normally represents an A4C instance; For more details, please check the Spring OIDC A4C plugin at [Res8]⁠

• A4C_SPRING_OIDC_ROLES (RW) - the roles given to the users created on the fly on A4C; Use CSV style approach, that is list of elements with single quote, separated by one and only one comma eg 'ADMIN','APPLICATION MANGER','DEV, OPS' at least one element is necessary; For more details, please check the Spring OIDC A4C plugin at [Res8]⁠

• A4C_ORCHESTRATOR_URL (RW) - the URL of the orchestrator that allows the deployment of the topologies created in A4C

• A4C_ORCHESTRATOR_ENABLE_KEYSTORE (RW) - enable the keystore creation and loading for those cases when the orchestrator certificate is not signed by a recognized authority

• A4C_ORCHESTRATOR_KEYSTORE_PASSWORD (RW) - the default password for the Java keystore containing the orchestrator certificate

• A4C_ORCHESTRATOR_KEY_PASSWORD (RW) - the password to be used to protected the certificate inside the Java keystore

• A4C_ORCHESTRATOR_PEM_CERT_FILE (RW) - the file name of the orchestrator certificate file that is available on A4C_CERTS_ROOT_PATH

• A4C_ORCHESTRATOR_PEM_KEY_FILE (RW) - the file name of the orchestrator certificate key file that is available on A4C_CERTS_ROOT_PATH

• A4C_RUNTIME_DIR (RW) - the container directory that holds the runtime data generated by A4C during runtime like logs, uploads, and the Elastic Search cluster data

• A4C_RESET_CONFIG (RW) - Caution! Enable the A4C settings manager execution with old data present; This will delete everything from the directory set in the A4C_RUNTIME_DIR and will let the settings manager re-create the A4C config files; Either "true" or "false"; Disabled for the time being

• A4C_ADMIN_USERNAME (RW) - the username of the default admin user of A4C

• A4C_ADMIN_PASSWORD (RW) - the password of the default admin user of A4C

• A4C_ENABLE_SSL (RW) - enable SSL mode for A4C; HTTP will be disabled; Either "true" or "false"

• A4C_KEY_STORE_PASSWORD (RW) - the password of the keystore used by A4C

• A4C_KEY_PASSWORD (RW) - the password of the key that is added using the PEM file refered by the ENV A4C_PEM_KEY_FILE

• A4C_PEM_CERT_FILE (RW) - the name of the PEM file with the certificate used to secure the A4C instance (just the name, without path)

• A4C_PEM_KEY_FILE (RW) - the name of the PEM file with the key used to secure the A4C instance (just the name, without path)

• A4C_JAVA_XMX_MEMO (RW) - the maximum size of the Java heap memory allocated to the A4C process; it must respect the Java nomenclature see [Res9]⁠ [[]]

⁠Settings Manager ⇑⁠

We included a small Java utility program that runs each time the container is started. It takes some of the values defined by the Docker ENV variables listed earlier, and modifies the config files of the a4c instance. The A4C_ADMIN_USERNAME defines a custom name for the admin user. This utility writes it in the a4c_install_path/a4c_install_dir/config/alien4cloud-config.yml. The code is in the alien4cloud-settings-manager folder.

We also use this application to convert the certificates provided as detailed in the next sections to a Java storing format. You can protect this way the a4c communication with SSL.

⁠SSL ⇑⁠

Skip this section if you don't want to deploy a secured A4C instance.

If you want to activate the HTTPS protocol for the A4C instance, you must set the A4C_ENABLE_SSL ENV to "true", without double quotes. Furthermore, you must mount an external volume containing a ca pem certificate (with the file name controlled by the A4C_PEM_CERT_FILE ENV) and a ca pem key (with the file name controlled by the A4C_PEM_KEY_FILE ENV). Finally, since the previous two files must be just file names, you can control the path inside the container for the mount using the A4C_CERTS_ROOT_PATH ENV. You can control the Java keystore password by the means of the A4C_KEY_STORE_PASSWORD ENV, and set the password for your key using the A4C_KEY_PASSWORD ENV. Take a look at the examples in the Deployment section.

⁠Deployment ⇑⁠

You can get the docker image from DockerHub IndigoDC repo [Res5]⁠.

• Get the Docker container by running:

docker pull indigodatacloud/alien4cloud-deep

• Create the deployment using:

docker run -d --name alien4cloud-deep  -p ${A4C_PORT_HTTP}:${A4C_PORT_HTTP} -e ${A4C_SPRING_OIDC_CLIENT_ID}=<IAM client ID> -e ${A4C_SPRING_OIDC_CLIENT_SECRET}=<IAM client secret> -e ${A4C_SPRING_OIDC_ROLES}=<list of roles>  indigodatacloud/alien4cloud-deep

where the A4C_PORT, A4C_SPRING_OIDC_CLIENT_ID, A4C_SPRING_OIDC_CLIENT_SECRET, A4C_SPRING_OIDC_ROLES run parameter values are explained in the Docker Variables section of this README.

• Follow the logs (and wait for the web app to start up):

docker logs -f alien4cloud-deep

• You can also mount the directory containing the runtime data generated by A4C, e.g.:

docker run -d --name alien4cloud-deep  -p ${A4C_PORT_HTTP}:${A4C_PORT_HTTP} -e ${A4C_SPRING_OIDC_CLIENT_ID}=<IAM client ID> -e ${A4C_SPRING_OIDC_CLIENT_SECRET}=<IAM client secret> -e ${A4C_SPRING_OIDC_ROLES}=<list of roles> -e A4C_RUNTIME_DIR=${A4C_RUNTIME_DIR_MINE} -v /mnt/a4c_runtime:${A4C_RUNTIME_DIR_MINE} indigodatacloud/alien4cloud-deep

where the A4C_VOLUME_DIR, A4C_PORT, A4C_SPRING_OIDC_CLIENT_ID, A4C_SPRING_OIDC_CLIENT_SECRET, A4C_SPRING_OIDC_ROLES run parameter values are explained in the Docker Variables section of this README.

• In order to secure the A4C connection you can use the following:

docker run -d --name alien4cloud-deep  -p ${A4C_PORT_HTTPS}:${A4C_PORT_HTTPS} -e ${A4C_SPRING_OIDC_CLIENT_ID}=<IAM client ID> -e ${A4C_SPRING_OIDC_CLIENT_SECRET}=<IAM client secret> -e ${A4C_SPRING_OIDC_ROLES}=<list of roles> -e A4C_RUNTIME_DIR=${A4C_RUNTIME_DIR_MINE} -v <path to directory that will hold A4C's runtime data>:${A4C_RUNTIME_DIR_MINE} -e A4C_CERTS_ROOT_PATH=${A4C_CERTS_ROOT_PATH_MINE} -v <path to certificates root>:${A4C_CERTS_ROOT_PATH_MINE} -e A4C_PEM_CERT_FILE=<ca cert file name> -e A4C_PEM_KEY_FILE=<ca private key file name> -e A4C_KEY_PASSWORD=<password used to sign the certificate> -e A4C_ENABLE_SSL=true indigodatacloud/alien4cloud-deep

where you have to specify the secure port, the root path where the certificates are on the host and its mapping inside the container, the names of the key and certificates pems and to enable SSL

⁠Upgrade ⇑⁠

The upgrade process involves pulling the desired version of the container alien4cloud-deep_ from the DockerHub repository [Res5]⁠. The new version can be executed along with the old one, provided that there are no conflicts. We do not support sharing of any kind of the data related to a container between two running at the same time. This is especially true for the runtime data generated by Alien4Cloud.

A4C stores the runtime generated data in the path set in the variable A4C_RUNTIME_DIR. You should be able to mount the runtime path of the newer version of the alien4cloud-deep Docker distribution in the same location as the old one. Alien4Cloud should be able to load the existing data, however a straightforward upgrade process is not always guaranteed due to the multiple versions released. All in all, be sure to backup your data before any upgrade.

⁠REST API ⇑⁠

A4C supports a REST API [Res4]⁠ that can be used for many operations. We list some examples in the following paragraphs.

• (MUST BE DONE FIRST) authenticate & set general variables

 # Set the A4C server
export ALIEN_URL='http://example.com:8088'
 # Set the cookie file path
export ALIEN_COOKIE='/tmp/a4c cookie.txt'
 # Authenticate, check if the file
 # This call protects the username and password by using the read call
 # When you execute the following line, the shell will wait for two values separated by the ENTER key, before executing the curl call
 # The first value is the username, the second is the password

curl -k -c "${ALIEN_COOKIE}" "$ALIEN_URL/login" --data-urlencode "username=$( read -s U; echo $U )" --data-urlencode "password=$( read -s P; echo $P )" --data-urlencode "submit=Login" -XPOST -H 'Content-Type: application/x-www-form-urlencoded'

• create a new user (you can select either combination of the roles; we list all available)

 # Create new user; the cookie must be from an ADMIN type user
curl -k -b "${ALIEN_COOKIE}" -XPOST -H 'Content-Type: application/json; charset=UTF-8' -H 'Accept: application/json, text/plain, */*' --data '{"email": "a@a", "firstName": "fn", "lastName": "ln", "password": "pass new", "roles": ["ADMIN", "APPLICATIONS_MANAGER", "ARCHITECT", "COMPONENTS_BROWSER",  "COMPONENTS_MANAGER"], "username": "tst2"}' "$ALIEN_URL/rest/v1/users"

• update existing user

 # Set the username you want to change; you can use the read call as when you authenticated yourself
export ALIEN_USERNAME='tst2'
 # Update user; the cookie must be from an ADMIN type user
curl -k -b "${ALIEN_COOKIE}" -XPUT -H 'Content-Type: application/json; charset=UTF-8' -H 'Accept: application/json, text/plain, */*' --data '{"email": "a@a", "firstName": "fn", "lastName": "ln", "password": "pass new", "roles": ["ADMIN", "APPLICATIONS_MANAGER", "ARCHITECT", "COMPONENTS_BROWSER",  "COMPONENTS_MANAGER"], "username": "${ALIEN_USERNAME}"}' "$ALIEN_URL/rest/v1/users/${ALIEN_USERNAME}"

⁠Usage ⇑⁠

Once Alien4Cloud started, you can connect to it using a supported browser [Res3]⁠. Before anything else, one should take a look at the [Res2]⁠.

⁠Plugin Activation ⇑⁠

The IndigoDataCloud Orchestrator plugin comes preinstalled in the Docker container. It should also be activated by default. Please check the list from Administration -> Plugins. The switch on the right should be green.

⁠Instance Creation ⇑⁠

First and foremost, one should create and instance of the DEEP - IndigoDataCloud Orchestrator plugin. Next, the values for the parameters should be set (explanation for each in the Plugin Parameters subsection. Finally, on the same page, one has to create a location. We support only one location, that is Deep Orchestrator Location.

⁠Users & Authorization ⇑⁠

The DEEP orchestrator plugin uses the token obtained when the user logs in with the IAM instance. Due to the token based authorization of the orchestrator used in DEEP, the local users cannot connect to the orchestrator.

A local user can be added using the Administration -> Users -> New User functionality. The same location allows the modification of various user specific settings. The web application stores the information locally. If you change it in the IAM, you have to manually update it for each A4C instance.

The A4C user roles are not taken in consideration by the plugin. The admin should consider the proper role(s) for a user only in the context of the A4C running instance. In order to allow a user that is not an admin to modify his/her own settings, the role USER_SPECIFIC_SETTINGS_EDITOR must be enabled.

A non-admin user cannot use a location, unless an admin adds the username to the list of authorized users for each location. This is done by going to a Administration -> Orchestrators -> instance of the orchestrator -> Location(s) -> Security -> Authorize users. The user(s) selected from the list appear on the Security page. They can now access the instance & location offered by the orchestrator plugin. If the access must be revoked for whatever reason, the red delete button de-authorizes a user for the selected combination of orchestrator instance and location.

⁠Plugin Parameters ⇑⁠

• callbackUrl - The callback URL for the orchestrator sent by the plugin when deploying a topology
• orchestratorEndpoint - The endpoint of the orchestrator used to deploy the topologies.
• orchestratorEndpointCert - The certificate of the orchestratorEndpoint server. Take a look at Obtain Certificate subsection to learn how to obtain the certificate.
• orchestratorPollInterval - Alien4Cloud tries to obtain the history of events every number of seconds. This parameter sets that number.
• importIndigoCustomTypes - Depending on the status of the work, there can be a different location of the indigo types definition file that is sent to the orchestrator. This field allows the admin to specify which TOSCA types file is used for all deployments that go through the IndigoDCOrchestrator plugin. Please keep in mind that this field affects only the future deployments. This happens because it is read when a topology is deployed.

⁠Obtain Certificate ⇑⁠

In order to obtain the certificates used by the plugin and stored in the tokenEndpointCert, orchestratorEndpointCert, and iamHostCert respectively, you should follow the steps described in this subsection for each different server. We refer to servers as in the following unit: *.domain.extension. Two variables may point towards the same server, but with different paths. deep-paas.cloud.ba.infn.it/rest/v1 and deep-paas.cloud.ba.infn.it/token point towards the same domain/subdomains with different paths, therefore you only have to execute the following procedure once, for deep-paas.cloud.ba.infn.it. You also need the port (normally 443 for HTTPS), as you'll see next.

1. Install openssl

apt-get install openssl

1. Get the key from the server (example for the development server managing the orchestrator). Use echo to terminate the connection

echo "Q" | openssl s_client -showcerts -servername deep-paas.cloud.ba.infn.it -connect deep-paas.cloud.ba.infn.it:443 > crt.tmp

1. Copy a certificate from crt.tmp. This file may contain more than one, any is valid. You need the characters between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----, for example to get all certificates separated by new lines:

grep -m 1 -ozP '(?<=-----BEGIN\ CERTIFICATE-----)(\n|.)+?(?=-----END\ CERTIFICATE-----)' crt.tmp > crt_solos.tmp

1. Remove the new lines from one certificate with your prefered approach, e.g.:

cat crt_solos.tmp | tr -d '\n'

1. This BASE64 string can now be used with the app

⁠Testin