SENSE Network Resource Manager (SENSE-NRM) Copyright (c) 2018-2020, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved.

If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Innovation & Partnerships Office at [email protected].

NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so.

Fri May 8 11:26:40 PDT 2020

[email protected]

2 Requirements

· Apache

% sudo yum install httpd mod_ssl mod_wsgi

· mod_wsgi

· Python 3.6 (or Anaconda3)

· FLASK

· RDFlib

% pip install flask_restful

% pip install rdflib

3 GitHub source repo

· https://github.com/esnet/sense-nrm-oscars

4 Sample ESnet service endpoint

· https://sense-nrm.es.net:8443/sense-rm/api/sense/v1/models

5 Apache configuration

5.1 Apache installation for python 3 and Anaconda3

On Centos, use yum to install apache with python3 modules.

e.g.

yum install httpd (or yum update httpd)

yum install python3.x86_64

yum install python36u-mod_wsgi

yum install mod_ssl

From the Anaconda archive, download python 3.6 version as the latest Anaconda with python 3.7 has a conflict with yum package mod_wsgi:

https://repo.continuum.io/archive/Anaconda3-5.2.0-Linux-x86_64.sh

5.2 apachectl with python from anaconda3 (python 3.6)

Add the following vars to the apachectl before configuration section.

Also, add -f option for the path to the httpd.conf, if necessary.
e.g. $HTTPD -k $ARGV -f /home/asim/nrm/apache/conf/httpd.conf

PYTHONPATH="/usr/local/pkg/anaconda3/lib/python3.6/site-packages"

export PYTHONPATH

LD_LIBRARY_PATH="/lib:/usr/lib:/usr/local/lib:/usr/lib64:/usr/local/pkg/anaconda3/lib"

export LD_LIBRARY_PATH

5.2.1 apachectl with python from anaconda2 (python 2.7)

Add the following vars to the apachectl before configuration section.

Also, add -f option for the path to the httpd.conf, if necessary.
e.g. $HTTPD -k $ARGV -f /home/asim/nrm/apache/conf/httpd.conf

PYTHONHOME="/usr/local/pkg/anaconda2"

export PYTHONHOME

PYTHONPATH="/usr/local/pkg/anaconda2/lib/python2.7/site-packages"

export PYTHONPATH

LD_LIBRARY_PATH="/lib:/usr/lib:/usr/local/lib:/usr/local/pkg/anaconda2/lib"

export LD_LIBRARY_PATH

5.3 httpd.conf configuration

· Update or add the following to the httpd.conf

· Replace red italic (e.g. *_PATH) with actual path

· Timeout 600

5.3.1 For log rotation

ErrorLog "|/usr/sbin/rotatelogs -l LOG_PATH/error_log.%Y-%m-%d-%H%M%S 10M"

CustomLog "|/usr/sbin/rotatelogs -l LOG_PATH/access_log.%Y-%m-%d 86400" common

</IfModule>

5.3.2 For WSGI

WSGISocketPrefix APACHE_PATH

WSGIDaemonProcess sensenrm user=LOGIN group=users processes=2 threads=15

WSGIScriptAlias / NRM_PATH/sensenrm_service.wsgi

WSGIScriptReloading On

WSGIProcessGroup sensenrm

WSGIApplicationGroup %{GLOBAL}

Options ExecCGI

SetHandler wsgi-script

Require all granted

</Directory>

5.3.3 permanent forwarding from 80 to 443 with virtual hosts

Include APACHE_PATH/conf/extra/httpd-vhosts.conf

5.3.4 Secure (SSL/TLS) connections

Include APACHE_PATH/conf/extra/httpd-ssl.conf

5.3.5 httpd-ssl.conf configuuration

· Update or add the following to the httpd-ssl.conf

Listen 443

SSLProtocol all -TLSv1.2

SSLProxyProtocol all -TLSv1.2

ErrorLog "|/usr/sbin/rotatelogs -l APACHE_PATH/logs/error_log.%Y-%m-%d-%H%M%S 10M"

TransferLog "|/usr/sbin/rotatelogs -l APACHE_PATH/logs/access_log.%Y-%m-%d 86400"

# Request header setting

RequestHeader set X-SSL-Client-S-DN-CN "%{SSL_SERVER_S_DN_CN}s"

RequestHeader set X-SSL-Client-S-DN "%{SSL_CLIENT_S_DN}s"

RequestHeader set X-SSL-Client-Verify "%{SSL_CLIENT_VERIFY}s"

RequestHeader set X-SSL-Client-Cert "%{SSL_CLIENT_CERT}s"

RequestHeader set X-Real-IP "%{REMOTE_ADDR}s"

RequestHeader set X-MYHOST "%{HTTP_HOST}s"

# SSL certs setting

SSLCertificateFile "HOSTCERT_FILE_PATH/sensenrm-cert.pem"

SSLCertificateKeyFile "HOSTKEY_FILE_PATH/sensenrm-key.pem"

SSLCACertificatePath "CA_DIR_PATH/certificates"

SSLVerifyClient require

SSLVerifyDepth 10

# Custom log setting

CustomLog "|/usr/sbin/rotatelogs -l APACHE_PATH/logs/ssl_request_log.%Y-%m-%d 86400" \

"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

</VirtualHost>

5.3.6 httpd-vhosts.conf

· Update or add the following to the httpd-vhosts.conf

ServerName SERVER_HOST_NAME

Redirect permanent / https://SERVER_HOST_NAME/

</VirtualHost>

6 SENSE-NRM Server configuration

6.1 Edit sensenrm_config.py

6.1.1 NRM server host configuration

nrm_config = {

"host": "NRM_SERVER_HOSTNAME",

"port": NRM_SERVER_PORT_NUMBER,

"urnprefix": "urn:ogf:network:es.net:2019", # Model URN prefix

"debug": 9 # debug level [0-9]

}

6.1.2 NRM Service related configuration

nrm_service = {

"l3vpn_model_insert": ./l3vpn-insert-190904.txt", # Static L3VPN model insert path

"poll_duration": 60, # in minutes. OSCARS available topology polling period

"default_delta_lifetime": 24 # in hours

}

6.1.3 NRM DB configuration

nrmdb_config = {

"type": "sqlite",

"url": "PATH_TO_SQLITE_FILE/nrm.db"

}

6.1.4 OSCARS access configuration

oscars_config = {

"url": "OSCARS_SERVER_HOST:PORT_NUMBER",

"default_user": "USER_LOGIN",

"default_passwd": "USER_PASSWD",

"default_token": "USER_TOKEN",

"default_dn": "USER_DN"

}

6.1.5 NRM user access configuration

users_config = {

"admin" : "ADMIN_DN",

"mapfile" : "NRM_USER_MAPPING_FILE_PATH"

# MAPFILE format: DN group

# e.g. "/DC=org/DC=OSG/O=OSG/OU=People/CN=Alex Sim 1116" default

}

6.1.6 Configurations for WSGI

· Must be the same as in httpd-ssl.conf

ssl_config = {

"capath": "CA_DIR_PATH/certificates",

"hostcertpath": "SERVICE_CERT_FILE_PATH/nrm-cert.pem",

"hostkeypath": "SERVICE_KEY_FILE_PATH/nrm-key.pem",

"httpsverify": False

}

7 NRM Log directory (existing) path

log_config = {

"basepath": "NRM_LOG_DIR_PATH/logs"

}

7.1 Edit sensenrm_service.wsgi

· Update to where all the NRM service python files are

sys.path.insert(0, "NRM_SERVER_DIR_PATH")

8 Host or User certificate

8.1 Host certificate

· You can generate a local service certificate with a local CA.

· You can also submit a request to a Certificate Authority (CA) to acquire a host certificate.

· NRM should accept all IGTF-accredited CAs (https://repo.opensciencegrid.org/cadist/).

· An instruction from Open Science Grid on how to get a host certificate:

https://opensciencegrid.org/docs/security/host-certs/

8.2 User certificate

· You will have to obtain your user certificate using the CILogon web UI: https://cilogon.org.

· An instruction from Open Science Grid on how to get a user certificate:

https://opensciencegrid.org/docs/security/user-certs/

· If you use a locallly generated user certificate, your CA certificate needs to be sent to the NRM service so that the service can add to the CA path to accept your user cert.

o The decision to add to the CA path depends on the site service admin.

9 NRM server

· HTTPS is only supported for all interfaces

· Start the NRM server

o % sudo apachectl start -e debug

· Restart/stop the NRM server

o % sudo apachectl restart -e debug

o % sudo apachectl stop

· Checking apache log if log dir is customized

o % tail APACHE_PATH/logs/error_log

9.1 Simple client testing with samples

· curl -i https://NRM_HOST_NAME:PORT/sslinfo \

--cert "USER_CERT_PATH/user-cert.pem" \

--key "USER_KEY_PATH/user-key.pem" \

--capath "CA_CERTS_DIR_PATH/certificates"

· wget https://NRM_HOST_NAME:PORT/sslinfo \

--certificate="USER_CERT_PATH/user-cert.pem" \

--private-key="USER_KEY_PATH/user-key.pem" \

--ca-directory="CA_CERTS_DIR_PATH/certificates"

· e.g.

curl -i https://dev-sense-nrm.lbl.gov:443/sslinfo \

--cert "/home/users/nrmuser/my-user-cert.pem" \

--key "/home/users/nrmuser/my-user-key.pem" \

--capath "/etc/grid-security/certificates"

wget https://dev-sense-nrm.lbl.gov:443/sslinfo \

--certificate="/home/users/nrmuser/my-user-cert.pem" \

--private-key="/home/users/nrmuser/my-user-key.pem" \

--ca-directory="/etc/grid-security/certificates"

10 To run multiple NRM servers

In case multiple NRM servers need to be run (e.g. testbed instance) on a new port, the following items need to be changed.

· sensenrm_service.wsgi to indicate a new path
e.g. sys.path.insert(): /home/nrm/nrm -> /home/nrm/tbnrm

· sensenrm_config.py

o nrm_config for port and urnprefix
e.g. port: 443 -> 8443
e.g. urnprefix: urn:ogf:network:es.net:2013 -> urn:ogf:network:es.net:2019

o nrm_service for shorter default lifetime
e.g. default_delta_lifetime: 168 -> 24

o oscars_config for testbed OSCARS connection
e.g. url: oscars-web.es.net:443 -> oscars-dev2.es.net:443

o nrmdb_config for new db file path
e.g. "url": "/home/nrm/nrm/nrmdb.db" -> "/home/nrm/tbnrm/nrmtb.db"

o log_config for new log path

e.g. "basepath": "/home/nrm/nrm/logs" -> "/home/nrm/tbnrm/logs"

· httpd.conf for various paths and new port info

· https-ssl.conf for SSL port and various paths

· https-vhosts.conf for new port info

· You may use the same SSL certs.

11 Python client command-line tool

Detailed client information is here: https://sdm.lbl.gov/nrm/readme-nrm-client.html

% python sensenrm_client_esnet.py -h