Introduction

This document details the procedure for creating an eagle-i institutional node as a virtual server (or instance) in the Amazon Elastic Compute Cloud (or EC2). Once created, the eagle-i node will operate entirely in the cloud. However, you will retain administrative responsibility over its operation and maintenance, and in particular, you will be responsible for running upgrade scripts when new eagle-i software is released. We do not expect these tasks to be complex, though basic Unix skills are desirable. This solution is ideal for institutions that want to evaluate eagle-i or participate in the eagle-i network but do not have easy access to a data center service. Naturally, the AWS service will incur operational costs (for pricing details, consult the AWS website).

The installation procedure is simple and does not require specialized technical skills. It will allow you to get an eagle-i node up and running in a short amount of time. For a production system, you may need to involve your IT department, in a limited way

...

.

A new AMI will be available for subsequent releases, likely with a slight delay after the release.

...

First time installation

Getting ready

This procedure can be used to create an evaluation/development or a production eagle-i node:

...

...

...

1. Getting ready

1. Prerequisites

2.

As a pre-prerequisite, you will need to decide which type of installation you will need, evaluation/development or production eagle-i node.

Prerequisites

...

1. Status
   title evaluation/development node

   may use the amazon-generated public hostname

2. Status
   colour Blue title production node

    will need to have a DNS record once you obtain an IP address from EC2
   • Decide on a good host name. It will determine the namespace of your Linked Open Data, and it shouldn't be changed once data exists in production.
   • Examples of existing host names: harvard.eagle-i.net, eagle-i.ea.vanderbilt.edu

...

1. Status
   title evaluation/development node

    may use the self-signed certificate provided by the AMI

2. Status
   colour Blue title production node

    needs an X509 certificate in PEM format

...

1. Sign up for AWS
2. Sign up for the EC2 service; this will require that you provide credit card information.

...

• If you're using Linux, you know what this is about already
• In MacOSX you can simply use the Terminal application that is installed by default (look in your Applications folder, under Utilities)
• In Windows we recommend downloading and installing PuTTY (a remote login client that can handle SSH keys) or cygwin (a full Unix toolset)

...

 Installation Procedures

1. Allocate EC2 Resources
2. Create an eagle-i node

...

3. Customize eagle-i node

...

1. Prepare to Customize eagle-i Node
2. Customize eagle-i Repository
3. Create system users
4. Customize public SPARQL endpoint
5. Update node configuration file

3. Verify installation

1. Verify the eagle-i node is up and running

4. Production instance only

There are additional steps that need to happen in order to make your instance production ready. For more information about elastic IP address, please see the Amazon Documentation: Elastic IP Addresses.

1. Allocate an elastic IP address
2. Associate the elastic IP to your public hostname
3. Install your SSL Certificate

1. Allocate EC2 Resources

1.1 Select a US availability zone/geographic region

Amazon allocates EC2 resources (IP addresses, virtual hardware) in specific facilities that are meant to cover different geographic regions (also called availability zones). We support three zones: US East (N. Virginia), US West (Oregon) and US West (N. California).

1. Go to your EC2 Dashboard
2. Using the pull down list at the top right hand corner of the dashboard (next to your user name); choose that which is most appropriate to your institution's location.
   Image Removed

...

For more detailed information on creating and using a key pair with your EC2 instances, please see AWS Documentation: EC2 Key Pairs

...

...

...

2.1 Launch Instance of the eagle-i AMI

For more detailed information on how to launch an EC2 Instance from an AMI, please see AWS Documentation: Launching an Instance.

1. Go to your EC2 Dashboard.
2. Click on the Launch Instance button
3. Select the Community AMIs tab on the left
4. Search for 'eaglei'

5. Select the latest eagle-i AMI

   Panel
   borderColor Grey title Latest eagle-i AMI
   eaglei-4.3.0-20160605 - ami-39e81654 Root device type: ebs Virtualization type: hvm

6. Choose an Instance Type that is appropriate for your installation. For the available instance types as of August 2016, we recommend:

   1. Status
      title evaluation/development node

       t2.micro

   2. Status
      colour Blue title production node

       m3.medium

7. Click on the Review and Launch button
8. Click on Edit Security groups
9. Click on the Add Rule button
10. Select HTTPS from the drop down
11. Click on the Review and Launch button
12. Click on the Launch button
13. Select the key pair that was created in 1.2
14. Click on the Launch Instance button

2.2 Wait for Instance to complete initialization

For more detailed information about the instance lifecycle, please see AWS Documentation: EC2 Instance Lifecycle.

...

1. Click on the EC2 Instance ID in the Launch Status page
2. Go to your EC2 Dashboard and click on Running Instances

...

...

Image RemovedImage Removed

Customize eagle-i node

...

...

...

1. Prepare to

...

1.1 Connect to the eagle-i node

1. Go to your EC2 Dashboard.
2. Click on Running Instances
3. Select the newly created EC2 Instance
4. Click on the Connect button and follow the instructions to SSH to the node
   Image Removed

1.2 Set required environment variables

In the SSH terminal:

1. Set the environment variable REPO_HOME to point to the repository home directory

   • export REPO_HOME=/opt/eaglei/repo

2. Set the environment variable SPARQLER_HOME to point to the SPARQLer home directory

   • export SPARQLER_HOME=/opt/eaglei/sparqler

3. Verify the variables have been set correctly
   • echo ${REPO_HOME}
   • echo ${SPARQLER_HOME}


    

   Code Block
   language bash theme Emacs title Set environment variables linenumbers true
   [root@ip-172-31-54-208 repo]# export REPO_HOME=/opt/eaglei/repo/ [root@ip-172-31-54-208 repo]# export SPARQLER_HOME=/opt/eaglei/sparqler/ [root@ip-172-31-54-208 repo]# echo ${REPO_HOME} /opt/eaglei/repo/ [root@ip-172-31-54-208 repo]# echo ${SPARQLER_HOME} /opt/eaglei/sparqler/

2. Customize repository

The following assumes that you are in the SSH terminal connected to your ec2 instance.

2.1 Stop the running tomcat instance

1. Go to the tomcat directory

   • cd /opt/apache-tomcat-7.0.39/

2. Stop tomcat using the shutdown wrapper script

   • sudo -su tomcat bin/shutdownwrapper.sh

3. Verify tomcat has been shutdown

   • ps aux | grep tomcat

[root@ip-172-31-54-208 log]# cd /opt/apache-tomcat-7.0.39/
[root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bin/shutdown.sh
Using CATALINA_BASE:   /opt/apache-tomcat-7.0.39
Using CATALINA_HOME:   /opt/apache-tomcat-7.0.39
Using CATALINA_TMPDIR: /opt/apache-tomcat-7.0.39/temp
Using JRE_HOME:        /opt/jdk1.8.0_66
Using CLASSPATH:       /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar
[root@ip-172-31-54-208 apache-tomcat-7.0.39]# ps aux | grep tomcat
root      1646  0.0  0.0 103312   876 pts/0    S+   13:02   0:00 grep tomcat
 
#### The following means that tomcat is still running ####
tomcat    1674 93.2 11.4 3599164 116800 pts/0  Sl   13:08   0:03 /opt/jdk1.8.0_66/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat-7.0.39/conf/logging.properties 
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1536m -XX:+PrintGCDetails 
-Xloggc:/opt/apache-tomcat-7.0.39/logs/tomcat-gc.log -Djava.endorsed.dirs=/opt/apache-tomcat-7.0.39/endorsed 
-classpath /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar 
-Dcatalina.base=/opt/apache-tomcat-7.0.39 -Dcatalina.home=/opt/apache-tomcat-7.0.39 
-Djava.io.tmpdir=/opt/apache-tomcat-7.0.39/temp org.apache.catalina.startup.Bootstrap start

2.2 Modify configuration.properties for the repository

...

• cd ${REPO_HOME}

...

• vim configuration.properties

...

• eaglei.repository.namespace

...

• eaglei.repository.title
• eaglei.repository.logo

1. Customize eagle-i Node
2. Customize eagle-i Repository
3. Create system users
4. Customize public SPARQL endpoint
5. Update node configuration file

4. Verify installation

...

[root@ip-172-31-54-208 ~]# cd ${REPO_HOME}
[root@ip-172-31-54-208 repo]# vim configuration.properties
 
##### configuration.properties file ####
eaglei.repository.namespace = http://ec2-54-175-59-6.compute-1.amazonaws.com/i/
eaglei.repository.title = AMI Test Repository
eaglei.repository.logo = https://alaska.qa.eagle-i.net:8443/sweet/images/eaglei-medium-blue.png

2.3 Prepare the repository for customization

1. Go to the repository home directory
   • cd ${REPO_HOME}
2. Delete the existing derby database
   • rm -fr db/
3. Run the prepare-install script in the etc directory.  Replace REPOADMIN and REPOADMINPW with the credentials for the repository administrator.
   • bash etc/prepare-install.sh REPOADMIN REPOADMINPW ${REPO_HOME}

4. Modify the ownership of the derby database to be owned by the tomcat user

   • chown -R tomcat:tomcat db/

    

   Code Block
   language bash theme Emacs title Prepare repository linenumbers true
   [root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# rm -r db/ [root@ip-172-31-54-208 repo]# bash etc/prepare-install.sh ami-admin-user ami-admin-pw ${REPO_HOME} _RUNJAVA set to "/opt/jdk1.8.0_66/jre/bin/java" java version "1.8.0_66" Java(TM) SE Runtime Environment (build 1.8.0_66-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode) ---all superuser logins--- ami-admin-user [root@ip-172-31-54-208 repo]# chown -R tomcat:tomcat db/

2.4 Start tomcat

1. Go to the tomcat directory

   • cd /opt/apache-tomcat-7.0.39/

2. Start tomcat using the startup wrapper script

   • sudo -su tomcat bin/startupwrapper.sh

3. Wait for tomcat to finish startup.

   Code Block
   language bash theme Emacs title Start tomcat linenumbers true
   [root@ip-172-31-54-208 opt]# cd apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bash bin/startupwrapper.sh Waiting for Tomcat to startup ... Tomcat startup finished in ~120 seconds

2.5 Finish the repository customization

1. Go to the repository home directory
   • cd ${REPO_HOME}
2. Run the finish install script in the etc directory.  Replace REPOADMIN and REPOADMINPW with the credentials for the repository administrator.
   • bash etc/finish-install.sh REPOADMIN REPOADMINPW repository-url-prefix

3. Run the upgrade script in the etc directory. Replace REPOADMIN and REPOADMINPW with the credentials for the repository administrator.

   • bash etc/upgrade.sh REPOADMIN REPOADMINPW repository-url-prefix 

   Code Block
   language bash theme Emacs title Finish repo linenumbers true
   [root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# bash etc/finish-install.sh ami-admin-user ami-admin-pw https://ec2-54-175-59-6.compute-1.amazonaws.com User metadata created. Updating data model ontology from jar, please wait... Updated data model ontology from jar. [root@ip-172-31-54-208 repo]# bash etc/upgrade.sh ami-admin-user ami-admin-pw https://ec2-54-175-59-6.compute-1.amazonaws.com ***** Checking and Upgrading Repository's Internal Graphs INFO: Skipping, internal graph is already up to date: http://eagle-i.org/ont/repo/1.0/ INFO: Skipping, internal graph is already up to date: http://eagle-i.org/ont/repo/1.0/NG_Query .....lots of output and stuff here .....   OK - specials all succeeded. rm -rf /tmp/repo-upgrade-9784

2.6 Verify repository has been customized correctly

In a browser, navigate to the repository admin console and log in using the repository administrator credentials.

Image Removed

3. Create and configure additional system users

3.1 Create the additional system users

Additional system users are needed for several automated processes. The following users need to be created:

1. Global user. This is an automated agent responsible for keeping the cached copy of commonly used resources up to date with the Commonly Used Resources' repository. This agent caches only the minimal information needed for the SWEET application.
   
   • Role: Automated Agent
   • Corresponding properties: eaglei.datatools.globals.user & eaglei.datatools.globals.password
2. Anonymous user. This agent is responsible for retrieving published information for using the profiles integration and for the resource listings on the institution (node) home page.
   
   • Role:none
   • Corresponding properties: eaglei.anonymousExtended.user & eaglei.anonymousExtended.password

3.2 Add the additional system users to the properties file

In the SSH terminal:

1. Go to the eagle-i configuration directory
   • cd /opt/eaglei/conf/
2. Open the eagle-i-apps-credentials.properties file in the text editor of your choice.
3. Add the credentials for the two system user mentioned above to the file
4. Save your changes.

[root@ip-172-31-54-208 sparqler]# cd /opt/eaglei/conf/
[root@ip-172-31-54-208 conf]# vim eagle-i-apps-credentials.properties
 
##### eagle-i-apps-credentials.properties file ####
# GLOBAL SERVICE
eaglei.datatools.globals.user=dtgu
eaglei.datatools.globals.password=dtgp

# Catalyst service
eaglei.anonymousExtended.user=eaeu
eaglei.anonymousExtended.password=eaep

4. Customize public sparql endpoint

The following assumes that you are in the SSH terminal connected to your ec2 instance.

4.1 Stop the running tomcat instance

1. Go to the tomcat directory

   • cd /opt/apache-tomcat-7.0.39/

2. Stop tomcat using the shutdown wrapper script

   • sudo -su tomcat bin/shutdownwrapper.sh

3. Verify tomcat has been shutdown

   • ps aux | grep tomcat

[root@ip-172-31-54-208 log]# cd /opt/apache-tomcat-7.0.39/
[root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bin/shutdown.sh
Using CATALINA_BASE:   /opt/apache-tomcat-7.0.39
Using CATALINA_HOME:   /opt/apache-tomcat-7.0.39
Using CATALINA_TMPDIR: /opt/apache-tomcat-7.0.39/temp
Using JRE_HOME:        /opt/jdk1.8.0_66
Using CLASSPATH:       /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar
[root@ip-172-31-54-208 apache-tomcat-7.0.39]# ps aux | grep tomcat
root      1646  0.0  0.0 103312   876 pts/0    S+   13:02   0:00 grep tomcat
 
#### The following means that tomcat is still running ####
tomcat    1674 93.2 11.4 3599164 116800 pts/0  Sl   13:08   0:03 /opt/jdk1.8.0_66/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat-7.0.39/conf/logging.properties 
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1536m -XX:+PrintGCDetails 
-Xloggc:/opt/apache-tomcat-7.0.39/logs/tomcat-gc.log -Djava.endorsed.dirs=/opt/apache-tomcat-7.0.39/endorsed 
-classpath /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar 
-Dcatalina.base=/opt/apache-tomcat-7.0.39 -Dcatalina.home=/opt/apache-tomcat-7.0.39 
-Djava.io.tmpdir=/opt/apache-tomcat-7.0.39/temp org.apache.catalina.startup.Bootstrap start

4.2 Modify configuration.properties for the public sparql endpoint

...

• cd ${SPARQLER_HOME}

...

• vim configuration.properties

...

• eaglei.repository.namespace

...

• eaglei.repository.title
• eaglei.repository.logo

...

[root@ip-172-31-54-208 ~]# cd ${SPARQLER_HOME}
[root@ip-172-31-54-208 sparqler]# vim configuration.properties
 
##### configuration.properties file ####
eaglei.repository.namespace = http://ec2-54-175-59-6.compute-1.amazonaws.com/i/
eaglei.repository.title = AMI Test Repository
eaglei.repository.logo = https://alaska.qa.eagle-i.net:8443/sweet/images/eaglei-medium-blue.png

4.3 Prepare the public sparql repository for customization

1. Go to the repository home directory
   • cd ${REPO_HOME}
2. Run the prepare-install script in the etc directory.  Replace SPARQLADMINUSER and SPARQLADMINPW with the credentials for the public sparqler repository administrator.
   • bash etc/prepare-install.sh SPARQLADMINUSER SPARQLADMINPW ${REPO_HOME} sparqler-users.derby

3. Modify the ownership of the derby database to be owned by the tomcat user

   • chown -R tomcat:tomcat db/ 

   Code Block
   language bash theme Emacs title Prepare repository linenumbers true
   [root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# bash etc/prepare-install.sh sparqler-user sparqler-pw ${REPO_HOME} sparqler-users.derby _RUNJAVA set to "/opt/jdk1.8.0_66/jre/bin/java" java version "1.8.0_66" Java(TM) SE Runtime Environment (build 1.8.0_66-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode) ---all superuser logins--- sparqler-user [root@ip-172-31-54-208 repo]# chown -R tomcat:tomcat db/

4.4 Update eagle-i property files with public sparqler information

1. Go to the eagle-i configuration directory
   • cd /opt/eaglei/conf/
2. Using the text editor of your choice, edit the eagle-i-apps.properties file
   • vim eagle-i-apps.properties
3. Add the following two properties to the eagle-i-apps.properties file to define the source and target repository for the public sparql endpoint:
   
   • eaglei.sparqler.source.URL, this will be your main repository URL
   • eaglei.sparqler.target.URL, this will be the URL of your public sparql endpoint. Typically it is a concatenation of the main repository URL with 'sparqler'
4. Save your changes.
5. Using the text editor of your choice, edit the eagle-i-apps-credentials.properties file.
6. Add the credentials used when running the prepare-install.sh script for the sparqler to the eagle-i-apps-credentials.properties file.
   • eaglei.sparqler.target.user
   • eaglei.sparqler.target.password

7. Save your changes

   Code Block
   language bash theme Emacs title Update Sparqler configs linenumbers true
   [root@ip-172-31-54-208 sparqler]# cd /opt/eaglei/conf/ [root@ip-172-31-54-208 conf]# vim eagle-i-apps.properties ##### eagle-i-apps.properties file ##### ## SPARQLER ### URL of source-repository (from which the sparqler reads public information): eaglei.sparqler.source.URL = https://ec2-54-175-59-6.compute-1.amazonaws.com/ ### URL of target (i.e., sparqler) repository: eaglei.sparqler.target.URL = https://ec2-54-175-59-6.compute-1.amazonaws.com/sparqler/ ########################################   [root@ip-172-31-54-208 conf]# vim eagle-i-apps-credentials.properties ##### eagle-i-apps-credentials.properties file ##### eaglei.sparqler.target.user=sparqler-user eaglei.sparqler.target.password=sparqler-password ####################################################

4.5 Start tomcat

1. Go to the tomcat directory

   • cd /opt/apache-tomcat-7.0.39/

2. Start tomcat using the startup wrapper script

   • sudo -su tomcat bin/startupwrapper.sh

3. Wait for tomcat to finish startup.

   Code Block
   language bash theme Emacs title Start tomcat linenumbers true
   [root@ip-172-31-54-208 opt]# cd apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bash bin/startupwrapper.sh Waiting for Tomcat to startup ... Tomcat startup finished in ~115 seconds

4.6 Finish the public sparql repository customization

1. Go to the repository home directory
   • cd ${REPO_HOME}
2. Run the finish install script in the etc directory.  Replace SPARQLADMINUSER and SPARQLADMINPW with the credentials for the public sparqler repository administrator.
   • bash etc/finish-install.sh SPARQLADMINUSER SPARQLADMINPW sparqler-repository-url-prefix

   Code Block
   language bash theme Emacs title Finish repo linenumbers true
   [root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# bash etc/finish-install.sh sparqler-user sparqler-pw https://ec2-54-175-59-6.compute-1.amazonaws.com/sparqler User metadata created. Updating data model ontology from jar, please wait... Updated data model ontology from jar

4.7 Verify the public sparql endpoint has been customized correctly

In a browser, navigate to the public sparql endpoint admin console and log in using the public sparqler repository administrator credentials.

Image Removed

5. Update the local-node.xml file

The local-node.xml file provides information to the applications about the node. In the SSH terminal:

1. Verify the eagle-i node is up and running

...

5. Production instance only

There are additional steps that need to happen in order to make your instance production ready. For more information about elastic IP address, please see the Amazon Documentation: Elastic IP Addresses.

...

...

1. Allocate an elastic IP address

...

1. Associate the elastic IP to your public hostname

...

...

1. Install your SSL Certificate

...

Getting a time out error when trying to navigate to the application in a browser

Double check your security groups and make sure that inbound traffic on HTTPS is allowed.

Error when running finish-install.sh : FAILED to create metadata for user, HTTP status=302

...