This document details the procedure for creating an eagle-i institutional node as a virtual server (or instance) in the Amazon Elastic Compute Cloud (or EC2). Once created, the eagle-i node will operate entirely in the cloud. However, you will retain administrative responsibility over its operation and maintenance, and in particular, you will be responsible for running upgrade scripts when new eagle-i software is released. We do not expect these tasks to be complex, though basic Unix skills are desirable. This solution is ideal for institutions that want to evaluate eagle-i or participate in the eagle-i network but do not have easy access to a data center service. Naturally, the AWS service will incur operational costs (for pricing details, consult the AWS website).
The installation procedure is simple and does not require specialized technical skills. It will allow you to get an eagle-i node up and running in a short amount of time. For a production system, you may need to involve your IT department, in a limited way.
What this is: an automated mechanism for instantiating an eagle-i node in the Amazon Cloud |
This procedure can be used to create an evaluation/development or a production eagle-i node:
Note that an evaluation or development node cannot be converted to a production node. |
As a pre-prerequisite, you will need to decide which type of installation you will need, evaluation/development or production eagle-i node.
You may need to involve your IT department to obtain the first 2 prerequisites for a . |
Public host name
Throughout this procedure, you will be using the AWS Management Console, and in particular the EC2 Dashboard and the Cloud Formation Dashboard. You may want to familiarize yourself with the console and bookmark it: https://console.aws.amazon.com |
For an evaluation node, there is no need to customize the installation any further. Using the public DNS as your hostname, you may now go to a browser and navigate to the eagle-i node to begin entering data, searching data, accessing the repository, etc. See verify the eagle-i node is up and running for more information on where to navigate to. |
There are additional steps that need to happen in order to make your instance production ready. For more information about elastic IP address, please see the Amazon Documentation: Elastic IP Addresses.
Please note that all the EC2 resources described below need to be allocated in the same availability zone |
Amazon allocates EC2 resources (IP addresses, virtual hardware) in specific facilities that are meant to cover different geographic regions (also called availability zones). We support three zones: US East (N. Virginia), US West (Oregon) and US West (N. California).
For more detailed information on creating and using a key pair with your EC2 instances, please see AWS Documentation: EC2 Key Pairs
.pem
extension (you may be prompted by your browser to select a location). Store it in a dedicated directory to which you will come back later, e.g. /my-home/aws/keys
For more detailed information on how to launch an EC2 Instance from an AMI, please see AWS Documentation: Launching an Instance.
Select the latest eagle-i AMI
eaglei-4.3.0-20160605 - ami-39e81654 Root device type: ebs Virtualization type: hvm |
Choose an Instance Type that is appropriate for your installation. For the available instance types as of August 2016, we recommend:
t2.micro
m3.medium
For more detailed information about the instance lifecycle, please see AWS Documentation: EC2 Instance Lifecycle.
For an evaluation node, there is no need to customize the installation any further. Using the public DNS as your hostname, you may now go to a browser and navigate to the eagle-i node to begin entering data, searching data, accessing the repository, etc. See verify the eagle-i node is up and running for more information on where to navigate to. |
In the SSH terminal:
Set the environment variable REPO_HOME to point to the repository home directory
export REPO_HOME=/opt/eaglei/repo
Set the environment variable SPARQLER_HOME to point to the SPARQLer home directory
export SPARQLER_HOME=/opt/eaglei/sparqler
echo ${REPO_HOME}
echo ${SPARQLER_HOME}
[root@ip-172-31-54-208 repo]# export REPO_HOME=/opt/eaglei/repo/ [root@ip-172-31-54-208 repo]# export SPARQLER_HOME=/opt/eaglei/sparqler/ [root@ip-172-31-54-208 repo]# echo ${REPO_HOME} /opt/eaglei/repo/ [root@ip-172-31-54-208 repo]# echo ${SPARQLER_HOME} /opt/eaglei/sparqler/ |
The following assumes that you are in the SSH terminal connected to your ec2 instance.
Go to the tomcat directory
cd /opt/apache-tomcat-7.0.39/
Stop tomcat using the shutdown wrapper script
sudo -su tomcat bin/shutdownwrapper.sh
Verify tomcat has been shutdown
ps aux | grep tomcat
[root@ip-172-31-54-208 log]# cd /opt/apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bin/shutdown.sh Using CATALINA_BASE: /opt/apache-tomcat-7.0.39 Using CATALINA_HOME: /opt/apache-tomcat-7.0.39 Using CATALINA_TMPDIR: /opt/apache-tomcat-7.0.39/temp Using JRE_HOME: /opt/jdk1.8.0_66 Using CLASSPATH: /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar [root@ip-172-31-54-208 apache-tomcat-7.0.39]# ps aux | grep tomcat root 1646 0.0 0.0 103312 876 pts/0 S+ 13:02 0:00 grep tomcat #### The following means that tomcat is still running #### tomcat 1674 93.2 11.4 3599164 116800 pts/0 Sl 13:08 0:03 /opt/jdk1.8.0_66/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat-7.0.39/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1536m -XX:+PrintGCDetails -Xloggc:/opt/apache-tomcat-7.0.39/logs/tomcat-gc.log -Djava.endorsed.dirs=/opt/apache-tomcat-7.0.39/endorsed -classpath /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar -Dcatalina.base=/opt/apache-tomcat-7.0.39 -Dcatalina.home=/opt/apache-tomcat-7.0.39 -Djava.io.tmpdir=/opt/apache-tomcat-7.0.39/temp org.apache.catalina.startup.Bootstrap start |
configuration.properties
for the repositorycd ${REPO_HOME}
vim configuration.properties
eaglei.repository.namespace
eaglei.repository.title
eaglei.repository.logo
Save the changes.
[root@ip-172-31-54-208 ~]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# vim configuration.properties ##### configuration.properties file #### eaglei.repository.namespace = http://ec2-54-175-59-6.compute-1.amazonaws.com/i/ eaglei.repository.title = AMI Test Repository eaglei.repository.logo = https://alaska.qa.eagle-i.net:8443/sweet/images/eaglei-medium-blue.png |
cd ${REPO_HOME}
rm -fr db/
REPOADMIN
and REPOADMINPW
with the credentials for the repository administrator.bash etc/prepare-install.sh REPOADMIN REPOADMINPW ${REPO_HOME}
Modify the ownership of the derby database to be owned by the tomcat user
chown -R tomcat:tomcat db/
[root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# rm -r db/ [root@ip-172-31-54-208 repo]# bash etc/prepare-install.sh ami-admin-user ami-admin-pw ${REPO_HOME} _RUNJAVA set to "/opt/jdk1.8.0_66/jre/bin/java" java version "1.8.0_66" Java(TM) SE Runtime Environment (build 1.8.0_66-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode) ---all superuser logins--- ami-admin-user [root@ip-172-31-54-208 repo]# chown -R tomcat:tomcat db/ |
Go to the tomcat directory
cd /opt/apache-tomcat-7.0.39/
Start tomcat using the startup wrapper script
sudo -su tomcat bin/startupwrapper.sh
Wait for tomcat to finish startup.
[root@ip-172-31-54-208 opt]# cd apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bash bin/startupwrapper.sh Waiting for Tomcat to startup ... Tomcat startup finished in ~120 seconds |
cd ${REPO_HOME}
REPOADMIN
and REPOADMINPW
with the credentials for the repository administrator.bash etc/finish-install.sh REPOADMIN REPOADMINPW repository-url-prefix
Run the upgrade script in the etc directory. Replace REPOADMIN
and REPOADMINPW
with the credentials for the repository administrator.
bash etc/upgrade.sh REPOADMIN REPOADMINPW repository-url-prefix
[root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# bash etc/finish-install.sh ami-admin-user ami-admin-pw https://ec2-54-175-59-6.compute-1.amazonaws.com User metadata created. Updating data model ontology from jar, please wait... Updated data model ontology from jar. [root@ip-172-31-54-208 repo]# bash etc/upgrade.sh ami-admin-user ami-admin-pw https://ec2-54-175-59-6.compute-1.amazonaws.com ***** Checking and Upgrading Repository's Internal Graphs INFO: Skipping, internal graph is already up to date: http://eagle-i.org/ont/repo/1.0/ INFO: Skipping, internal graph is already up to date: http://eagle-i.org/ont/repo/1.0/NG_Query .....lots of output and stuff here ..... OK - specials all succeeded. rm -rf /tmp/repo-upgrade-9784 |
In a browser, navigate to the repository admin console and log in using the repository administrator credentials.
Additional system users are needed for several automated processes. The following users need to be created:
eaglei.datatools.globals.user & eaglei.datatools.globals.password
eaglei.anonymousExtended.user & eaglei.anonymousExtended.password
In the SSH terminal:
[root@ip-172-31-54-208 sparqler]# cd /opt/eaglei/conf/ [root@ip-172-31-54-208 conf]# vim eagle-i-apps-credentials.properties ##### eagle-i-apps-credentials.properties file #### # GLOBAL SERVICE eaglei.datatools.globals.user=dtgu eaglei.datatools.globals.password=dtgp # Catalyst service eaglei.anonymousExtended.user=eaeu eaglei.anonymousExtended.password=eaep |
The following assumes that you are in the SSH terminal connected to your ec2 instance.
Go to the tomcat directory
cd /opt/apache-tomcat-7.0.39/
Stop tomcat using the shutdown wrapper script
sudo -su tomcat bin/shutdownwrapper.sh
Verify tomcat has been shutdown
ps aux | grep tomcat
[root@ip-172-31-54-208 log]# cd /opt/apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bin/shutdown.sh Using CATALINA_BASE: /opt/apache-tomcat-7.0.39 Using CATALINA_HOME: /opt/apache-tomcat-7.0.39 Using CATALINA_TMPDIR: /opt/apache-tomcat-7.0.39/temp Using JRE_HOME: /opt/jdk1.8.0_66 Using CLASSPATH: /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar [root@ip-172-31-54-208 apache-tomcat-7.0.39]# ps aux | grep tomcat root 1646 0.0 0.0 103312 876 pts/0 S+ 13:02 0:00 grep tomcat #### The following means that tomcat is still running #### tomcat 1674 93.2 11.4 3599164 116800 pts/0 Sl 13:08 0:03 /opt/jdk1.8.0_66/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat-7.0.39/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1536m -XX:+PrintGCDetails -Xloggc:/opt/apache-tomcat-7.0.39/logs/tomcat-gc.log -Djava.endorsed.dirs=/opt/apache-tomcat-7.0.39/endorsed -classpath /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar -Dcatalina.base=/opt/apache-tomcat-7.0.39 -Dcatalina.home=/opt/apache-tomcat-7.0.39 -Djava.io.tmpdir=/opt/apache-tomcat-7.0.39/temp org.apache.catalina.startup.Bootstrap start |
configuration.properties
for the public sparql endpointcd ${SPARQLER_HOME}
configuration.properties
file in a text editor of your choicevim configuration.properties
eaglei.repository.namespace
eaglei.repository.title
eaglei.repository.logo
Save the changes.
[root@ip-172-31-54-208 ~]# cd ${SPARQLER_HOME} [root@ip-172-31-54-208 sparqler]# vim configuration.properties ##### configuration.properties file #### eaglei.repository.namespace = http://ec2-54-175-59-6.compute-1.amazonaws.com/i/ eaglei.repository.title = AMI Test Repository eaglei.repository.logo = https://alaska.qa.eagle-i.net:8443/sweet/images/eaglei-medium-blue.png |
cd ${REPO_HOME}
SPARQLADMINUSER
and SPARQLADMINPW
with the credentials for the public sparqler repository administrator.bash etc/prepare-install.sh SPARQLADMINUSER SPARQLADMINPW ${REPO_HOME} sparqler-users.derby
Modify the ownership of the derby database to be owned by the tomcat user
chown -R tomcat:tomcat db/
[root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# bash etc/prepare-install.sh sparqler-user sparqler-pw ${REPO_HOME} sparqler-users.derby _RUNJAVA set to "/opt/jdk1.8.0_66/jre/bin/java" java version "1.8.0_66" Java(TM) SE Runtime Environment (build 1.8.0_66-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode) ---all superuser logins--- sparqler-user [root@ip-172-31-54-208 repo]# chown -R tomcat:tomcat db/ |
cd /opt/eaglei/conf/
eagle-i-apps.properties
fileim eagle-i-apps.properties
eagle-i-apps.properties
file to define the source and target repository for the public sparql endpoint:eaglei.sparqler.source.URL
, this will be your main repository URLeaglei.sparqler.target.URL
, this will be the URL of your public sparql endpoint. Typically it is a concatenation of the main repository URL with 'sparqler'eagle-i-apps-credentials.properties
file.eagle-i-apps-credentials.properties
file.eaglei.sparqler.target.user
eaglei.sparqler.target.password
Save your changes
[root@ip-172-31-54-208 sparqler]# cd /opt/eaglei/conf/ [root@ip-172-31-54-208 conf]# vim eagle-i-apps.properties ##### eagle-i-apps.properties file ##### ## SPARQLER ### URL of source-repository (from which the sparqler reads public information): eaglei.sparqler.source.URL = https://ec2-54-175-59-6.compute-1.amazonaws.com/ ### URL of target (i.e., sparqler) repository: eaglei.sparqler.target.URL = https://ec2-54-175-59-6.compute-1.amazonaws.com/sparqler/ ######################################## [root@ip-172-31-54-208 conf]# vim eagle-i-apps-credentials.properties ##### eagle-i-apps-credentials.properties file ##### eaglei.sparqler.target.user=sparqler-user eaglei.sparqler.target.password=sparqler-password #################################################### |
Go to the tomcat directory
cd /opt/apache-tomcat-7.0.39/
Start tomcat using the startup wrapper script
sudo -su tomcat bin/startupwrapper.sh
Wait for tomcat to finish startup.
[root@ip-172-31-54-208 opt]# cd apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bash bin/startupwrapper.sh Waiting for Tomcat to startup ... Tomcat startup finished in ~115 seconds |
cd ${REPO_HOME}
SPARQLADMINUSER
and SPARQLADMINPW
with the credentials for the public sparqler repository administrator.bash etc/finish-install.sh SPARQLADMINUSER
SPARQLADMINPW
sparqler-repository-url-prefix
[root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# bash etc/finish-install.sh sparqler-user sparqler-pw https://ec2-54-175-59-6.compute-1.amazonaws.com/sparqler User metadata created. Updating data model ontology from jar, please wait... Updated data model ontology from jar |
In a browser, navigate to the public sparql endpoint admin console and log in using the public sparqler repository administrator credentials.
local-node.xml
fileThe local-node.xml
file provides information to the applications about the node. In the SSH terminal:
There are additional steps that need to happen in order to make your instance production ready. For more information about elastic IP address, please see the Amazon Documentation: Elastic IP Addresses.
Troubleshooting
Double check your security groups and make sure that inbound traffic on HTTPS is allowed.
The repository url must match exactly. Typically this happens when http is typed instead of https.