This document details the procedure for creating an eagle-i institutional node as a virtual server (or instance) in the Amazon Elastic Compute Cloud (or EC2). Once created, the eagle-i node will operate entirely in the cloud. However, you will retain administrative responsibility over its operation and maintenance, and in particular, you will be responsible for running upgrade scripts when new eagle-i software is released. We do not expect these tasks to be complex, though basic Unix skills are desirable. This solution is ideal for institutions that want to evaluate eagle-i or participate in the eagle-i network but do not have easy access to a data center service. Naturally, the AWS service will incur operational costs (for pricing details, consult the AWS website).
The installation procedure is simple and does not require specialized technical skills. It will allow you to get an eagle-i node up and running in a short amount of time. For a production system, you may need to involve your IT department, in a limited way.
What this is: an automated mechanism for instantiating an eagle-i node in the Amazon Cloud |
This procedure can be used to create an evaluation/development or a production eagle-i node:
Note that an evaluation or development node cannot be converted to a production node. |
As a pre-prerequisite, you will need to decide which type of installation you will need, evaluation/development or production eagle-i node.
You may need to involve your IT department to obtain the first 2 prerequisites for a . |
Public host name
Throughout this procedure, you will be using the AWS Management Console, and in particular the EC2 Dashboard and the Cloud Formation Dashboard. You may want to familiarize yourself with the console and bookmark it: https://console.aws.amazon.com |
For an evaluation node, there is no need to customize the installation any further. Using the public DNS as your hostname, you may now go to a browser and navigate to the eagle-i node to begin entering data, searching data, accessing the repository, etc. See verify the eagle-i node is up and running for more information on where to navigate to. |
Please note that all the EC2 resources described below need to be allocated in the same availability zone |
Amazon allocates EC2 resources (IP addresses, virtual hardware) in specific facilities that are meant to cover different geographic regions (also called availability zones). We support three zones: US East (N. Virginia), US West (Oregon) and US West (N. California).
For more detailed information on creating and using a key pair with your EC2 instances, please see AWS Documentation: EC2 Key Pairs
.pem
extension (you may be prompted by your browser to select a location). Store it in a dedicated directory to which you will come back later, e.g. /my-home/aws/keys
For more detailed information on how to launch an EC2 Instance from an AMI, please see AWS Documentation: Launching an Instance.
Select the latest eagle-i AMI
eaglei-4.3.0-20160605 - ami-39e81654 Root device type: ebs Virtualization type: hvm |
Choose an Instance Type that is appropriate for your installation. For the available instance types as of August 2016, we recommend:
t2.micro
m3.medium
For more detailed information about the instance lifecycle, please see AWS Documentation: EC2 Instance Lifecycle.
For an evaluation node, there is no need to customize the installation any further. Using the public DNS as your hostname, you may now go to a browser and navigate to the eagle-i node to begin entering data, searching data, accessing the repository, etc. See verify the eagle-i node is up and running for more information on where to navigate to. |
In the SSH terminal:
Set the environment variable REPO_HOME to point to the repository home directory
export REPO_HOME=/opt/eaglei/repo
Set the environment variable SPARQLER_HOME to point to the SPARQLer home directory
export SPARQLER_HOME=/opt/eaglei/sparqler
echo ${REPO_HOME}
echo ${SPARQLER_HOME}
[root@ip-172-31-54-208 repo]# export REPO_HOME=/opt/eaglei/repo/ [root@ip-172-31-54-208 repo]# export SPARQLER_HOME=/opt/eaglei/sparqler/ [root@ip-172-31-54-208 repo]# echo ${REPO_HOME} /opt/eaglei/repo/ [root@ip-172-31-54-208 repo]# echo ${SPARQLER_HOME} /opt/eaglei/sparqler/ |
The following assumes that you are in the SSH terminal connected to your ec2 instance.
Go to the tomcat directory
cd /opt/apache-tomcat-7.0.39/
Stop tomcat using the shutdown wrapper script
sudo -su tomcat bin/shutdownwrapper.sh
Verify tomcat has been shutdown
ps aux | grep tomcat
[root@ip-172-31-54-208 log]# cd /opt/apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bin/shutdown.sh Using CATALINA_BASE: /opt/apache-tomcat-7.0.39 Using CATALINA_HOME: /opt/apache-tomcat-7.0.39 Using CATALINA_TMPDIR: /opt/apache-tomcat-7.0.39/temp Using JRE_HOME: /opt/jdk1.8.0_66 Using CLASSPATH: /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar [root@ip-172-31-54-208 apache-tomcat-7.0.39]# ps aux | grep tomcat root 1646 0.0 0.0 103312 876 pts/0 S+ 13:02 0:00 grep tomcat #### The following means that tomcat is still running #### tomcat 1674 93.2 11.4 3599164 116800 pts/0 Sl 13:08 0:03 /opt/jdk1.8.0_66/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat-7.0.39/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1536m -XX:+PrintGCDetails -Xloggc:/opt/apache-tomcat-7.0.39/logs/tomcat-gc.log -Djava.endorsed.dirs=/opt/apache-tomcat-7.0.39/endorsed -classpath /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar -Dcatalina.base=/opt/apache-tomcat-7.0.39 -Dcatalina.home=/opt/apache-tomcat-7.0.39 -Djava.io.tmpdir=/opt/apache-tomcat-7.0.39/temp org.apache.catalina.startup.Bootstrap start |
configuration.properties
for the repositorycd ${REPO_HOME}
vim configuration.properties
eaglei.repository.namespace
eaglei.repository.title
eaglei.repository.logo
Save the changes.
[root@ip-172-31-54-208 ~]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# vim configuration.properties ##### configuration.properties file #### eaglei.repository.namespace = http://ec2-54-175-59-6.compute-1.amazonaws.com/i/ eaglei.repository.title = AMI Test Repository eaglei.repository.logo = https://alaska.qa.eagle-i.net:8443/sweet/images/eaglei-medium-blue.png |
cd ${REPO_HOME}
rm -fr db/
REPOADMIN
and REPOADMINPW
with the credentials for the repository administrator.bash etc/prepare-install.sh REPOADMIN REPOADMINPW ${REPO_HOME}
Modify the ownership of the derby database to be owned by the tomcat user
chown -R tomcat:tomcat db/
[root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# rm -r db/ [root@ip-172-31-54-208 repo]# bash etc/prepare-install.sh ami-admin-user ami-admin-pw ${REPO_HOME} _RUNJAVA set to "/opt/jdk1.8.0_66/jre/bin/java" java version "1.8.0_66" Java(TM) SE Runtime Environment (build 1.8.0_66-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode) ---all superuser logins--- ami-admin-user [root@ip-172-31-54-208 repo]# chown -R tomcat:tomcat db/ |
Go to the tomcat directory
cd /opt/apache-tomcat-7.0.39/
Start tomcat using the startup wrapper script
sudo -su tomcat bin/startupwrapper.sh
Wait for tomcat to finish startup.
[root@ip-172-31-54-208 opt]# cd apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bash bin/startupwrapper.sh Waiting for Tomcat to startup ... Tomcat startup finished in ~120 seconds |
cd ${REPO_HOME}
REPOADMIN
and REPOADMINPW
with the credentials for the repository administrator.bash etc/finish-install.sh REPOADMIN REPOADMINPW repository-url-prefix
Run the upgrade script in the etc directory. Replace REPOADMIN
and REPOADMINPW
with the credentials for the repository administrator.
bash etc/upgrade.sh REPOADMIN REPOADMINPW repository-url-prefix
[root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# bash etc/finish-install.sh ami-admin-user ami-admin-pw https://ec2-54-175-59-6.compute-1.amazonaws.com User metadata created. Updating data model ontology from jar, please wait... Updated data model ontology from jar. [root@ip-172-31-54-208 repo]# bash etc/upgrade.sh ami-admin-user ami-admin-pw https://ec2-54-175-59-6.compute-1.amazonaws.com ***** Checking and Upgrading Repository's Internal Graphs INFO: Skipping, internal graph is already up to date: http://eagle-i.org/ont/repo/1.0/ INFO: Skipping, internal graph is already up to date: http://eagle-i.org/ont/repo/1.0/NG_Query .....lots of output and stuff here ..... OK - specials all succeeded. rm -rf /tmp/repo-upgrade-9784 |
In a browser, navigate to the repository admin console and log in using the repository administrator credentials.
Additional system users are needed for several automated processes. The following users need to be created:
eaglei.datatools.globals.user & eaglei.datatools.globals.password
eaglei.anonymousExtended.user & eaglei.anonymousExtended.password
In the SSH terminal:
[root@ip-172-31-54-208 sparqler]# cd /opt/eaglei/conf/ [root@ip-172-31-54-208 conf]# vim eagle-i-apps-credentials.properties ##### eagle-i-apps-credentials.properties file #### # GLOBAL SERVICE eaglei.datatools.globals.user=dtgu eaglei.datatools.globals.password=dtgp # Catalyst service eaglei.anonymousExtended.user=eaeu eaglei.anonymousExtended.password=eaep |
The following assumes that you are in the SSH terminal connected to your ec2 instance.
Go to the tomcat directory
cd /opt/apache-tomcat-7.0.39/
Stop tomcat using the shutdown wrapper script
sudo -su tomcat bin/shutdownwrapper.sh
Verify tomcat has been shutdown
ps aux | grep tomcat
[root@ip-172-31-54-208 log]# cd /opt/apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bin/shutdown.sh Using CATALINA_BASE: /opt/apache-tomcat-7.0.39 Using CATALINA_HOME: /opt/apache-tomcat-7.0.39 Using CATALINA_TMPDIR: /opt/apache-tomcat-7.0.39/temp Using JRE_HOME: /opt/jdk1.8.0_66 Using CLASSPATH: /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar [root@ip-172-31-54-208 apache-tomcat-7.0.39]# ps aux | grep tomcat root 1646 0.0 0.0 103312 876 pts/0 S+ 13:02 0:00 grep tomcat #### The following means that tomcat is still running #### tomcat 1674 93.2 11.4 3599164 116800 pts/0 Sl 13:08 0:03 /opt/jdk1.8.0_66/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat-7.0.39/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1536m -XX:+PrintGCDetails -Xloggc:/opt/apache-tomcat-7.0.39/logs/tomcat-gc.log -Djava.endorsed.dirs=/opt/apache-tomcat-7.0.39/endorsed -classpath /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar -Dcatalina.base=/opt/apache-tomcat-7.0.39 -Dcatalina.home=/opt/apache-tomcat-7.0.39 -Djava.io.tmpdir=/opt/apache-tomcat-7.0.39/temp org.apache.catalina.startup.Bootstrap start |
configuration.properties
for the public sparql endpointcd ${SPARQLER_HOME}
configuration.properties
file in a text editor of your choicevim configuration.properties
eaglei.repository.namespace
eaglei.repository.title
eaglei.repository.logo
Save the changes.
[root@ip-172-31-54-208 ~]# cd ${SPARQLER_HOME} [root@ip-172-31-54-208 sparqler]# vim configuration.properties ##### configuration.properties file #### eaglei.repository.namespace = http://ec2-54-175-59-6.compute-1.amazonaws.com/i/ eaglei.repository.title = AMI Test Repository eaglei.repository.logo = https://alaska.qa.eagle-i.net:8443/sweet/images/eaglei-medium-blue.png |
cd ${REPO_HOME}
SPARQLADMINUSER
and SPARQLADMINPW
with the credentials for the public sparqler repository administrator.bash etc/prepare-install.sh SPARQLADMINUSER SPARQLADMINPW ${REPO_HOME} sparqler-users.derby
Modify the ownership of the derby database to be owned by the tomcat user
chown -R tomcat:tomcat db/
[root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# bash etc/prepare-install.sh sparqler-user sparqler-pw ${REPO_HOME} sparqler-users.derby _RUNJAVA set to "/opt/jdk1.8.0_66/jre/bin/java" java version "1.8.0_66" Java(TM) SE Runtime Environment (build 1.8.0_66-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode) ---all superuser logins--- sparqler-user [root@ip-172-31-54-208 repo]# chown -R tomcat:tomcat db/ |
cd /opt/eaglei/conf/
eagle-i-apps.properties
fileim eagle-i-apps.properties
eagle-i-apps.properties
file to define the source and target repository for the public sparql endpoint:eaglei.sparqler.source.URL
, this will be your main repository URLeaglei.sparqler.target.URL
, this will be the URL of your public sparql endpoint. Typically it is a concatenation of the main repository URL with 'sparqler'eagle-i-apps-credentials.properties
file.eagle-i-apps-credentials.properties
file.eaglei.sparqler.target.user
eaglei.sparqler.target.password
Save your changes
[root@ip-172-31-54-208 sparqler]# cd /opt/eaglei/conf/ [root@ip-172-31-54-208 conf]# vim eagle-i-apps.properties ##### eagle-i-apps.properties file ##### ## SPARQLER ### URL of source-repository (from which the sparqler reads public information): eaglei.sparqler.source.URL = https://ec2-54-175-59-6.compute-1.amazonaws.com/ ### URL of target (i.e., sparqler) repository: eaglei.sparqler.target.URL = https://ec2-54-175-59-6.compute-1.amazonaws.com/sparqler/ ######################################## [root@ip-172-31-54-208 conf]# vim eagle-i-apps-credentials.properties ##### eagle-i-apps-credentials.properties file ##### eaglei.sparqler.target.user=sparqler-user eaglei.sparqler.target.password=sparqler-password #################################################### |
Go to the tomcat directory
cd /opt/apache-tomcat-7.0.39/
Start tomcat using the startup wrapper script
sudo -su tomcat bin/startupwrapper.sh
Wait for tomcat to finish startup.
[root@ip-172-31-54-208 opt]# cd apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bash bin/startupwrapper.sh Waiting for Tomcat to startup ... Tomcat startup finished in ~115 seconds |
cd ${REPO_HOME}
SPARQLADMINUSER
and SPARQLADMINPW
with the credentials for the public sparqler repository administrator.bash etc/finish-install.sh SPARQLADMINUSER
SPARQLADMINPW
sparqler-repository-url-prefix
[root@ip-172-31-54-208 sparqler]# cd ${REPO_HOME} [root@ip-172-31-54-208 repo]# bash etc/finish-install.sh sparqler-user sparqler-pw https://ec2-54-175-59-6.compute-1.amazonaws.com/sparqler User metadata created. Updating data model ontology from jar, please wait... Updated data model ontology from jar |
In a browser, navigate to the public sparql endpoint admin console and log in using the public sparqler repository administrator credentials.
local-node.xml
fileThe local-node.xml
file provides information to the applications about the node. In the SSH terminal:
cd /opt/eaglei/conf/
local-node.xml
fileim local-node.xml
Follow the instructions in the comments to update the <node></node>
section. Specifically, update lines 14, 21, 24, 26, 29.
<local> <!-- Global node information is required to access centrally curated resources --> <!-- This section should be used as is, unless otherwise directed --> <global URL="http://global.eagle-i.net"> <institutionList> <institution URI="http://purl.org/eagle-i/app-ext/#ERO_0000065_29" graphURI="test29"> <namespace>http://global.eagle-i.net/i/</namespace> <label>Centrally Curated Resources</label> <logoSrc>https://www.eagle-i.net/logos/logo_global.png</logoSrc> </institution> </institutionList> </global> <node URL="https://ec2-54-175-59-6.compute-1.amazonaws.com/"> <!-- REQUIRED: url for the node --> <institutionList> <!-- REQUIRED: URI and graphURI for the institution/repository --> <!-- NB: URI will eventually be allocated by eagle-i's global repository; --> <!-- for the time being, choose a URI in your repository's instance namespace. --> <!-- This URI doesn't need to be resolvable, it is simply an identifier --> <!-- NB: graphURI is required but is not currently used. --> <institution URI="http://unique.key.for/insitution" graphURI="inst"> <!-- REQUIRED: This is the namespace that the instances are in for this repository. --> <!-- The namespace needs to be a well formed url --> <namespace>http://ec2-54-175-59-6.compute-1.amazonaws.com/i/</namespace> <!-- REQUIRED: This is the label that is used in the UI display. --> <label>AMI test repository</label> <!-- OPTIONAL: Logo to be displayed for institution. --> <!-- If not supplied, the default is blank square. --> <logoSrc>https://alaska.qa.eagle-i.net:8443/sweet/images/eaglei-medium-blue.png</logoSrc> </institution> </institutionList> </node> </local> |
Log in with one of the users you created and verify you can access the SWEET workbench, create a test organization and publish it, verify it appears in search after being published, etc. You may want to compare your screens with our training node: https://training.eagle-i.net/sweet and https://training.eagle-i.net/institution
There are additional steps that need to happen in order to make your instance production ready. For more information about elastic IP address, please see the Amazon Documentation: Elastic IP Addresses.
BEFORE PROCEEDING with a instance:
|
The install procedure above initially configures eagle-i with a self-signed certificate; this is acceptable for an evaluation or developmentenvironment, but not for a production instance. In order to finalize the installation of a production instance, please follow the steps below.
These files are security-sensitive. Please make sure they are transferred to you in a secure manner (e.g. a memory stick, or using the scp command) and delete them from your personal machine once they are installed. If in doubt, please ask for assistance of your IT department. |
Obtain, from the person who purchased the certificate, the following files:
key.pem
cert.crt
ca.crt
Assuming the files are located in a directory /my-home/aws/cert
on your machine, here are two methods for securely transferring the files to your EC2 instance:
/opt/eaglei/install.
skc-macbook:~ sophia$ ssh -i "aws-eaglei-ami.pem" root@ec2-52-207-177-113.compute-1.amazonaws.com Last login: Wed Aug 10 13:19:32 2016 from 134.174.140.154 [root@ip-172-31-54-208 ~]# mkdir /opt/eaglei/install |
In another terminal window, go to the directory containing the certificate files
cd /my-home/aws/cert
Do a SCP of the files to the newly created install folder on your EC2 instance (substitute your own file names and public hostname)
scp -i "aws-eaglei-ami.pem" key.pem cert.crt ca.crt root@ec2-52-207-177-113.compute-1.amazonaws.com
cd /my-home/aws/cert scp -i "aws-eaglei-ami.pem" key.pem cert.crt ca.crt root@ec2-52-207-177-113.compute-1.amazonaws.com:/opt/eaglei/install/. |
In the SSH terminal,
sh /bin/cert-install.sh -b ca.crt -c cert.crt -k key.pem
cert-install.sh
script is finished.Using an online SSL validation service, such as http://www.geocerts.com/ssl_checker, enter your public hostname to verify that the certificate has been installed correctly
Be sure to remove the security sensitive files used for installation
[root@ip-172-31-54-208 install]# rm ca.crt cert.crt key.pem |
Troubleshooting
Double check your security groups and make sure that inbound traffic on HTTPS is allowed.
The repository url must match exactly. Typically this happens when http is typed instead of https.