This document details the procedure for creating an eagle-i institutional node as a virtual server (or instance) in the Amazon Elastic Compute Cloud (or EC2). Once created, the eagle-i node will operate entirely in the cloud. However, you will retain administrative responsibility over its operation and maintenance, and in particular, you will be responsible for running upgrade scripts when new eagle-i software is released. We do not expect these tasks to be complex, though basic Unix skills are desirable. This solution is ideal for institutions that want to evaluate eagle-i or participate in the eagle-i network but do not have easy access to a data center service. Naturally, the AWS service will incur operational costs (for pricing details consult the AWS website).
The installation procedure is simple and does not require specialized technical skills. It will allow you to get an eagle-i node up and running in a short amount of time. For a production system, you may need to involve your IT department, in a limited way.
What this is: an automated mechanism for instantiating an eagle-i node in the Amazon Cloud, and for performing subsequent upgrades
This procedure can be used to create an evaluation/development or a production eagle-i node:
Note that an evaluation or development node cannot be converted to a production node.
As a pre-prerequisite, you will need to decide which type of installation you will need, evaluation/development or production eagle-i node.
You may need to involve your IT department to obtain the first 2 prerequisites for a .
Public host name
Throughout this procedure, you will be using the AWS Management Console, and in particular the EC2 Dashboard and the Cloud Formation Dashboard. You may want to familiarize yourself with the console and bookmark it: https://console.aws.amazon.com
Please note that all the EC2 resources described below need to be allocated in the same availability zone
Amazon allocates EC2 resources (IP addresses, virtual hardware) in specific facilities that are meant to cover different geographic regions (also called availability zones). We support three zones: US East (N. Virginia), US West (Oregon) and US West (N. California).
If you are installing a instance: ask the administrator of your domain to create a DNS record that maps the public hostname you previously selected to this IP address. Your domain administrator is usually somebody in your IT department. Creation of the eagle-i node (step 2) will fail if your public hostname does not resolve to this IP address.
For more detailed information on creating and using a key pair with your EC2 instances, please see AWS Documentation: EC2 Key Pairs
.pemextension (you may be prompted by your browser to select a location). Store it in a dedicated directory to which you will come back later, e.g.
Before proceeding: if you are creating a instance, make sure that your public hostname resolves to the elastic IP address created in 1.2. You can use an online service to check, for example: http://www.whatsmydns.net/
For more detailed information on how to launch an EC2 Instance from an AMI, please see AWS Documentation: Launching an Instance.
Select the latest eagle-i AMI
eaglei-4.3.0-20160605 - ami-39e81654
Root device type: ebs Virtualization type: hvm
Choose an Instance Type that is appropriate for your installation. For the available instance types as of August 2016, we recommend:
For more detailed information about the instance lifecycle, please see AWS Documentation: EC2 Instance Lifecycle.
For an evaluation node, there is no need to customize the installation any further. You may now go to a browser and navigate to the eagle-i node to begin entering data, searching data, accessing the repository, etc.
In the SSH terminal, go to the tomcat directory -
Stop tomcat using the shutdown script -
sudo -su tomcat bin/shutdown.sh
Verify tomcat has been shutdown -
ps aux | grep tomcat
[root@ip-172-31-54-208 log]# cd /opt/apache-tomcat-7.0.39/ [root@ip-172-31-54-208 apache-tomcat-7.0.39]# sudo -su tomcat bin/shutdown.sh Using CATALINA_BASE: /opt/apache-tomcat-7.0.39 Using CATALINA_HOME: /opt/apache-tomcat-7.0.39 Using CATALINA_TMPDIR: /opt/apache-tomcat-7.0.39/temp Using JRE_HOME: /opt/jdk1.8.0_66 Using CLASSPATH: /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar [root@ip-172-31-54-208 apache-tomcat-7.0.39]# ps aux | grep tomcat root 1646 0.0 0.0 103312 876 pts/0 S+ 13:02 0:00 grep tomcat #### The following means that tomcat is still running #### tomcat 1674 93.2 11.4 3599164 116800 pts/0 Sl 13:08 0:03 /opt/jdk1.8.0_66/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat-7.0.39/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1536m -XX:+PrintGCDetails -Xloggc:/opt/apache-tomcat-7.0.39/logs/tomcat-gc.log -Djava.endorsed.dirs=/opt/apache-tomcat-7.0.39/endorsed -classpath /opt/apache-tomcat-7.0.39/bin/bootstrap.jar:/opt/apache-tomcat-7.0.39/bin/tomcat-juli.jar -Dcatalina.base=/opt/apache-tomcat-7.0.39 -Dcatalina.home=/opt/apache-tomcat-7.0.39 -Djava.io.tmpdir=/opt/apache-tomcat-7.0.39/temp org.apache.catalina.startup.Bootstrap start
Click continue to accept the defaults. The next window will provide entry fields for a few parameters necessary to configure the EC2 instance and the eagle-i software. They are described below, in alphabetical order. NB Unfortunately, we cannot control the order in which these parameters are presented in the Cloud Formation window, so the list below will not necessarily match the order you see.
Once you have entered values for all these parameters, click continue and skip the following dialogue window (Add Tags) by clicking continue again. If any of the parameter values is invalid, a red error message will appear, at the bottom of the dialogue window. If this is the case, return to the parameter screen and enter a valid value. Once all your parameters validate, you will be presented with a summary of the information you provided. If the information is correct, click continue. A screen indicating the stack creation is in progress will appear:
You may close the Cloud Formation dialogue window.
Monitor the progress of your stack creation in the Cloud Formation dashboard -- detailed progress will be shown in the Events tab (hit refresh to get updated views). Once the stack is created, its status will be CREATE_COMPLETE.
At this point you have an EC2 instance that is in the process of booting.
If you are installing an evaluation or development instance, take note of the public hostname that was dynamically assigned. You will find this in the Outputs tab of the Cloud Formation Dashboard, in the PublicDnsName row. The name will be in the amazonws.com domain, for example:
ec2-54-225-140-48.compute-1.amazonaws.com. Note how ugly this name is, which is why we do not recommend it for a production system - it would need to be typed by your users and it would appear in your Linked Open Data.
You will be able to manage your EC2 instance from the EC2 Dashboard. In the left navigation bar, open the Instances section and select Instances. You will see one row with your instance information. Check the box on the left and click on the Actions button to obtain a menu of possible actions.
As soon as the EC2 instance boots, the eagle-i installation process automatically starts. It takes care of downloading, installing and configuring eagle-i prerequisites (Java, Tomcat, Postfix) and eagle-i software. When this process completes, which will take a little while, you will have an eagle-i node up and running.
If you would like to monitor the progress of this step, start an SSH session with your newly created instance (see Appendix 1). Otherwise, get a coffee and skip the rest of this section.
From your SSH terminal, issue the following command:
tail -f /var/log/bootstrap.log
You will see the current progress of the installation procedure. You will see the message
bootstrap.sh: finished when the procedure completes.
Log in with one of the users you created and verify you can access the SWEET workbench, create a test organization and publish it, verify it appears in search after being published, etc. You may want to compare your screens with our training node: https://training.eagle-i.net/sweet and https://training.eagle-i.net/institution
If your EC2 instance is successfully created, but your eagle-i node does not come up, SSH into your instance and look for error messages in the log (follow instructions in section 2.3. Monitor eagle-i installation). Some common errors are listed below. If you hit an error that is not listed yet, please let us know.
If you are installing a production environment and your hostname, at the time of installation, did not resolve to your elastic IP address, you will see a message like the following:
FAILED: user-specified hostname 'eagle-i.miskatonic.edu' doesn't resolve to ip address '220.127.116.11'
To get past this error:
Double check with your DNS administrator that the correct information was used. DNS changes may take time to propagate, so make sure your hostname resolves within the EC2 environment by SSHing into your instance and issuing the
host command. The answer should indicate that the host has your IP address, for example:
host eagle-i.miskatonic.edu eagle-i.miskatonic.edu has address 18.104.22.168
However, if there is not answer, the DNS mapping has not yet propagated and you will need to wait a bit longer.
The eagle-i bootstrapping script that runs when the EC2 instance starts downloads all the files it needs from
open.med.harvard.edu. If for some reason the download server cannot be reached, you will see an error message in the logs, for example:
bootstrap.sh: checkout eaglei-ansible repository svn: PROPFIND of '/svn/eagle-i-install/!svn/bln/217': could not connect to server (https://open.med.harvard.edu) 2013-04-17T17:22:56Z: bootstrap.sh: FAILED: error checking out eaglei-ansible repository: 1
This lack of connectivity is likely temporary. Wait a little while and reboot the EC2 instance to restart the bootstrapping procedure (see the Maintenance Tasks section for instructions on rebooting your instance).
The install procedure above initially configures eagle-i with a self-signed certificate; this is acceptable for an evaluation or development environment, but not for a production instance. In order to finalize the installation of a production instance, please follow the steps below.
Obtain, from the person who purchased the certificate, the following files:
You will need to copy these three files into your EC2 instance, to the directory
These files are security-sensitive. Please make sure they are transferred to you in a secure manner (e.g. a memory stick, or using the scp command) and delete them from your personal machine once they are installed. If in doubt, please ask for assistance of your IT department.
For example, assuming the files are named as above and located in the directory
/my-home/aws/cert, issue the following command in your terminal (substitute your own file names and public hostname):
cd /my-home/aws/cert scp -i /my-home/aws/keys/eagle-i-key.pem key.pem cert.crt ca.crt firstname.lastname@example.org:/opt/eaglei/install/.
Follow the instructions in the section Transferring files with PSCP at the end of the AWS/PuTTY guide.
SSH into your EC2 instance, as described in Appendix 1.
In your SSH session, run the following commands (substitute the actual names of your files):
cd /opt/eaglei/install sh /bin/cert-install.sh -b ca.crt -c cert.crt -k key.pem
At the prompt, enter the key's password. When the script finishes, tomcat will restart. After restart, verify that your certificate is correctly installed by entering your public hostname in an online SSL validation service, such as http://www.geocerts.com/ssl_checker
Finally, remove the security-sensitive files used for installation, e.g.:
rm ca.crt cert.crt key.pem
Your EC2 instance contains scripts to upgrade the eagle-i software upon new releases and patches. You will need to manually trigger this process, as follows:
cd /opt/eaglei/install/ansible source setup.sh && ansible-playbook -e "upgrade=true" install.yml
Your EC2 instance can be started and stopped from the EC2 Dashboard. In the left navigation bar, open the Instances section and select Instances. You will see one row with your instance information. Check the box on the left and click on the Actions button to obtain a menu of possible actions.
Most maintenance tasks require that you SSH into your EC2 instance, see Appendix 1.
If you need to modify your eagle-i configuration (for example, to add a Google analytics ID) edit one or several of the following files, as appropriate. See the Repository Configuration Guide, and the Application Configuration Guide.
/opt/eaglei/conf/eagle-i-apps.properties /opt/eaglei/conf/eagle-i-apps-credentials.properties /opt/eaglei/conf/whoami.xml /opt/eaglei/conf/repo/configuration.properties /opt/eaglei/conf/sparqler/configuration.properties
Your eagle-i node is set to back up its repository data every day. The backups are located in
/opt/eaglei/repo/backup/. To restore a backup, use the
move-everything.sh script located in
/opt/eaglei/repo/etc. For more information, see the Repository Installation, Upgrade and Administration Guide.
It is very easy to delete EC2 instances that are used for evaluation or as development environments and create new ones, so experimentation is encouraged. Please note that when you no longer need an EC2 instance, you need to delete all its associated resources (which incur charges individually, even if they are not attached to an instance). The easiest way to do so is as follows:
For both methods, use as server name your public hostname (or your elastic IP address, if the hostname is not working) and as username
The ssh command will not accept a key that has broad file system permissions. Open a terminal and change to the directory where you've stored the key, e.g.:
Issue the following command to set the correct permissions:
chmod 700 . chmod 600 *
Issue the ssh command, specifying the root user, for example: (you will need to substitute the full path of your key and the public hostname of your instance)
ssh -i /my-home/aws/keys/eagle-i-key.pem email@example.com
At this point you have a remote terminal session with your EC2 instance.
See also this guide at AWS documentation central: Connecting to Linux/UNIX Instances Using SSH
Follow this guide at AWS documentation central: Connecting to Linux/UNIX Instances from Windows Using PuTTY. In the connection window, enter your public hostname (either the amazon-provided or that which you mapped to your elastic IP address) and when prompted for a user, use
Double check your security groups and make sure that inbound traffic on HTTPS is allowed.