Skip to content

LapDevelopment_ServerDeployment

Jump to bottom
MilenKouylekov edited this page Feb 1, 2016 · 34 revisions

Background

The instructions on this page are intended to provide a step by step installation of a lap+galaxy instance with:

  • Abel Cluster Support
  • FEIDE Authetification
  • POSTGRES Data Storage
  • Gold Accounting

MongoDB

As a prerequisite to LAP installation, there must be a MongoDB database available for access by the LAP user. Some notes on how MongoDB is configured on the LAP servers is available on the LapDevelopment/MongoDB, but for local installations

  • Install MongoDB via your package manager, e.g.

      yum -y install mongodb-server

  • Confirm that the database directory (/var/lib/mongodb/ by default) is available;

  • Optionally, review MongoDB settings (e.g. in /etc/mongodb.conf and /etc/sysconfig/mongod);

  • Start the server and optionally enable automated start-up, e.g.

      /etc/init.d/mongodb start

By default, MongoDB initially allocates database space relatively generously (at around three gigabytes, it appears). If disk space is at a premium (as can be the case on a laptop :-), consider adding the --smallfiles option to the start-up sequence of the MongoDB server.

Download LAP

On the (non-Galaxy) LAP side, the following components are required: the LAP Tree, Library, tool descriptions, and (optionally) operational scripts.

  cd /home/laportal
  svn co http://svn.emmtee.net/lap/trunk/tree
  svn co http://svn.emmtee.net/lap/trunk/library
  svn co http://svn.emmtee.net/lap/trunk/tools
  svn co http://svn.emmtee.net/lap/trunk/operation

Create Environment Variables File

Create a file with the name env.sh in /home/laportal that contains all the environment variables needed to run LAP

  export LAPTREE=/home/laportal/tree
  export LAPLIBRARY=/home/laportal/library
  export LAPSTORE=mongodb://127.0.0.1:27017/lapstore

Galaxy

In early 2016, there are several versions of Galaxy in use with LAP. However, development focus now shifts towards the 2015.03 release, and there is a pre-configured snapshot available in SVN with local extensions (e.g. custom datatypes) and the basic configuration to use the LAP Tool descriptions (assuming the above locations):

  cd /home/laportal
  svn co http://svn.emmtee.net/lap/trunk/local/galaxy

When using this version, no symbolic links should be required (assuming the LAP components reside in the recommended location), i.e. config/galaxy.ini already points to the LAP tool configuration:

  tool_config_file = /home/laportal/tools/config.xml
  tool_path = /home/laportal/tools

To start Galaxy:

  cd /home/laportal/galaxy
  ./run.sh > main.log

At this point, with a bit of luck, the built-in Galaxy web server will be listening at http://localhost:8080.

Except for (currently, a small number of) LAP-specific extensions, this Galaxy version corresponds to the following release:

  git clone https://github.com/galaxyproject/galaxy/
  cd galaxy
  git checkout release_15.03

Installing Postgres

PostgresSQL is the database used to store users data. The database is provided by UIO DB Hotell

To install Postgres on Fedora/Redhat use the following command:

dnf install postgresql postgresql-server postgresql-devel postgresql-contrib

For test purposes you can initialize postgres server on local machine that can be used by galaxy. This server will be replaced in the production and development version of LAP. The local installation of posgres should have an enabled ssl.

Enable SSL in Galaxy

The following commands are used to enable SSL connection in Galaxy. This connection is required by the Hotell database servers. To do this you must recompile the module psycopg2. The recompilation is done with the following procedure.

1. Install the postgresql client and server (we need the server only to compile the postgresql ssl module!! Don't start the server)

yum install postgresql-server.x86_64 postgresql-devel.x86_64 postgresql.x86_64

2. If missing install gcc and redhat-rpm-config

yum install gcc redhat-rpm-config

3. Install Python devel

yum install python-devel.x86_64

4. Create a directory for all additional eggs/packs for galaxy

cd /home/laportal/galaxy
mkdir additional_packs
cd additional_packs

5. Download the python pack needed for the SSL connection with the DB. It might already exist but it is not compiled for SSL.

The Galaxy developers insisted on this particular version of psycopg2 as it is working fine. The newer ones might be working as well but I haven't tested them, so we better stick to this one.

wget http://eggs.g2.bx.psu.edu/psycopg2/psycopg2-2.0.13.tar.gz

6. Unpack the tar, cd into it

tar xvzf psycopg2-2.0.13.tar.gz 
cd psycopg2-2.0.13

Modify the setup.py as follows: just before the line

from distutils.core import setup, Extension

add

import pkg_resources
from scramble_lib import *

7. Before you scramble, prepare ez_setup.py file needed by the setup.py in the next step. Download ez_setup.py

wget http://peak.telecommunity.com/dist/ez_setup.py

Put the file in /home/laportal/galaxy/scripts/scramble/lib/ next to the file scramble_lib.py

cp ez_setup.py ../../scripts/scramble/lib/

8. Scramble the egg.

PYTHONPATH=/home/laportal/galaxy/scripts/scramble/lib python setup.py egg_info --tag-build=_<POSTGRES_VERSION>_static bdist_egg

9. The egg must be placed in Galaxy default egg directory. If /home/laportal/galaxy/eggs directory does not exist make it:

mkdir /home/laportal/galaxy/eggs

10. The generated psycopg egg put in default Galaxy egg directory.

cp dist/psycopg2-2.0.13_<POSTGRES_VERSION>_static-py2.7-linux-x86_64-ucs4.egg ../../eggs/

11. Update the file egs.ini with the correct versions of psycopg2 and postres

Install Apache

The Apache server serves as a proxy and aggregate for both galaxy and gold interfaces. It is used also from the module for Feide authentication.

1. Install apache server and appropriate mods.

2. Edit the file /etc/httpd/conf/httpd.conf. Redirect the root to SSL 443 port.

<VirtualHost _default_:80>
  RewriteEngine on
  ReWriteCond %{SERVER_PORT} !^443$
  RewriteRule ^/(.*) https://%{HTTP_HOST}/$1 [NC,R,L]
</VirtualHost>

Update the CGI section.

<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options  +ExecCGI
    AddHandler cgi-script .cgi .pl 
    Order allow,deny
    Allow from all
</Directory>

4. Generate SSL keys for the certificate check if mod_ssl and open_ssl are installed, if not install them.

yum install mod_ssl
yum install openssl.x86_64

Create keys and certificates

# Generate private key 
openssl genrsa -out ca.key 1024

# Generate CSR 
openssl req -new -key ca.key -out ca.csr

# Generate Self Signed Key
openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt

# Move the files to the correct locations
cp ca.crt /etc/pki/tls/certs/galaxy.server.crt
cp ca.key /etc/pki/tls/private/galaxy.server.key
cp ca.csr /etc/pki/tls/private/galaxy.server.csr

5. Create Virtual Host for galaxy in /etc/httpd/conf.d/ssl.conf

NameVirtualHost *:443

LoadModule ssl_module modules/mod_ssl.so
LoadModule mod_auth_memcookie_module modules/mod_auth_memcookie.so

<VirtualHost *:443>
  ServerName hyperbole.uio.no

   <Proxy balancer://balancer_lap_clarino>
       BalancerMember http://localhost:8080
   </Proxy>

  SSLEngine on
  SSLProtocol all -SSLv2 -SSLv3
  SSLCertificateFile /etc/pki/tls/certs/galaxy.server.crt
  SSLCertificateKeyFile /etc/pki/tls/private/galaxy.server.key

  # SSL stuff goes here, check with your security guy at the Tech Dept.

  RewriteEngine on
  RewriteRule ^/static/style/(.*) /home/laportal/galaxy/static/june_2007_style/blue/$1 [L]
  RewriteRule ^/static/scripts/(.*) /home/laportal/galaxy/static/scripts/packed/$1 [L]
  RewriteRule ^/static/(.*) /home/laportal/galaxy/static/$1 [L]
  RewriteRule ^/favicon.ico /home/laportal/galaxy/static/favicon.ico [L]
  RewriteRule ^/robots.txt /home/laportal/galaxy/static/robots.txt [L]


</VirtualHost>

Home | Forum | Discussions | Events

Clone this wiki locally