Installing Madrigal for the first time

Doc home

Madrigal home

Installing Madrigal for the first time

Prerequisites

Madrigal is meant to be installed on any Unix-like OS, including Mac and Linux. Most of the prerequisites for Madrigal are pre-installed in any modern unix distribution. They are:

1. A C and a FORTRAN compiler. The free GNU compilers may be downloaded from the GNU Website .
2. Autotools (automake, autoconf, aclocal, libtoolize, autoheader)
3. A web server. See instructions below to run a Django application (which Madrigal is) on apache.
4. The environmental variable MADROOT should be set to the path to the Madrigal installation directory. Its recommended this be done in your login script (eg, .bash_profile) because you will need it to run administrative scripts later.
5. With Madrigal 3, the installation uses a version of python 3 you install yourself. The easiest way I have found to install python with scientific tools is to use either apt-get (for ubuntu), or the Anaconda python distribution, or pip. The following python modules need to be installed:
   1. django (version 1.11 or greater)
   2. pip and setuptools
   3. numpy
   4. scipy
   5. matplotlib
   6. netCDF4
   7. h5py
   8. sqlparse
   9. filelock
   10. bootstrap3 (can be installed via pip install django-bootstrap3)
   11. aacgmv2 (can be installed via pip install aacgmv2)
       

In addition, you need to have Autotools (automake, autoconf, aclocal, libtoolize, autoheader) installed, along with h5repack (may be part of h5utils package).

Installation instructions

Madrigal can be installed on any unix server with a web server configured for Django. If you want to link your data in with data on other Madrigal servers, please notify the OpenMadrigal administrator. In general you do not need root permission to install madrigal once the prerequisites listed above are installed.

1. Create a directory to be used as your Madrigal root directory on your unix server. This directory will be referred to as MADROOT.
2. Create the environment variable MADROOT with that directory path.
3. Download the latest Madrigal distribution file, madrigal*.tar.gz from the OpenMadrigal distribution page to MADROOT.
4. gunzip madrigal*.tar.gz
5. tar -xf madrigal*.tar
6. Repeat the 3 steps above to get and untar the Sample Experiments file experiments.tar.gz.
7. Make sure $MADROOT/bin/python links to the version of python with the required modules listed above. For example, if you are using anaconda, you might run <ln -s //anaconda/bin/python $MADROOT/bin/python>.
8. If desired, copy any data from your old $MADROOT/experiments directory to this new $MADROOT/experiments directory.
9. In madroot, there will now be a file called madrigal.cfg.template. Copy it to madrigal.cfg, and edit all the configuration parameters, as described in the Editing madrigal.cfg section below.
10. Configure your web server to run the Madrigal Django application. Instruction for apache can be found in the Configuring Apache for a Madrigal Django Application section below. If you are running a webserver other than apache, please consult the Django documentation for your particular webserver.
11. Edit the file MADROOT/metadata/siteTab.txt with a unique id to include your site. The format of siteTab.txt is described here. Note that if your site will be running https, the third field will begin https://. If you want to be an official Madrigal site that other Madrigal sites share data with, request the OpenMadrigal administrator to assign you this id. If this is just a test site, use any id not already in the siteTab.txt file. Finally, make sure the last column of the line for your site ends ,3.0 . This tells all sites that you site is now a Madrigal 3 site. For example:

    999,mad31,localhost,madrigal,cgi-bin/madrigal,madrigal/servlets,Bill Rideout,MIT Haystack Observatory,Route 40,,Westford,MA,01886,USA,1-781-981-5624,brideout@haystack.mit.edu,3.0

12. Be sure to cd to MADROOT before running the following step. Then you should be able to complete the installation simply by typing
    

    <sudo> bash installMadrigal  &> install.log &

    The sudo is required only if installing modules on your default version of python requires sudo. There may be a long pause when running updateMaster near the end of the installation since the instParmTab.txt metadata file is being built for the first time by examining every data file, but future calls to updateMaster will be much faster since only new experiments are examined. Help with any installation errors is available from the OpenMadrigal administrator.
13. If there were no errors, your madrigal installation should be running at the url given by MADSERVERROOT.
14. If you want to receive notices about updates to Madrigal, sign up for the openmadrigal-admin mailing list under http://cedar.openmadrigal.org/openmadrigal.
15. Set the script madroot/bin/updateMaster up as a cron job to run once a day.
16. If you want to add any documentation pages specific to your site to the Madrigal documentation pages, see the other admin tasks section of this manual.
17. If you want to add your own rules of the road to the Madrigal experiment page, see the other admin tasks section of this manual.

Editing the madrigal.cfg file

• MADROOT - Madrigal root directory (absolute). This must be set as an environmental variable.
• MADSERVER - Web server for accessing Madrigal. For https, add https:// to the beginning. If not given, uses http
• MADSERVERROOT - document virtual directory relative to MADSERVER. If the full Madrigal Url is the same as the MADSERVER field (for example, if the url is http://madrigal.hao.ucar.edu/), then set this field to a period. (Example: MADSERVERROOT = . )
• SITEID - Site ID - Must be unique and same as in siteTab.txt
• HTMLSTYLE - Body style of html pages
• INDEXHEAD - Heading in the top-level Madrigal page
• CONTACT - Mailto link of contact person(s) for this madrigal installation. Multiple email addresses may be included if separated by commas.
• MAILSERVER - Name of mailserver (possibly localhost if running sendmail)
• PLOTBUTTONLABEL - By default the button to show additional documentation is labelled Plots/Docs. Change this line for a different label.
  

Configuring Apache for a Madrigal Django Application

Some configuration of apache is required to run a Django application like Madrigal. If you are running a webserver other than apache, please consult the Django documentation for your particular webserver. We will configure apache to run Madrigal in mod_wsgi daemon mode for improved performance.

The first step is to make sure mod_wsgi is installed as part of your apache webserver. If not, see mod_wsgi installation directions. Once installed, add the line

LoadModule wsgi_module modules/mod_wsgi.so

near the top of the httpd.conf file. Next add the follow section, with MADROOT, MADSERVERROOT and YOUR_SERVER replaced with values for your server from the madrigal.cfg file described below. Note that if MADSERVERROOT is zero length (that is, if the full Madrigal Url is the same as the MADSERVER field - for example, if the url is http://madrigal.hao.ucar.edu/) then /MADSERVERROOT just becomes / unlike in the madrigal.cfg file when it becomes a period. Set NUM_CPU to about one half available on your server.

SetEnv PYTHONPATH MADROOT/source/madpy/djangoMad/
WSGIDaemonProcess YOUR_SERVER python-path=MADROOT/source/madpy/djangoMad processes=NUM_CPU display-name=%{GROUP} lang='en_US.UTF-8' locale='en_US.UTF-8'
WSGIProcessGroup YOUR_SERVER
WSGIScriptAlias /MADSERVERROOT MADROOT/source/madpy/djangoMad/djangoMad/wsgi.py process-group=YOUR_SERVER
Alias /static/ MADROOT/source/madpy/djangoMad/madweb/static/
WSGIApplicationGroup %{GLOBAL}

 <Directory MADROOT/source/madpy/djangoMad/madweb/static>

Require all granted

</Directory>
<Directory MADROOT/source/madpy/djangoMad/djangoMad>

<Files wsgi.py>  

Require all granted

</Files>

</Directory>

You may also need to add to your apache configuration file SetEnv LD_LIBRARY_PATH $MADROOT/lib, possibly along with other library paths.

Finally, you need to edit the file MADROOT/source/madpy/djangoMad/djangoMad/settings_production_template.py, and save it as MADROOT/source/madpy/djangoMad/djangoMad/settings_production.py. The lines to be edited are:

SECRET_KEY = '*****'
ALLOWED_HOSTS = ['madrigal3.haystack.mit.edu']
ADMINS = (('Bill Rideout', 'brideout@haystack.mit.edu'),)
EMAIL_HOST = 'hyperion.haystack.mit.edu'
MANAGERS = (('Bill Rideout', 'brideout@haystack.mit.edu'),)
STATIC_URL = '/static/'

The STATIC_URL line only needs to be edited if MADSERVERROOT is not empty; that is, if your madrigal is installed in a subdirectory. For example, if your url is http://madrigal.sri.com/madrigal, you would change that line to be STATIC_URL = '/madrigal/static/'. But if your url is http://madrigal.sri.com, you would leave that STATIC_URL line as is.

You can generate a new secret key for your site at http://www.miniwebtool.com/django-secret-key-generator/, which is recommended for security reasons. The other lines are self-explanatory.

When you are done, be sure to restart apache for the changes to take effect. Note that because this Madrigal Django application runs in daemon mode, it will be necessary to restart the web server if you ever update the python code.

Throttling Madrigal admin emails

Sometimes Madrigal users write scripts that cause numerous admin emails, and in addition malicious users can also generate numerous admin emails. If you would like to throttle the number of Madrigal admin emails you receive, see Throttling admin emails

Installing Madrigal for the first time

Doc home

Madrigal home