Remote data access programming tutorial - Introduction

The easiest way to use the Madrigal remote data access API is to simply let the web interface write the script you need for you. Just choose the Access data pull-down menu and choose Create a command to download multiple exps. Then follow the instructions, and you will have the command you need to download whatever you want from Madrigal.

The rest of this tutorial is for those who want to go beyond those automatically created scripts. Madrigal now exposes all the information and capabilities it has as web services, which allows easy access to Madrigal data from any computer on the internet using any platform (Unix, Windows, Mac, etc). Madrigal's web services are basically cgi scripts with simple output that allows easy parsing of the information. Any language that supports the HTTP standard can then access any Madrigal site. We have written remote API's using python and Matlab, but almost any language can be used.

Note that this approach of remotely accessing Madrigal data has been always possible before by parsing the html output meant to be displayed in a web browser (this general programming method is referred to as "screen scraping"). However, not only is this parsing difficult; but the code often breaks when the user interface is modified in any way. With web services the returned cgi scripts are designed to be both simple to parse and stable.

The web services are not implemented according to the SOAP or XMLRPC standard since not all scripting languages have support for these standards (or for XML parsing). Instead they use the simple approach of returning data requested via a query as a delimited text file. Note that the remote access examples written in python and Matlab work on any platform that python and/or Matlab supports.

Use of the remote access methods will be much easier if you understand how Madrigal data is organized. Review the section, "How does Madrigal organize data?" if you have not previously read it.

If you are interested in accessing the capabilities of Madrigal via some other language than Matlab or python, read the following section on Madrigal web services. Also, you might want to read the section on Madrigal web services if you find something you want to add either the remote python or Matlab API's. Otherwise, you can go skip ahead to the section on the remote Matlab API or the remote python API. These API's are available for download from the OpenMadrigal site.