BioimageInstallation - BioImage Wiki (internal)

Bioimage installation notes

This is a page of notes for installing Bioimage on a new system.

Contents:

Prerequisites
Bioimage software installation
Building a new database
Configuration
Start BioImage
Using BioImage
BioImage static web pages
File layout
Setting up a Ruby/Rails server with Postgres and d2r bridge
Installing ImageMagick and RMagick
1. There are a few ruby files in cvs in the bulkuploader module
Mapping CSV files into the db
Future thoughts

For information about Linux system configuration practices and security, see:

1. Prerequisites

For example system, andros has recent software, milos2 has older system with Apache server.

Target server system:

Linux or Windows operating system.
Tomcat V5.5.9 (or earlier V5 per Milos); most recent recommended (but not 5.5.12). Jetty is also believed to work as the applet server.
Apache server (optional; probably wise for public facing service, unnecessary for internal use). Milos currently uses 1.3.27.
Java runtime 1.4.1 or 1.5
Postgresql server (somewhere; may be separate machine)
CVS client (e.g. TortoiseCVS)
Ant to build Tomcat kit from CVS files

1.1. Specific installation prerequisites

I am working on a non-development !BioImage installation on a Windows host, for which the starting configuration is:

Windows XP pro
Java runtime 1.5.0_05
Tomcat 5.5.9
Access to Postgresql server (paros.zoo.ac.uk)
CVS client to obtain files (e.g. TortoiseCVS)
Ant to build Tomcat kit from CVS files

1.2. Tomcat installation notes

Run the Tomcat installer

Create JAVA_HOME and CATALINA_HOME environment variables; e.g.

JAVA_HOME=C:\Program Files\Java\jdk1.5.0_05
CATALINA_HOME=C:\Program Files\Apache Software Foundation\Tomcat 5.5

Install the Tomcat admin servlet: simply unpack the Admin download package into the Tomcat installation directory, so that it is added to the files already installed. (This was installed by default in older versions of Tomcat.

* Add the following to the web.xml file under /var/tomcat (ie the tomcat web.xml file, not the application web.xml file under /var/tomcatXX/WEB-INF)

<mime-mapping>

<extension>ico</extension> <mime-type>image/x-icon</mime-type>

</mime-mapping>

To check that Tomcat is running, browse to http://localhost:8080/ and http://localhost:8080/admin/. The latter will require entry of the admin username and password created during the inital Tomcat installation.

1.3. Ant installation notes

Ant requires that JDK, not just JRE, is installed, or many Ant tasks are not supported
Unpack the Ant binary distribution into a directory
Set environment variable ANT_HOME to reference that directory

Add the Ant binary directory to the path; e.g.

ANT_HOME=E:\Program Files\Ant
PATH=%PATH%;E:\Program Files\Ant\bin

2. Bioimage software installation

Bioimage files are in cvs, at ext:<username>@paros.zoo.ox.ac.uk/data/cvsroot/bioimage2/, or http://paros.zoo.ox.ac.uk/cgi-bin/viewcvs.cgi/bioimage2/.

Installation procedure:

Extract bilimage files from CVS into a working directory (CVS location above)

Edit build.xml: uncomment the line <property name="local" value="true" /> if copying from a local copy on the server and alter the property definitions in the section headed "Live server details" to reflect the target server; e.g.

<!--live server variables--> 
<property name="server.user.live" value="bioimage" />
<property name="server.port.live" value="" />
<property name="axis.dest.live" value="${server.dest.live}" />
<property name="server.manager.name.live" value="manager" />
<property name="server.manager.pass.live" value="vidos92" />
<property name="server.host.live" value="rodos.zoo.ox.ac.uk" />
<property name="server.root.live" value="C:/DevUtils/Apache Software Foundation/Tomcat 5.5/" />

Create a command window, set the the current directory to be that into which the BioImage source was extracted from CVS, and run Ant to create the target deployment; e.g.
- ```
C: 
cd C:\CVSFiles\bioimage2
ant upload_all_live
```

(Note the target contains underscores, not hyphens. Use ant -p to see list of targets in the Ant file.)

This should copy all required files to the target system, but in case of problems, compare the target directories with Andros and figure out what is missing. Directories most likely to be problematic are tomcat/common/lib stuff and web/inf/lib.

(Maybe need to ensure database driver files are installed in the right location - postgesql driver in $CATALINA_HOME/common/lib)

3. Building a new database

Last changed jan 2006 - now matches the new 8.0 installation.

To build a new database
- make a new home directory - mkdir db
- initdb -D db
- /usr/local/psql/bin/postmaster -D db -i (-i flag needed for remote connection)
- psql -p5432 template1
- template1= #CREATE DATABASE jenatest;
- as root cd /usr/local/src/postgresql-8.0.5/contrib/tsearch2
- gmake
- gmake install
- su postgres (note - stay in tsearch directory)
- psql -p5432 jenatest < tsearch2.sql
- createlang plpgsql jenatest;
If postgres is already installed we can simply drop the database and rebuild it:
- psql -p5432 template1
- template1= #DROP DATABASE jenatest;
  - (No clients can be connected - this means all tomcat installations that might be connected to the database have to be stopped. If the database still will not drop, kill the postgres server by doing the following as user postgres:
  - /usr/local/pgsql/bin/pg_ctl stop -D /data/users/bioimage/db -W -m fast followed by
  - /usr/local/pgsql/bin/pg_ctl start -o "-i" -D /data/users/bioimage/db -W )
- template1= #CREATE DATABASE jenatest;
- as user postgreq:
  - ```
  cd /usr/local/src/postgresql-8.0.5/contrib/tsearch2 
  psql -p5432 jenatest < tsearch2.sql
  createlang plpgsql jenatest;
```

there is some additional setup to do with tsearch2. First you need to install ispell - which can be downloaded from http://www.cs.hmc.edu/~geoff/ispell.html#ftp-sites. I downloaded this into the bioimage home directory, ran make and make install as described in the readme. This creates a british.med file in /home/bioimage/ispell_******/languages/british which I copied to /usr/local/lib. I needed to change the name of this file from british.med to english.med.

If the database (identified in the bioimage.properties file) is empty, it will be rebuilt from the data file (also identified in bioimage.properties) when tomcat is restarted. There is now code in the rebuild method of Ontology Manager that handles the adding of functions and triggers and the indexing.
- we can now query the database with something like:

SELECT id, title FROM study WHERE idxabstract @@ 'aquaporin'::tsquery;

3.1. Reloading the database from an ontology file

To reload a database from an ontology file, stop the server and restart it;

As user postgres:

# /usr/local/pgsql/bin/pg_ctl stop -D /data/users/bioimage/db -W -m fast
# /usr/local/pgsql/bin/pg_ctl start -D /data/users/bioimage/db -W
# psql -p5432 jenatest
jenatest# \c template1
jenatest# drop DATABASE jenatest;
jenatest# create database jenatest;
jenatest# \q
#

3.2. Database backup

This *should* now be simply a matter of following the instructions on the postgres site - need to check (Old instructions applied to previous postgres version and have been removed - they are still available on the old wiki if needed)

3.3. Character encoding

Some notes on Character Encoding Database must be set up with LATIN1 encoding - so in psql create database bioimageproxy with encoding='LATIN1';

All xml,xsl and jsp files should be encoded in UTF-8 - HOWEVER, stylesheets that are taking database content and outputting it to screen should use <xsl:output encoding="ISO-8859-1"/> What we think happens here is that this handles the LATIN1 encoded database content correctly. When the LATIN1 encoded output gets wrapped by the sitemesh decorator (which uses a UTF-8 charset) the LATIN1 encoded output of the stylesheet gets translated to UTF-8, and displayed properly in a browser using UTF-8 encoding.

struts error messages are often very abstruse - what they actually mean is listed here http://www.geocities.com/dustinmarx/SW/struts/errors.html

4. Configuration

Two main Tomcat files:

server.xml in Tomcat conf directory (e.g. /var/jakarta-tomcat-5.5.4/conf/server.xml)
bioimage/WEB-INF/web.xml (e.g. /var/jakarta-tomcat-5.5.4/bioimage/WEB-INF/web.xml)

The second of these (web.xml) should not need changing. There is a server.xml.example file in the config folder below the root directory of the cvs download - this can be used as a starting point Edit server.xml to edit the <Resource> entries for BioImage. (It is possible that these may be defined as XML external entities - look at the XML DTD to see where the external files are kept.)

Also, there are BioImage files called config/bioimage.properties, config/log4j.properties and config/ont-policy.rdf.

4.1. server.xml

Note that failure to edit the server.xml file may result in strange errors and files created unexpectedly at the root level in the containing file system. The file server.xml.example is in the root of the Bioimage installation in Tomcat, and may be used as the basis of a Tomcat configureation file if no other applications are installed. Otherwise, cut-and-paste the relevant sections into the existing server.xml file. Either way, some edits will be needed:

There should be an entry like this:

<Resource name="jenatest" scope="Shareable" type="javax.sql.DataSource" 
          url="jdbc:postgresql://129.67.24.149:1127/jenatest"
          validationQuery="select 1;" maxIdle="10" maxActive="40" 
          driverClassName="org.postgresql.Driver"
          maxWait="5000" 
          factory="org.apache.commons.dbcp.BasicDataSourceFactory"
          username="bioimage" 
          password="******" />

This defines a JNDI data source called "jenatest" within the Tomcat environment. The name is arbitrary, but see below.

Later, the BioImage host and application context are defined, :

<Engine name="Catalina" defaultHost="andros.zoo.ox.ac.uk" debug="0">
  <Host name="andros.zoo.ox.ac.uk" appBase="bioimage" debug="99"
        unpackWARs="true" autoDeploy="true"
        xmlValidation="false" xmlNamespaceAware="false">

  :

    <Context cachingAllowed="false" 
             charsetMapperClass="org.apache.catalina.util.CharsetMapper" 
             crossContext="true" docBase="" path="" 
             reloadable="true" useNaming="true">
        <Environment name="bioimageProperties" 
                    value="C:/DevUtils/Apache Software Foundation/Tomcat 5.5/bioimage/config/bioimage.properties" 
                    type="java.lang.String" override="false"/>
        <ResourceLink global="jenatest" name="jenatest" type="javax.sql.DataSource"/>
        <Resource name="owlbasePool" auth="Container" 
                  type="org.apache.commons.pool.impl.GenericObjectPool" 
                  factory="org.bioimage.admin.pool.OwlbaseFactory"/>
        <Resource name="checkPointPool" auth="Container" 
                  type="org.apache.commons.pool.impl.GenericObjectPool" 
                  factory="org.bioimage.security.CheckPointFactory"/>
    </Context>
  </Host>
</Engine>

Note that this contains a link to the previously defined "jenatest" data source, which is located and used by the BioImage application.

Edits required are:

Change the host name in the <Engine> and <Host> elements.
URL of the JDBC connection in the resource description.
Path to the BioImage properties file in the <Environment> element of the context description (note that a full path name is required here - a relative path does not work).
For a deployed application, set reloadable="false" in the context description (i.e. tell Tomcat not to look for code changes, which in turn can trigger reloading of the application.

4.2. config/bioimage.properties

Note the location of the BioImage properties file (config/bioimage.properties in the example above, which contains many entries, including some like these:

#BioImage Properties
#===================

# Linux:
#bioimage.tomcat.base=/var/tomcat5/bioimage
#bioimage.tomcat.base_uri=file:///var/tomcat5/bioimage

# Windows:
bioimage.tomcat.base=C:/DevUtils/Apache Software Foundation/Tomcat 5.5
bioimage.tomcat.base_uri=file:/C:/DevUtils/Apache%20Software%20Foundation/Tomcat%205.5

#-------------------------------------------------------------------------------
#admin related props
#-------------------------------------------------------------------------------
admin.pool.min-idle=4
admin.dumplocation="C:/DevUtils/Apache Software Foundation/Tomcat 5.5/bioimage/dump.rdf"

#-------------------------------------------------------------------------------
#image display props
#-------------------------------------------------------------------------------
display.maxImages=200
display.imagesPerPage=30
display.maxStudies=100
display.studiesPerPage=10
display.maxDatasets=100
display.datasetsPerPage=10
display.xsl="C:/DevUtils/Apache Software Foundation/Tomcat 5.5/bioimage/default.xsl"

#-------------------------------------------------------------------------------
#database connectivity props
#-------------------------------------------------------------------------------
db.bioimagedata.datasource=jenatest
conn.host=devel.bioimage.org
conn.stylesheet="C:/DevUtils/Apache Software Foundation/Tomcat 5.5/bioimage/treeDisplay.xsl"

#-------------------------------------------------------------------------------
#owlbase related props
#-------------------------------------------------------------------------------
owlbase.metadata=file:/C:/DevUtils/Apache%20Software%20Foundation/Tomcat%205.5/bioimage/config/ont-policy.rdf
owlbase.defaultwriter=org.bioimage.owlbase.output.OwlbaseWriter
owlbase.cache=n
owlbase.representatives=C:/DevUtils/Apache Software Foundation/Tomcat 5.5/bioimage/config/representatives.xml
owlbase.ontology=bioimage
owlbase.baseontology=bioimagedata

#these are the reference uris - local versions are set in ont-policy.rdf
owlbase.bioimage.uri=http://www.bioimage.org/ontology/bioimage
owlbase.bioimage.datauri=http://www.bioimage.org/ontology/bioimagedata
owlbase.bioimage.datafile=file:/C:/DevUtils/Apache%20Software%20Foundation/Tomcat%205.5/bioimage/ontology/bioimagedata.owl

owlbase.bioimage.idprefix=bioimage
owlbase.bioimagedata.storage.type=PostgreSQL

#PostgreSQL
owlbase.bioimagedata.storage.drv=org.postgresql.Driver
owlbase.bioimagedata.storage.db=jenatest
owlbase.bioimagedata.storage.url=jdbc:postgresql://paros.zoo.ox.ac.uk:1127/jenatest
owlbase.bioimagedata.storage.user=bioimage
owlbase.bioimagedata.storage.pass=bi7jk9
owlbase.bioimage.reasoner=5

#-------------------------------------------------------------------------------
#submission related props
#-------------------------------------------------------------------------------
submission.default.authorNo=org.bioimage.submission.DefaultAuthorNo
submission.uploader=org.bioimage.submission.upload.BasicUploader
submission.uploader.basedir=C:/DevUtils/Apache Software Foundation/Tomcat 5.5/bioimage/uploadtest
submission.extractor.SourceMob=org.bioimage.submission.extract.BasicSourceMobExtractor
submission.logdir=C:/DevUtils/Apache Software Foundation/Tomcat 5.5/logs
order.submission.stylesheet=C:/DevUtils/Apache Software Foundation/Tomcat 5.5/bioimage/all_xsl/submission_ordering.xsl

#-------------------------------------------------------------------------------
#transform related props
#-------------------------------------------------------------------------------
stylesheet.studydetails=C:/DevUtils/Apache Software Foundation/Tomcat 5.5/bioimage/all_xsl/studyDetails_boxml.xsl
stylesheet.authordetails=C:/DevUtils/Apache Software Foundation/Tomcat 5.5/bioimage/all_xsl/authorDetails_boxml.xsl
stylesheet.imagedetails=C:/DevUtils/Apache Software Foundation/Tomcat 5.5/bioimage/all_xsl/studyDetails_boxml.xsl

#-------------------------------------------------------------------------------
#misc. props
#-------------------------------------------------------------------------------
news.item=DTGED image annotations

Possible edits needed include:

change filenames and directory names to reflect the local installation
owlbase.bioimagedata.storage.db must refer to the data source resource described in server.xml
owlbase.cache controls whether class signatures are cached; i.e. a local copy of properties for each class, used to prevent repeated database accesses for otherwise static information. Turned off (n) for development, turn on (y) for deployment.

Other values can be changed from the Admin page within BioImage.

You might find it worth saving a copy of the edited bioimage.properties file before starting Bioimage, as this file is re-written by BioImage and loses its formatting and any helpful comments.

4.3. config/log4j.properties

The log4j properties file config/log4j.properties contains the location of the log file - which may also need changing.

4.4. config/ont-policy.rdf

This file indicates alternate locations for retrieving OWL ontology definition files.

There are two lines that may need editing:

<OntologySpec> 
  <!-- local version of the OWL BioImage Ontology -->
  <publicURI rdf:resource="http://www.bioimage.org/ontology/bioimage" />
  <altURL rdf:resource="file:/c:/DevUtils/Apache%20Software%20Foundation/Tomcat%205.5/bioimage/ontology/bioimage.owl"/>
  <language  rdf:resource="http://www.w3.org/TR/owl-features/#term_OWLDL" />
  <prefix    rdf:datatype="&xsd;string">bioimage</prefix>
  </OntologySpec>

<OntologySpec>
  <!-- local version of the OWL BioImage Ontology -->
  <publicURI rdf:resource="http://www.bioimage.org/ontology/bioimagedata" />
  <altURL rdf:resource="file:/c:/DevUtils/Apache%20Software%20Foundation/Tomcat%205.5/bioimage/ontology/bioimagedata.owl"/>
  <language  rdf:resource="http://www.w3.org/TR/owl-features/#term_OWLDL" />
  <prefix    rdf:datatype="&xsd;string">bioimagedata</prefix>
</OntologySpec>

Change the <altURL> rdf:resource= atrributes to reflect the local installation.

5. Start BioImage

Start Tomcat to activate the BioImage server. E.g., run /var/jakarta-tomcat-5.0.24/bin/catalina.sh, or start Tomcat monitor from the Start -> All programs -> Apache Tomcat 5.5 menu, then right-click the Apache Tomcat icon (usually at the bottom-right of the Windows screen) and select Start service.

6. Using BioImage

Browse to http://hostname:8080/, and the BioImage front page should be displayed.

Seach and Submit links on front page are key links into BioImage functionality, which will (if needed) request username/password information.

To access the admin page, append admin to the URI, e.g. http://hostname:8080/admin.

7. BioImage static web pages

These are on milos in directory /var/tomcat5/bioimage/pub. The front page is index.jsp.

8. File layout

The file layout should be determined by the ant build script. However, several things (wrongly set file permissions, missing files) can result in the layout missing key files, or carelessly putting them in the wrong place. The following is a check list taken from a working installation:

/var/tomcat5/biomage/WEB-INF/lib Note that there are two copies of commons-logging, one in the tomcat common lib folder, and one in the bioimage folder. Both are required - this is not an error.

-rwx------  1 bioimage bioimage  358273 Nov  8 08:55 antlr.jar
-rwx------  1 bioimage bioimage  730762 Jan 17 14:54 arq.jar
-rwx------  1 bioimage bioimage 1224774 Jan 17 14:52 axis.jar
-rwx------  1 bioimage bioimage  188671 Nov  8 08:55 commons-beanutils.jar
-rwx------  1 bioimage bioimage  168446 Nov  8 08:55 commons-digester.jar
-rwx------  1 bioimage bioimage   22379 Nov  8 08:55 commons-fileupload.jar
-rwx------  1 bioimage bioimage   63980 Jan 17 14:01 commons-lang.jar
-rwx------  1 bioimage bioimage   38015 Nov  8 08:55 commons-logging.jar
-rwx------  1 bioimage bioimage   84462 Nov  8 08:55 commons-validator.jar
-rwx------  1 bioimage bioimage   56404 Jan 17 15:05 cos.jar
-rwx------  1 bioimage bioimage   26881 Jan 17 13:54 dom.jar
-rwx------  1 bioimage bioimage 2450757 Aug 13  2004 icu4j.jar
-rwx------  1 bioimage bioimage   21519 Jan 25 09:33 image_taglib.jar
-rwx------  1 bioimage bioimage   65425 Nov  8 08:55 jakarta-oro.jar
-rwx------  1 bioimage bioimage  191881 Jan 17 13:54 jaxen-full.jar
-rwx------  1 bioimage bioimage   27049 Jan 17 13:54 jaxp-api.jar
-rwx------  1 bioimage bioimage   35658 Jan 17 14:52 jaxrpc.jar
-rwx------  1 bioimage bioimage    6727 Jan 17 13:54 jdbc2_0-stdext.jar
-rwx------  1 bioimage bioimage  127118 Jan 17 14:55 jdom.jar
-rwx------  1 bioimage bioimage 2238661 Jan 17 14:50 jena.jar
-rwx------  1 bioimage bioimage   21128 Jan 17 13:54 jstl.jar
-rw-r--r--  1 bioimage bioimage  352668 Jan 24 18:09 log4j-1.2.8.jar
-rwx------  1 bioimage bioimage  305434 Jan 17 13:54 mail.jar
-rwx------  1 bioimage bioimage  283333 Jan 17 14:59 ng4j.jar
-rwx------  1 bioimage bioimage   14916 Jan 17 15:01 rdftwig.jar
-rwx------  1 bioimage bioimage   18402 Jan 17 14:52 saaj.jar
-rwx------  1 bioimage bioimage   26437 Jan 17 13:54 sax.jar
-rwx------  1 bioimage bioimage   23563 Jan 17 13:54 saxpath.jar
-rwx------  1 bioimage bioimage  100414 Jan 20 19:02 sitemesh-2.0.1.jar
-rwx------  1 bioimage bioimage  515490 Jan 17 13:54 standard.jar
-rwx------  1 bioimage bioimage  544774 Jan 25 10:23 struts.jar
-rwx------  1 bioimage bioimage   10518 Jan 17 14:01 struts-legacy.jar
-rwx------  1 bioimage bioimage  147464 Jan 17 14:58 stxx.jar
-rwx------  1 bioimage bioimage 1271111 Jan 17 13:54 xalan.jar
-rwx------  1 bioimage bioimage 1202684 Jan 17 13:54 xercesImpl.jar
-rwx------  1 bioimage bioimage   76924 Jan 17 15:05 xmlform.jar

/var/tomcat5/common/lib

-rw-r--r--  1 root root  41534 Nov  7  2004 aspectjrt-1.2.1.jar
-rw-r--r--  1 root root 258735 Jan 28  2005 cactus-1.7.jar
-rwx------  1 root root 298136 Aug 13  2004 commons-collections.jar
-rwx------  1 root root 107631 Dec 20  2004 commons-dbcp-1.2.1.jar
-rw-r--r--  1 root root 112341 Oct 29  2004 commons-el.jar
-rw-r--r--  1 root root 225375 Oct 11  2004 commons-httpclient-2.0.2.jar
-rw-r--r--  1 root root  38015 Jun 16  2004 commons-logging-1.0.4.jar
-rwx------  1 root root  42492 Dec 20  2004 commons-pool-1.2.jar
-rw-r--r--  1 root root 411014 Jun  4  2005 httpunit-1.6.jar
-rw-r--r--  1 root root 382011 Oct 29  2004 jasper-compiler.jar
-rw-r--r--  1 root root 916757 Oct 29  2004 jasper-compiler-jdt.jar
-rw-r--r--  1 root root  76043 Oct 29  2004 jasper-runtime.jar
-rw-r--r--  1 root root  50493 Oct 29  2004 jsp-api.jar
-rw-r--r--  1 root root 121070 Sep 14  2002 junit-3.8.1.jar
-rw-r--r--  1 root root 352668 Aug 13  2004 log4j-1.2.8.jar
-rwx------  1 root root  16277 Jan 17 14:16 naming-config-0.8.jar
-rwx------  1 root root  32747 Jan 17 14:16 naming-core-0.8.jar
-rwx------  1 root root  18534 Jan 17 14:16 naming-factory-0.8.jar
-rw-r--r--  1 root root 154101 Oct 29  2004 naming-factory-dbcp.jar
-rwx------  1 root root   6810 Jan 17 14:16 naming-java-0.8.jar
-rwx------  1 root root   8280 Jan 17 14:16 naming-management-0.8.jar
-rwx------  1 root root  48884 Jan 17 14:16 naming-resources-0.8.jar
-rwx------  1 root root 195719 Sep 24  2004 pg74.1jdbc2ee.jar
-rwx------  1 root root  97523 Nov  1 16:42 servlet-api.jar

/var/tomcat5/bioimage/WEB-INF

bioimage.properties
bioimage.properties.andros
bioimage.properties.rodos
c.tld
c-rt.tld
decorators.xml
fmt.tld
fmt-rt.tld
index.jsp
log4j.properties
ont-policy.rdf
ont-policy.rdf.andros
ont-policy.rdf.rodos
ont-policy-test.rdf
representatives.xml
sitemesh.xml
sitemesh-decorator.tld
sitemesh-page.tld
sql.tld
sql-rt.tld
struts-bean.tld
struts-config.xml
struts-config-errors.xml
struts-config-retrieval.xml
struts-config-submission.xml
struts-html.tld
struts-logic.tld
struts-nested.tld
struts-template.tld
struts-tiles.tld
stxx.properties
stxx-transforms.xml
validator-rules.xml
web.xml
x.tld
x-rt.tld

In web.xml we have the following - obviously the definitive version is the one in cvs - this is just here for reference. Do not be tempted to try and tidy up the struts taglibs by using the more recent uri syntax - it doesn't seem to work properly yet.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.4" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee   http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">
  <filter>
    <filter-name>getsession</filter-name>
    <filter-class>org.bioimage.GetSession</filter-class>
  </filter>
  <filter>
    <filter-name>sitemesh</filter-name>
    <filter-class>com.opensymphony.module.sitemesh.filter.PageFilter</filter-class>
    <init-param>
      <param-name>debug.pagewriter</param-name>
      <param-value>false</param-value>
    </init-param>
  </filter>
  <filter-mapping>
    <filter-name>getsession</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>
  <filter-mapping>
    <filter-name>sitemesh</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>
  <servlet>
    <servlet-name>Init</servlet-name>
    <servlet-class>org.bioimage.admin.Init</servlet-class>
    <init-param>
      <param-name>log4j-init-file</param-name>
      <param-value>log4j.properties</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet>
    <servlet-name>admin</servlet-name>
    <servlet-class>org.bioimage.admin.BioImageAdministration</servlet-class>
  </servlet>
  <servlet>
    <servlet-name>redirect</servlet-name>
    <servlet-class>org.bioimage.RedirectOnError</servlet-class>
  </servlet>
  <servlet>
    <servlet-name>action</servlet-name>
    <servlet-class>org.apache.struts.action.ActionServlet</servlet-class>
    <init-param>
      <param-name>config</param-name>
      <param-value>/WEB-INF/struts-config.xml</param-value>
    </init-param>
    <init-param>
      <param-name>validating</param-name>
      <param-value>false</param-value>
    </init-param>
    <init-param>
      <param-name>debug</param-name>
      <param-value>99</param-value>
    </init-param>
    <init-param>
      <param-name>detail</param-name>
      <param-value>99</param-value>
    </init-param>
    <init-param>
      <param-name>stxxInit</param-name>
      <param-value>/WEB-INF/stxx.properties</param-value>
    </init-param>
    <init-param>
      <param-name>retrieval</param-name>
      <param-value>/WEB-INF/struts-config-retrieval.xml</param-value>
    </init-param>
    <init-param>
      <param-name>submission</param-name>
      <param-value>/WEB-INF/struts-config-submission.xml</param-value>
    </init-param>
    <init-param>
      <param-name>errors</param-name>
      <param-value>/WEB-INF/struts-config-errors.xml</param-value>
    </init-param>
    <init-param>
      <param-name>config/retrieve</param-name>
      <param-value>/WEB-INF/struts-config-retrieve.xml</param-value>
    </init-param>
    <load-on-startup>2</load-on-startup>
  </servlet>
  <!-- Servlet Mappings -->
  <servlet-mapping>
    <servlet-name>admin</servlet-name>
    <url-pattern>/admin</url-pattern>
  </servlet-mapping>
  <servlet-mapping>
    <servlet-name>redirect</servlet-name>
    <url-pattern>/redirect</url-pattern>
  </servlet-mapping>
  <servlet-mapping>
    <servlet-name>action</servlet-name>
    <url-pattern>*.do</url-pattern>
  </servlet-mapping>
        
        <!-- The Welcome File List -->
        <welcome-file-list>
                <welcome-file>index.jsp</welcome-file>
        </welcome-file-list>
        <!-- Tag Library Descriptors -->
        <jsp-config>
        <taglib>
  <taglib-uri>/WEB-INF/struts-bean.tld</taglib-uri>
  <taglib-location>/WEB-INF/struts-bean.tld</taglib-location>
</taglib>

<taglib>
  <taglib-uri>/WEB-INF/struts-html.tld</taglib-uri>
  <taglib-location>/WEB-INF/struts-html.tld</taglib-location>
</taglib>

<taglib>
  <taglib-uri>/WEB-INF/struts-logic.tld</taglib-uri>
  <taglib-location>/WEB-INF/struts-logic.tld</taglib-location>
</taglib>

<taglib>
  <taglib-uri>/WEB-INF/struts-template.tld</taglib-uri>
  <taglib-location>/WEB-INF/struts-template.tld</taglib-location>
</taglib>
        <taglib>
                <taglib-uri>image.tld</taglib-uri>
                <taglib-location>/WEB-INF/lib/image_taglib.jar</taglib-location>
        </taglib>

<taglib>
    <taglib-uri>sitemesh-page</taglib-uri>
    <taglib-location>/WEB-INF/sitemesh-page.tld</taglib-location>
 </taglib>    
 <taglib>
    <taglib-uri>sitemesh-decorator</taglib-uri>
     <taglib-location>/WEB-INF/sitemesh-decorator.tld</taglib-location>
 </taglib> 
                </jsp-config>   
        <!-- Resource Environment References -->

        <resource-env-ref>
                <description>Object factory and pool for OntologyManager instances.</description>
                <resource-env-ref-name>owlbaseFactory</resource-env-ref-name>
                <resource-env-ref-type>org.bioimage.owlbase.OwlbaseFactory</resource-env-ref-type>
        </resource-env-ref>

        <resource-env-ref>
                <description>Object factory and pool for Checkpoint instances.</description>
                <resource-env-ref-name>checkPointPool</resource-env-ref-name>
                <resource-env-ref-type>org.bioimage.security.CheckPointFactory</resource-env-ref-type>
        </resource-env-ref>

        <resource-ref>
                <description>Public DB Connection</description>
                <res-ref-name>bioimagepublic</res-ref-name>
                <res-type>javax.sql.DataSource</res-type>
                <res-auth>Container</res-auth>
        </resource-ref>
        <resource-ref>
                <description>Public DB Connection</description>
                <res-ref-name>jenatest</res-ref-name>
                <res-type>javax.sql.DataSource</res-type>
                <res-auth>Container</res-auth>
        </resource-ref>
        <resource-ref>
                <description>Proxy DB Connection</description>
                <res-ref-name>bioimageproxy</res-ref-name>
                <res-type>javax.sql.DataSource</res-type>
                <res-auth>Container</res-auth>
        </resource-ref>
</web-app>

9. Setting up a Ruby/Rails server with Postgres and d2r bridge

These are some notes on an experiment to set up a simple ruby/rails server that allows users to input some data to a relational database - I wanted to find out how long it would take to get something up and running this way - it turns out to be very quick (<2 days) to build a simple database, a few forms, and set up d2r to convert the data into rdf that can be dumped into bioimage.

I installed Ruby 1.84 on andros using the instructions at http://wiki.rubyonrails.com/rails/pages/RailsOnFedora but using wget ftp://ftp.ruby-lang.org/pub/ruby/ruby-1.8.4.tar.gz to get the most recent tarball. Installed fastcgi even though this is not really necessary at this stage

for postgres stuff i used http://wiki.rubyonrails.org/rails/pages/TutorialStepOnePostgresql

the stuff at http://www.onlamp.com/pub/a/onlamp/2005/01/20/rails.html is very good too

I opened port 3000 in IPtables by adding the line -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 3000 -j ACCEPT

to the end of /etc/sysconfig/iptables

by the end of day 1 i had a database table i could input data to.

next day downloaded d2r, and read the documentation. Installing was a bit fiddly, because of the postgres/mysql differences, but there is now a database creationg file and an xml mapping file that work with postgres in cvs

10. Installing ImageMagick and RMagick

Good instructions are at http://rmagick.rubyforge.org/install-faq.html#win - the key bit is

It's easy. Kaspar has already done all the work so you don't need to. The rmagick-win32 gem includs a pre-compiled version of ImageMagick and the most commonly used delegate libraries. Download the rmagick-win32 gem from the RMagick project page on RubyForge and unzip it into a temporary directory. (Note: this gem can't be installed remotely!) Run this command (where 1.x.x is the RMagick version number)

essentially you

gem install RMagick-win32-1.x.x-mswin32.gem

Then run

postinstall.rb

I then needed to copy RMagick.rb and RMagick.so into C:\ruby\lib\ruby\site_ruby to get it all to work

10.1. There are a few ruby files in cvs in the bulkuploader module

http://paros.zoo.ox.ac.uk/viewcvs/viewcvs.cgi/bulkuploader/

loadFromCSV.rb contains a script for reading from a CSV file and uploading the data to the database It's a very hacky thing - intended in its current form to be used only for bulk import. Both the filenames and model are hard wired into the code. However, the code is simple, easy to read, and all you need is this class. It calls the Converter class in convert.rb to extract the image metadata, create the thumbnails etc.

Converter.convert accepts an input and output filename and converts from the input format to the output format based on the filetype

Converter.thumbnail does the same, but also resizes to 200px square

Both methods also tidy up filenames by removing spaces and replacing them with underscores

convertfiles.rb contains a Filelist class that takes a directory and returns an array of files of a given type in the directory and its subdirectiries

There are three unit test files in the test directory. We should pull these together into a test suite.

11. Mapping CSV files into the db

TO RUN D2R C:\>set CLASSPATH=%CLASSPATH%;C:\d2r\lib\d2rmap-V03.jar;C:\workspace\bioimage3\WebRoot\WEB-INF\lib\jena.jar;c:\workspace\bioimage3\common_lib\log4j-1.2.8.jar;c:\d2r\lib\jakarta-oro-2.0.8.jar;c:\d2r\lib\xercesImpl.jar;c:\d2r\lib\xmlParserAPIs.jar;c:\workspace\bioimage3\common_lib\pg74.1jdbc2ee.jar;C:\workspace\graphl\graphl.jar;c:\workspace\bioimage3\WebRoot\WEB-INF\lib\commons-logging.jar

C:\>path = %path%;c:\d2r\lib

C:\>java de.fuberlin.wiwiss.d2r.D2rProcessor -map:c:\workspace\bioimage3\test\dtgedMap.xml -output:bioimagedata.rdf

This runs the D2R processor on the database - whose details are hard coded into dtgedMap.xml and outputs an RDF document that can be used to load a new bioimage database.

12. Future thoughts

http://wiki.rubygarden.org/Ruby/page/show/ScriptingExcel has some stuff about using ruby to read directly from an excel file ...

-- GrahamKlyne 2005-11-22 14:18:46