Old version: the latest version of this page is being maintained at
http://milos2.zoo.ox.ac.uk/ibrgtech/index.php/Zakynthos_setup.
Zakynthos Installation Notes
This page contains notes about the installation and setup of Zakynthos, which is intended to provide a base for infrastructure server components.
Contents:
- Basics
- Extras installed
- Component installation notes
- Updating VMware and VMware tools
- Enabling backups for MySql
- Cloning Zakynthos for a new system
- Adding a new virtual disk
- Late fixes
1. Basics
Scientific Linux 4.4, with:
Default runlevel 3 (X-windows installed but not running)
Apache HTTPD
Postfix mail relay
MySql
Note: when installing under VMware, re-run the system-confic-display script after VMware tools have been installed. This seems to fix some problem with switching between the X-window session and text-only console sessions, and also problems switching between runlevels 5 and 3.
2. Extras installed
PHP 5.20
phpMyAdmin
Media Wiki
Semantic Media Wiki
Drupal
WebCalendar
3. Component installation notes
3.1. Installing SSHBlack
See:
http://wiki.oss-watch.ac.uk/InstallingSshblack
I had to adjust the init.d/sshblack script to take account of a different directory for the Perl script file coming from the V2.8 SSHBlack kit.
3.2. Assign MySql root password
See:
http://www.cyberciti.biz/faq/mysql-change-root-password/
If you have never set a root password for MySQL, the server does not require a password at all for connecting as root. To setup root password for first time, use mysqladmin command at shell prompt as follows:
mysqladmin -u root password NEWPASSWORD
However if you want to change (or update) a root password, then you need to use following command:
mysqladmin -u root -p oldpassword newpass
3.2.1. Reset MySql root password
See:
http://www.cyberciti.biz/tips/recover-mysql-root-password.html
Quick summary for root password reset:
# service mysqld stop # mysqld_safe --skip-grant-tables & # mysql -u root mysql> use mysql; mysql> update user set password=PASSWORD("....") where User='root'; mysql> flush privileges; mysql> quit; # service mysqld stop # service mysqld start # mysql -u root -p Enter password: .... mysql> quit;
The full log looks like this:
# service mysqld stop Stopping MySQL: [ OK ] # mysqld_safe --skip-grant-tables & [1] 21601 # Starting mysqld daemon with databases from /var/lib/mysql # mysql -u root Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 to server version: 5.0.22 Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> use mysql; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> update user set password=PASSWORD("....") where User='root'; Query OK, 2 rows affected (0.02 sec) Rows matched: 2 Changed: 2 Warnings: 0 mysql> flush privileges; Query OK, 0 rows affected (0.01 sec) mysql> quit; Bye # service mysqld stop STOPPING server from pid file /var/run/mysqld/mysqld.pid 071101 12:29:17 mysqld ended Stopping MySQL: [ OK ] [1]+ Done mysqld_safe --skip-grant-tables # service mysqld start Starting MySQL: [ OK ] # mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 to server version: 5.0.22 Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> quit;
3.3. Installing PHP 5.20 from source
yum install httpd-devel
Download and unpack PHP source to working directory
./configure --with-apxs2=/usr/sbin/apxs --enable-mbstring --with-mysql --with-mysqli=/usr/bin/mysql_config
make
make install
The make install adds the following line to /etc/httpd/conf/httpd.conf that duplicates a line in /etc/httpd/conf.d/php.conf, and which should be removed:
LoadModule php5_module /usr/lib/httpd/modules/libphp5.so
The ./configure, make and make install steps can be repeated at any time to update the PHP setup, e.g. with different options.
3.3.1. Warning
I have found that automatic system updates by YUM can create a new file /etc/httpd/conf.d/php.conf that attempts to configure PHP4 to handle requests for PHP files, which conflicts with using PHP5. This leads to obscure errors about mysql.so and segment faults reported in /var/log/httpd/error_log when trying to access facilities based on PHP and a MySql database. The problem can be fixed by deleting (or renaming) the php.conf file.
My /etc/httpd/conf.d/php5.conf file looks like this:
# # PHP is an HTML-embedded scripting language which attempts to make it # easy for developers to write dynamically generated webpages. # LoadModule php5_module modules/libphp5.so # # Cause the PHP interpreter to handle files with a .php extension. # AddType application/x-httpd-php .php .php5 # AddType application/x-httpd-php-source .phps # # Add index.php to the list of files that will be served as directory # indexes. # DirectoryIndex index.php
3.4. Installing Media Wiki
See:
http://www.mediawiki.org/wiki/Manual:Installation (was:
http://meta.wikimedia.org/wiki/Help:Installation)
Download kit (see
http://www.mediawiki.org/wiki/Manual:Downloading_MediaWiki) Unpack files to web server directories and ensure the config subdirectory is writeable by apache, e.g.:
cd /var/www/html tar xvfz tar xvfz /storage1/kit/NETKIT/MediaWiki/mediawiki-1.11.0.tar.gz ln -s mediawiki-1.11.0/ wiki chown -R root: mediawiki-1.11.0 chown -R apache: mediawiki-1.11.0/config
With the database password to hand, run the mediawiki installation script: browse to the wiki front page (e.g.
http://zakynthos.zoo.ox.ac.uk/wiki/), then click on the "set up the wiki" link. Move file config/LocalSettings.php to the parent directory:
mv config/LocalSettings.php .
Edit LocalSettings.php:
$wgEnableUploads = true;
Sort file/directory protections:
chown -R apache: images chmod go-w LocalSettings.php
The wiki should now be usable.
3.4.1. Updating a wiki user's password
Login as root on the host system.
Start the mysql client with the appropriate credentials:
/usr/bin/mysql -u root -p
(enter the password when prompted.)
Issue the following command:
USE wikidb; UPDATE iw_user SET user_password = MD5(CONCAT(user_id, '-', MD5('somepass'))) WHERE user_name = 'someuser';
(changing "somepass" and "someuser" as appropriate.)
3.5. Installing Drupal
See:
http://drupal.org/node/260
Take note of the bit about creating the Drupal database in MySql - it is easily overlooked.
The mysqli client interface for PHP (see PHP section above) is needed to overcome a bug in Drupal 4.7.4 regarding maintaining a session login identity.
Initial configuration of Drupal to do somnething useful isn't obvious. As a Drupal admin, go to the modules menu item and enable some useful modules. I started with the blog module.
3.6. Installing WebCalendar
Download and unpack the kit to a working directory. Rename to /var/www/html/calendar. Change ownership so that apache can write to the directory tree.
Issue the following commands to create and configure the WebCalendar database and database user account using MySql utilities:
mysqladmin -u root -p create webcalendar
mysql --user=root mysql mysql> GRANT ALL PRIVILEGES ON webcalendar.* TO webcaluser@localhost IDENTIFIED BY '(password)' WITH GRANT OPTION; mysql> FLUSH PRIVILEGES; mysql> QUIT
Create a file /etc/http/conf.d/WebCalendar.conf:
<Location /calendar> php_value magic_quotes_gpc 1 </Location>
and restart the web server (use service httpd restart or apachectl -k restart).
Now browse to
http://zakynthos.zoo.ox.ac.uk/calendar/ - the browser should be redirected to the install/setup page.
After the initial installation has been done, browsing to
http://zakynthos.zoo.ox.ac.uk/calendar/ should display a calednar login. Log in as admin, password admin, and change the password.
3.7. Configuring Mailman
Mailman is installed as part of the SL 4.4 system.
Mailman is configured using some of the the instructions in
http://www.gnu.org/software/mailman/mailman-install/index.html. Relevant files are:
/usr/lib/mailman/... - software installation
/usr/lib/mailman/Mailman/mm_cfg.py - configuration
/var/lib/mailman/... - data files
/etc/postfix/main.cf - postfix configuration
Edit /usr/lib/mailman/Mailman/mm_cfg.py, adding the following line at the end:
MTA = 'Postfix'
Run /usr/lib/mailman/bin/genaliases. This creates new aliases and aliases.db files in /etc/mailman/.
Modify /etc/postfix/main.cf to contain this line:
alias_maps = hash:/etc/aliases,hash:/etc/mailman/aliases
Note that the alias_database entry should not be changed.
Set up the mailman cron jobs, which are described in file /usr/lib/mailman/cron/crontab.in, by copying this file to /etc/cron.d/mailman. Do not use the crontab command with this file, as the mailman username in this file is interpreted as a shell command, and gives rise to many errors in the cron logs.
Start mailman on boot, and immediately:
[chkconfig --list mailman chkconfig mailman on service mailman start
Finally, uncomment the last line of /etc/httpd/conf.d/mailman.conf edit the host name into it:
RedirectMatch ^/mailman[/]*$ http://zakynthos.zoo.ox.ac.uk/mailman/listinfo
Browsing to
http://zakynthos.zoo.ox.ac.uk/mailman/listinfo should display MailMan's front page.
4. Updating VMware and VMware tools
Basically, installing a new version of VMware workstation removes the previous version, but sufficient configuration is retained that any old VMs are visible to the new installation.
To update VMware tools, follow the procedure for instlling VMware tools:
http://www.vmware.com/support/ws55/doc/ws_newguest_tools_upgrade.html
http://www.vmware.com/support/ws55/doc/ws_newguest_tools_linux.html
Here are the key steps:
Power on the virtual machine.
After the guest operating system has started, prepare your virtual machine to install VMware Tools.
Choose VM > Install VMware Tools.
As root (su -), mount the VMware Tools virtual CD-ROM image, change to a working directory (for example, /tmp), uncompress the installer, then unmount the CD-ROM image:
mount /media/cdrom1 cd /tmp
Install the RPM package:
rpm -Uhv /media/cdrom1/VMwareTools-5.0.0-<xxxx>.i386.rpm umount /media/cdrom1
Configure VMware Tools:
-
vmware-config-tools.pl
-
Respond to the questions the installer displays on the screen. Press Enter to accept the default value.
In an X terminal, launch the VMware Tools background application.
vmware-toolbox &
5. Enabling backups for MySql
NOTE: These instructions are still under test
First, enable binary logging from the MySql server: edit file /etc/my.cnf so that section [mysqld] contains the line:
log-bin = /var/log/mysql/mysql-bin.log
Also create a directory /var/log/mysql/, and sets its owner to be mysql. Now restart mysqld.
Create a file /root/.my.cnf containing the following lines, where <password> is the MySql database root password:
[client] password=<password>
Ensure this file is set to be readable only by user root:
chown root:root /root/.my.cnf chmod 600 /root/.my.cnf
Create a script file /usr/local/<host>-dump-mysql, where <host> is a name distinguishing this host or MySql instance:
#!/bin/sh (-) # # Backup selected databases from running MySql # Assumes binary logging is enabled # tgtdir="/var/log/mysql/" host="delos" first="--flush-logs" # Remember list of old log files oldlogs=`echo /var/log/mysql/mysql-bin.[!i]*` for db in mysql wikidb drupaldb webcalendar do echo "Backing up database $db to ${tgtdir}dump_${host}_${db}.sql.bz2 ($first)" # Delete old backup if test -e ${tgtdir}oldlogs/dump_${host}_${db}.sql.bz2 ; then rm ${tgtdir}oldlogs/dump_${host}_${db}.sql.bz2 fi # Rename current backup as old backup if test -e ${tgtdir}dump_${host}_${db}.sql.bz2 ; then mv ${tgtdir}dump_${host}_${db}.sql.bz2 ${tgtdir}oldlogs/dump_${host}_${db}.sql.bz2 fi # Create new backup mysqldump --master-data=2 $first $db | bzip2 -c > ${tgtdir}dump_${host}_${db}.sql.bz2 first="" done # Now move old log files away if test -n "`echo /var/log/mysql/oldlogs/mysql-bin.*`" ; then rm /var/log/mysql/oldlogs/mysql-bin.* fi mv ${oldlogs} /var/log/mysql/oldlogs/ # # End.
Finally, arrange to run this script as a weekly cron job, preferably at a time when the server can be expected to be lightly loaded - for the example below, 01:00AM on Sunday mornings. Create a file /etc/cron.d/mysqldump with the following contents:
# Dump mysql database every week (Sunday 01:00AM) 0 1 * * sun root /usr/local/bin/delos-dump-mysql
6. Cloning Zakynthos for a new system
Shutdown the Zakynthos system, and take a VMware snapshot if necessary.
Use the VMware snapshot manager to create a clone copy (preferred to a linked clone).
Now start the newly copied VM and rename all the references to "zakynthos", its IP address, and other placeholder names to values for the new system:
System IP address: this is invoked automatically when the new system is booted.
The X-window network setting interface gives options to set hostname and up to 3 DNS servers.
After using this to change the machine name, reboot
/etc/hosts
Remove old log files
Change root password
Reboot
updatedb (for 'locate')
Change MySql passwords
root
wikiuser
drupaluser
webcaluser
Apache:
Edit file /var/www/html/index.html
Media Wiki:
Change database password: edit file /var/www/html/wiki/LocalSettings.php
Change admin password: login as "WikiSysop" and use User Preferences page to change password
Rename host name on main page
Drupal:
Change database password: edit file /var/www/html/drupal/sites/default/settings.php
Change main admin account: log in as druid and use account setting menu item to change password
If necessary, change the site description using administer/settings/general settings.
WebCalendar:
Change database password: edit file /var/www/html/calendar/includes/settings.php
Change the admin password (use admin/users for this).
Postfix:
Edit: /etc/postfix/main.cf
Rerun newalias
Mailman:
Edit /etc/httpd/conf.d/mailman.conf
From directory /usr/lib/mailman run this command:
bin/withlist -l -r fix_url mailman -v
In directory /etc/mailman, run /usr/lib/mailman/bin/genaliases to regenerate file /etc/mailman/aliases.db. (The host name is embedded in this file.) Check the file permissions allow mailman group to update the resulting file.
Change the mailman site password: from directory /usr/lib/mailman, run bin/mmsitepass
Change mailman list password: use something like:
http://delos.zoo.ox.ac.uk/mailman/admin/mailman
Rebuild PHP from source (the host name is embedded somewhere in libphp5.so).
After all this, some references to the old system name remain in the MySql database files: wikidb, drupaldb, user.MYD,user.MYI, ibdata1.
Checklist of passwords for installed facilities:
System root user
MySql users (use phpMyAdmin):
root
wikiuser
drupaluser
webcaluser
Media Wiki users:
WikiSysop
Drupal users (use Drupal admin menu):
druid (admin)
WebCalendar users (use admin account):
admin
Mailman:
mailman site password
mailman list password
7. Adding a new virtual disk
Stop the virtual machine
Create a new virtual disk using the VMWare "console" window
Restart the virtual machine
Use fdisk to create a new partition on the new disk (/dev/sd?, where '?' is 'b', 'c', etc.)
Create a new primary partition occupying the entire virtual disk; default partition type is Linux
Check partition details, save and exit fdisk
Create a file system on the new disk:
mkfs -t ext3 /dev/sd?1
Edit fstab, creating a new entry for the new disk based on an existing entry, e.g.
/dev/sdb1 /data ext2 defaults 0 2
/dev/sdc1 /eprints ext3 defaults 0 2
The last two numbers are a dump flag (non-zero if file system should be dumped), and pass number for fsck processing (root file system first, then all others).
Mount the new disk:
mount /dev/sd?1
8. Late fixes
(In due course, some or all of these should be installed in the zakynhtos base system.)
Comment out entry for gate_news in /usr/lib/mailman/cron/crontab.in -- seems to generate bad mailman command. (Why?) I now think this is a red herring, and the problem was caused by trying to use the file with the crontab command, rather than simply placing it in /etc/cron.d. Note that crontab and files in /etc/cron.d additionally specify a username under which the script is executed.
Restrict anonymous access to the Media Wiki installation; in /var/www/html/wiki/LocalSettings.php:
$wgGroupPermissions['*' ]['createaccount'] = false; $wgGroupPermissions['*' ]['read'] = true; $wgGroupPermissions['*' ]['edit'] = false; $wgGroupPermissions['*' ]['createpage'] = false; $wgGroupPermissions['*' ]['createtalk'] = false;
Stop and disable cupsd:
service cups stop chkconfig cups off chkconfig --list cups
Rewrite drupal URIs to clean form. Add file /etc/httpd/conf.d/drupal.conf:
# Define URI rewriting for drupal, to allow 'clean' URIs without ?q=<node> <Directory /var/www/html/drupal> RewriteEngine on RewriteBase /drupal RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule ^(.*)$ index.php?q=$1 [L,QSA] </Directory>
Then enable clean URIs through Drupal's Administer -> Settings -> General menu.
8.1. Changing mailman host name
With the new host name resolved by DNS, edit /usr/lib/mailman/Mailman/mm_cfg.py to reflect the new name; e.g.
# Override name: fqdn = "imageweb.zoo.ox.ac.uk" DEFAULT_URL_HOST = fqdn DEFAULT_EMAIL_HOST = fqdn
Run fix_url to fix up the new URIs; e.g.
/usr/lib/mailman/bin/withlist -l -r fix_url imageweb-announce /usr/lib/mailman/bin/withlist -l -r fix_url imageweb-private
Check the mailinglist admin interface to see that the new hostname shows as the preferred hostname.
8.2. Installing TSM
Install TSM, e.g.
rpm -i /mnt/hgfs/kits/tux/TSM/tsm-client-b4-5.3.0.4-3.i386.rpm
Edit /opt/tivoli/tsm/client/ba/bin/dsm.sys:
Specify the machine name for backup
Change schedule log name to /var/log/dsmsched.log
Edit /opt/tivoli/tsm/client/ba/bin/dsm.opt to specify the filesystems to backup; e.g.
Domain / /boot
Edit incl.excl to exclude live MySql database files; e.g.
Exclude.dir /var/lib/mysql
Initial backup using dsmc: use incr command.
Check that the file /opt/tivoli/tsm/client/ba/bin/dsm.sys contains the line, adding it if necessary:
ManagedServices Schedule
Copy the dsmcad service control file to /etc/init.d, and install the service to start automatically in the appropriate run levels:
cp /opt/tivoli/tsm/client/ba/bin/dsmcad-init /etc/init.d/dsmcad chmod a+x /etc/init.d/dsmcad chkconfig --add dsmcad chkconfig --list dsmcad
Start the client accdeptor daemon (CAD):
service dsmcad start
8.3. Increasing the PHP file upload size
Rename /etc/php.ini, and create a symlink to /usr/local/lib/php.ini
mv /etc/php.ini /etc/php.ini.old ln -s /usr/local/lib/php.ini /etc/php.ine
Edit php.ini to allow larger file uploads:
post_max_size = 20M
and
upload_max_filesize = 10M
Restart HTTPD server
Log in as drupal administrator, and increase allowed file upload size and total upload size for authenticated users (administer > settings > upload > Settings for authenticated user).
8.4. Receiving incoming mail
(See also:
http://www.cpqlinux.com/hostname.html)
Request DNS changes so that MX routing for imageweb.zoo.ox.ac works OK. In the Oxford environment, this amounts to something like this:
Please apply DNS changes to make imageweb.zoo.ox.ac.uk the main host name for A and MX records at 129.67.24.54, and delos.zoo.ox.ac.uk an alternative name for imageweb.zoo.ox.ac.uk, corresponding to: delos.zoo.ox.ac.uk. 86400 IN CNAME imageweb.zoo.ox.ac.uk. imageweb.zoo.ox.ac.uk. 86400 IN A 129.67.24.54 imageweb.zoo.ox.ac.uk. 86400 IN MX 9 oxmail.ox.ac.uk.
Edit /etc/hosts to include imageweb as host name; e.g.:
root@imageweb# less /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost 129.67.24.54 imageweb.zoo.ox.ac.uk imageweb 129.67.24.54 delos.zoo.ox.ac.uk delos
Edit interface definitions so that imageweb.zoo is a recognized interface for postfix incoming mail. The command hostname shows the current host name. ifconfig deals with network interface address configuration. File /etc/sysconfig/network holds the permanent network name, e.g.:
# less /etc/sysconfig/network NETWORKING=yes HOSTNAME=imageweb.zoo.ox.ac.uk
Note: changing the host name as above may require the TSM password to be reset. Use the self-registration pages at
https://register.oucs.ox.ac.uk/self/index to reset the server-side password, then use the dsmc program (and maybe perform an incr operation?) to reset the client-side stored password. Edit postfix configuration file /etc/postfix/main.cf to use hostname imageweb instead of delos. Also, allow any incoming mail for this domain or host. Here are some relevant lines that have been customized, though many are not related specifically to the receiving mail domain:
myhostname = imageweb.zoo.ox.ac.uk mydomain = zoo.ox.ac.uk inet_interfaces = $myhostname, localhost mydestination = $myhostname, localhost.$mydomain, localhost mynetworks_style = host relayhost = [oxmail.ox.ac.uk] alias_maps = hash:/etc/aliases,hash:/etc/mailman/aliases alias_database = hash:/etc/aliases
File /var/log/maillog records all incoming connections the the mail receiver, with information about the MAIL FROM and RCPT TO addresses of any messages accepted.
8.5. No Ethernet interface in cloned system
We have observed that ion the process of cloning a system, the new system's ethernet interface (with new IP address, etc) may not start at boot.
A quick fix may be this command:
ifup eth0
Some or all of the following files are used when bringing up the network after a reboot:
/etc/modprobe.conf
/etc/sysconfig/network-scripts/ifcfg-eth0
/etc/sysconfig/networking/devices/ifcfg-eth0
/etc/sysconfig/networking/profiles/default/ifcfg-eth0
On our test system, ifcfg-eth0 looks like this:
# Advanced Micro Devices [AMD] 79c970 [PCnet32 LANCE] DEVICE=eth0 ONBOOT=yes BOOTPROTO=none IPADDR=129.67.24.151 NETMASK=255.255.252.0 GATEWAY=129.67.27.254 HWADDR=00:0C:29:5F:5F:90 TYPE=Ethernet
On our test system, modprobe.conf looks like this:
alias eth0 vmnics alias scsi_hostadapter mptbase alias scsi_hostadapter1 mptscsi alias scsi_hostadapter2 mptfc alias scsi_hostadapter3 mptspi alias scsi_hostadapter4 mptsas alias scsi_hostadapter5 mptscsih install ipw3945 /sbin/modprobe --ignore-install ipw3945 ; sleep 0.5 ; /sbin/ipw3945d --quiet remove ipw3945 /sbin/ipw3945d --kill ; /sbin/modprobe -r --ignore-remove ipw3945 alias snd-card-0 snd-ens1371 options snd-card-0 index=0 install snd-ens1371 /sbin/modprobe --ignore-install snd-ens1371 && /usr/sbin/alsactl restore >/dev/null 2>&1 || : remove snd-ens1371 { /usr/sbin/alsactl store >/dev/null 2>&1 || : ; }; /sbin/modprobe -r --ignore-remove snd-ens1371 alias usb-controller uhci-hcd # Added by VMware Tools install vmnics /sbin/modprobe vmxnet; /sbin/modprobe pcnet32; /bin/true alias char-major-14 es1371
-- GrahamKlyne 2006-11-20 14:17:06
BioImage Wiki (internal)