GenePattern Goodies/Badies

GP installer fails to launch, Java throws exception:

Preparing to install...
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...

Launching installer...

An internal LaunchAnywhere application error has occured and this application cannot proceed. (LAX)

Stack Trace:
java.lang.IllegalArgumentException: Malformed \uxxxx encoding.
        at java.util.Properties.loadConvert(Properties.java:569)
        at java.util.Properties.load0(Properties.java:392)
        at java.util.Properties.load(Properties.java:342)
        at com.zerog.common.java.util.PropertiesUtil.loadProperties(DashoA10*..)
        at com.zerog.lax.LAX.(DashoA10*..)
        at com.zerog.lax.LAX.main(DashoA10*..)

After some googling around, I discovered that Launch Anywhere often screws around with PS1, so changing this for the current shell resolves the issue:

$ export PS1=">"

Some basic ways to access GP's HSQLDB:

You must have RC configuration file(s) in the home directory either sqltool.rc or dbmanager.rc:

# This is for a hsqldb Server running with default settings on your local
# computer (and for which you have not changed the password for "SA").
urlid gp
url jdbc:hsqldb:hsql://127.0.0.1:9001/xdb
username sa
password

Then sqltool (commandline) or DatabaseManager (GUI) can be called:

java -cp hsqldb.jar org.hsqldb.util.DatabaseManager --urlid gp

If GP is installed on NFS share, it fails to start due to the HSQLDB failure.

This is NOT GP's issue. It's a HSQLDB issue. The workaround is disble HSQLDB file locking by presenting these in DB's properties file:

pwbcad@ubuntu:data$ cat db0.properties
hsqldb.nio_data_file=false
hsqldb.lock_file=false
...

UPDATE: the above method is for version between 1.7.2. and 1.8.0. Couldn't find out which version the GP uses. But tried replace the hsqldb.jar with the 1.8.10 version, now GP can start successfully.

SGE in PWBC

Below is the output from installing SGE via apt-get:

Creating config file /etc/default/gridengine with new version
Setting up gridengine-master (6.2u5-1ubuntu1) ...
Initializing cluster with the following parameters:
 => SGE_ROOT: /var/lib/gridengine
 => SGE_CELL: default
 => Spool directory: /var/spool/gridengine/spooldb
 => Initial manager user: sgeadmin
Initializing spool (/var/spool/gridengine/spooldb)
Initializing global configuration based on /usr/share/gridengine/default-configuration
Initializing complexes based on /usr/share/gridengine/centry
Initializing usersets based on /usr/share/gridengine/usersets
Adding user sgeadmin as a manager
Cluster creation complete
Setting up libxp6 (1:1.0.0.xsf1-2build1) ...
Setting up lesstif2 (1:0.95.2-1) ...
Setting up gridengine-qmon (6.2u5-1ubuntu1) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place

When running "qstat-f" on exec host, it complains:

error: commlib error: got select error (No route to host)
error: unable to contact qmaster using port 6444 on host "pwbclinuxlab.garvan.unsw.edu.au"

pwbclinuxlab.garvan.unsw.edu.au is the ex-qmaster which has been removed. Even in the new qmaster, add the exec host again, the exec host still remember the old one. Because it's the string hardcoded in:

/var/lib/gridengine/default/common/act_qmaster

When running "qstat-f" on exec host, it has another complain...

error: commlib error: access denied (client IP resolved to host name "". This is not identical to clients host name "")
error: unable to contact qmaster using port 6444 on host "sgeqmast01.garvan.unsw.edu.au"

Read Things to think about before installing Grid Engine.

This is very likely related to DNS. SGE requires both forward and reverse DNS queries. So make sure DNS server has been setup properly. In case of DNS server setup is too difficult, adding proper entries to /etc/hosts will fix the issue.

Remember: qmaster's hosts file must contain all SGE hosts' (qsub host, qexec host etc) record. Any other SGE host must contain qmaster's record.

127.0.0.1       localhost
129.94.136.232  sgeqexec01.garvan.unsw.edu.au   sgeqexec01
129.94.136.230  sgeqmast01.garvan.unsw.edu.au   sgeqmast01

Ubuntu 10.10 Base Image Installation Note

On the new VMware platform, I need to create a Ubuntu base image, which will be used as a deployment template for other groups. The base image has to be basic, flexible and secure.

Partioning the virtual hard disk, 36GB, LVM is used.

I create a volume group called VG01 on the 36GB HDD. From there I use the follow partion schema. There is no dedicated partition for /home because the users store the data to NFS space.

• 210MB /boot ext3
• 15.1GB / ext4
• rest swap

Partitioning Strategies

The Linux Logical Volume Manager

Enable OpenSSH

Just remember to select it during the installation.

Build up the min security FIRST

First modify the entries in the following files:

root@pwbclinuxlab:~# cat /etc/hosts.allow
# /etc/hosts.allow: list of hosts that are allowed to access the system.
#                   See the manual pages hosts_access(5) and hosts_options(5).
#
# Example:    ALL: LOCAL @some_netgroup
#             ALL: .foobar.edu EXCEPT terminalserver.foobar.edu
#
# If you're going to protect the portmapper use the name "portmap" for the
# daemon name. Remember that you can only use the keyword "ALL" and IP
# addresses (NOT host or domain names) for the portmapper, as well as for
# rpc.mountd (the NFS mount daemon). See portmap(8) and rpc.mountd(8)
# for further information.
#

#Allow servers & admin
...
root@pwbclinuxlab:~# cat /etc/hosts.deny
# /etc/hosts.deny: list of hosts that are _not_ allowed to access the system.
#                  See the manual pages hosts_access(5) and hosts_options(5).
#
# Example:    ALL: some.host.name, .some.domain
#             ALL EXCEPT in.fingerd: other.host.name, .other.domain
#
# If you're going to protect the portmapper use the name "portmap" for the
# daemon name. Remember that you can only use the keyword "ALL" and IP
# addresses (NOT host or domain names) for the portmapper, as well as for
# rpc.mountd (the NFS mount daemon). See portmap(8) and rpc.mountd(8)
# for further information.
#
# The PARANOID wildcard matches any host whose name does not match its
# address.
#
# You may wish to enable this to ensure any programs that don't
# validate looked up hostnames still leave understandable logs. In past
# versions of Debian this has been the default.
# ALL: PARANOID
ALL: ALL

Optional: change root password

sudo passwd

Optional: enable RSA SSH login

Password-less SSH via PuTTY

Config the network interface for internet access, DNS, hostname etc

Ubuntu Networking Configuration Using Command Line

Update the apt source.list so apt-get sources from Arrnet.

root@pwbclinuxlab:~# cat /etc/apt/sources.list
#
# deb cdrom:[Ubuntu-Server 10.10 _Maverick Meerkat_ - Release amd64 (20101007)]/ maverick main restricted

#deb cdrom:[Ubuntu-Server 10.10 _Maverick Meerkat_ - Release amd64 (20101007)]/ maverick main restricted
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

#deb http://au.archive.ubuntu.com/ubuntu/ maverick main restricted
#deb-src http://au.archive.ubuntu.com/ubuntu/ maverick main restricted
deb http://mirror.aarnet.edu.au/ubuntu/ maverick main restricted
deb-src http://mirror.aarnet.edu.au/ubuntu/ maverick main restricted

## Major bug fix updates produced after the final release of the
## distribution.
#deb http://au.archive.ubuntu.com/ubuntu/ maverick-updates main restricted
#deb-src http://au.archive.ubuntu.com/ubuntu/ maverick-updates main restricted
deb http://mirror.aarnet.edu.au/ubuntu/ maverick-updates main restricted
deb-src http://mirror.aarnet.edu.au/ubuntu/ maverick-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
#deb http://au.archive.ubuntu.com/ubuntu/ maverick universe
#deb-src http://au.archive.ubuntu.com/ubuntu/ maverick universe
#deb http://au.archive.ubuntu.com/ubuntu/ maverick-updates universe
#deb-src http://au.archive.ubuntu.com/ubuntu/ maverick-updates universe
deb http://mirror.aarnet.edu.au/ubuntu/ maverick universe
deb-src http://mirror.aarnet.edu.au/ubuntu/ maverick universe
deb http://mirror.aarnet.edu.au/ubuntu/ maverick-updates universe
deb-src http://mirror.aarnet.edu.au/ubuntu/ maverick-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
#deb http://au.archive.ubuntu.com/ubuntu/ maverick multiverse
#deb-src http://au.archive.ubuntu.com/ubuntu/ maverick multiverse
#deb http://au.archive.ubuntu.com/ubuntu/ maverick-updates multiverse
#deb-src http://au.archive.ubuntu.com/ubuntu/ maverick-updates multiverse
#deb http://mirror.aarnet.edu.au/ubuntu/ maverick multiverse
#deb-src http://mirror.aarnet.edu.au/ubuntu/ maverick multiverse
#deb http://mirror.aarnet.edu.au/ubuntu/ maverick-updates multiverse
#deb-src http://mirror.aarnet.edu.au/ubuntu/ maverick-updates multiverse

## Uncomment the following two lines to add software from the 'backports'
## repository.
## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
# deb http://au.archive.ubuntu.com/ubuntu/ maverick-backports main restricted universe multiverse
# deb-src http://au.archive.ubuntu.com/ubuntu/ maverick-backports main restricted universe multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
# deb http://archive.canonical.com/ubuntu maverick partner
# deb-src http://archive.canonical.com/ubuntu maverick partner

## Uncomment the following two lines to add software from Ubuntu's
## 'extras' repository.
## This software is not part of Ubuntu, but is offered by third-party
## developers who want to ship their latest software.
# deb http://extras.ubuntu.com/ubuntu maverick main
# deb-src http://extras.ubuntu.com/ubuntu maverick main

deb http://security.ubuntu.com/ubuntu maverick-security main restricted
deb-src http://security.ubuntu.com/ubuntu maverick-security main restricted
deb http://security.ubuntu.com/ubuntu maverick-security universe
deb-src http://security.ubuntu.com/ubuntu maverick-security universe
deb http://security.ubuntu.com/ubuntu maverick-security multiverse
deb-src http://security.ubuntu.com/ubuntu maverick-security multiverse

Install VMware Tools

Do not get the one from atp-get. It is unsupported by VMware. Follow the VMware official guide instead.

VMware Tools Installation Guide For Operating System Specific Packages

VMwareTools

Install other useful utilities like htop etc

Enable LDAP auth

LDAP feature can be enabled on another new image which is a clone of this one, so that I can have two images: with or without LDAP.

LDAPClientAuthentication

OpenLDAP Server

X11, Xming Confusion: Which X Server is Serving You?!

When I tried to start qmon, a X11 application from SGE, it gave my errors:

Warning: Cannot convert string "-adobe-helvetica-medium-r-*--14-*-*-*-p-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-helvetica-bold-r-*--14-*-*-*-p-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-helvetica-medium-r-*--20-*-*-*-p-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-helvetica-medium-r-*--12-*-*-*-p-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-helvetica-medium-r-*--24-*-*-*-p-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-courier-medium-r-*--14-*-*-*-m-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-courier-bold-r-*--14-*-*-*-m-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-courier-medium-r-*--12-*-*-*-m-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-helvetica-medium-r-*--10-*-*-*-p-*-*-*" to type FontStruct
X Error of failed request:  BadName (named color or font does not exist)
  Major opcode of failed request:  45 (X_OpenFont)
  Serial number of failed request:  343
  Current serial number in output stream:  354

According to many posts like this one, suggest to add required fonts from the repository then add to X11 path. Makes perfect sense. But I stuck in the step that adding fonts to X11 path by using xset fp+:

unused@pwbclinuxlab:~$ xset fp+ /usr/share/fonts/X11/100dpi
xset:  bad font path element (#90), possible causes are:
    Directory does not exist or has wrong permissions
    Directory missing fonts.dir
    Incorrect font server address or syntax

I tried many ways to fix it, like permissions, created fonts.dir by using mkfontdir (See here). Nothing helps. I had to try to find X11's conf file to see if anything there, but I couldn't find any!!! I checked X11 directories under /etc, /usr all empty or no binary there!! Now I realized that there is no X11 server running.

I almost forgot, I use PuTTY in a Windows platform, with Xming running, and now I am almost sure, Xming is the X server I need to modify, not the one on the server!! I found the font support installation from sourceforge called Xming-fonts-7-5-0-25-setup.exe, installed it, everything is fine!!

Password-less SSH via PuTTY

I need to connect to a Ubuntu 10.10 server via PuTTY, password-lessly.

You can generate the key pair from two places:

• On the Unbuntu server by using ssh-keygen. Detail
• On the local machine, using a utility called PuTTYgen

But using either way alone won't work because:

1. PuTTY only understands the private key file generated by PuTTYgen. PuTTY complains if I select a private key generated by ssh-keygen (on the server).
2. Ubuntu OpenSSH doesn't understand the public key generated by PuTTYgen. The server will give you Server refused our key if I copy the public key generated by PuTTYgen.

The solution is tricky but simple: all I need to do is, generate the key pair on the server by using ssh-keygen. Copy the generated private key to local machine, and load it to PuTTYgen, then convert it into PuTTY format. That's all!