svn trick: importing part of repository into another

Importing part of repository into another

svnadmin dump svndir/archiveA > ~/archiveA.svn_dump
cat ~/archiveA.svn_dump | svndumpfilter include subdirA > ~/archiveA-subdirA.svn_dump
svnadmin load svndir/archiveB < ~/archiveA-subdirA.svn_dump

Feb 03, 2011

svnimport (/linuxtricks)
2011-02-03, 10:28 [edit]

System Administration Tricks (mostly on linux)

Feb 03, 2011

home (/linuxtricks)
2011-02-03, 10:01 [edit]

Mailman users

Send a message with body "who to listname-request@listserver.

Jun 01, 2009

mailmanusers (/linuxtricks)
2009-06-01, 12:24 [edit] Michele's wikiblog


There's no gfortran, so we use Intel. Version 10 is not recognized by f2py (at least), so use 9.1. Linking does not find e_wsfe in the gnu-compiled liblapack and libblas, so we recompile those with Intel. Linking then complains about vmldExp2, which is found in Intel's libsvml.

LAPACK with Intel

The options required in are

FORTRAN  = ifort
OPTS     = -O3
NOOPT    = -O3 -fltconsistency -fp_port
LOADER   = ifort

the -fltconsistency is required to avoid a hang on the tests.

Feb 16, 2009

cosmomldc (/linuxtricks)
2009-02-16, 11:24 [edit] Michele's wikiblog

[insert password, then select /bin/bash; login again to get the new shell]
[do "module list", and "module unload" all the intel and mpich modules]
module load mpich-tcp-gnu
mkdir install
cd install
tar zxvf Python-2.5.1.tgz 
cd Python-2.5.1
./configure --prefix=$HOME
make install
cd ..
rm -rf Python-2.5.1
cd ~
cat >> .profile
[type "export PATH=$HOME/bin:$PATH";
 this will be effective at next login, so also run this in the terminal
 on some configurations this may need to go in .bashrc]
cd install
python --prefix=$HOME
easy_install numpy
tar zxf synthLISA-1.3.4c.tar.gz
cd synthLISA-1.3.4
./ swig --prefix=$HOME
python install --prefix=$HOME
cd ..
rm -rf synthLISA-1.3.4
[the following takes a long time, run in the background]
cd ~
cvs -d login
[CVS password: XXX]
cvs -d co qm
cvs -d co smbh
cd ~/install
tar zxf gsl-1.9.tar.gz
cd gsl-1.9
./configure --prefix=$HOME
make install
cd ..
rm -rf gsl-1.9
tar zxf fftw-3.1.2.tar.gz
cd fftw-3.1.2
./configure --prefix=$HOME
make install
make clean
./configure --prefix=$HOME --enable-float
make install
cd ..
rm -rf fftw-3.1.2
export PKG_CONFIG_PATH=$HOME/lib/pkgconfig
cd ~
export LSCSOFT_USER=${LSCSOFT_USER:-"anonymous"}
export LSCSOFT_SRCDIR=${LSCSOFT_SRCDIR:-"${HOME}/src/lscsoft/"}
mkdir -p ${LSCSOFT_SRCDIR}
cvs -d:pserver:${LSCSOFT_USER} login
[CVS password: lscsoft]
cvs -d:pserver:${LSCSOFT_USER} co lal
export LSCSOFT_SRCDIR=${LSCSOFT_SRCDIR:-"${HOME}/src/lscsoft/"}
export LAL_PREFIX=${LAL_LOCATION:-"${HOME}/opt/lscsoft/lal"}
./configure --prefix=${LAL_PREFIX}
make install
cat >> ~/.profile
[type ". $HOME/opt/lscsoft/lal/etc/"]
. $HOME/opt/lscsoft/lal/etc/
cd ~/smbh
python --prefix=$HOME
cat >> ~/.profile
 type "PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$HOME/lib/pkgconfig"]
cd ~/qm
python install --prefix=$HOME --rpath
cd ~/install
tar zxf mpi4py-0.5.0.tar.gz
cd mpi4py-0.5.0
python install --prefix=$HOME
python build_exe
python install_exe --prefix=$HOME
cd ..
[and then use the mpi4py installed in $HOME/bin instead of the system-provided mpirun]
[to install matplotlib]
cd ~/install
tar zxf libpng-1.2.22.tar.gz 
cd libpng-1.2.22
./configure --prefix=/home/vallis; make; make install
cd ..
rm -rf libpng-1.2.22
tar zxf matplotlib-0.90.1.tar.gz
cd matplotlib-0.90.1
[edit to add '/home/vallis' to the 'linux' and 'linux2' include dirs]
python install --prefix=$HOME
cd ..
rm -rf matplotlib-0.90.1
[get PyMC, patched version, from Michele]
tar zxvf PyMC-1.3.3.tar.gz 
cd PyMC-1.3.3
python install --prefix=$HOME
cd ..
rm -rf PyMC-1.3.3

Nov 28, 2007

cosmosmbh (/linuxtricks)
2007-11-28, 15:22 [edit]

Installing Jeeves

Ok, I'm making a diary of the steps I've taken to install Jeeves, my new server. Let's see…

→ Installed Red Hat Enterprise Linux AS U3. Partitioned with separate /, /var, /opt, /home.

→ Given it most packages imaginable.

→ Once the installer rebooted, ran bootstrap-el (after adding to correct netgroup and making yp on Newman, and reloading nfs on daffy), ran up2date (installing all available upgrades, including kernel; a couple of dependencies were not satisfied, excluded those).

→ Ran cfagent a couple of times; fixed ssh_users.amarok on Newman.


→ Downloaded Trac 0.9.6

→ Downloaded ClearSilver 0.10.3. Installed with --prefix=/usr and --with-python=/usr/bin/python.

→ The standard Subversion package does not have Python bindings. Removed it (rpm -e), and downloaded Subversion 1.3.2. Ran ./configure --prefix=/usr; make; make install; make swig-py. Will probably need to export PYTHONPATH=/usr/lib/svn-python.

→ Installed Silvercity 0.9.5.

→ Ran python install for Trac.

→ Got SQLite 3.3.7, ran ./configure --prefix=/usr --enable-threadsafe. Before making it, had to install tcl-devel-8.4.7-2.i386.rpm.

→ Installed pysqlite 2.3.2.

→ Created svn archive with svnadmin create /home/svn/foo.

→ Created project with trac-admin /home/projects/foo initenv.

→ Tested on builtin webserver with tracd --port 8000 /home/projects/foo. It's a miracle, but it works!

Trac again

→ svn seems to have troubles from the command line. Redirected /usr/lib/ and /usr/lib/ to /usr/lib/ instead of /usr/lib/

→ Downloaded cvs2svn 1.4.0 from Ran make install.

→ Converted bcv and synthlisa archives. To use svn without a server, do

svn co svn+ssh://

→ After much fussing on mod_python, concluded that it does not play well with suexec (which I'd like to use to keep projects under my username). Went on to fastcgi.

→ Downloaded mod_fastcgi-2.4.2 from To make, had to first install pcre-devel-4.5-3.2.RHEL4.i386.rpm, httpd-devel-2.0.52-28.ent.i386.rpm, copy Makefile.AP2 to Makefile, and run make top_dir=/usr/lib/httpd.

→ After much suffering with mod_python, cgi, fastcgi and all the damned bunch, decided to go with the tracd standalone server. Written a simple /etc/init.d/tracd.

→ Made an authentication entry for me with htdigest -c users vallis

→ Given myself all trac permissions with trac-admin /home/projects/synthlisa permission add vallis TRAC_ADMIN

→ Removed all anonymous permissions with this script

Aug 28, 2006

jeeves (/linuxtricks)
2006-08-28, 17:43 [edit]

OpenPBS: freeing busy nodes

If an OpenPBS node stays busy (as seen with pbsnodes -l -a) without apparent reason (the pbs_mom is up, the load is low…), one can try to free it by issuing the command

set node [nodename] state=free

in qmgr.

Apr 27, 2006

openpbs-freenode (/linuxtricks)
2006-04-27, 10:41 [edit]

Linux trick: authenticated SMTP

To do simple password authentication, sendmail servers will need the lines


in their file (compiled to /etc/mail/ by doing make in /etc/mail). [Above and below, the first quote character is the "ssh evaluate command" type, the second is the standard straight quote.]

I have not been able to do SMTPS (SMTP over SSL). Clearly opening port 465 and doing

DAEMON_OPTIONS(‘Port=smtps, Name=TLSMTA, M=s’)dnl 

is a start, but there still seems to be a problem with certificates.

A sneaky bug in some sendmail upgrades (in particular, sendmail-8.12.11-4.24.1.legacy) seems to break SMTP authentication. To fix it, try

/usr/sbin/alternatives --set mta /usr/sbin/sendmail.sendmail

See this bug report.

Mar 24, 2006

smtpauth (/linuxtricks)
2006-03-24, 12:05 [edit]

Linux tricks: installing Darwin Streaming Server

→ Untar the archive somewhere, and cd into it.

→ Run ./Install, and create an administrator name and password.

→ Connect to http://localhost:1220, create an mp3 password (what for), refuse/accept secure administration, set the Movies directory (perhaps /home/movies), refuse/accept streaming on port 80.

→ Done! More might be needed to relay broadcasts (add to these notes).

→ To ensure launch at boot try something like ln -s /root/DarwinStreamingSrvr5.5-Linux/ /etc/rc.d/rc5.d/S86qtss

Jan 14, 2006

darwinstream (/linuxtricks)
2006-01-14, 20:57 [edit]

Linux trick: installing OpenPBS

These are the steps to install OpenPBS on a linux cluster. The following refers to version 2.3.16.

Download and unpack the distribution. Run configure with the appropriate options: default settings are to install the package in /usr/local (and in particular /usr/local/bin and /usr/local/sbin; this should be fine for our purposes); to have the working directory (PBS_HOME) in /usr/spool/pbs; to have fifo as the scheduler. Configure with --with-scp (for all things to run smoothly, root on the server should be able to ssh into the execution hosts without password, and vice versa).

Patches needed with gcc 3.2.2 (before configuring); in buildutils/makedepend-sh, modify the eval $CPP… command at line 576 to include grep -v ">$" | \ after grep -v "$s\$" | \. Modify src/lib/Liblog/pbs_log.c by adding #include at the top. Same for src/server/svr_connect.c.

Cd into doc and make install to install the man pages.

Make and make install. If you have mounted /usr/local on other machines for access to Server/Executable/Client applications you may need to change permissions on /usr/local/sbin/pbs_mom. Each slave will also need a minimal pbs_mom directory structure. Copy buildutils/pbs_mkdirs from the source directory to the execution hosts, then on each of them run

sh pbs_mkdirs mom
sh pbs_mkdirs aux
sh pbs_mkdirs default

In order, run pbs_server, pbs_sched, pbs_mom (they are in /usr/local/sbin). The first time only, start pbs_server with the -t create option.

Before starting the Moms, you need to create the config file PBS_HOME/mom_priv/config, containing (assuming your pbs server is tweedledee)

$logevent 0x1ff
$clienthost tweedledee

Update [20040207]: Rajarshi Guha suggests adding also

$max_load 2.0
$ideal_load 1.0
$usecp /home

where the first line is the fully qualified hostname of the server, the second and the third indicate the load above which a mom will not take a job (and the load that it will wait to have before taking new jobs, after refusing one), and the last line directs the moms to copy log files with cp (over NFS, assuming it's available) in case scp does not work. This config should go also to all the execution hosts. When this is done, start pbs_mom on all the hosts.

Update [20040207]: When everything is started, run qmgr and setup some options:

set s
set s
set s query_other_jobs=true
create q standard queue_type=e
set q standard resources_min.cput=1,resources_max.cput=12:00:00
set q standard resources_default.cput=30:00
set q standard enabled=true, started=true
set s scheduling=true

Next, still in qmgr, add all the nodes with

create node node-1
create node node-2

Update [20040207]: I like to have scripts in /usr/local/bin that execute a command and copy a file over the cluster: they would look something like

for host in ‘cat /usr/spool/PBS/server_priv/nodes`; do
        echo -n "$host "
        ssh $host "$*"


for host in `cat /usr/spool/PBS/server_priv/nodes`; do
        echo -n "$host "
        scp $1 $host:$2

[You’ll probably need to change permissions on the nodes file.] Now you're good to go!

Jan 03, 2006

openpbs (/linuxtricks)
2006-01-03, 14:43 [edit]

Creating SSL certificates for IMAP/Apache in RedHat

Worked out in RedHat, but might work elsewhere. Caution, these are my partial recollections. Should test it again to make sure it works. Maybe when I reinstall the mail server…

We work in /usr/share/ssl. Edit the openssl.cnf configuration file, modify dir in CA_default to /usr/share/ssl, and set the default place for new certs to $dir/certs. Change the defaults for countryName, etc., so that we won't have to type them in repeatedly.

Touch /usr/share/ssl/index.txt, and echo '100001' into /usr/share/ssl/serial.

We create a root certificate for our server. We don't really want to shell out money to a proper authority, right?

openssl req -new -x509 -extensions v3_ca -keyout private/cakey.pem \
    -out cacert.pem -days 3650 -config ./openssl.cnf

Now the root certificate consists of /usr/share/ssl/cacert.pem and /usr/share/ssl/private/cakey.pem; the first is the public part, which we can distribute to people who want to trust us.

Now go to certs and create a request for the imapd certificate:

openssl req -new -nodes -out imapd-req.pem -keyout imapd-key.pem \
    -days 3650 -config ../openssl.cnf

and actually get the certificate with

openssl ca -out imapd-cert.pem -config ../openssl.cnf \
    -infiles imapd-req.pem

(-infiles must be last!) We'll need the passkey used to create the CA. Last, merge the imapd key and certificate into the final signed certificate:

cat imapd-key.pem imapd-cert.pem > imapd.pem

Should be all! Imapd knows to look for /usr/share/ssl/certs/imapd.pem. See also how to use these certificates in OS X

The same certificates can be used for Apache, but the .key files goes into /etc/httpd/conf/ssl.key/server.key; the .pem files goes into /etc/httpd/conf/ssl.crt/server.crt.

Mmh… maybe /usr/share/ssl/misc/CA does all of the above?

Big thanks go to the kind folks at flatmtn.

Aug 26, 2005

sslcerts (/linuxtricks)
2005-08-26, 00:47 [edit]

Unix trick: stopping processes in cfengine

I often use the define/shellcommands (final) trick, but it seems the following is an alternative, especially for demons that don't require /etc/service stop:

    "lpd" signal=kill

Feb 07, 2005

cfenginekill (/linuxtricks)
2005-02-07, 14:02 [edit]

OpenPBS trick: maximum cpu time

In OpenPBS, it is a good rule to set the default resource allocations of the execution queues to the same value of the maximum resource allocations. For example, in qmgr:

set queue myqueue resources_max.cput = 72:00:00
set queue myqueue resources_min.cput = 00:00:01
set queue myqueue resources_default.cput = 72:00:00

otherwise the default value is considered as the maximum for a given job that does not request more via something like qsub -l cput=…, and the job will be killed when it reaches the default value (of time, memory, etc.)

Nov 27, 2004

openpbs-maxcput (/linuxtricks)
2004-11-27, 12:00 [edit]

Linux trick: enabling SpamAssassin

To enable SpamAssassin (as preconfigured in the standard RedHat 9.0 distribution) on a user by user basis, add the lines

:0fw: spamassassin.lock
| /usr/bin/spamassassin

to .procmailrc in your home directory (assuming that sendmail is configured to use procmail as the local delivery agent). The incoming spam should then be tagged (X-Spam-Flag: YES) and the subject line should be changed to include [SPAM]. Personal preferences can be found in .spamassassin.

A better strategy for large-scale deployment would be to replace /usr/bin/spamassassin above with /usr/bin/spamc (which might need the spamd daemon running). But we'll leave this for the future.

Nov 11, 2003

spamassassin (/linuxtricks)
2003-11-11, 21:57 [edit]

Sendmail trick: processing mail to virtusers

First, the configuration file should point to the virtual user database; to do this, include a line

FEATURE(‘virtusertable’,‘hash /etc/mail/virtusertable.db’)dnl

The virtual user database itself should be contained, in clear, in the file /etc/mail/virtusertable; each line should contain a virtual address ( and the translated address. The .db file is created with makemap hash virtusertable.db < virtusertable, but running make in the /etc/mail directory should be enough.

Second, the should contain the line


and the /etc/mail/local-host-names file should contain all the domains for which we are accepting mail.

Now run make in /etc/mail to transfer your changes to the configuration file, reload sendmail, and have fun.

Nov 11, 2003

virtusers (/linuxtricks)
2003-11-11, 20:39 [edit]

  Login:   Password:   [rss] [cc] [w3] [css]

© M. Vallisneri 2012 — last modified on 2010/01/29

Tantum in modicis, quantum in maximis