How do use itertools in Python to build permutation or combination

Posted on November 9, 2012 by Thomas Cokelaer

There is a python module dedicated to permutations and combinations called itertools.

import itertools

The permutation function allows you to get permutation of N values within a list, where order matters. For instance, selecting N=2 values with [1,2,3] is done as follows:

>>> print list(itertools.permutations([1,2,3], 2))
[(1, 2), (1, 3), (2, 1), (2, 3), (3, 1), (3, 2)]

If order is not important, you can use combinations:

>>> print list(itertools.combinations([1,2,3], 2))
[(1, 2), (1, 3), (2, 3)]

Then, you may want to build all possible arrays of N values taken from a list of possible values. For instance, you may want to build a vector of length N=3 made of 0 and 1. This can be done with the cartesian product function:

>>> print list(itertools.product([0,1], repeat=3))
[(0, 0, 0), (0, 0, 1), (0, 1, 0), (0, 1, 1), (1, 0, 0), 
(1, 0, 1), (1, 1, 0), (1, 1, 1)]
Posted in Python | Tagged | 6 Comments

yum update failure: Cannot retrieve repository metadata

Posted on November 2, 2012 by Thomas Cokelaer

Sometimes, using “yum update”, the process fails with an error message similar to::

    Error: Cannot retrieve repository metadata (repomd.xml) for repository:
    rpmfusion-free-updates. Please verify its path and try again

You could hack your yum repository in /etc/yum.repo.d but the best solution is simply to use the yum option “–disablerep
o”:

sudo yum --disablerepo=rpmfusion-free-updates --disablerepo=rpmfusion-nonfree-updates --disablerepo=rpmfusion-free upd
ate
Posted in Linux | Tagged , | Leave a comment

SED: insert text at the top of a bunch of files

Posted on November 2, 2012 by Thomas Cokelaer

at a line at the top of a file

Imagine, you have a set of files ending in .txt, and you want to add the statement #header at the top of each of them. Then, simply type:

ls *.txt | xargs  sed  -e '1i #header'

Replace sed -e by sed -i -e to replace the file content, otherwise it is like a dry run.

Posted in Linux | Tagged | Leave a comment

quick SVN tutorial for end-users

Posted on November 1, 2012 by Thomas Cokelaer

There are lots of resources on the web about SVN. so, this page is just a quick summary of the 4-5 commands you will need to start with.

The following commands are for linux users (and therefore Mac users who have access to a linux command line interface).

A SVN repository is a centralised repository within an external server. You, as a user can access to this repository to obtain all its contents and to commit changes or new content.

So, first, you need the name of the repository. Let us suppose that the server is https://server and that the name of the repository is project.

Then, you can check out the entire repository on your hard disk inside a directory called project (you can name it differently if you want):

svn co https://server/project project

Go into the project directory. You can add a new file:

svn add example.py

edit your file and when you are happy with it, you can just commit it into the server so that other users can access to it:

svn commit example.py -m "some meaningful comments for bookkeeping"

Note that we used the option “-m” to provide a comment.

If you want to delete a file,

svn delete example.py

Right now, it is only deleted locally, so if you want this change to be reflected on the server, you must now commit the changes:

svn commit -m "delete a file"

Keep in mind that you can perform these commands on several files at the same time. For instance,

svn add file1.py file2.py
svn commit file1.py file2.py

If you simply type

svn commit

without files, then SVN will search for all the new files and all the modified files and will commit them on the server.

You may want to know what are the new or modified files before a commit. This can be done with the following command:

svn status

Finally, one of the interest of SVN is that you can track the difference between your directory and the server directory. For instance if you’ve modified your file example.py and wonder how different it is from the server, type

svn diff example.py

If you can not commit a file, usually it is because you do not work on the lastest version of the file. So,
you must update first:

svn update

This happens when one of your collaborators committed a new version of the file you are working on.

To perform a dry-run update in order to see if someone modified a file, type

svn status -u
Posted in Computer Science | Tagged | Leave a comment

Tkinter missing dependency

Posted on October 31, 2012 by Thomas Cokelaer

Under Fedora 17, I did an import of Tkinter package but got this error in a python shell:

Traceback (most recent call last):
  File "slider-demo-1.py", line 1, in <module>
    from Tkinter import *
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 39, in <module>
    import _tkinter # If this fails your Python may not be configured for Tk

I first install tkinter (i686) with yumex and it do not help, I then install tk-devel: no success. I finally installed another tkinter (x86_64) and then it solved the problem.

So, the solution is:

sudo yum install tkinter* tk-devel
Posted in Linux, Python | Tagged | Leave a comment

Bioconductor: first installation

Posted on October 28, 2012 by Thomas Cokelaer

Bioconductor is a website that gathers open source software for bioinformatics in the R language.

You can install a Bioconductor package using their interface. As an example, let us install the RBGL package. First, start a R session, and then just type:

source("http://www.bioconductor.org/biocLite.R")
biocLite("RBGL")

If you are installing a package for the first time and started R as a user (not root), then you will not have permission to install package in the standard directory. However, R will ask you to create a new directory to install the new package. Just say yes to the questions.

Dependencies will be managed by R automatically.

Posted in R | Leave a comment

Installing JPype to use Java from Python

Posted on October 26, 2012 by Thomas Cokelaer

JPype is a Python package that allows you to call Java from Python.

First, download the zip file (I got version 0.5.4.2). I will use a virtual environment here, so the syntax is simply:

cd /home/user/Downloads # path where the zip file was downloaded
unzip JPype-0.5.4.2.zip
cd JPype-0.5.4.2
python setup.py install

It may work out of the box, but this was not the case for me…I got a compilation error about a missing header (nji.h). In fact, the README-LINUX.txt gives you a hint for the solution. First, locate nji.h by typing this command in a shell:

locate nji.h
<pre>
 
I got this answer: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/include/jni.h telling me that my JAVA home is /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/
 
Then, edit the file setup.py, search for the function called setupLinux and update the following line 
 
<pre lang="python">
self.javaHome = '/usr/lib/jvm/java-1.5.0-sun-1.5.0.08' # Ubuntu linux
<pre>
 
with the proper JAVA path found abobe:
 
<pre lang="python">
self.javaHome = '/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/'

Try again the installation, which should work now

python setup.py install

Now, time for testing, enter a python shell and type:

>>> from jpype import *
>>> startJVM(getDefaultJVMPath(), "-ea", "-Djava.class.path=/tmp/Jpype/sample")
>>> java.lang.System.out.println("Hello World!!")
Hello World!!
>>> shutdownJVM()

Note that you must set your JAVA_HOME environment variable before to reflect what you changed in the setup.py. In my case, I typed:

export JAVA_HOME="/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/"
Posted in Linux, Python | Tagged , , , | 4 Comments

How to get processor information (linux)

Posted on October 17, 2012 by Thomas Cokelaer

Simply type

cat /proc/cpuinfo

You can also try

cat /proc/meminfo
Posted in Linux | Tagged | Leave a comment

Fedora 17: post installation

Posted on October 15, 2012 by Thomas Cokelaer

Fedora 16 crashed after an update. Good time for a fresh installation of Fedora 17.

The installation process provides most of the basic tools (browser, editor…) but some favorites are missing… Here is what I did for the post installation

Packages installation with yumex

First, update your existing packages.

sudo yum update

Then install yumex that provides a user interface to yum:

sudo yum install yumex

You can install this package also to speed up the yum searches:

sudo yum install yum-fastestmirror

Firefox related

Firefox requires the flash plugin for the videos. Many web sites (e.g., Fedora FAQs) provides help on this subject. You need to install the plugin yourself. First, you need to add the repositories:

sudo rpm -ivh http://linuxdownload.adobe.com/adobe-release/adobe-release-i386-1.0-1.noarch.rpm
sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-adobe-linux

and then the plugin:

sudo yum install flash-plugin nspluginwrapper.x86_64 nspluginwrapper.i686 alsa-plugins-pulseaudio.i686 libcurl.i686

Firefox has also many addons. I use the following ones:

xmarks
firebug
adblock plus

Install VLC player

VLC is a versatile mp3/video player but it requires to add a repository to the existing yum repositories, which is done as follows:

su -c 'rpm -Uvh http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm'
sudo yum install vlc

How to install acroread

First, you need to install the repositories

sudo rpm -ivh http://linuxdownload.adobe.com/adobe-release/adobe-release-i386-1.0-1.noarch.rpm
sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-adobe-linux

and then for Fedora 17 (on x86_64 bit system, 32-bit dependencies is also installed) type:

sudo yum install nspluginwrapper.i686 AdobeReader_enu

Gnome issues in Fedora 17

where is the desktop ?

Install the following package:

sudo yum install gnome-tweak-tool

Launch gnome-tweak-tool, select Desktop. Set “Have File Manager handle the desktop” to ON.

How to maximize the windows

By default, there are no more minimize/maximize buttons. again , you can start gnome-tweak-tool.

Then, go to Shell and see the “Arrangements of buttons on the titlebar” options.

some useful software for software development and scientific purposes (or not)

Then, the entire command to install the packages I use is:

sudo yum install PyQt4 git-svn thunderbird virtualenv 
sudo yum install qhull* freeglut*  bison flex CGAL-devel sip-devel PyQt4-devel  qt4-devel
sudo yum install dvipng qscintilla* boost boost-devel boost-python R R-core 
sudo yum install readline readline-devel flex-static 
sudo yum install gimp azureus
sudo vim gvim kile patch wget dos2unix meld

Python packages

sudo easy_install ipython python-virtual scons nose sphinx rpy2 numpy matplotlib scipy

unrar

To install some tools such as unrar, you need another repository:

sudo rpm -Uvh http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm

and

sudo yum install unrar

texlive and inconsolata missing fonts

sudo rpm -i http://jnovy.fedorapeople.org/texlive/2012/packages.fc17/texlive-release.noarch.rpm
yum clean all
yum install texlive # if not already installed.
yum update
yum install tex-inconsolata, tex-inconsolata-font

References

Reference for Gnome desktop issues: link1.

Other references: smashing web

Posted in Linux | Tagged | Leave a comment

Upgrading R and rpy2

Posted on October 9, 2012 by Thomas Cokelaer

When upgrading R version, you must recompile rpy2.

Let us take a case study based on the upgrade of R2.15 to R2.15.1.

First, you need to download the source from the main page http://www.r-project.org/

Once downloaded, get the source tar ball and unzipped it.

You should have a file called R-2.15.1.tar.gz and if unzipped, a directory called R-2.15.1

cd R-2.15.1
./configure --enable-R-shlib
make
sudo make install

Second, try:

sudo easy_install rpy2

if it does not work, get the source and compile rpy2 from the source.

Posted in Computer Science | Tagged , | Leave a comment