Python header

This post is about what to put on top of your Python script. You may have seen Python script starting with something like:

#!/usr/local/bin/python
# coding: latin-1

and wonder what it is useful for or do you need it.

The first line make your script possibly executable by telling your system where to find the python executable. Under linux, if you changed the permission of the file to executable (chmod 755 script.py) they you can simply type the name of the script (without python in front of it).

The second line is related to the characters that will be found ib the script.By default the encoding is utf-8 but if you want to use another one you must specify it.

The encoding must be specified on the first or second line of the script and using one of these syntax:

#!/usr/local/bin/python
# coding: latin-1
#!/usr/local/bin/python
# -*- coding: latin-1 -*-

Reference: pep-0263

Posted in Python | Tagged | Leave a comment

R language: object oriented programming

I’m used to object oriented programming in languages such as C++ and Python and I should say that the R syntax is not the most intuitive I’ve seen. Besides, it is quite difficult to find a good documentation about it. I’m not going to write another exhaustive attempt describing OOP in R but since I had to figure out in many different places how to write a simple class, let me summarize what I found.

First of all, the most relevant and well written document I found is a PDF document called “A (not So) short Introduction to S4, object oriented programming in R, by Christophe Genolini”.

In R, there are 2 OOP syntax called S3 and S4. Stick to S4, which is the newest one.

Then, keep in mind that when you will declare a method, you must know if it already exists in the standard library otherwise you will have first to create it in two steps; we’ll come to that in a moment.

So, let us be pragmatic and jump directly to the class design.

Definition

First, you create a new class by using the setClass funtion:

 setClass("MyClass",
     #can contains attributes of different types:
     representation(
         myMatrix="matrix",
         myList="list"),
     ## validity method (optional I think)
     validity=function(object) {
         msg <- NULL
         nrow <- nrow(myMatrix(object))
 
         if (nrow <= 1)
         {
             msg <- "'nrow must be stricly larger than 1"
        }
         if (is.null(msg)) TRUE else msg
     })

The class definition allows to add attributes and perform sanity checks.

Constructor

Then, you need a constructor. It can be done directly with the new function:

new("MyClass", myMatrix=matrix(c(1,2,3,4), nrow=2), myList=list(data=c(1,2)))

or via a user-defined constructor:

 MyClass <-function(data_matrix, data_list){
     # you can do all sort of sanity checks here or add a dispatcher,...
     new("MyClass", myMatrix=data_matrix, myList-data_list )
 }

Accessors

Now, for the accessors, you can do the following:

## accessors
myMatrix <- function(obj, ...) obj@myMatrix
myList <- function(obj, ...) obj@myList

Methods

First, the way to call a method associated to an object is:

method(obj)

which is quite different from many other languages where you would type:

obj.method()

What took me a while to understand is that there are 2 ways of defining a method depending it is has already been created or not in another class. For instance, the plot methods exists in the R language, so if you want to create your own plot method, you use setMethod function:

setMethod("plot", "MyClass", function(x, y, ... ){
    # do some plotting using the x argument that is your object
    })

The 2 first arguments are pretty obvious: the name of the method and the name of the class. Although here, the second argument is misleading. It is not the name of the class name but the list of arguments (the signature). In our case, it is an object “MyClass”. Then, the third argument must be function(x,y,…). Where does it come from ? It looks like it must be the same as the already declared method. Remember that plot already exists in the R language. If you type

args(plot)

you will get

function (x, y, ...)

This is what you must use as your third argument. Since you provided as a 2nd argument a MyClass object, x will be replaced by the object and y can just be ignored. What if you want to have several inputs ?

You can design a new method as follows by changing the signature of the second argument:

 setMethod("plot", signature(x="CNOlist", y="CNOlist"), function(x, y, ... ){
     # do some plotting with x and y objects
 })

Another simple example to overload the length method:

 setMethod("length", "CNOlist", function(x) length(x@timepoints))

Now, if you want to create a method that does not exist (e.g., “newMethod”) you must create what is called a Generic.

      setGeneric(
         name="newMethod",
         def=function(object, optional_arg1=1){standardGeneric("newMethod")}
     )

Now you can create the method itself

 setMethod("newMethod", "MyClass", 
     definition=function(object, optional_arg1=1){
         #do something with your object and the optional arg 
     }
 )

One issue is that by using the setGeneric you may replace an existing generic method with the same name. So it is recommended to check its existence using

if (isGeneric("MyMethod")==FALSE){
     setGeneric(
         name="MyClass",
         def=function(object){standardGeneric("MyClass")}
     )
 }

or to lock it :

 lockBinding("MyMethod", .GlobalEnv)

There are of course many more features and tricks when designing R objects. The documentation of the class/method is not easy either and of course if you create a package, you need to set your NAMESPACE file properly. Something like:

# MyClass class
exportClasses(MyClass)
exportMethods("plot", "myMethod", "length")
Posted in Computer Science | Tagged | Leave a comment

Configuring wireless under RaspberryPi xbmc

Here are the instructions to configure a wireless (belkin).

Most of the instructions were found in the reference [1] but I got some issues with the wicd-curses installation, for which there is a solution later in this post.

First, follow these commands:

  • Exit XBMC. (bottom button)
  • Log into the terminal by typing your username and password for the media center user.
  • Type sudo apt-get update
  • Type sudo apt-get install wicd-curses

Here, I got some errors related to dbus. Here is the relevant part of the error message:

insserv: Service dbus has to be enabled to start service wicd
insserv: exiting now!
dpkg: error processing wicd (--configure):
 subprocess installed post-installation script returned error exit status 1

I was able to fix this error by typing:

sudo update-rc.d -f dbus defaults

Then,

Type sudo reboot
After reboot exit XBMC again.
Log into the terminal by typing your username and password for the media center user.
Type sudo wicd-curses

You may get this error message after the last command:

[sudo] password for xbmc:
Can't connect to the daemon, trying to start it automatically...
Traceback (most recent call last):
  File "/usr/share/wicd/curses/wicd-curses.py", line 1022, in <module>
    setup_dbus()
  File "/usr/share/wicd/curses/wicd-curses.py", line 1014, in setup_dbus
    dbus_ifaces = dbusmanager.get_dbus_ifaces()
  File "/usr/lib/pymodules/python2.6/wicd/dbusmanager.py", line 36, in get_dbus_ifaces
    return DBUS_MANAGER.get_dbus_ifaces()
  File "/usr/lib/pymodules/python2.6/wicd/dbusmanager.py", line 62, in get_dbus_ifaces
    if not self._dbus_ifaces: connect_to_dbus()
  File "/usr/lib/pymodules/python2.6/wicd/dbusmanager.py", line 48, in connect_to_dbus
    return DBUS_MANAGER.connect_to_dbus()
  File "/usr/lib/pymodules/python2.6/wicd/dbusmanager.py", line 79, in connect_to_dbus
    proxy_obj = self._bus.get_object("org.wicd.daemon", '/org/wicd/daemon')
  File "/usr/lib/pymodules/python2.6/dbus/bus.py", line 244, in get_object
    follow_name_owner_changes=follow_name_owner_changes)
  File "/usr/lib/pymodules/python2.6/dbus/proxies.py", line 241, in __init__
    self._named_service = conn.activate_name_owner(bus_name)
  File "/usr/lib/pymodules/python2.6/dbus/bus.py", line 183, in activate_name_owner
    self.start_service_by_name(bus_name)
  File "/usr/lib/pymodules/python2.6/dbus/bus.py", line 281, in start_service_by_name
    'su', (bus_name, flags)))
  File "/usr/lib/pymodules/python2.6/dbus/connection.py", line 620, in call_blocking
    message, timeout)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.ServiceUnknown: The name org.wicd.daemon was not provided by any .service files

You can fix it by playing with the services:

sudo /etc/init.d/networking stop
sudo service wicd start
Then,  sudo /etc/init.d/wicd restart

Now try again to start the wicd-curses interface and configure the wireless network. The interface is not brilliant but you should be able to figure out how to navigate within it.

  • Use the arrow keys to highlight the wireless network you want to connect to and press the right cursor key to configure it.
  • Scroll down and check ‘Automatically connect to this network’.
  • Scroll down to the bottom and enter the wireless key. Press enter to select different protocol (WPA, WPE, …)
  • Press F10 to save. You should be back at the list of networks with your network highlighted.
  • Press shift and C to connect to the network.
  • Wait a while and check the status in the bottom left corner. Once it works, press q to quit.
  • Type sudo reboot

References [1] http://www.cenobite.eu/index.php?option=com_content&view=article&id=64:how-to-enable-wireless&catid=35:xbmc-live

Posted in raspberryPi | Tagged , | Leave a comment

raspberryPi: installing xbmc

Here is the procedure I followed to install raspbmc (reference)

Let us say that you want to install the raspbmc distribution on a new sd card. Be aware that the content of the sd card may be overwritten..

First insert the sd card into your laptop. Then, let us get the installer script (Python script provided by the web site here above). Type:

wget http://svn.stmlabs.com/svn/raspbmc/testing/installers/python/install.py

And execute the script:

python install.py

As an example, the output I got is

Raspbmc installer for Linux and Mac OS X
http://raspbmc.com
----------------------------------------
Please ensure you've inserted your SD card, and press Enter to continue.
 
Enter the 'Disk' you would like imaged, from the following list:
Disk /dev/sda: 250.1 GB, 250059350016 bytes
Disk /dev/sdb: 500.1 GB, 500107862016 bytes
Disk /dev/mapper/vg-lv_swap: 14.7 GB, 14663286784 bytes
Disk /dev/mapper/vg-lv_root: 53.7 GB, 53687091200 bytes
Disk /dev/mapper/vg-lv_home: 431.2 GB, 431208005632 bytes
Disk /dev/mmcblk0: 32.0 GB, 31993102336 bytes
 
Enter your choice here (e.g. 'mmcblk0' or 'sdd'):

Here, it depends on your system. I inserted a 32G sd card, which device is called mmcblk0. type the appropriate name and enter. The raspbmc distribution will be installed on the card.

Once done, time to go to the raspberryPi. Power it off. insert the sd card you have just prepared into your raspberryPi. Power it on and wait for the raspbmc distribution to finalise the installation. It should work out of the box. You are ready to play with bmc.

Posted in raspberryPi | Tagged , | 1 Comment

Setup a server on your raspberryPi

This is a summary of what I did to setup a server on my raspberry Pi. This is based on previous post on the same topic and this link:

http://www.instructables.com/id/Raspberry-Pi-Web-Server/

Before starting, make sure that your raspberry software are updated and let us install the VIM editor as well:

sudo apt-get update
sudo apt-get install vim

This post is not about vim but if you want to have a go, here are a few commands. First, type the character i to enter into the insert mode to be able to start editing. To save, press ESC key followed by :w. To enter the insert mode again; type i. To save and quit, type ESC followed by :wq. That’s it for the vim tutorial. Let us go back to the raspberry server.

hostname

The first step proposed in the link above is to setup a hostname for your computer. I should admit that it did not work for me but I do not mind to use the IP address. I’ll put the information for bookkeeping.

First, let us edit the dhclient.conf file

sudo -i
vim /etc/dhcp/dhclient.conf

Now look for the line “#send host-name “xxxxxx”;” (on the raspberry version I used, xxxxxx is andare.fugue.com), and remove the “#” symbol from in front of it. Now press Esc key, and type “:wq” to save the changes.

Back in the shell, type:

hostname xxxxxx  # where "xxxxxx" is what you want to call your server

Finally for this step, type

ifdown eth0
ifup eth0

Now to reboot the system type

halt

Once you have started the raspberry pi again, change the password::

sudo -i
passwd pi

Install apache

sudo apt-get install apache2 php5 libapache2-mod-php5

you may get a few warnings (could not reliably determine the server’s full qualified name for servername). You may ignore them. I got the warning but the server works.

The group may be already setup but it does not cost anything to check:

sudo groupadd www-data
sudo usermod -g www-data www-data

then open a browser and type http://localhost

To enable htaccess files you must modify the config files::

sudo vim /etc/apache2/sites-enabled/000-default"

You’ll see a line like the following:

AllowOverride None

change it to

    AllowOverride ALL

Mysql

sudo apt-get install mysql-server mysql-client php5-mysql

It may take a while. When asked for a password, enter one and confirm it.

install an FTP server

In order to upload files on your FTP server, type in

sudo chown -R pi /var/www

Then, install this tool:

sudo apt-get install vsftpd

And edit its configuration file:

sudo vim /etc/vsftpd.conf

Search through the file and change the following lines:

anonymous_enable=YES Change To anonymous_enable=NO

and uncomment these lines (remove the hash )

#local_enable=YES
#write_enable=YES

Also, add a line to the bottom of the file:

force_dot_files=YES

Now restart the FTP server with

    sudo service vsftpd restart

Now you are able to connect to your Raspberry Pi with an FTP client, using the following information:

Host: (Hostname you set up or IP address)
Username: pi
Password: (Password you set previously)
Port: 21

CGI Python script configuration

edit the file /etc/apache2/sites-enabled/000-default and adapt the section related to CGI script as follows:

 ScriptAlias /cgi-bin/ /var/www/cgi-bin/
        <Directory "/var/www/cgi-bin">
                AllowOverride None
                Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
                Order allow,deny
                Allow from all
                AddHandler cgi-script .py
        </Directory>

Create the directory and change the permission:

mkdir /var/www/cgi-bin
chmod 755 /var/www/cgi-bin

You can try this sctipt:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import cgi
 
def main():
    print "Content-type: text/html\n"
    form = cgi.FieldStorage()
    print "<h1>Test photo</h1>"
main()
Posted in raspberryPi | Tagged , , | 1 Comment

Web Services in Biology

There are quite a lot of web services in biology that provides various information about genes, pathways, ontology and so on. I found it quite difficult to look for the relevant services. There is a nice portal though, which is called biocatalogue and provides a “curated catalogue of Life Science Web Services”. See biocatalogue website for more information.

So, what is a web services anyway ? From wikipedia, “A Web service is a method of communication between two electronic devices over the World Wide Web. A Web service is a software function provided at a network address over the web or the cloud, it is a service that is “always on” as in the concept of utility computing.”

Generally, it has an interface described in XML format, more precisely, a Web Services Description Language (aka WSDL). Other systems interact with the Web service in a manner prescribed by its description using SOAP messages.

There are two major classes of Web services:

  • REST-compliant Web services, in which the primary purpose of the service is to manipulate XML representations of Web resources
  • arbitrary Web services, in which the service may expose an arbitrary set of operations
Posted in Life Science | Tagged | Leave a comment

Solving a Linear Programming problem with Python (Pulp)

Linear Programming is a type of optimisation where an objective function should be maximised given some constraints. Let us consider the following simple problem (from The GNU Linear Programming Kit, Part 1). Let us say that you want to maximize profits by selling wood soldiers (denoted x1) and wood trains (denoted x2) given that the margin is 3$ for one soldier and 2$ for one train, you want to maximize the profit. In addition, you have the following constraints per week:

  • a soldier requires 2 hours of finishing labour. a train requires 1 hour of finishing labour. You have only 100 hours of finishing labour avaiable weekly
  • a soldier requires 1 hour of carprentry labour. Same for a train. You have only 80 hours of carpentry labour available weekly
  • demand for soldiers not more than 40 per week

The constraints can be transformed into equations:

  2*x1 + x2 <= 100
  x1 + x2 <= 80
  x1 <=40

and of course, x1>=0 and x2=0 otherwise there is nothing to optimise.

In The GNU Linear Programming Kit, Part 1, the author uses glpk to solve this problem. However, I found this Python library called pulp that provides a nice interface to glpk and other libraries. I’m going to solve the problem with pulp. First, we need to install glpk. Under Fedora, you need to install glpk and glpk-utils :

sudo yum install glpk, glpk-utils

Then, the following python script using pulp solves the problem. The code is self explanatory:

from pulp import *
 
# declare your variables
x1 = LpVariable("x1", 0, 40)   # 0<= x1 <= 40
x2 = LpVariable("x2", 0, 1000) # 0<= x2 <= 1000
 
# defines the problem
prob = LpProblem("problem", LpMaximize)
 
# defines the constraints
prob += 2*x1+x2 <= 100 
prob += x1+x2 <= 80
 
 
# defines the objective function to maximize
prob += 3*x1+2*x2
 
# solve the problem
status = prob.solve(GLPK(msg=0))
LpStatus[status]
 
# print the results x1 = 20, x2 = 60
value(x1)
value(x2)

It works like a charm ! x1=20, x2=60 as expected.

Posted in Data analysis | Tagged , | 7 Comments

Raspberry pi: first steps

My first raspberry pi arrived !!
I pluged in a USB mouse and keyboard, the SD card I bought at the same time (it has already a linux system on it), a hdmi cable to the first screen I found and a wire connection. Finally, let us plugin the power cable. Some green light blinking (the system is booting) and here it is: some linux booting on the screen. It works out of the box. Literaly 5 minutes to set up.

Let try some stuff. Looks like a debian linux operating system. Python provided. Let us lick on LXTerminal in the top right corner to get a X terminal.

sudo apt-get update
sudo apt-get ipython

Let us start ipython now:

import webbrowser
webbrowser.open("http://www.raspberrypi.org")

Great. Great!!!

Posted in raspberryPi | Tagged | Leave a comment

How use R to build permutation or combination

In a previous post, I used itertools in Python to create combination and permutation using the standard module itertools. What about using R language to perform the same task.

The permutation function allows you to get permutation of N values within a list, where order matters. For instance, selecting N=2 values with [1,2,3] is done as follows:

>>> print list(itertools.permutations([1,2,3], 2))
[(1, 2), (1, 3), (2, 1), (2, 3), (3, 1), (3, 2)]

using R, you can use the combinations package (could not find anything in the standard R library but I’m sure it exists…)

If order is not important, you can use combinations:

>>> print list(itertools.combination([1,2,3], 2))
[(1, 2), (1, 3), (2, 3)]

using R, you can use:

> combn(seq(1:3), 2)
  [,1] [,2] [,3]
[1,]    1    1    2
[2,]    2    3    3

Then, you may want to build all possible arrays of N values taken from a list of possible values. For instance, you may want to build a vector of length N=3 made of 0 and 1. This can be done with the cartesian product function:

>>> print list(itertools.product([0,1], repeat=3))
[(0, 0, 0), (0, 0, 1), (0, 1, 0), (0, 1, 1), (1, 0, 0), 
(1, 0, 1), (1, 1, 0), (1, 1, 1)]

In R, you can use the expand function

>>>expand.grid(rep(list(0:1), 3)) 
Var1 Var2 Var3
1    0    0    0
2    1    0    0
3    0    1    0
4    1    1    0
5    0    0    1
6    1    0    1
7    0    1    1
8    1    1    1

In other word, expand.grid can be used to build a permutation with repetitions.

Posted in Python, R | Tagged , | Leave a comment

Fly with Python

xkcd python

References: http://xkcd.com/353/

Posted in Python | Tagged | Leave a comment