conda and bioconda channel

Posted on February 23, 2016 by Thomas Cokelaer

Anaconda ( https://www.continuum.io/downloads ) provides a Python distribution with more than 400 of the most popular Python packages for science, math, engineering, and data analysis. Anaconda installs an executable called conda that will be your entry point.

Anaconda solves two problems.

First it is a virtual environment similar to virtualenv, which is used in the Python community. The advantage I found with compared to a tool such as virtualenv is that you can easily have any Python environment on your system (e.g. 2.6, 3.4 and so on) without the need to have to install the specific Python version yourself (see other posts on this subjet , , .

Second, installation of packages (e.g. numpy) to be found on the conda website will be fast and smooth since packages are pre-compiled: no compilation errors (in principle) and no time spent waiting for the compilation to be finished. This is very valuable since it also means that your software can be installed by anybody who will use anaconda (and therefore the same pre-compiled libraries as yours).

conda can download more than 400 packages but there are much more out there ! First, you can still use pip to install any package to be found on website.
Besides, conda provides a mechanism that ease the creation of parallel channels. One of them dedicated to biology is called bioconda. The bioconda channel can be added to your environment by typing

conda config --add channels r
conda config --add channels bioconda

You can then install a tool such as bwa (sequence alignment) as follows:

conda install bwa

Again the great advantages here will be that bwa will be installed in a second (time to download the package) and the executable ready to be used !

I mentionned the fact that you can use different version of Python. How does that work ? You need to create an environment. A default one is provided once Anaconda is installed but you can create as many as you want by naming them and setting the version of Python to be used. For instance, let us create an environment with python3.5:

conda create --name py35 python=3.5

You can then activate it as follows:

source activate py35

You can check that you have indeed the correct python version by starting a python shell and check the version

For more information, visit those two sites:

– https://bioconda.github.io/
– https://www.continuum.io/downloads

Posted in biology, Python | Tagged , , | 1 Comment

pypi upload failed (401)

Posted on February 6, 2016 by Thomas Cokelaer

On a new computer after a fresh installation, I tried to upload a new version of an existing package on the pypi repository. Of course, I have already a username/password and uploaded packages in the past. However, in the source directory, when typing

python setup.py sdist upload

I got the following error message:

Upload failed (401): You must be identified to edit package information

I just forgot to create a special file named .pypirc in the HOME directory with this content:

[pypi]
username:your_username
password:your_password
Posted in Python | Tagged | Leave a comment

Installing Fedora 23 in dual-boot and post-installation

Posted on February 5, 2016 by Thomas Cokelaer

I’ve just installed Fedora 23 from scratch in dualboot (with an existing windows partition). The installation is better and better and you do not need to create the partition under windows. Just downloaded the iso file from fedora website and burnt it on a DVD. Then reboot.

The installation was done on a DELL latitude E6540. To boot on the DVD, press F12 key while you still see the DELL logo. There, you should see an option to boot from the DVD first instead of the hard drive.

Then Fedora installation should now guide you intuitively. The only tricky part will be that you should create a partition, shrinking the existing partition to be able to install Fedora.

I won’t enter into more details and I encourage you to visit the fedora website would you have any question.

Here below are some of the post-installation similarly to what I did in another post related to F19 Fedora19.

One main difference is that the yum tool has an alternative named dnf.

First, update the packages:

dnf update

Firefox related

Firefox has also many addons. I use the following ones:

xmarks
firebug
adblock plus

Acroread

wget http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i486linux_enu.rpm
 
dnf install AdbeRdr9.5.5-1_i486linux_enu.rpm
 
dnf install nspluginwrapper.i686 libcanberra-gtk2.i686 adwaita-gtk2-theme.i686 PackageKit-gtk3-module.i686

Other useful tools

dnf install vlc p7zip gnome-tweak-tool

Gnome specific

The minimize/maximize windows can be added to windows. First, start gnome-tweak-tool.

Then, go to Windows. You should see a button to activate the min/max buttons.

Some packages for development

dnf install git vim meld gvim ipython python-virtualenv R-core R-devel PyQt4-devel  autoconf
readline-devel
Posted in Linux | Tagged | Leave a comment

python: certificate verified failed

Posted on January 17, 2016 by Thomas Cokelaer

I was just trying to download a file on github (raw) using wget package (or curl) under Python2.7.11 and Windows 7. I got a certificate error as follows:

IOError: [Errno socket error] [ SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

As an example, here is the code with a Python shell

import wget
wget.download("https://valid_url_link")

This is not an issue with the tool wget since I got the same message in a windows shell with curl.

After quite a lot of time on stackoverflow and other resources, I finally found a solution on a github page (https://github.com/servo/servo/issues/5917). his may work for other systems so here is the solution that worked for me:

import wget
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
wgte.download("https://valid_url_link")
Posted in Python | Tagged , | 22 Comments

Installing a perl module under linux

Posted on June 24, 2015 by Thomas Cokelaer

Sometimes you need to use perl… and quickly you need to install a missing dependencies.
you can do it manually but it can be cumbersome. Here is another way using cpanminus tools:

curl -L http://cpanmin.us | perl - --sudo App::cpanminus

And then install (in a shell) the missing file. e.g.:

cpanm Games-Go-GoPair-1.001.tar.gz
Posted in Computer Science | Tagged | Leave a comment

Starting with docker and creating an image with ipython

Posted on May 14, 2015 by Thomas Cokelaer

On my fedora box, I followed the instructions from the official web page: https://docs.docker.com/installation/fedora/

sudo yum -y install docker
sudo systemctl start docker
sudo systemctl enable docker
sudo docker run -i -t fedora /bin/bash

Once the new shell appears (bash), you can type for instance:

yum install ipython

Note that sudo does not work here (I presume that by default you have root permission).

You can now exit your docker container (exit)

Note that to start your fedora container it takes a fraction of a second ! Note also the size (250Mb) . To check that type:

sudo docker images
Posted in Computer Science, Linux, Uncategorized | Tagged | Leave a comment

git error: cannot do a partial commit during a merge.

Posted on February 11, 2015 by Thomas Cokelaer

After a pull command, I ended up with a conflict in a file, which I fixed by manually editing the file (search for the <<< signs). The merge is fine and therefore tried to commit the file again but got this error: 

fatal: cannot do a partial commit during a merge.

Using :

git commit -a my_filename

did not work, instead use

git commit -i my_filename

From the git documentation, the -i should be used to conclude a conflicted merge.

Posted in Computer Science | Tagged | 5 Comments

How to update a forked repository (github)

Posted on December 11, 2014 by Thomas Cokelaer

I followed those steps and it worked like a charm using github interface assuming there was no changes between the fork and main repository

  • Open your fork on GitHub.
  • Click on Pull Requests.
  • Click on New Pull Request.
    By default, GitHub will compare the original with your fork (there shouldn’t be anything different)
  • Click on switching the base (if no changes were made in the fork) or click Edit and switch the base manually.
    Now GitHub will compare your fork with the original, and you should see all the latest changes.
  • Click on Click to create a pull request for this comparison (set name to something like “Update from original”).
  • Click on Send pull request.
  • Scroll down and click Merge pull request and finally Confirm merge. You will be able to merge the request automatically.

reference: how-to-update-github-forked-repository

Posted in Software development, Uncategorized | Tagged , | 1 Comment

Moving from SVN to git: $Id:$ equivalent

Posted on December 11, 2014 by Thomas Cokelaer

In SVN (and CVS), you could include special strings such as

$Id:$

and after a commit, the revision number would appear after “Id:”, which was a convenient way to retrieve the revision for instance in a print statement.

Apparently, with GIT, the equivalent does not exist by default. Indeed somebody could refer to version 1.1 of a file but your
version 1.1 of that file is different and so it would not make sense to keep track of a revision.

Yet, there is way to include the hash from GIT. First, go the repository you are working one and add (if you want to add $Id:$ in a Python code):

cat .git/info/attributes 
# see man gitattributes
*.py ident

Then put $Id$ somewhere in the file (on top generally)

Then commit. Note you will need to delete the file and check it out again:

git commit foo.sh
rm foo.sh
git co foo.sh

You should now see the $Id: followed by a commit hash string:

# $Id: 143e77a7f07343b14 $

reference: http://stackoverflow.com/questions/384108/moving-from-cvs-to-git-id-equivalent

Posted in Software development, Uncategorized | Tagged | 1 Comment

migrating a SVN sub-directory to GIThub

Posted on November 12, 2014 by Thomas Cokelaer

Follow-up on a previous about migrating an entire SVN repository to github keeping the history:

For a sub-directory within a SVN, you should use

git svn clone svn+ssh://path/to/repo/ --trunk=trunk/directory --branches=branches/*/directory --tags=tags/*/directory

without the –stdlayout option.

Posted in Computer Science | Tagged , | Leave a comment