Reading space-separated key:value text file into bash array

Using pip to install Python dependencies

Recently I ran into problems of incompatibilities in dependencies, especially when running Python-based scripts (numpy & pandas, essentially the scipy suite).

The easiest way to avoid these troubles (I hope it works for you too!) is to install them using pip rather than your Linux distributions repository (eg. through Ubuntu's apt-get install).

Bioinformatics on a CentOS 7 workstation

Recently the lab upgraded to a Dell Precision Tower 7910 which comes with two Intel® Xeon(R) E5-2670 v3 CPUs. Decided to try CentOS 7 because Ubuntu 12.04 and 14.04 were giving some (yet unknown) problems. I'll list here the things I did for a basic setup to be able to do bioinformatics and statistics stuff, just in case anyone needs help.

The main difference is how packages (rpm vs. deb) are installed (yum vs. apt-get) and also perhaps what comes pre-installed with each system.

Installation
CentOS 7 comes with a graphical installer, that gives you a list of options of what you want. I went for the GNOME desktop option.

Partitioning: by default it will go with the LVM scheme using the XFS filesystem. I modified mine to ext4 (you can read more here), and the layout is like this (for 2TB HDD, sda has Windows 7/8 pre-installed).

Filesystem                Size  Used Avail Use% Mounted on
/dev/sdb4                  48G  261M   46G   1% /
devtmpfs                   63G     0   63G   0% /dev
tmpfs                      63G  290M   63G   1% /dev/shm
tmpfs                      63G   18M   63G   1% /run
tmpfs                      63G     0   63G   0% /sys/fs/cgroup
/dev/sdb3                  96G  6.6G   85G   8% /usr
/dev/sdb5                  48G   58M   46G   1% /tmp
/dev/sdb7                  20G  1.2G   17G   7% /var
/dev/sdb2                 494M   94M  401M  19% /boot
/dev/sdb1                 200M  9.6M  191M   5% /boot/efi
/dev/sdb8                 1.6T   31G  1.5T   3% /home
/dev/sda3                 1.9T  239G  1.6T  13% /run/media/diana/OS

Packages and repositories
You will need the EPEL repo for many packages

wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm
rpm -ivh epel-release-7-5.noarch.rpm

Some key packages I had to install to bring the fresh installation up to speed.

sudo yum install ntfs-3g libxml2 libxml2-devel libcurl libcurl-devel scipy

Bioinformatics/stats-related

More to come soon.

Installing the UCSC Genome Browser on the Synology NAS

This post is constantly updated as I progress through the (hopefully will culminate in working) process.

1. Read instructions here which brought me to here

2. Did the QuickStart
2a. Created databases hgcentral and hg19 via phpMyAdmin
2b. Populated directories (/volume1/@database/mysql/) via rsync commands listed

DETOUR #1! Requirements. I already have apache and mysql, but upon checking, do not have make, gcc and the rest. Off to install (if it is possible).

3. Fortunately it can be
3a. Install optware ipkg
http://www.vspecialist.co.uk/how-to-install-ipkg-on-a-synology-nas/
http://pcloadletter.co.uk/2011/08/02/serviio-0-6-on-synology-nas/
3b. Remember to add /opt/bin and /opt/sbin to your PATH
3c. Run ipkg install crosstool-native optware-devel gcc

Installation instructions

4. [Under Step1] Add options to httpd.conf

4a. I'm not sure if mysql devel packages are in Synology, so I'll skip this step first

Synology folder locations

Correct as of DSM 5.1

Apache : /usr/syno/apache/ has moved to /etc/httpd/

Migrating from Unfuddle (Subversion) to BitBucket (Git)

Tried to use the script provided by Atlassian but it failed as the default Unfuddle setup does not have the trunk/branch/tag structure.

Used svn2git instead with the following steps:

1. Create an authors.txt to map svn to git usernames (check https://github.com/nirvdrum/svn2git for instructions)

2. Run svn2git
svn2git -v http://svn.example.com/path/to/repo --username dlow --trunk / --nobranches --notags --authors authors.txt

3. Create repo on Bitbucket

4. Sync your repo
git remote add origin https://user@bitbucket.org/user/your_git_repo.git
git push -u origin --all

This should preserve all your previous commits from svn over to BitBucket (or Git in general).

If you use Eclipse as your IDE, the following video is useful for setting up EGit and going through some basics of Git.

Choosing colors

While not specifically for R, ColorBrewer gives you an excellent reference for color range for different number of bins as well. Fantastic for heatmaps.