Subsetting/ extracting reads from bam files

1. Regions (make an index first, then select your regions)
samtools index bam_file 
samtools view -b file.bam chr1 > file_chr1.bam 
samtools view -b file.bam chr1:1-10000 > file_chr1_1_10000.bam 

2. Regions given in bed file
samtools view -bL regions.bed file.bam > file_regions.bam 

3. First n reads
samtools view -h file.bam | head -n 10000 | samtools view -bS - > file_n10000.bam

Book review : Programming the Semantic Web

I picked up this book wanting to learn more on NoSQL that I had to implement in developing some packages. This book serves well as a beginners guide : why we need semantics, and compares current ideas and structure of SQL databases to the more flexible and abstract forms that it is trying to promote.

For me it was written in a very logical progression and concepts were introduced in a very good pace. The book goes on to talk about RDF and OWL together with how they can be implemented (O'Reilly has another book Learning SPARQL which may be a logical progressing after this book). No doubt it is not going to make you a pro on the subject, but it was sufficient for me to understand the basics of semantic data structures.

Few things you might want to take note:

1. this book uses Python, so a basic understanding would be very helpful to understand how things are implemented
2. the website semprog.com is not available anymore, but you can get the code package at http://examples.oreilly.com/9780596153823/Programming-the-Semantic-Web.zip

Overall, a good introductory book.

PS: If you don't like to read, youtube now has a lot of channels talking about NoSQL and semantics including O'Reilly themselves, so you might want to check that out.

Version control systems

The type of version control system (Git vs SVN vs CVS vs Mercurial) used would likely not have much impact while working within a single lab environment (with one or two coders). I started thinking more about this when moving some of my more "public" code from an SVN-based Unfuddle over to GitHub. Git is more suited when working in a larger group, and then there are the concept or forking/pulling is great. The only thing stopping me from moving wholesale is the pricing option for private repos (Unfuddle is free). I also like that Bioconductor can bridge its SVN repo with Github (so I can share my code easily as well) and I also enjoy the whole user experience of the site.

As for now, I'm on both. I'm thinking of moving my private repos to BitBucket, which supports Git and Mercurial. The ability to work with a local repo (Git) while on the road is very useful.

Some further self-reading : http://stackoverflow.com/questions/871/why-is-git-better-than-subversion

Adding the optional IH tag to SAM files

One of the major complaints about the 2 most often used aligners BWA and Bowtie is its failure to report the NH or IH tag.

The IH tag is an indicator of the number of stored alignments in the SAM file that contains the current query (i.e. the read). This is meaningful for multi-mapped reads if you want to know to how many locations the same read has been mapped (eg. assuming your Bowtie parameter "k" has been set to more than 1).

I've written an awk oneliner that will add this tag to your SAM file. What it does is to iterate the file twice, first to tabulate counts, and second to write the extra tag.

Fixing svn in RStudio (Mac OS)

After updating Rstudio and R to version 3.0.3, I lost the "svn" option under version control.

Rstudio started with these messages (a clue to fixing the problem!)

During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C"
2: Setting LC_COLLATE failed, using "C"
3: Setting LC_TIME failed, using "C"
4: Setting LC_MESSAGES failed, using "C"

The Internationalization of the R.app was causing this problem and a simple

system("defaults write org.R-project.R force.LANG en_US.UTF-8")

on the R command line and restarting Rstudio was all that was needed.

Filtering FASTQ files for unique reads

Filtering for duplicate reads in fastq files may be important if your application requires considering unique entries for counting etc.

Brent Pederson wrote a very quick script utilizing Bloom filters for this purpose (read more at : http://hackmap.blogspot.sg/2010/10/bloom-filter-ing-repeated-reads.html). The installation process might not be clear for those not familiar with code, so I'll try and explain the process step-by-step here.

To run the fastq_unique.py script, you'ld need three things:

1. Perl module Bloom Faster
• either install through cpan or manual download
2. Python module nose (pybloomfaster tests)
• installation directions on the nose page
3. Brent's wrapper pybloomfaster
• download the master zip

• sudo python setup.py install

Installing python modules with setuptools or pip

Remember to set your http (and https) proxy!

Running into errors like this:

sudo pip install nose
Cannot fetch index base URL http://pypi.python.org/simple/

Or this:

sudo easy_install nose
Scanning index of all packages (this may take a while)
Reading http://pypi.python.org/simple/
Download error: [Errno -2] Name or service not known -- Some packages may not be found!

Is simply a matter of setting your http proxy because PYPI redirects to https. Check your environment by:

env | grep -i http

If it returns empty, nothing has been set. Set them using:

set http_proxy=http://localhost:8080
set https_proxy=http://localhost:8080

And it should now work.