Big Data solutions in genomics

Are big data solutions for genomics all about increasing processing speed / handling it's size? Perhaps the most immediate application of big data solutions (or in most of these cases, Hadoop) is optimization or distributed computing. Examples of these are:

• Hadoop-BAM library for the scalable manipulation of aligned next-generation sequencing data in the Hadoop distributed computing framework
• CloudDOE a user-friendly tool for deploying Hadoop clouds and analyzing high-throughput sequencing data with MapReduce
• CloudBurst parallel read mapping 
• DistMap short read mapping on a Hadoop cluster
• SeqPig scalable scripting for large sequencing data sets in Hadoop
• Crossbow  implementation of exisiting tools (Bowtie, SoapSNP) to run on a parallel cloud cluster
• SeqAlto read alignment and resequencing
• Myrna Cloud-scale differential gene expression for RNA-seq

These are great examples of processing NGS data quicker and over a cluster/cloud/parallel system.

Can it do more? I would really like to see (and also develop myself if I can!) genomics solutions using these principles for:

• Statistical analysis
• Network inference
• Associations
• Metagenomics

Perhaps it is a good time to also talk about the application of non-relational databases (NoSQL) solutions. Genomic data is sparsely distributed (at least to me, in terms of storing metadata as well). For example, if you look at chromatin-bound entities (ChIP-sequencing), not all have the same bound sites. Similarly, a gene expression experiment will store data differently than a DNA mutation one. They do however share similar data, and more often than not, we have to combine them.

 I leave you with the videos of the Big Data in Biomedicine Conference 2014, which talks a lot on what kind of big data is being generated, what kind of computer technology is available to provide solutions, and a little bit at the end on the statistics and machine learning (best part!).

Data Analysis in R : PCA

Good article on principle component analysis in R, how to do feature selection, etc. "Big data" is a hot topic at the moment, and it's relevance in bioinformatics cannot be ignored.

http://www.r-bloggers.com/introduction-to-feature-selection-for-bioinformaticians-using-r-correlation-matrix-filters-pca-backward-selection/

Installing Shiny server

Adding column entries to DataFrame objects

The DataFrame class comes from the IRanges package and is used by several other packages (eg. Rsamtools) to read in bam files.

Simply cbind-ing a column to the DataFrame object will change it to the more conventional data.frame class.

So, instead of 
cbind(object1,newcolumn)

do:
cbind(object1,DataFrame(newcolumn))

to ensure that the result is still in the DataFrame class.

Fixing svn in RStudio (Mac OS)

After updating Rstudio and R to version 3.0.3, I lost the "svn" option under version control.

Rstudio started with these messages (a clue to fixing the problem!)

During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C"
2: Setting LC_COLLATE failed, using "C"
3: Setting LC_TIME failed, using "C"
4: Setting LC_MESSAGES failed, using "C"

The Internationalization of the R.app was causing this problem and a simple

system("defaults write org.R-project.R force.LANG en_US.UTF-8")

on the R command line and restarting Rstudio was all that was needed.

R Shiny + qPCR calculator

The problem with R (and many other platform that involves some level of coding/commands for execution) for biologists is : how do I run/use a package? How do we, as developers, "package" them for mass consumption? Several years ago, I started with creating GUI interfaces to provide a point and click platform for my scripts in R, but this also requires the user to install R, have the relevant libraries installed.

Shiny takes care of that, putting your scripts on the web, and most importantly, frees the user from having to install anything.

As an exercise on building apps, I've created a simple qPCR calculator and plot generator.


More good things to come.

R package writing tips

S4 union classes
To create a class that accepts different class objects:

setClassUnion("AorB", c("classA", "classB")) 

Saving data files in R packages
They need to be compressed to avoid warnings when running R CMD check

save(a,b,c,file="savefile.rda",compress='xz') 

Make R package zip (for Windows) on Linux 
make destination folder eg. testand write to test

mkdir test 
R CMD INSTALL -l test pkgname 
cd test 
zip -r pkgname pkgname


R package documentation writing
If you are generating Sweave files from Lyx (Rnw), add

%\VignetteIndexEntry{Your-Title}

to the top of your Rnw file