Category: bioruby

Translating a nucleotide sequence in six frames with bioruby

Bioruby offers a very easy and simple way to translate nucleotide sequences.

seq= Bio::Sequence::NA.new("acctatagctctagcta")
seq.translate

We know that there are six posible reading frames for any given nucleotide sequence. Generally the longests Open reading frame is taken to be the correct frame, when we do not have information about the possible protein that is encoded by a given gene. By default the translate method performs translation in the first frame but it can take an argument that defines the translation frame

seq.translate(2) #translate using the second reading frame.

Given a long list of sequences how do we quickly determine the correct reading frame. We would want to have method to translate a given  sequence in all frames and pick the longest reading frame. Assuming that the correct reading frame has no stop codons, we can write a quick method to perform  the six frame translation.

 def longest_reading_frame(sequence)
  orfs = [] #a container for orfs(open reading frames)
  #translate a sequence in all 6 frames
   6.times do |frame|
   translated = Bio::Sequence::NA.new(sequence).translate(frame + 1)
   stop_codons = translated.scan(/\*/).size
    orfs << translated if stop_codons == 0
   end
  orfs[0]
end

This method uses an array to collect all translated sequences that contain no stop codons and returns the first sequence in the array. This might not scale very well for very long sequences but that will be a post for another day!

Happy Biology!

Advertisements

Converting sequence data from csv to fasta format

Many  times I find someone storing sequence data in excel Workbooks.(insert scream here) This is usually followed by a request which goes like this,

Someone: ” I will send you some sequences and then we can perform xyz analysis please?”

Me: “Are they in fasta format?”

Someone: “No, they are in Excel ”

Me: (supressing a laugh) “Ok, do you mind to convert them to Fasta and then we can do xyz?”

Someone:(with a wiggle on the face)  “How do I do that?, Is there a windows  program to do that?”

Me: (feeling superman-nish) “eeh we can create a quick script in perl or Ruby, I prefer Ruby … but you should lean some basic perl or Ruby…. and run away from windows. :)”

Me: “Save your data as CSV(File ->Save As-> csv),  then send me that file”

So here is a very simple script that reads a csv file and creates a fasta file using Ruby.

You need to specify the path to the input csv file and the output fasta file, the column number that contains the name of the sequence and the column number that contains the sequence data in the csv file.

require 'csv'
# read a csv file and create a fasta file
def csv_to_fasta(csv_file,output_file,name_col,seq_col)
  File.open(output_file,'w') do |file|
  count = 0
  CSV.foreach(csv_file) do |row|
   sequence_id = row[name_col]
   seq = row[seq_col]

  count = count+1
  puts sequence_id
  file.puts ">#{sequence_id} \n#{seq}"
 end
 puts "#{count} sequences processed"
end
csv_file    = "#{ENV['HOME']}/path_to_csv_file.csv"
fasta_file  = "#{ENV['HOME']}/path_to_fasta_file.fasta"

seq_name_col = 0 #assumes the first column contains the names
seq_data_col = 1 #second column contains the seq data

csv_to_fasta(csv_file,fasta_file,seq_name_col,seq_data_col)

Happy biology!

My first Bioruby plugin calculates the isoelectric point of a protein

Late last year,  there was a lot of talk about creating a plugin system for Bioruby. The idea is that more people can start to develop bioinformatics libraries using the Ruby language and the libraries can leverage on the bioruby framework. Bioruby maintainers can then concentrate on yet to be defined “core” parts of the library to ensure compatibility and support for the plugins.Together with Pascal Bentz we have created a library to calculate the Isoelectric point of a protein given a Pka set and an  amino acid sequence of a peptide/protein. The project lay domant for a while at github until now! I am happy to release my first bioruby plugin, bio-isoelectric point! Download it at rubygems.org Fork it and check the usage at github

Examples

require 'bio'
require 'bio-isoelectric_point'
protein_seq = Bio::Sequence::AA.new("KKGFTCGELA")

#what is the protein charge at ph 14?
charge = protein_seq.charge_at(14) #=>-2.999795857467562

#calculate the ph using dtaselect pka set and round off to 3 decimal places
isoelectric_point = protein_seq.isoelectric_point(‘dtaselect’, 3) #=>8.219

# calculate the isoelectric point pH with a custom set
custom_pka_set = { “N_TERMINUS” => 8.1,
“K” => 10.1,
“R” => 12.1,
“H” => 6.4,
“C_TERMINUS” => 3.15,
“D” => 4.34,
“E” => 4.33,
“C” => 8.33,
“Y” => 9.5
}
iep_ph = protein_seq.isoelectric_point(custom_pka_set, 3) #=> 8.193

This gem supports the following Pka sets, as well as allowing a user to provide a custom Pka set.

    * dta_select
    * emboss
    * rodwell
    * wikipedia
    * sillero

Happy biology!


A ruby class for screen-scraping plasmodb database

Plasmodb is the primary resource for retrieving Plasmodium falciparum genomic data and information. Unfortunately this database has no API or XML service to request or query its  information from a programmer’s point of view or for easy automation of sequence information retrieval.  Recently I needed to download a long list of Plasmodium falciparum genomic, Protein and other information for a set of genes. Been lazy to click and open the webpage for each gene in my list. I wrote this in ruby.

It would be great if Plasmodb  would provide an easy way  of automated sequence retrieval. A webservice or an XML output format would do. Screen scraping is not a very efficient approach.  Here we use Scrapi which  is an HTML scraping toolkit for Ruby. It uses CSS selectors to write easy, maintainable scraping rules to select, extract and store data from HTML content.

#A class to fetch information from plasmodb using the scrapi API
##TODO handle  Scraper::Reader::HTTPUnspecifiedError
class Plasmodb
   #retrives a information  using the gene_id
   #returns a structure obj
  def fetch_by_gene_id(var_name)
    begin
      scraper = Scraper.define do
        process "div#genomicSequence pre",    :genomic_sequence  => :text
        process "div#transcriptSequence pre", :mrna_sequence =>:text
        process "div#proteinSequence pre",    :protein_sequence  =>:text
        process "div#Aliases td>table",       :aliases =>:text
        result :protein_sequence,:aliases,:mrna_sequence,:genomic_sequence
        end

     search_link="http://plasmodb.org/plasmo/showRecord.do?
               name=GeneRecordClasses.GeneRecordClass&source_id="+var_name+"&project_id=PlasmoDB"
     uri = URI.parse(search_link)
     @query = scraper.scrape(uri)

    rescue Scraper::Reader::HTTPUnspecifiedError
      "None"
    end
  end
  #returns the predicted protein sequence
  def protein_sequence
    @query.protein_sequence.chomp
  end
#  Returns the genomic sequence
  def genomic_sequence
    @query.genomic_sequence.chomp
  end
  #returns Aliases
  def aliases
    @query.aliases
  end
  #returns the mrna sequence
  def mrna_sequence
    @query.mrna_sequence.chomp
  end
end

#Use the class to fetch information.
require 'rubygems'
require 'bio'
require 'scrapi'

file = "/home/george/genes_list.txt" #a file containing a list of accession numbers.
#one accession number per line

plasmo = Plasmodb.new #initialize a plasmodb class instance

#Read the file and process each accession number.
File.readlines(file).each do |line|
  line.chomp!
  plasmo.fetch_by_gene_id(line)  #fetches the information from Plasmodb.
  #print a fasta entry for the protein sequence
  puts Bio::Sequence.new(plasmo.protein_sequence).output(:fasta,:header=>line)
  puts Bio::Sequence.new(plasmo.genomic_sequence).output(:fasta,:header=>line)
end

#another example
#p = Plasmodb.new
#p.fetch_by_gene_id("PFD0020c")
#puts p.genomic_sequence

Bio-graphics, BioSQL and Rails part 2

In  part 1 of this series we created a rails application and connected it to a BioSQL database. We also overwrote the rails convections to accommodate our legacy schema.

To understand the BioSQL schema, please review the documentation here. A brief overview of is as follows. Every record we enter into our database is a ‘bioentry’ and goes to the bioenty table. A bioentry can be composed of the following entities: the record’s public name, public accession and version, its description and an identifier field.

The actual sequence data is stored in the biosequence table which contains raw sequence information associated with a bioentry, and alphabet information (‘protein’, ‘dna’, ‘rna’). This is because not all records in our database need to be associated with a raw sequence. Additional sequence information is stored in the seqfeature table together with other qualifiers.

The location of each seqfeature (or sub-seqfeature) is defined by a location entity, describing the stop and start coordinates and strand. This information is stored in the location table.

In our rails application we are going to create some models and a few controllers. In RESTful language, we are actually creating resources. In this example we will be very simplistic and just create a biodatabase, taxon, bioentry, biosequence, seqfeature, location resources. We will also create associations between them in their model classes. But before that delete the index.html file from your rails application public folder and add the following line to your configurations/routes.rb file

 map.root :controller => "biosequences"

To quickly create the models, controllers, associated views and a test suite for each of our resources, just run the rails generate scaffold command, passing the name of the model as an argument. For example,

generate scaffold Bioentry

will create a bioentry model, a bioentries_controller, associated views (index,show,edit and new), a migration file, though in our case we do not need it. When you finish scaffolding, the routes.rb file should have the following resources declared.

  map.resources :seqfeatures
  map.resources :locations
  map.resources :bioentries
  map.resources :biosequences
  map.resources :taxon
  map.resources :biodatabases

Let us create some mandatory associations for the models.

Edit the /models/biodatabase.rb file by adding the following

 has_many :bioentries #a biodatabase is associated with many bioentries
 validates_uniqueness_of :name  #The name foe each biodatabase is unique!

Edit the /models/bioentry.rb file by adding the following

    belongs_to :biodatabase
    belongs_to :taxon
    has_one :biosequence

Edit the /models/taxon.rb and add

   has_one :bioentry

Edit the /models/biosequence.rb file by adding:

  set_primary_key :bioentry_id #biosequence uses bioentry_id as a primary key!
  belongs_to :bioentry

edit the /models/location.rb file by adding:

 belongs_to :seqfeature

Edit the /models/seqfeature.rb file by adding:

  belongs_to :bioentry
  has_many :locations

Note that most likely you will be adding huge files to the database. BioSQL comes with a set of  perl scripts to enable you do that. Until bioruby 1.3 is released you will have to use the perl scripts to add huge datasets. All the documentation to do that is available from the BioSQL website. I used a perl script load_ncbi_taxonomy.pl to load taxon data to my database. This script comes with the BioSQL. (It did not seem to work on my system, I will sort that later)

To make this post shorter and get to the meat of it, i will assume that you have some existing data in your biosql database. If not, create some dummy data to populate, the biodatabase, bioentry,biosequence, seqfeature and location tables. In Part 3, I will show you how to create the necessary views to populate the database. After all biologists don’t want to interact with raw SQL queries and sometimes have no idea of running scripts, however they are very web savy!

Edit the /biosequences/show.html.erb to look as follows:

<h2><%= @biosequence.bioentry.name%>(<%= @biosequence.alphabet %>)</h2>
<p>Sequence</p>
<%= @biosequence.seq %><br/>


<%= link_to 'Edit', edit_biosequence_path(@biosequence) %> 

Now navigate to http://localhost:3000/biosequences/1

and then navigate to http://locahost:3000/biosequences/1.xml The XML version of your sequence is also available!

Lets add some ability to render graphics for the sequences.

Add the following lines at the top of the biosequence.rb model file

 require 'stringio'
 require 'base64' 

In the biosequence.rb model class, create a new method called draw_graphic.

def self.draw_graphic(value)
      #get the name and length of the main feature to be drawn
     main_feature = Bioentry.find(value)
     len = main_feature.biosequence.length.to_i
     name = main_feature.name

    #create a Biographics panel and add a track
      @my_panel = Bio::Graphics::Panel.new(len,:width=> 900)
      @track = @my_panel.add_track("#{name}",:glyph=>'directed_generic')

     #specify the range for the main feature
     main_feature_range = "1..#{len}"
      @track.add_feature(Bio::Feature.new("#{name}",main_feature_range), :label=>" ")

    #write the output to memory
        output = StringIO.new
        @my_panel.draw(output)
        return output.string
  end

This method will be called by an action method in biosequence_controller.rb file.

  def to_image
    begin
      image = Biosequence.draw_graphic(Biosequence.find(params[:id]))
      send_data(image, :filename => "graphic.svg", :disposition => "inline")
    rescue  ActiveRecord::RecordNotFound
      add_error("Error:Attempt to call image without specifying a biosequence  ID")
      redirect_to :action=>'index'
    end
  end

We add a rescue block to capture record not found errors. In RESTful applications a controller is limited to seven actions. So we need to add a collection to our biosequence resource in routes.rb. This is how we do it.

  map.resources :biosequences,:collection=>{:to_image=>:get}

Now we need to modify our /biosequences/show.html.erb file, to enable rendering of the graphic. For that we will create a helper method so that our show.html.erb view is ‘clean’. In helpers/biosequences_helper.rb file, add the following code

  def render_image(feature_obj)
     image_tag(url_for({:action=>'to_image',:id=>feature_obj}))
  end

And in the /views/biosequences/show.html.erb file add the following line of code

<%= render_image(@biosequence) %><br/>

Now assuming  that you have a biosql database with valid data, navigate to

http://localhost:3000/biosequences/show/1

screenshort

screenshort

The above is a screen shot from my example application while I was writing this tutorial.

The source code for this example  application is available from github

For a full review of the methods available for biographics please check the project’s git repository and the rdoc.

Bio-graphics, BioSQL and Rails part 1

In these series I will show you how to quickly add graphics support to a bioinformatics database rails application. We are going to use the biographics library by Jan Aerts, the BioSQL database schema, and rails 2.2.2 (also works with 2.3.2)  In this simple example we want to represent a sequence as a graphic, such that we can view it in a web browser more or less the way Gbrowse works. Each main feature has different subfeatures at different locations along it.

——————————————————————- main feature

——- ——– ———— ——– ——    subfeatures

We need to have the following installed, rails 2.1.1, bio 2.1, biographics 1.4 all available as gems and a database based on BioSQL schema.

We need to download the BioSQL schema located here. The latest version as of this writing is BioSQL v1.0 (code-named Tokyo) release, v1.0.1. Create a database called biosql_development. I am on Ubuntu Linux with Mysql 5.0.

george:>mysql -u george -p
:enter password
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 27
Server version: 5.0.51a-3ubuntu5.4 (Ubuntu)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> create database biosql_development;
Query OK, 1 row affected (0.02 sec)

mysql> 

I have created a database called biosql_development. Why am i not using migrations? The reason is that BioSQL has some agreed standards on table names and schema convection which are not compatible with rails database creation and table naming conventions. However Rails allows us to override these default convections, when working with legacy databases, as will be our case.

After creating the database, load the BioSQL schema to the empty database. First we need to tell mysql which database to use.

mysql> use biosql_development;

then load the schema

mysql> source /home/george/Desktop/downloadsfolder/biosql-1.0.1/sql/biosqldb-mysql.sql;
Query OK, 0 rows affected, 1 warning (0.48 sec)

Query OK, 0 rows affected (0.15 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected, 1 warning (0.01 sec)

Query OK, 0 rows affected (0.03 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected, 1 warning (0.01 sec)

 ........ trucated
mysql>

Now we need to create a rails application and connect to this database.

I use the Netbeans IDE development environment for creating ruby and rails applications. Go ahead and create a rails application and specify to use mysql as the database adapter.

To connect to our legacy database, we need to override some convections. First disable table plurulization, and tell rails that the table primary name is named as tablename_id as opposed to just the id column expected by rails. To do that

Create a new file in your application configurations/initializers directory called override_rails.rb (you can call it whatever).

 class ActiveRecord::Base
  self.pluralize_table_names = false

  self.primary_key_prefix_type = :table_name_with_underscore
 end

The two lines above tells ActiveRecord not to expect the table names to be plural and that the primary key for each table is named as tablename_id format.

Also create another one called external_libraries.rb in the initializers directory, as you can tell this is where I want to put my require statements for loading external libraries.

require 'rubygems'

#load the bioinformatics library
require 'bio'

#load the biographics library
require 'bio-graphics'

#load the sql views extension library
gem 'rails_sql_views'
require 'rails_sql_views'

This file loads our gems. The rails_sql_views gem allows us to create views and access them by creating models corresponding to the views.

At this point if you run rake db:schema:dump, we will have a rails based BioSQL schema and which we can conveniently use to create a BioSQL database on any Relational database that rails supports and this includes Microsoft SQL server, DB2, Oracle, SQLlite and a host of others. All that would be required is to change the database.yml file to suit the adapter of choice and then execute rake db:schema:load to load the BioSQL schema.

Please note that if your are using rails 2.2.2,  you may want to comment the lines

unless Kernel.respond_to?(:gem)
  Kernel.send :alias_method, :gem, :require_gem
end

in rails_sql_views(0.6.1), otherwise running db:schema:dump will cause rake to abort.

In the next part I will describe how to create the necessary resources for our RESTful(Representational State Transfer) bioinformatics web application and rendering of the graphics.


Approximate string matching metrics with amatch

Most often in sequence analysis we want to compare how  similar two sequences are. How can we quantify similarity by using a metric? That was my question yesterday and I went hunting for a ruby implementation for such metrics. Luckily I got a library called amatch which is an approximate string matching extension for ruby! amatch implements the following metrics:

Hamming distance, Levenshtein edit distance,longest subsequence common to two strings,longest substring common to two strings,sellers distance and pair distance which is based on the number of adjacent character pairs, that are contained in two  strings.

Hamming distance

This is the number of characters that are different between two strings. This is not recommended for the majority of string based information retrieval. Very similar strings can sometimes be given high hamming distances.

Leveshtein edit distance

Is defined as the minimal costs involved in transforming one string into another by using  deletion, insertion and substitution of a character to one of the strings. The algorithm can associate a cost for performing each of the operations and for this metric it is usually 1.

Longest common substring

This is define as the contiguous chain of characters that exists in both strings. The longer the substring the better the match between the two strings. The problem with this approach is that if a difference was introduced in the middle of one string, the distance will be longer that if the same difference was introduced at the beginning of one of the strings.

Longest common Subsequence

The longer the common sub sequence is, the more similar the two strings will be. In this case a sub sequence does not have to be contiguous.

Look at the documentation for more explanations of the metrics and algorithms.

To use the library you need to first install the gem. I installed it on my Linux box running Ubuntu and ruby 1.8.6.

sudo gem install amatch

Then in script,

require 'rubygems'

require 'amatch'
include Amatch
require  'bio'
#with bioruby it would be easy to compare two sequence entries  for example
seq_obj1 = Bio::Sequence.auto("actagatatttgat")
seq_obj2 = Bio::Sequence.auto("gccagatagttaat")

#calculate the hamming distance
 m = Hamming.new(seq_obj1.to_seq)
 m.match(seq_obj2.to_seq)
#=> 

#calculate pair-distances between the two sequences
pair_distance_obj = PairDistance.new(seq_obj1.seq)
pair_distance_obj.match(seq_obj2.seq)
 #=>
# note that you can just substitute the strings directly to the metric object creation method
without creating the sequence objects!

Note that amatch  failed to install on windows XP with the following error

Building native extensions.  This could take a while…
ERROR:  Error installing amatch:
ERROR: Failed to build gem native extension.

C:/ruby-1.8.6/ruby/bin/ruby.exe extconf.rb install amatch
creating Makefile

nmake
‘nmake’ is not recognized as an internal or external command,
operable program or batch file.

Although i have nmake installed on my windows machine. I will look at that later.

Happy string matching!