Speeding up Matplotlib plotting times for real-time monitoring purposes

Working on a side project I had to plot a parameter read from a nano-second-range sensor, and naturally I got curious how fast I can push Matplotlib. Needless to say, Matplotlib is meant to be used for publication quality graphs, and has been never meant to be used for performance plotting.

Matplotlib refresh rate with and without Blit

Continue reading Speeding up Matplotlib plotting times for real-time monitoring purposes

Transfer dynamic many-column labeled data from MySQL to Excel-like layout - MySQL Pivot Table

Excel is a de facto tool for day-to-day matters inside the lab (unfortunately). Say you are collecting information from several sources (or better you have automated that process into a multithreaded crawler) and curating them into one beautiful excel sheet. One problem that may occur quite early is that you end up with a huge excel sheet, and as result your biologist will start going insane or ask you to link an item to something in the database. Hence, the next logical step is to transfer your information into a relational database.

An example of data source: http://www.ncbi.nlm.nih.gov/gene/2218,

enter image description here

"Official Symbol", "Locus tag" being the labels. Continue reading Transfer dynamic many-column labeled data from MySQL to Excel-like layout - MySQL Pivot Table

Outlier detection in nonlinear least-square regression

On a weekend project I was dealing with fitting a multivariate non-linear kinetic rate regression highly sensitive to outliers. Normally people select those points by hand, and my automation mind thought how good is it to do it by "machine". Generally I looked at two methods that are widely used, I) generalized ESD test (Rosner, 1983), and II) Robust regression and Outlier removal (ROUT) methods used in Prism/Graphpad software (Motulsky and Brown, 2006). So here I start with ESD, which was easier to implement, and maybe in future I try out ROUT.


The generalized (extreme Studentized deviate) ESD test is used to detect one or more outliers in a univariate data set that follows an approximately normal distribution (Rosner, 1983). It is an easy to use method which all it requires is to set a maximum number of outlier candidates and a significance threshold.

Continue reading Outlier detection in nonlinear least-square regression

A simple and dirty batch job scheduler in python

Last week I needed to feed jobs defined in an internal web interface to the computation engine. Below is the code I wrote, which looks at the MySQL table for the defined jobs, and executes a command line code. Can it be done better? let me know.

Let's import the necessary libraries first:

import sys, os, time
from time import gmtime, strftime
import string
import MySQLdb as mdb
import atexit
import threading
import subprocess
import shlex

Next, define how to connect to mysql:

# Connect to DB -----------
db_host = ""
db_db = "test"
db_user = "root"
db_password = "tooth"

con = mdb.connect(db_host, db_user,
        db_password, db_db);

app_path = "/Users/danialt/software/"
db_port = 3306
pr = {}

Continue reading A simple and dirty batch job scheduler in python

TikZ: A better way to draw scientific and technical diagrams

In preparation of my last paper I needed to draw a detailed complex diagram. Since I use LaTeX and source control (git) I did not want to use Adobe Illustrator (which I don't have a license), Inkscape (great package, though still very limited for complex diagrams), and other common ones such as Corel. Needless to say that all their formats are binary and not source-control friendly. After a quick search I found TikZ. It had a wow effect on me right away. In TikZ, you can use the full power of LaTeX (as in typesetting), and in addition enhance your diagrams with the power of programming. You can create this kind of diagrams in a minute: Tikz fun diagram Continue reading TikZ: A better way to draw scientific and technical diagrams

Mechanics and Substrate Transport of Moving Biofilm Structures -- thesis available online


If there is a beauty of nature more fascinating than the vast variety of her species, it is the immense ability of them to adapt to the least likely livable environments. In this work, we look at the ways biofilms adapt to the harsh conditions of living in fast water flows by numerically studying the biophysical consequences of their special form and possible related function. As a special case, we look at the biofilm streamers, which are clusters of microbial aggregates connected to a tail elongated from the cluster in the direction of the flow. Streamers have a seemingly similar shape to streamlined bodies, a configuration intended to reduce the fluid drag force.

Experimentally it is also observed that the streamers oscillate (flap) in the flow, which suggests that the form may provide higher mixing around the biofilm structure. The question that naturally arises is whether the streamer form is an adaptation mechanism providing a function or a passive formation? Obviously, every familiar form or behavior of an organism, e.g., the streamlined form of sperms, does not enforce an adaptation mechanism in that particular organism, e.g., drag reduction. Therefore, we take these hints and numerically construct a model of a single biofilm streamer to weigh the contribution of the fluid-induced oscillations and the special form on its physical and biological performance.

In the context of this work, a state-of-the-art two-dimensional fluid-structure interaction model of biofilm streamers, coupled with mass transfer of a dissolved substrate is developed. This model numerically calculates the transient deformation of the streamer with simultaneous substrate transport and uptake using moving mesh finite element method.

thesis cover

We show that the streamlined form of the biofilm streamers reduces the fluid forces acting on the structure significantly. In addition, the periodic deformation (oscillation) of the flexible body increases the substrate transport into the biofilm compared to the static (immobile) case. Overall, we propose that the special morphology of the streamers, regardless of the formation process, is a successful strategy in reducing the fluid forces biofilms experience, and increases their overall biological fitness by providing relatively higher substrate transport especially in the tail section.

Download it here

On the cover of Biotechnology and Bioengineering journal

Yes, the titles says it already. Look for B&B Volume 105-6! It is really a good feeling when your work pays off and is appreciated -- I mean on the visuals of course, why would else they select it for the cover? :-).

enter image description here * Cover Legend Visualization of the simulated oscillatory movement of biofilm streamers induced by flow. Left: Time snapshots of the moving mesh; Right: Water velocity field. See related article by Taherzadeh et al., Biotechnology and Bioengineering, Volume 105, Number 3 (February 15, 2010), pages 600–610. (Image courtesy of Danial Taherzadeh)

Biofilm streamers and the drag of biological flexible structures in flow

In nature, biological systems such as biofilms exhibit a streamlined form exposed to harsh changing environments. Also, these structures have an interesting structure in common. They have a base attached to the substratum and a tail that flaps in the flow. These two characteristics: streamlined shape and flapping tail behavior have been the focus of my recent study (Taherzadeh et al., 2009).