Talk at Novo Nordisk STAR Symposium 2011: "Inferential protein structure determination using chemical shifts"

Slides from the talk:
http://dl.dropbox.com/u/17435887/STAR_symposium_2011_Anders_Christensen.pdf

MC simulation of Protein G (2OED) folding. Crystalk structure in cyan, MC simulation in green:
http://dl.dropbox.com/u/17435887/2OED_fold.avi
http://www.youtube.com/watch?v=jQVtEFYAuWE&feature=feedu

Energies used were Profasi force field energy and CamShift 1.35 MD energy.
I used PyMol to generate .png files from the MC samples and mencode to merge the .png files into an .avi file.

PDB V3 to V2 file converter and CamShift 1.35 notes

I stubmled upon a program called CamShift 1.35, which predicts chemical shifts from a coordinates in a PDB file. However, everytime I tried to supply it with a seemingly  standard PDB file, adhereing to every PDB naming convention I could find, CamShift 1.35 would complain about missing atoms and used an internal force-field to add hydrogens and whatnot at coordinates that were not correspoding to the coordinates I had supplied.

A little trick I found, was to uncomment the lines 278-279 in the camshift-1.35/bin/camshift.cpp file:

//  for(int r=0;r<rep.size();r++)
//    cout<<"\t"<<p.atom_name(rep[r],Almost::Protein::BASE)<<"\n";

 and the recompile. This will add the names of all atoms that are added. Please note, that protonation states are adjusted entirely by CamShift, so even if you give it a proper PDB file you will have things added, such as H-gamma on cysteine residues, even despite of actual cys-bridges in the supplied PDB file.

At any rate, by doing the above, I discovered that the CamShift 1.35 code uses the old PDB v2 naming convention (which was deprecated during the last millennium). Sadly I couldn't find any file converter that would let convert a standard (currently PDB v3) PDB file to an older v2 type. Not even my all-time favourite file-converter OpenBabel was capable of this. Long story short, I found the proper naming conversions and put it in a short python script.

Usage is like this:

$ python pdb_v3_to_v2.py myfile.pdb

This will print the new file to standard out put. If you want it in a new file, simply use the awesomeness of shell:

$ python pdb_v3_to_v2.py myfile.pdb > outfile.pdb

Since it's written in standard Python, it will work on any platform (Windows, UNIX, Linux, MacOS, etc) and you need not worry about compiling, just a working Python interpreter is enough.


Have fun converting. Shoot me a message, if you find any bugs or quirks. Download the program here:

http://dl.dropbox.com/u/17435887/pdb_v3_to_v2.py

(click link or picture to download)

YouTube Lecture: Novel Enzymes, Rapid Structure Determination, and an Online Computer Game

This is a very interesting video about the general field of computational protein structure prediction - the field in which I am involved. It's a talk by Professor David Baker, Dept. of Biochemistry, University of Washington - probably better known as the guy behind stuff like ROSETTA and FoldIt.

 

Installing MOPAC2009 on Ubuntu 10.04 LTS - Lucid Lynx (And keep source codes open!)

If you try and install MOPAC2009 on a standard Ubuntu 10.04 LTS system, you will get a very weird problem. I unzipped MOPAC2009 in /opt/mopac as described in the installation guide. However, when I tried to run the executable I got a very odd message:

-bash: ./MOPAC2009.exe: No such file or directory

How odd is that? I could see the executable with ls and I even tabbed my way to the full filename. Unrelated to the actual error message, it turns out, that the MOPAC2009.exe executable is compiled as 32-bit, and not 64-bit as you'd expect in 2011, and it is actually here the problem lies - not a missing file! First, let me give the remedy, then my rant. You need to install a set of 32-bit compatible libraries that will let 32-bit code run. Since we're running Ubuntu, this is of course easy as pie!

sudo apt-get install ia32-libs

But very odd, that I get a complaint saying "No such file or directory"! Really! I was going nuts and had started blaming the NFS file system. A big thanks to my favorite web-site ubuntuforums.org, where I found other possible causes for missing files, other than actual missing files.

However, the take home message of my post here really is: Start distributing source codes! I'm sure it's possible for users to compile Mopac (written in Fortran 90/95), if a decent Makefile is included. This would be the only drawback in distributing source codes. Another thing is, that binaries makes it impossible to optimize code at compiletime and link to optimal BLAS routines, etc.

 But these are not the big issues here. The things that really matter are: (1) You will never discover bugs from binaries and (2) you will never know what exact setting your algorithms are using from binaries. What if you needed to slightly tweak a parameter somewhere? What is you get unexpected results, and you know the input is good? You should of course turn to the source code! I have myself found bugs/quirks in both Dalton and GAMESS.

I had a Dalton calculation that kept crashing, and found a coefficient which was mis-typed. I quickly contacted the authors and things were solved SAME DAY, and the fix lives on in the new DALTON2011!

In GAMESS (and this is an even better story!) I discovered how to have semi-empirical methods run in parallel after snooping around in the code for a couple of days, looking for the Fock-matrix diagonalization routines. Turns out, that the only thing that prevents parallel semi-empirical methods is a flag in the code - remove that and voila!

Unfortunately cheerful stories as these will never happen to programs such as MOPAC. And I'm willing to bet money, that Mr. MOPAC would really prefer to receive a bug report along with a code patch, rather than just the bug reports.

MOPAC2009 is even free for academic use, and Jimmy Stewart (AKA Mr. MOPAC) has even been helpful in getting us started with a GAMESS implementation of his newest PM6 method. Thanks Jimmy!

UPDATE: Another thing concerning installation of MOPAC2009. The installation guide recommends using 777 permission for the installation. This is dangerous, and means that EVERYONE has write permissions to that folder. Stick with 755, which only gives the owner write permission - otherwise that file is an easy target for a hacker. 777 - bad, unless really necessary !

Debian Clusters for Education and Research: The Missing Manual

This is the recommendation of a great site I just found. I wish I had found it earlier! I have just been setting up a research/education cluster at Center for Molecular Computational Sciences at KU, with everything from NIS to NFS and Torque and even Sun Ray terminals.

This great site guides you through everything you need, in order to have a full, working cluster - only using free software. The author calls it "The Missing Manual", and I couldn't agree more!

LINK:

Debian Clusters for Education and Research: The Missing Manual

My Talk at the COMS Seminar August 18, 2011.

Here are the slides and videos I showed at my recent talk at a COMS seminar. COMS is the virtual Center for Comutational Molecular Sciences at KU. I'm using youtube to share the videos. The videos were created using the excellent PyMOL MovieSchool. Remember to have FreeMOL installed when you wish to save the movie as an MPEG video file. I'm using the public folder in my DropBox to share the slides. Couldn't be much easier ..

... or could it be any easier? In my next presentation I will include a link to this blog using a QR code on the last slide like this (created using http://qrcode.kaywa.com/ - for free):

 Towards protein structures that agree
with spectroscopic data

Chemical shifts assisted protein structure refinement

Download the slides from my talk here:

http://dl.dropbox.com/u/17435887/2011_August_18.pdf

Protein G MC simulations - OPLS/AA with NMR restraints

Top: OPLS/AA + CamShift 1.35
Bottom: OPLS/AA + H(N) Chemical Shifts

Compiling GAMESS with CUDA (GPU support)

As I mentioned in a previous post, much of the mathematics involved in quantum, chemistry can be formulated to be massively parallelized and implementation exists so you can run most types of calculations on hundreds or thousands of cores. I've heard people from Pacific Northwest National Laboratory running parallel coupled cluster CCSD calculations on 225,000 processors at 1.3 PFlops!

Very few people have access to hundreds or thousands of individual CPU cores at home or work, but most of us have access to hundreds of cores on the GPU that drives the computer graphics. So let's use our GPUs for massively parallel quantum computing, dawg!

This post tells you how to compile the newest GAMESS (Oct 10, 2010, R1) so it runs RHF 2-electron integrals using an NVIDIA GPU.

I will follow up with a post on the theory behind the GPU parallel 2-el integral routines and a post on GAMESS/GPU benchmark. I've used a lot of the pointers in the gamess/libqc/aaa.readme.1st file. Let me know if you found this helpful.

In order to do compile GAMESS with CUDA support (using this guide), there are a few requirements:

• I am going assume 64-bit Linux, because I only tried this on that platfrom.
• You need a copy of GAMESS from at least Oct 10, 2010. I'm further more going to assume that you already have properly installed said version of GAMESS and that your installation works. I'm assuming it's installed in  ~/gamess
• You need a NVIDIA graphics card which is CUDA enable (all newer NVIDIA cards are) with a working set of drivers.
• The GNU gcc and g++ compilers. I've only been able to make this work with 4.1 and 4.3, however. These are included the repositories of all major Linux releases. Other versions may work, but these are "guarateed" to work. Guaranteed, as in they potentially may compile the code. Get these via "sudo apt-get install gcc-4.3 g++-4.3", "yum install", etc. or the ditto of your distro.
• CUDA Toolkit version 2.3. You can download it here. GAMESS does not work with the current version (which is 3.2, at the time of writing. Get the version that matches your Linux the most. I used the Ubuntu 9.10 version on a Ubuntu 10.04 LTS system and the RedHat Enterprise Linux 5.3 version on a CentOS 5.4 system - both worked just fine!
• A compiled and version of Boost 1.43. I tried some other versions, but they didn't work. 1.43 is "guaranteed" to work.
• You need an auxiliary code preprocessor named "cheetah" to do stuff to the C++ source code. I used the newest of 2.4.4.

I can't really help you with installing the NVIDIA drivers, but there are many guides floating around on the interwebs. It is likely that the one that came with your Linux works, but it may not. In that case, get a newer driver. The method to do this is highly OS dependent. Unfortunately, you can't use the open source nouveau nv driver - you need to run the real (and closed) McCoy from NVIDIA.

UPDATE: I suggest running the newest beta driver from NVIDIA, since it includes some CUDA specific improvements:

wget http://us.download.nvidia.com/XFree86/Linux-x86_64/270.18/NVIDIA-Linux-x86_64-270.18.run

Step 1) Installing and modifying Boost 1.43 from the source. First make a temporary directory and get Boost:

mkdir ~/temp
cd temp
wget http://downloads.sourceforge.net/project/boost/boost/1.43.0/boost_1_43_0.tar.bz2
tar xjf boost_1_43_0.tar.bz2
cd boost_1_43_0/

Now, compile and install Boost. I always use ~/programs/ for my installation. Boost uses it's own configure and make scripts, and takes about 5-10 minutes to compile on my machine.

./bootstrap.sh --prefix=/home/andersx/programs/boost_1_43_0
./bjam
./bjam install 

nvcc (the CUDA compiler) requires three minor modifications of the Boost header files, in order to compile code linked to Boost.

Step 1.1) Change /home/andersx/programs/boost_1_43_0/include/boost/mpl/aux_/integral_wrapper.hpp line 59 from:

#if BOOST_WORKAROUND(__EDG_VERSION__, <= 243)

to

#if BOOST_WORKAROUND(__EDG_VERSION__, <= 243) || defined(__CUDACC__)

Step 1.2) Change /home/andersx/programs/boost_1_43_0/include/boost/mpl/size_t_fwd.hpp around line 22, so that one original line is in an 'else' clause:

#if defined(__CUDACC__)
   typedef std::size_t std_size_t;
   template< std_size_t N > struct size_t;
#else
   template< std::size_t N > struct size_t;
#endif

Step 1.3) Change /home/andersx/programs/boost_1_43_0/include/boost/mpl/size_t.hpp  around line 23, so that four original lines are in an else clause:

#if defined(__CUDACC__)
  #define AUX_WRAPPER_VALUE_TYPE std_size_t  
  #define AUX_WRAPPER_NAME size_t    
  #define AUX_WRAPPER_PARAMS(N) std_size_t N 
#else  
  //typedef std::size_t std_size_t;    
  #define AUX_WRAPPER_VALUE_TYPE std::size_t    
  #define AUX_WRAPPER_NAME size_t    
  #define AUX_WRAPPER_PARAMS(N) std::size_t N 
#endif

Step 2) Download and install Cheetah. This is pretty much straight forward:

cd ~/temp

wget http://pypi.python.org/packages/source/C/Cheetah/Cheetah-2.4.4.tar.gz

tar xvf Cheetah-2.4.4.tar.gz

cd Cheetah-2.4.4/
python setup.py install

Step 3) Get the CUDA Toolkit v. 2.3:

You can follow this link and get the version which matches your Linux the closest. I used the Ubuntu 9.10 version on a Ubuntu 10.04.2.

cd ~/temp 

wget http://developer.download.nvidia.com/compute/cuda/2_3/toolkit/cudatoolkit_2.3_linux_64_ubuntu9.04.run

sh ./cudatoolkit_2.3_linux_64_ubuntu9.04.run

This installs the nvcc compiler. I used the default path which is /usr/local - you may of course put nvcc wherever you want. This can be useful, if you want the newest nvcc for another program and have different versions installed simultaneously.

You then need to add the following lines to your ~/.bashrc file in order to always be able to find the CUDA compiler and libraries:

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

then write

source ~/.bashrc

or log out and back in to set the environment variables properly.

Step 4) You are now ready to compile the GPU Fock routines (called libqc). These are located in the libqc directory in the GAMESS directory. You may need to add a --mtune=native option. I found it needed with gcc-4.3, but using the option didn't work with gcc-4.1, YMMV. "native" tells your compiler to optimize the code for the local machine at compilation time.

 

cd ~/gamess/libqc

if you use gcc-4.3 and g++-4.3:

CC=gcc-4.3 CXX=g++-4.3 CXXFLAGS='-O2 -DNDEBUG -msse3 -ffast-math -ftree-vectorize -mtune=native' ./configure --with-gamess --with-integer8 --with-boost=/home/andersx/programs/boost_1_43_0 --prefix=/home/andersx/gamess/libqc

if you use gcc-4.1 and g++-4.1:

CC=gcc-4.1 CXX=g++-4.1 CXXFLAGS='-O2 -DNDEBUG -msse3 -ffast-math -ftree-vectorize' ./configure --with-gamess --with-integer8 --with-boost=/home/andersx/programs/boost_1_43_0 --prefix=/home/andersx/gamess/libqc

If the configure script ran without problems, you're now ready to compile libqc:

make

make install

This took my PC around 20 minutes. I thought it had crashed, but there was one file which just took 15 minutes to compile. Last thing to do in this step is making libqc visible to your compiler and GAMESS. Add the following line to ~/.bashrc and source it as in step 3.

export LD_LIBRARY_PATH=/home/andersx/gamess/libqc:$LD_LIBRARY_PATH

Step 5) Linking GAMESS to libqc:

In the ~/gamess/comp file, you need to set this flag to true:

GPUCODE=true 

Now, recompile the rhfuhf GAMESS routine with libqc:

./comp rhfuhf

Finally, in the GAMESS linking script (~/gamess/lked) you need to link to libqc properly. Set the following flag to true:

GPUCODE=true

Then change the library pathnames at the second place the string 'GPUCODE' appears. The three paths must point to your libqc, CUDA, and Boost libraries.

Finally, you need to link a new executable. Type the following:

./lked gamess gpu 

If linking was successful, there is now an executable named gamess.gpu.x.

Congratulations: You're now able to do 2-el integrals and Fock-matrix formation GPU parallel.