Justin's Code Haus
Ramblings of a compiler engineer

AMD APP: Getting Device Assembly

Sometimes it is useful to look at the intermediate and assembly code for GPU programs.  This can lead to some interesting performance insights, especially for compiler writers.  Unfortunately, the AMD APP SDK is a bit limited on Linux, and the AMD APP KernelAnalyzer, which conveniently dumps the AMDIL and Device ISA for an OpenCL kernel, is not available on Linux.  However, digging through the AMD APP OpenCL Programming Guide, one finds an environment variable that can be used for the same purpose: GPU_DUMP_DEVICE_KERNEL.

According to the programming guide, this environment variable can take one of three values:

1 Save intermediate IL files in local directory.
2 Disassemble ISA file and save in local directory.
3 Save both the IL and ISA files in local directory.

Therefore, if you run your OpenCL program with:

$ GPU_DUMP_DEVICE_KERNEL=3 ./my-program

You will get two files in your local directory: [kernel-name]_[device-name].il and [kernel-name]_[device-name].isa, which contain AMDIL and Device ISA disassembly, respectively.

Posted Thu 09 February 2012 by Justin Holewinski in Programming (GPU, OpenCL, AMD)

LLVM 3.0: PTX Backend

NOTE: The information is this article only applies to LLVM 3.0 and 3.1. As of LLVM 3.2, the PTX back-end has been replaced with the NVPTX back-end.

With the release of LLVM 3.0, the PTX back-end is now in a fairly usable state.  It even integrates with the Clang OpenCL front-end to produce correct PTX code usable by the nVidia OpenCL run-time.  However, please note that the back-end is still experimental and there are unimplemented features.  As always, please post any questions to the llvm-dev mailing list.

In this post, I aim to give a quick overview of how to use the back-end to compile OpenCL kernels.

As an example, consider the following matrix multiplication routine written in OpenCL:

#define BLOCK_SIZE 16

void matmul(__global float* A, __global float* B, __global float* C) {

  __local float scratchA[BLOCK_SIZE][BLOCK_SIZE];
  __local float scratchB[BLOCK_SIZE][BLOCK_SIZE];

  int globalX = get_global_id(0);
  int globalY = get_global_id(1);
  int size = get_global_size(0);
  int k;
  float sum = 0.0f;
  int numBlocks = size / BLOCK_SIZE;
  int b;

  int tidX = get_local_id(0);
  int tidY = get_local_id(1);

  for(b = 0; b < numBlocks; ++b) {
    // Populate a cache for A/B
    int x;
    int y;

    x = b * BLOCK_SIZE + tidX;
    y = globalY;

    scratchA[tidY][tidX] = A[y * size + x];

    x = globalX;
    y = b * BLOCK_SIZE + tidY;

    scratchB[tidY][tidX] = B[y * size + x];


    for(k = 0; k < BLOCK_SIZE; ++k) {
      float myA;
      float myB;

      myA = scratchA[tidY][k];
      myB = scratchB[k][tidX];

      sum += myA * myB;


  C[globalY * size + globalX] = sum;

We can use the libclc library, written by Peter Collingbourne, to provide the OpenCL built-in functions for Clang.  This library will map OpenCL built-in functions to target-specific functions in the LLVM IR that the PTX back-end knows how to handle.  If $LIBCLC points to the download of libclc, then you can invoke Clang with:

clang -ccc-host-triple ptx32
  -Xclang -target-feature -Xclang +ptx23
  -Xclang -target-feature -Xclang +sm20
  -I$LIBCLC/include/generic -I$LIBCLC/include/ptx
  -include clc/clc.h -Dcl_clang_storage_class_specifiers
  -O3 matmul_kernel.cl -S -o matmul_kernel.ptx

The options can be a bit verbose at the moment, but practically all of them can be placed in a wrapper script.  Clang will compile the kernel and emit the generated PTX code to matmul_kernel.ptx.  This code can then be loaded as an OpenCL binary kernel using the nVidia OpenCL SDK, using the clCreateProgramWithBinary function.  As an added bonus, the performance is about the same as if the kernel was compiled using the nVidia OpenCL compiler!

Posted Fri 02 December 2011 by Justin Holewinski in Programming (GPU, LLVM, OpenCL)

Installing Matplotlib on OS X 10.7 with Homebrew

[edit: It looks like things have changed a bit since the release of 10.7, so your mileage may vary with this method.  This was written when 10.7 was brand new and most software was not yet updated for it.]

For those of you that do not know, Matplotlib is an excellent Python plotting library that allows you to create professional-quality plots for inclusion on web pages, Latex documents, Beamer presentations, Keynote presentations, and any other software that can import SVG, EPS, PNG, or virtually any graphic format.

However, getting matplotlib installed on Mac OS X 10.7 can be a bit tricky, especially if you are using Homebrew as your "package manager."  First off, Homebrew does not have packages for matplotlib, as well as some of its dependencies.  Additionally, the current Matplotlib release version (1.0.1 as of this post) does not compile out-of-the-box against libpng 1.5, which is included in the X11 distribution shipped with Mac OS X 10.7.

For previous versions of Mac OS X (10.6, 10.5), the usual way to install matplotlib was to install python, pkg-config, and gfortran with Homebrew, then install numpy and matplotlib through pip, ala:

$ brew install python
$ brew install gfortran
$ brew install pkg-config
$ easy_install pip
$ pip install numpy
$ pip install matplotlib

Unfortunately, as previously mentioned, all is not so easy in the world of Mac OS X 10.7, and the difficulty lies with libpng 1.5, installed with Mac OS X 10.7's version of X11. Briefly put, Matplotlib 1.0.1 is not compatible with libpng 1.5 due to a change in the API. Fortunately, the fix is already applied up-stream and will probably be a part of Matplotlib 1.0.2, or 1.1.0, or whatever the next released version is.

Until the next release, the Matplotlib sources in Git can be used. Instead of pulling the sources from the Matplotlib SourceForge site, you need to pull them from the Matplotlib GitHub site. I'm not sure if this GitHub site is "official," but is looks to be.

All that is needed is to build Matplotlib from source instead of using pip, so the installation procedure is now:

$ brew install python
$ brew install gfortran
$ brew install pkg-config
$ easy_install pip
$ pip install numpy
$ cd $HOME
$ git clone https://github.com/matplotlib/matplotlib.git
$ cd matplotlib
$ python setup.py build
$ python setup.py install

And now you're good to go! Hopefully this will become much easier with the next official release of Matplotlib.

Posted Thu 21 July 2011 by Justin Holewinski in General (Homebrew, Mac OS X, Matplotlib)

The Beauty of C++ Templates

Every so often, I'll get a random C++ question from a friend or colleague.  Most of the time the answers are trivial, at least for someone who has a history with the language.  Other questions make me stop and ponder, searching for the best "C++" way to do something.  Yesterday, the question was simple and the solution turned out to be equally simple, but getting to the solution made me stop and appreciate some of the cool things one can do with C++ templates.

The Problem

The problem was simple.  Suppose you have a C++ template class/struct that is parameterized by a single type, e.g.

template<typename T>
class my_data {
  // ...
  T element_;

The Solution

Now, the question is, "how do I write a method for this class/struct that maps the type of T to an enumeration value?"  For context, the real problem involved mapping T to an MPI data type, e.g. (float -> MPI_FLOAT), (double -> MPI_DOUBLE), etc..

The first thought for anyone familiar with containers may be to explicitly generate a map, e.g. std::map in this case, to hold all possible mappings from the C++ type (via typeid()) to the MPI type (really just an integer).  Such a solution is certainly valid and may be the best way to approach the problem in another language such as C# or Java.  After pondering the "C++" solution to the problem for a few minutes, my colleague and I came up with a fairly elegant solution involving templates.  Or, at least I found it quite elegant.

 * This struct wrappers the MPI data type value for the given C++ type.
 * Any valid MPI data type value must have a corresponding explicit template
 * instantiation below.
template<typename T>
struct mpi_type_wrapper {
  int mpi_type;

// Explicit instantiation for `float'
template <>
: mpi_type(MPI_FLOAT) {}

// Explicit instantiation for `double'
template <>
: mpi_type(MPI_DOUBLE) {}

The mpi_type_wrapper struct is a convenient way to convert an arbitrary C++ type to an equivalent MPI type.  All one has to do is declare a local variable of type mpi_type_wrapper<T> (with appropriate T) and read the value of its mpi_type field.  Of course, none of this is specific to MPI in any way.  The only requirement is that an explicit instantiation of the constructor must be provided for any C++ types that are to be converted.

Why This Solution?

This solution strikes me as elegant for two reasons.  First, it is a solution that would be difficult, if not impossible, to express in many other languages.  Second, and most interesting to me, there is no run-time overhead associated with this solution.  You can even compile this with RTTI turned off.  Any reasonable compiler automatically inlines the appropriate constructor, then constant propagation replaces any uses of the mpi_type field with the appropriate MPI_* enumeration value.  There is no memory overhead associated with explicitly keeping a map at run-time, nor any time overhead of performing a map look-up.  The final code just uses the constant value!  If you do not believe me, check out this example:

 * Some template class that needs to know the MPI_DataType value for its
 * template parameter type.
template<typename T>
struct some_type {
  void printType() {
    mpi_type_wrapper<T> wrap;

    printf("My Type: %d", wrap.mpi_type);

int main() {
  some_type<float> floatClass;
  some_type<double> doubleClass;


  return 0;

And the generated code?

  pushq %rbx
  leaq L_.str(%rip), %rbx
  movq %rbx, %rdi
  xorl %esi, %esi
  xorb %al, %al
  callq _printf
  movl $1, %esi
  movq %rbx, %rdi
  xorb %al, %al
  callq _printf
  xorl %eax, %eax
  popq %rbx


While this example is probably trivial for most experienced C++ programmers out there, including myself, I always find myself stopping and appreciating such solutions.  In this case, C++ templates provide such an elegant and efficient solution that I cannot help feeling giddy.

Posted Fri 01 April 2011 by Justin Holewinski in Programming (C++)