Run Vendored Binaries on Heroku

By Jon Saints - 12 May 2014

Summary

Often on Heroku your app might depend on a system package that is not installed in the base Heroku system. Including an external binary dependency in your Heroku app is easy. These binaries are called vendored binaries.

Here is a quick summary of how to do this:

Compile the binary on the heroku platform using a one-off dyno bash prompt heroku run /bin/bash
Copy the compiled package to your local git repo and place it in a folder .heroku/vendor

On your next push to heroku, the system path will recognize the binary dependencies in your path LD_LIBRARY_PATH.

Background

Recently I needed to install the python Shapely library on a Heroku app. Shapely has a dependency on the C Library GEOS which is not part of the base Heroku system.

Here is how I complied and vendored the GEOS binary for Heroku. You could use the same process to vendor a different binary.

Step #1: Setup your git repository and heroku app locally

I assume you have read and followed the appropriate Heroku Getting Started Guide for your language. I used Getting Started with Python on Heroku

Step #2: Compile binaries on Heroku

With your application configured, run bash on a Heroku process

heroku run /bin/bash

From the Heroku bash command prompt, download and extract the source code for the binary package you need to run on Heroku. In my case it was:

curl -O http://download.osgeo.org/geos/geos-3.4.2.tar.bz2
tar -xjvf geos-3.4.2.tar.bz2

Open the README or the INSTALL doc for your source code and follow the compile instructions with one change. Change the prefix for the install to be something easy to find in your /app folder.

cd geos-3.4.2
./configure --prefix=/app/.heroku/vendor
make
make install

When the compiling is finished you will have a working binary in the /app/scratch-space folder. In my case geos made three folders in the scratch-space folder: bin, lib, and include.

Create an zip archive of the binaries.

tar -czvf /app/geos-3.4.2-heroku.tar.gz .heroku/vendor

Step #3: Add the vendor library to your local source control

Next copy the geos-3.4.2-heroku.tar.gz to your local machine. I did this in two steps. You might have a better method. I used scp to copy the file to another server and then scp again copy the geos-3.4.2-heroku.tar.gz to my local machine.

With the geos-3.4.2-heroku.tar.gz on your local machine extract it into a folder .heroku/vendor in the root of your app. Heroku will automatically add the .heroku/vendor folder to your LD_LIBRARY_PATH.

# from my app root folder
mkdir .heroku
mkdir .heroku/vendor
tar -xzvf ~/Downloads/geos-3.4.2-heroku.tar.gz

Make sure you have the same folder structure as scratch-space in .heroku/vendor. In my case I have three folders: bin, lib, and include.

ls .heroku/vendor
bin include lib

Add the binaries to your git repo and deploy to Heroku. The binaries will be available to your app.

git add .
git commit -m "vendored binary for lib geos"
git push heroku master

Optional Step #4: Use buildpacks instead of adding binaries to local git repo

Instead of Step #3 you can also use the Multi Buildpack and the Vendored Libraries Buildpack build if you prefer not to have vendored binaries in your git repo.

You will want to modify your tar.gz file from Step #2 so that it extracts its contents to .heroku/vendor/BINARY_FILES_HERE.

Then copy your binary from Step #2 to amazon s3. Make the URL public.

Install Multi build pack support. See https://github.com/ddollar/heroku-buildpack-multi

Install Vendored Binary buildpack support and create a .vendor_urls file and add your S3 url to the tar.gz to it.

Note on Installing libgeos PHP bindings on Heroku

Use this command to find where php .so extentions are installed:

php-config --extension-dir

look for geos.so in the folder listed and include this in your zip file

tar -czvf /app/geos-3.4.2-heroku.tar.gz .heroku/vendor/ `php-config --extension-dir`/geos.so

Then add .heroku/php/etc/php/conf.d/geos.ini with

; GEOS extension
extension=geos.so

Commit all of this to your local source and push to heroku. NOTE: Everytime PHP is upgraded on heroku you will need to recompile the geos extension. The php-config --extension-dir will change with each upgrade as well.

Jon Saints