Fork me on GitHub
11 Nov 2009
Ruby C Extension Wrapup

This is a wrapup of my C extension series of blog posts.

I'm going to show the key points again on the example of my project Joker.

This is inteded for all the people who don't wanna hear the story, but rather want a brief explanation of how the code works.

The Toolchain

...consists of Jeweler, rake-compiler and rake-tester.

The Directory Structure

Here's an excerpt from a tree of Joker's project directory as it's currently on my system:

|-- Rakefile
|-- ext
|   `-- joker_native
|       |-- Joker.c
|       |-- Joker.h
|       |-- Wildcard.c
|       |-- Wildcard.h
|       |-- compile.c
|       |-- compile.h
|       |-- extconf.rb
|       |-- match.c
|       `-- match.h
|-- lib
|   `-- joker.rb
`-- test
    |-- c
    |   |-- test_compile.c
    |   `-- test_match.c
    `-- ruby
        `-- test_joker.rb

7 directories, 24 files


The extconf file is pretty straight forward. It only configures the directory of the extension and tells mkmf to create a Makefile for it:

require 'mkmf'

extension_name = 'joker_native'

The Rakefile

First we must configure Jeweler, so we can obtain the gemspec:

require 'jeweler'
jeweler_tasks = do |gem|                = 'joker'
    gem.summary             = 'Joker is a simple wildcard implementation that works much like Regexps'
    gem.description         = gem.summary               = ''
    gem.homepage            = ''
    gem.authors             = ['Fabian Streitel']
    gem.rubyforge_project   = 'k-gems'
    gem.extensions          = FileList['ext/**/extconf.rb']

    gem.files.include('lib/joker_native.*') # add native stuff

$gemspec         = jeweler_tasks.gemspec
$gemspec.version = jeweler_tasks.jeweler.version

Then, we can setup rake-compiler and rake-tester:

require 'rake/extensiontask'
require 'rake/extensiontesttask''joker_native', $gemspec) do |ext|
    ext.cross_compile   = true
    ext.cross_platform  = 'x86-mswin32'
    ext.test_files      = FileList['test/c/*']

CLEAN.include 'lib/**/*.so'

And include some workarounds for nasty problems (they might be solved some time in the future).

# Workaround for rake-compiler, which YAML-dump-loads the
# gemspec, which leads to errors since Procs can't be loaded
Rake::Task.tasks.each do |task_name|
    case task_name.to_s
    when /^native/

task :fix_rake_compiler_gemspec_dump do
    %w{files extra_rdoc_files test_files}.each do |accessor|
        $gemspec.send(accessor).instance_eval { @exclude_procs = }

And finally we can define some nice shortcuts ofr the compilation process:

desc("Build linux and windows specific gems")
task :gems do
    sh "rake clean build:native"
    sh "rake clean build:cross"
    sh "rake clean build"

task "build:native" => [:no_extconf, :native, :build] do
    file = "pkg/joker-#{`cat VERSION`.chomp}.gem"
    mv file, "#{file.ext}-i686-linux.gem"

task "build:cross" => [:no_extconf, :cross, :native, :build] do
    file = "pkg/joker-#{`cat VERSION`.chomp}.gem"
    mv file, "#{file.ext}-x86-mingw32.gem"

task :no_extconf do
    $gemspec.extensions = []

The Workflow

Now, you can write your C code, compile it with

rake compile

write some C tests and start them with

rake test:c

or run valgrind on one of the test executables with

rake test:valgrind:joker_native[test_compile]

or gdb it with

rake test:gdb:joker_native[test_compile]

If you're done with writing and testing you can build the pre-compiled stuff for your system and the cross compilation target platform with

rake native build
rake cross native build

and build the three package types (not compiled, pre-compiled and cross-compiled) and package them as separate gems all with as much as

rake gems