Rails integration headless browser testing: why Ajax may destroy your will to live

You're writing a few Ruby on Rails integration/features tests, using the headless browser tool of your choice - Capybara-wekit, Poltergeist, etc. Say you write a feature test that probes a route containing buggy code. Your code base will fall over, the test will fail, and the HTML readout of your page, accessed within Capybara by going

page.html

will show a huge amorphous blob of a stack trace. Annoying to debug, but it's something. So far so good.

Now imagine you've fixed whatever that first problem was, and you're merrily testing away, writing feature tests to ensure different bits of the page behave as they should. You arrive at a form, perhaps one which communicates with the server and spits back more page content. You write the relevant bits of form general-mess-about-y code using Capybara's Action methods, write a test to ensure the page content gets updated the way you think it should and run the test. Bam.

The test fails - the page content is unchanged.

Why? The server didn't spit out a stack trace. Is there something wrong with the browser response, then? You check the Javascript console messages - a tad difficult, considering you're doing headless browser testing, but with Capybara-webkit, simply check the

page.driver.console_messages

variable, which now surely is crammed with informative messages telling you what went wrong. Nothing again. How odd. Okay then, is the response executing at all? Add a

console.log("Hey, I'm alive!");

call or two to it. Test. Nothing. I went through all this and more. I added an

ap params

to the start of the Ajax controller response, and that worked just fine, so clearly the test form submission itself was working. (And by the way, you should totally check out the fantastic gem awesome_print). I spent several hours tearing my hair out, trying to figure out why the hell code would apparently silently fail like this. Finally, of course, I realised that there are two separate controller calls here, the initial browser page load and the Ajax response, and only the first shows stack traces in response to code errors. The second just glares at you, as I found to my cost. To find the problem, you could try wrapping the entire controller method in a begin/rescue/end block, just while you're troubleshooting, and printing the exception message. Crude but effective.

1 2 3 4 5 6 7

def create begin # Your incredibly insolent Rails code goes here rescue e ap e.message # Caught you end end

Making Capybara integration testing play nicely with jQuery Elastic

One of the coolest ways to test the hell out of your Rails app, or any Rack-based app, is Capybara - it opens an actual browser window, in your test environment, and automatically executes all the instructions you give it - going to whatever page you like, logging into your app, interacting with form elements, the lot, saving you the hassle of doing it manually every time you make the tiniest little change you think might cripple your app.

Elastic is a jQuery plugin. It dynamically expands textareas to hold their content. You know what it's like. You're typing in a textarea, and bam, you run out of space, scrollbars appear, and you have to scroll like mad to access the novel you're writing. Elastic changes all that. A hell of a useful plugin.

As I discovered over several agonizing hours recently, though, Capybara and Elastic sometimes do not play nicely together. See, when you type into an Elastic-ified textarea, it's not actually a textarea at all. It's a custom div made by the plugin. If you right-click it in Chrome and choose "Inspect Element", you might see something like this:

1 2 3 4 5 6

<div style="border: 1px solid rgb(161, 161, 161); display: none; font-family: sans-serif; font-size: 13px; font-weight: 400; line-height: normal; padding: 5px; position: absolute; white-space: pre-wrap; width: 470px; word-wrap: break-word;"> Your Content Here </div>

Okay, so what? you may ask. So you're typing into an autogenerated div instead of a textarea. Does this break the form the textarea's in? No. When you hit Submit, the plugin copies your expand-o-div's contents back into the original textarea, ensuring all is fine and dandy?

So what's the problem?

The problem comes to light when you're running Capybara tests to ensure that the textarea contains the content you think it does. Say you're putting together a comment submission form in Rails. The textarea HTML will look something like this:

1 2 3

<textarea id="comment_content" name="comment[content]"> Your Content Here </textarea>

Somewhere in your Javascript, you'll have a line saying

$('#comment_content').elastic()

And in your integration tests, you might have something saying

page.find('textarea[name="comment[content]"]')['value'].should eq "Hey look, I've typed this after calling elastic()"

And bam. It fails.

Failure/Error
 page.find('textarea[name="comment[content]"]')['value'].should eq
"Hey look, I've typed this after calling elastic()"

expected to find text "Hey look, I've typed this after calling elastic()"
actually found "Your Content Here.". The content <i>before</i> you 
called elastic(). I know why, and I could tell you why, but you're just a
silly human, you wouldn't understand.

If you're anything like me, you can spend several agonizing hours staring, frazzled and homicidal, at this infuriating message. Or you can be slightly smarter than I was, put two and two together, and realise that, hey, you need to trigger the form's submit() method to copy your expand-o-div's content back into the original textarea. Then you're golden.

Managing the the rvm-capistrano gem

Capistrano's a fantastic gem. Add it to your Rails app, spend only a very few short hours fiddling with the config settings in desperation, trying to get the damn thing working (github.com/capistrano/capistrano/wiki/2.x-From-The-Beginning saved me so so much time - don't worry about it mentioning Rails 2.x apps, the instructions work just fine for Rails 3.x apps too), and you can upload your development code to a production server and make it available to your baying masses of fans.

Just recently, though, I tried uploading my latest code update to PathWrangler, typed 'cap deploy' on the command line, and this got spat back at me:

/Users/[me]/.rvm/lib/rvm/capistrano.rb:5 in `<top (required)>': RVM - Capistrano integration
was extracted to a separate gem, install `gem install rvm-capistrano` and remove the
`$LOAD_PATH.unshift` line, note also the 'set :rvm_type, :user' is now the default (instead
of :system). (RuntimeError)

And following that, a lovely stack trace. Groovy.

A quick bit of Googling reveals this - Ruby Version Manager's discussion on how the integration of RVM functionality into Capistrano has now been moved into its own, separate gem, called 'rvm-capistrano'. Okay, fair enough I think to myself - I shall add that to my Gemfile, and try 'cap deploy' again.

Result:

/Users/[me]/.rvm/lib/rvm/capistrano.rb:5 in `<top (required)>': RVM - Capistrano integration
was extracted to a separate gem, install `gem install rvm-capistrano` and remove the
`$LOAD_PATH.unshift` line, note also the 'set :rvm_type, :user' is now the default (instead
of :system). (RuntimeError)

The same error again. Huh?

A bit of fiddling around reveals the problem - in my config/deploy.rb file, the hub of all Capistrano activity in a Rails app, you of course have 'require "rvm/capistrano"', to import RVM's Capistrano functionality. When the command-line error message says "remove the `$LOAD_PATH.unshift` line", it's trying to tell you that because the RVM-Capistrano stuff is now in its own gem, you no longer need to write 'require "rvm/capistrano". Delete it, and all will be well.

Getting jQuery-File-Upload to play nicely with Rails 3.x and Internet Explorer

So you're a Rails programmer. You'd like to implement Ajax-flavoured file uploads on your app. You may well have skimmed your eye over Flash-based solutions like SWFUpload or Uploadify, grappled with such delights as the Rails CSRF token problem, and quickly decided that Flash-based solutions suck huge leathery balls.

Then you discover the HTML5-based jQuery-File-Upload. Dear god. It's amazing. Not only is it HTML-based, so the browser will happily pass the session and CSRF tokens with Ajax requests, but it seems to promise everything. Asynchronous file uploads. Multiple files. Graceful failure for older browsers. A nice GUI. Even IE6 compatibility! It'll wash your car and send flowers to your spouse and egg the houses of your enemies and promise you the world. Amazing stuff.

You implement it on your site and start uploading images - perhaps working alongside the fantastic Rails image manipulation gem Paperclip - and everything's working wonderfully. Chrome, Firefox, Safari, it works beautifully. And then you try it on Internet Explorer, and the file upload form returns the following error:

SyntaxError: Invalid character

Eh? Doesn't the plugin's Github page say it should work on IE6+?

Why yes it does. See, the problem isn't Internet Explorer itself; it's that Rails needs a bit of massaging before the two will play nicely. In particular, the CSRF token rears its ugly head once again. The thing is, Chrome, Firefox, Safari - they all natively support XHR file uploads, as noted on the documentation. IE doesn't. jQuery-File-Upload works around this by invisibly swapping in an iframe-based form behind the scenes and using that instead.

Now that's just fine and dandy when you're using a vanilla web server that doesn't have CSRF protection - but the iframe form doesn't carry the Rails CSRF token in its form data, so when the file upload happens, Rails notes its absence, and duly throws a tantrum.

How to fix this? Simple. You'll need to whack the CSRF token on the end of the URL you use when uploading a photo. Here's how. If you followed jQuery-File-Upload's main.js code to initialize the plugin, the initialization call itself will look something like this:

1 2 3 4

$('#fileupload').fileupload({ dataType: 'json', acceptFileTypes: /(\.|\/)(gif|jpe?g|png|tiff)$/i });

Simply add the following code below, so now you've got this:

1 2 3 4 5 6 7 8 9 10

$('#fileupload').fileupload({ dataType: 'json', acceptFileTypes: /(\.|\/)(gif|jpe?g|png|tiff)$/i }); $('#fileupload').bind('fileuploadsend', function(event, data) { auth_token = $('meta[name="csrf-token"]').attr('content'); data.url = data.url + '?authenticity_token=' + encodeURIComponent(auth_token); $.blueimpUI.fileupload.prototype.options.send.call(this, event, data); });

On the 'file upload send' event, this function fires, whacking 'authenticity_token=[auth_token]' on the URL, and then continuing calling the jQuery-file-upload code as normal, but now with an amended URL.

That, however, is only half the problem. Once you've done this - and be sure to use Ctrl+R to refresh the page in IE9, rather than your standard F5, to clear the cache and load your new improved code - you'll notice that the request itself works and the relevant Rails controller fires, but IE insists on downloading the JSON response as a static file. This is a known issue - to get around it, you'll need to change the content type to 'text/plain' in your Rails file upload controller response code. Like so:

1 2 3 4

def create # your existing code render :json => [photo_json], :content_type => 'text/plain' end

And that's it! IE will now behave itself.