Exception Errno::ENOENT in PhusionPassenger

As Passenger fails so rarely there's not much documentation on the error messages, so I'm posting this for the benefit of googlers everwhere... Scroll to the bottom for the Cliffs notes.

Passenger is a fantastic piece of software, I can't praise it highly enough. It's brought Ruby deployments from the complexity of reverse proxying to a mongrel cluster up to being on par of ease with PHP, a difficult and noteworthy achievement that can only increase the adoption of Rack based technologies in the future.

That said, in very rare cases the Passenger spawner blows up in spectacular ways that the old Mongrel deployments never even dreamed of. When this happens all of your current and future Rack instances running on Passenger will respond to every request with a 500 error, your only course of action being a full Apache restart, and since it's so rare there's usually nothing that can be found on the net to fix it.

Here's an example from one of my company's Sinatra services:

*** Exception Errno::ENOENT in PhusionPassenger::Rack::ApplicationSpawner (No such file or directory - /tmp/passenger.19250/backends/backend.Hoa3NPf9xWIsBy3D4sBZtTKE1DgjhTKjbSpdwSPWW6FakeTwddzh2hwteMo6qYecQC
zgV) (process 8942):
      from /opt/ruby-enterprise-1.8.6-20090610/lib/ruby/gems/1.8/gems/passenger-2.2.4/lib/phusion_passenger/abstract_request_handler.rb:279:in `initialize'
      from /opt/ruby-enterprise-1.8.6-20090610/lib/ruby/gems/1.8/gems/passenger-
...snip...
[ pid=725 file=ext/apache2/Hooks.cpp:688 time=2009-11-24 06:31:02.381 ]:
Unexpected error in mod_passenger: Cannot spawn application '/opt/number_service': The spawn server has exited unexpectedly.
Backtrace:
   in 'virtual boost::shared_ptr Passenger::ApplicationPoolServer::Client::get(const Passenger::PoolOptions&)' (ApplicationPoolServer.h:471)
   in 'int Hooks::handleRequest(request_rec*)' (Hooks.cpp:485)

The root cause here is actually this ticket - the /tmp/passenger.****/ folder will be almost empty because some temp watcher cleaned it out in a quiet period. The ticket has been closed but unless you feel like building from source you'll have to wait until the next release to fix the issue.

Our fix was to move the Passenger temp directory out of /tmp in the Apache httpd.conf to stop temp watchers from nuking our deployments. Instructions on setting this up are found in the fantastic Passenger documentation.