Firebelly Design

November 05, 2009 by Brandon Weiss

A few weeks ago I started writing a long, incredibly vitriolic post about the state of web hosting. In it, I basically asserted that all hosting is crap, in that depending on what kind of hosting you choose, it is always one of these three things:

• Too unstable
• Too difficult
• Too expensive

And I lamented the fact that there existed no One Host to rule them all. There’s lots of good hosts, and even a few great hosts, but no awesome hosts.

The post started to get pretty long, chock-full of swear words, and eventually became so technical that it would have been unintelligible to all but only the geekiest of the geeks. Which is a problem because although I write about technical things, I try, when possible, to write about them as non-technically as possible so that people who aren’t developers can understand them as well.

So to that end I hacked it to pieces and started over. Here’s my explanation of the state of web hosting, the types of hosting available, and the problems with each:

Shared Hosting

Shared hosting makes up probably 99% of all the hosting in the world. If you have a personal site (and you’re not super famous) you’ve probably got it on a shared host. Shared hosting is awesome because everything is managed. The server, the stack (that’s the software on the server), everything. You don’t have to worry about security, or updating the server, or breaking something. And there’s a GUI for doing things like managing domains, users, databases, etc. The quality of the GUI varies, but I’ve found Dreamhost and Media Temple to both have awesome interfaces.

The problem with shared hosts is that they are, by definition, shared. You are sharing the resources with other sites, possibly hundreds. Someone else’s site getting slashdotted could easily make your site unreachable. And hosting a higher-traffic site of even moderate complexity is absolutely out of the question; it would just be unbearably slow.

Conclusion: Shared hosting is just too unstable.

Self-managed Hosting, with Control Panel

Self-managed hosting (with a control panel) is the next level up from shared hosting. Instead of physically sharing a server with other people, you get one of several virtualized servers. It’s kind of like how you can run Windows inside of OS X using VMWare or Parallels. It’s a server inside of a server. You’re still sharing the physical server itself, but only with other virtualized servers. This is awesome because your resources are protected. If you paid for 512 MB of RAM you get 512 MB of RAM. And no one’s hilarious new internet meme is going to take your site down. If they get slashdotted, their virtual server might crash, but the physical server will be fine, and thus your virtual server will be fine.

Most virtual private servers (VPS) come with a control panel, but they’re not the same as the (sometimes) custom ones you get on shared hosts. The most popular and common VPS control panel is Plesk, but it’s only the most popular because it doesn’t suck as much as the others, so that’s not saying much. Plesk somehow manages to make simple, everyday tasks fourteen times more complicated than they need to be. Half the time it’s faster to just use the command line, which defeats the whole point of having a control panel in the first place.

Recently Dreamhost came out with Dreamhost PS, which is a VPS, but it uses the custom control panel from their shared hosting! I’ve been testing it out, and it seems to be a nice fit for sites whose needs fall somewhere in between Shared Host and VPS, but I don’t know if it’s going to cut it for a really high-demand site.

Conclusion: Self-managed hosting with a control panel is just too difficult.

Self-managed Hosting, without Control Panel

Self-managed hosting (without a control panel) is the #1 choice for serious web developers. Often they’re referred to as slices. It’s the same thing as a VPS, but without any control panel. You do it all yourself from the command line. Some come with the stack pre-installed, but most just install the OS(Operating System) and the rest is up to you. You can install as much or as little as you need.

Developers love slices because they’re super cheap, super stable, and they have total control. Slicehost is probably the most popular of the self-managed hosts, and rightly so, because they’re really awesome. If you’re looking for totally self-managed hosting, Slicehost is it.

But here’s where I defer from most other developers; I think self-managed hosting is a complete waste of time and money. I’m a web developer, not a systems administrator. I want to write web applications, not configure and manage servers.

It’s ridiculous that I should have to spend four hours installing everything from scratch just because I need a new slice for a new client, and it’s even more ridiculous that I have to bill a client for that. And what happens when/if the client moves on? What happens if I move on? Is the next developer going to be savvy enough to use a self-managed host? There’s no standard way to configure a server, everything is always different depending on the OS and the developer. I could easily spend as much time writing down explanations of why a server is setup the way it is as I could spend setting up the server itself. And who’s paying for the maintenance and monitoring? Is the client going to want pay a few hundred dollars a month for me to make sure their slice is up-to-date and everything is running fine? Probably not. Which means over time the slice is going to have tons of unpatched security holes as new exploits are discovered. The older the slice, the more insecure it will become.

So why then, if Slicehost is such a waste of time, do so many web developers use it? Well, when choosing between a self-managed host with a control panel that is less stable but less time-consuming, or a self-managed host without a control panel, that is more stable but also more time-consuming, most developers choose the latter. They’re not necessarily wrong, but I’d say in general, clients would prefer to save money and be more secure, rather than have a slightly more stable server. But really, either way it’s a pain in the ass.

Conclusion: Self-managed hosting without a control panel is just too difficult.

Managed Hosting

Managed hosting is essentially the same as self-managed hosting, except you’re paying someone else to do all the systems administration, configuration and monitoring work that you would normally do. In some cases, this might make a lot of sense, in others, not so much. I think managed hosting is great, but it’s hard to justify the cost, especially when people have become so accustomed to spending $30 a month for hosting. They see $150 per month for the cheapest plan and they think that’s outrageous. But is it really? Time is money; depending on how much you charge, if you do even an hour of work a month on a server, you’re almost breaking even with managed hosting, and that doesn’t even factor in the higher quality of service you’re getting.

But, despite all these benefits, managed hosting is still going to cost too much for some people.

Conclusion: Managed hosting is just too expensive.

The One Host to Rule Them All

So then where is the One Host to Rule Them All? All I want is a host that will easily let me add a domain, create a database and deploy an application. Surely something like this must already exist, right? Something like Basecamp but for hosting. I want less, not more. How hard could that be?

Turns out, not that hard. I didn’t realize it until recently, but what I was envisioning was cloud hosting.

It’s kind of hard to explain what cloud hosting is or how it works, because it’s such a fundamental paradigm shift away from how we currently think about hosting, but it’s a system very similar to the power grid. You plug your appliance into the grid, it has access to as much or as little power as it needs, and you get billed on a monthly basis for however much you used. It’s just like that, but instead of an appliance you have an application, or a website.

I just came to this realization now, but cloud hosting isn’t really new; Amazon EC2 launched a beta in mid-2006 and went into official production in late 2008. So you’d think people would be all over this cloud computing thing. And when it was first announced, I was. I was really excited when Amazon EC2 launched their beta, right up until I realized how complicated it was (too complicated). It was an awesome service, it just needed a front-end; a better way of interacting with it.

Enter Heroku

A year or two ago I discovered a service called Heroku. It was in beta, and I thought I’d give it a try, but it totally confused me. It was Ruby on Rails hosting, and you could write or edit your app in your browser, or you could write it locally and then upload it. That didn’t make much sense to me, so I quickly stopped using it and then promptly forgot it even existed.

It wasn’t until Windy City Rails 2009 that I heard their name again, and I made a mental note to check out their site later. When I did, I was blown away. Not only is their site gorgeous, but after poking around a bit, it appears that they have managed to create dead-simple Ruby on Rails hosting leveraging Amazon’s EC2. A beautiful front-end to Amazon EC2’s awesome back-end.

Check this out:

$ sudo gem install heroku
$ heroku create sushi
Created http://sushi.heroku.com/
git@heroku.com:sushi.git

$ git push heroku master
-----> Heroku receiving push
-----> Rails app detected
-----> Launching..... done

Holy crap. That’s it? Three commands, in what, thirty seconds? And the best part: no setup, no configuration and no maintenance. By comparison, on Slicehost that would have taken me the better part of several hours, who knows how many terminal commands, and ongoing regular maintenance.

Conclusion: Heroku might be the One Host to rule them all.¹

¹ Well, at least for Ruby hosting.

Comments (2)

October 05, 2009 by Brandon Weiss

I come from a strictly web development background. I’m not a computer science major, nor have I ever really used any software development languages besides in the requisite C++/Java class that every developer seems to take in high school or college. Since I started on the web rather than in software, my first programming language was PHP, so I hadn’t even heard the term ‘stack trace’ until I started using Ruby on Rails.

For those not in the know, a stack trace is really just an error report. You run your application, something goes wrong and it spits out a stack trace so you can figure out what it is. The reason it’s called a ‘trace’ is because rather than just showing the line and file the error occurred in, it actually traces the path through the program that was taken to get to the error, which can be useful in debugging.

Reading a stack trace isn’t hard; it’s one of those things that once you know how seems extremely obvious, but at first is a little unclear, especially if you come from a PHP-based web development background like myself. Here’s an example:

dev:fb brandon$ script/server
=> Booting WEBrick
=> Rails 2.3.4 application starting on http://0.0.0.0:3000
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- acts_as_ferret (MissingSourceFile)
  from /usr/.../custom_require.rb:31:in `require'
  from /usr/.../dependencies.rb:156:in `require'
  from /usr/.../dependencies.rb:521:in `new_constants_in'
  from /usr/.../dependencies.rb:156:in `require'
  from /Users/brandon/Sites/fb/config/environment.rb:64
  from /usr/.../custom_require.rb:31:in `gem_original_require'
  from /usr/.../custom_require.rb:31:in `require'
  from /usr/.../dependencies.rb:156:in `require'
  from /usr/.../dependencies.rb:521:in `new_constants_in'
  from /usr/.../dependencies.rb:156:in `require'
  from /usr/.../commands/server.rb:84
  from /usr/.../custom_require.rb:31:in `gem_original_require'
  from /usr/.../custom_require.rb:31:in `require'
  from script/server:3

Woah, what the hell is all that? It looks intimidating, but it’s really not. You can ignore the ellipses in the paths; they aren’t normally there, I just cut the paths short for brevity’s sake.

So let’s start at the top. script/server is a script that starts up a very simple Ruby server called WEBrick (used for development only). You can see WEBrick starting to boot up and then a whole bunch of FAIL. The stack trace is ordered in reverse chronology, so it starts at the bottom and ends at the top. If you look at the bottom you can see the script I initially called. Then lots of stuff happens. These aren’t errors, it’s just the path that’s being taken through the program. After the line about the Rails application starting you’ll see a summary:

/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- acts_as_ferret (MissingSourceFile)

This was the most confusing part for me, because in PHP, an error generally looks something like this:

Warning: Invalid argument supplied for foreach() in /www/yourserver/html/index.php on line 42

This is fairly unambiguous. On line 42 in index.php the argument given to the foreach() is bad. But the stack trace summary isn’t as simple as that. If you look at the path, that’s not even a file in my application; it’s a file that’s part of RubyGems, the plugin system for Ruby. Obviously there’s not an error in RubyGems (well I suppose there could be, but let’s assume there isn’t). At the end of the summary, you’ll see the error itself:

no such file to load -- acts_as_ferret

A quick Google search shows that acts_as_ferret is a Ruby gem (plugin) for implementing full text search using Ferret. OK, so clearly the acts_as_ferret gem is missing. You could just sudo install gem acts_as_ferret and the next time you run script/server the error would most likely be gone. But something is amiss. Rails has this awesome dependency architecture where you can specify gems that your app requires, and if they aren’t there, Rails will nicely let you know that; it shouldn’t be throwing an error like this.

To find the location of the problem, you actually need to look for the last line (chronologically) in the stack trace that has a path referencing a file in your app. In my case there’s only one line, and it is:

from /Users/brandon/Sites/fb/config/environment.rb:64

If I go to line 64 in environment.rb I see this:

require 'acts_as_ferret'

And there’s the problem. The gem was being referenced incorrectly. In your environment.rb file the syntax for indicating a dependency would be something like:

config.gem 'acts_as_ferret'

Awesome, now when I run script/server I get:

dev:fb brandon$ script/server
=> Booting WEBrick
=> Rails 2.3.4 application starting on http://0.0.0.0:3000
Missing these required gems:
  acts_as_ferret  
  campaign_monitor  
  flickraw  

You're running:
  ruby 1.8.7.174 at /usr/local/bin/ruby
  rubygems 1.3.5 at /Users/brandon/.gem/ruby/1.8, /usr/local/lib/ruby/gems/1.8

Run `rake gems:install` to install the missing gems.

Comments (0)