Setting up an EC2 instance is complicated. Instances don't persist when you shut them down, so to get one with just what you want installed, you have to create your own image (AMI). This involves starting up a base image with the OS you want (e.g. Ubuntu 9.10 Server), installing the software you want, then bundling it. Except that's dodgy, because creating an image while services are running can have odd side-effects. So I found a solution which involves bundling a completely basic instance without anything installed, then downloading and unbundling it and running some funky commands to simulate running on it. Then you install the bits that you want and rebundle it.
Currently building one with lighttpd, MySQL 5.1 and ruby, plus assorted gems.