Dan Hulton's blog

If you do web development on Windows, you’re probably used to making do. XAMPP can go a long way, as can Cygwin, but once you want to start playing around with anything state-of-the-art, you’ll start feeling left behind. The solution? Set up a virtual machine running Linux, so you can take advantage of the latest and greatest in their native environments.

Set up Ubuntu

For this purpose, we’re going to be using VirtualBox – originally by Sun, recently(ish) purchased by Oracle. For my money, it still has the best mix of ease-of-use, features, and price (free). You can download the latest version here.

And while you’re at it, go download the latest version of Ubuntu Server. For this article, we’re using 11.10 (Oneiric Ocelot), which you can download here. Grab the 64-bit version.

Install VirtualBox. Before you run it the first time, you’ll want to right-click the icon and choose “Properties”. Then go to the “Compatability” tab and click the checkbox next to “Run this program as an administrator”. If you want any symlinks you create in Ubuntu to be recognized by Windows, you MUST be running the program as an administrator. Windows is funny like that.

Launch VirtualBox, then create a new virtual machine and set the Ubuntu ISO as the CD/DVD drive. (Under the Storage menu in Settings, click on the CD-looking thing under the IDE controller). Also, make sure to set up Networking as “Bridged”. Start up the virtual machine, then follow Ubuntu’s really slick install process. The only application you’ll want to install during setup is the OpenSSH server. We’ll install the rest ourselves.

When all is said and done, restart it and log in. Now the fun begins!

Set up your networking

Let’s set up a static IP address so we don’t have to keep figuring out what dynamic IP the machine obtained.

$ sudo nano /etc/network/interfaces

Edit the last part to look like this:

# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.0.151
        netmask 255.255.255.0
        network 192.168.0.0
        broadcast 192.168.0.255
        gateway 192.168.0.1

You’ll have to fiddle with those numbers a bit depending on your networking configuration. Ensure primarily that the address you supply isn’t taken by anyone else on your network, and that the gateway matches the one you get when you type “ipconfig” in Windows. Restart your network to get the changes recognized.

$ sudo /etc/init.d/networking restart

Once that’s working, start up Notepad as Administrator in Windows (right-click and “Run as administrator”). Then open up “C:\Windows\System32\drivers\etc\hosts” (no extension) and add your new machine’s IP address. Now you can do things like SSH directly to the hostname or type the hostname in your browser instead of remembering the IP address.

Install Apache

This is easy, and can be handled entirely through automated utilities built-in to Ubuntu.

$ sudo apt-get install --reinstall language-pack-en
$ sudo dpkg-reconfigure locales
$ sudo apt-get install gcc make wget cron curl
$ sudo apt-get install apache2 apache2-mpm-prefork apache2-prefork-dev apache2-utils apache2.2-common

One quick thing, to avoid getting the error “Could not reliably determine the server’s fully qualified domain name, using 127.0.1.1 for ServerName” every time we restart, we need to make a small change.

$ sudo nano /etc/apache2/httpd.conf

Add the following line and save.

ServerName localhost

Install MySQL

Nearly as easy. Start with the automated utilities.

$ sudo apt-get install mysql-server-5.1

No need to give MySQL a root password on your development box. Then edit the MySQL config file.

$ sudo nano /etc/mysql/my.cnf

Change the “bind-address” line in that file to point the the local server IP. (192.168.0.151 in the example above.)

Then, it’s worthwhile allowing root to log in from machines other than localhost. That way you can use whatever Windows GUI tool you like to log in to your virtual machine’s MySQL installation.

$ mysql -u root
GRANT ALL ON *.* TO 'root'@'%';
exit;
$ sudo service mysql restart

Install PHP

Now we could just use the built-in version of PHP, if we felt like taking the easy road. But that’s no fun. Let’s install PHP 5.4 RC2, so we can play around with all the new features. As an added bonus, if you get comfortable with compiling PHP now, you can recompile with whatever version and options you want later for when there’s a new release of PHP, and you don’t have to wait for your package maintainers to update.

First we need a bunch of build tools.

$ sudo apt-get build-dep php5
$ sudo apt-get install libxml2 libxml2-dev libzip-dev libbz2-dev curl libcurl4-openssl-dev libcurl3 libcurl3-gnutls libjpeg62 libjpeg62-dev libpng12-0 libpng12-dev libmcrypt-dev libmcrypt4 libxslt1-dev libxml2-dev

Next, grab and extract PHP 5.4.

$ cd ~/
$ mkdir tmp
$ cd tmp
$ wget http://downloads.php.net/stas/php-5.4.0RC2.tar.gz -O php-5.4.0RC2.tar.gz
$ tar xvfz php-5.4.0RC2.tar.gz

Now, we configure and compile. This takes a while. Feel free to grab a sandwich or something.

$ cd php-5.4.0RC2
$ ./configure --with-apxs2=/usr/bin/apxs2 --with-config-file-path=/etc/php5 --with-mysql=mysqlnd --enable-inline-optimization --disable-debug --enable-bcmath --enable-calendar --enable-ctype --enable-force-cgi-redirect --enable-ftp --with-gd --enable-memory-limit --disable-sigchild --enable-trans-sid --with-ftp --with-jpeg-dir=/usr --with-png-dir=/usr --with-zlib=yes --with-zlib-dir=/usr --with-openssl --with-xsl=/usr --with-gd --with-mcrypt=/usr --with-mhash=/usr --enable-mbstring=all --with-curl=/usr/bin --with-curlwrappers --enable-mbregex --enable-zend-multibyte --with-bz2=/usr --with-mime-magic --with-iconv --with-pdo-mysql=mysqlnd --enable-fileinfo --with-pear --enable-exif
$ make
$ sudo make -i install

Of course, now we have to ensure that Apache knows about PHP.

$ sudo ln -s /usr/local/bin/php /usr/bin/php
$ sudo nano /etc/apache2/mods-available/php5.conf

Add the following lines and save.

AddType application/x-httpd-php .php .phtml .php3
AddType application/x-httpd-php-source .phps

Now enable PHP (and mod-rewrite, while we’re at it).

$ sudo a2enmod php5
$ sudo a2enmod rewrite

It wouldn’t hurt to ensure that PHP is configured how we like it, either.

$ cd /etc
$ sudo mkdir php5
$ cd php5
$ sudo cp ~/tmp/php-5.4.0RC2/php.ini-production php.ini
$ sudo nano php.ini

Ensure the following are set:

date.timezone = America/Toronto (Or whatever is closest to you.)
short_open_tag = On
error_reporting = E_ALL
display_errors = On
log_errors = On
error_log = /var/log/php.log
max_execution_time = 30
memory_limit = 128M
mysql.default_socket = /var/run/mysqld/mysqld.sock

Now make sure that the logfile we specified exists and is writable.

$ sudo touch /var/log/php.log
$ sudo chmod a+rw /var/log/php.log

Finally, restart Apache!

$ sudo service apache2 restart

You should be able to visit the address of your new virtual machine (or the alias you set up in hosts) now and get your basic Apache display.

Extra step – VirtualHosts

I presume you’re going to want to develop multiple projects without having to create separate virtual machines for them all. That’s easy! We’ll just set up an Apache VirtualHost for each one.

$ sudo nano /etc/apache2/sites-available/{development url}

Add the following, then save.

<VirtualHost *:80>
    ServerName {development url}
    DocumentRoot /home/{user}{/{development url}/

    <Directory /home/{user}/{development url}/>
        AllowOverride All
    </Directory>
</VirtualHost>

Now enable your site and reload Apache.

$ sudo a2ensite {development url}
$ sudo service apache2 reload

Extra step – Shared folders

I bet you’re wondering how you’re supposed to get files on and off that machine, huh? Well, since you installed OpenSSH, you have options there, but there’s an easier way. You can just share that folder you were referencing above with a specific folder on your host machine. Make changes in Windows, and it will (usually) be reflected in the virtual machine. I’ve noticed a few times that when I’ve created a file, the Ubuntu host flips out and doesn’t properly recognize it, but a quick restart fixes it, and machine restarts are usually like ten seconds, max.

First, make sure that folder above exists on your virtual machine, then click on the “Devices” menu in VirtualBox, then click “Install Guest Additions”. This makes a virtual CD available to your virtual machine that contains the actual guest additions we’re going to install.

$ sudo mount /dev/cdrom /media/cdrom
$ cd /media/cdrom
$ sudo ./VBoxLinuxAdditions.run

Ignore the crap about x.org failing – we don’t have any GUI installed on the server, so whatever. Now shut down the machine (“sudo shutdown -h now”), and open up the Settings menu and click on Shared Folders. Click the little Folder/Plus icon on the right, and choose a Folder Path from your Windows box, and remember the Folder Name it gives you (or make up your own). Hit “OK” and start that machine back up.

We’re going to set up this machine to automatically mount that folder every time it boots up.

$ sudo nano /etc/init.d/rc.local

Add the following just after “do_start() {“:

    mount -t vboxsf {Folder Name} /home/{user}/{development url}

Now restart to check it out.

$ sudo restart -r now
# After restarting...
$ cd /home/{user}/{development url}
$ ls

You should see all the files from your Windows machine. If you don’t, well… ugh. Okay, I got this fixed by re-running the VBoxLinuxAdditions.run script, the re-mounting. No clue why that worked, but it worked, so… yeah. Not questioning it.

What Else Could We Possibly Do?

Hot damn, that’s a lot of stuff, isn’t it? Well, we still have Memcache, Memcached (yes, both), and Redis to install. These are all optional, but super-neat tools, and practically essential if you want to write anything that scales well these days.

For now though, you should have a good base to work from. If anything doesn’t work, let me know, but I followed along with my own VM while I was writing this, so I’m pretty confident I didn’t skip anything.

I understand I’m a little late to the game, but Memcached is friggin’ awesome.

(No, I’m not talking about Memcache, I’ve known about that for a while. I’m talking about Memcached, the younger, beefier, stupidly-named cousin of Memcache. Crazy new features, but named in a way that’s sure to guarantee dozens of Google search mismatches. Nice.)

Previously, if you wanted to be sure that your cache writes weren’t clobbering each other, you had to implement some form of locking system, which introduces the very healthy chance of deadlocks and locking timeouts. This kills the concurrency.

But with Memcached, you can do this really sweet thing called “cas()” which stands for “compare and set”. (Maybe. Depending on who you’re talking to.)

Simple usage:

$cas = null;
$m = new Memcached();
do {
    // Get user, get cas token byref
    $user = $m->get('user::4099', null, $cas);

    // Do something with your user
    $m->cas($cas, 'user::4099', $user);
} while ($m->getResultCode() != Memcached::RES_SUCCESS);

So when you’re getting your $user, you’re also getting a $cas token. This token is updated every time the cache entry is changed. When you go to write your user later, instead of calling set(), you call cas() and pass in your $cas token. Memcached will only update the user if $cas is equal to the cas token in memory for that cache entry. If it fails, it sets Memcached’s internal result code to RES_DATA_EXISTS.

Now obviously you’ll want to limit that do-while loop there to prevent infinite looping, and if you really wanted to be a speed demon, you could change that “null” in there to a callback function so this function would block less, but still – it’s pretty goddamn quick, and no need to wait for locks.

There’s just one problem. What if you need to work with MULTIPLE objects?

See, there’s a getMulti() and a setMulti(), but no casMulti(). This is even weirder, because getMulti() even passes back an array of $cas tokens. Googling for it produces complaints about its lack, or worse, people confused about the difference between Memcache and Memcached. Apparently it’s not even IN the Memcached server, which sort of blows my mind. I can think of all kinds of situations where you’d want two or more objects to be updated in-step.

So anyway, since this thing doesn’t exist natively, I wrote it.

Memcached::casMulti()

/**
 * Set an array of Memcached keys with cas tokens, fail on all if any don't match.
 *
 * @param array   $items      An array of items to store.  All items require "key", "value", and "cas" entries.
 * @param integer $expiration The expiration time, defaults to 0.
 */
function casMulti(array $items, int $expiration = 0) {
    $newCasToken = null;
    $successfullySet = array();

    // First load old versions of data, ensure cas values remain sound
    foreach ($items as $item) {
        $item['old_value'] = $this->get($item['key'], null, $newCasToken);

        // If any tokens have changed since we loaded them, WHOOPS
        if ($newCasToken !== $item['cas']) {
            throw new DataChangedException("Data changed while we were messing about.");
        }
    }

    foreach ($items as $item) {
        // On successful set
        if ($this->cas($item['cas'], $item['key'], $item['value'], $expiration)) {
            // Get new cas token
            $this->get($item['key'], null, $item['cas']);

            // Keep track of our successes
            $successfullySet[] = $item;

        // Failed set?
        } else {
            // Data has been modified, crap
            if ($this->getResultCode() === Memcached::RES_DATA_EXISTS) {
                // Loop through "successful" sets
                foreach ($successfullySet as $item) {
                    // Get new cas token
                    $this->get($item['key'], null, $newCasToken);

                    // If this item hasn't changed on us
                    if ($newCasToken !== $item['cas']) {
                        // Undo our changes
                        $this->cas($newCasToken, $item['key'], $item['old_value'], $expiration);
                    }
                }

                throw new DataChangedException("Data changed while we were messing about.");

            // Some other reason?
            } else {
                // Do something appropriate
                throw new Exception("Something else messed up.");
            }
        }
    }
}

Some caveats:

• This function assumes it’s part of a Memcached library, ideally one that extends PHP’s Memcached object, and thus $this is equal to a valid Memcached instance.
• Since I don’t have access to Memcached’s Result Object (there’s no setResultObject), it relies on Exceptions to report failures, and those pretty obviously haven’t been fleshed out.
• This is the biggie: I haven’t tested it yet. Like, not at all. I’m busy trying to set up a Virtual Machine to replace the shitty XAMPP-on-Windows thing I’ve been fighting with for years, but it’s a slow process. This will of course be updated once I get a chance to test it out, but I had an itch one evening and just had to scratch it and write this code. (I’ll also be updating this blog with the process used to create said Virtual Machine, should you choose to try it out yourself.)

With that said, let’s walk through it quickly.

You’re going to be passing in an array of items to compare-and-set, and that array should look something like:

array(
    [0] => array(
        "key" => "user::4099",
        "value" => "json-user-data-goes-here",
        "cas" => "big-long-cas-token-taken-from-get-or-getMulti",
    ),
    [1] => array(
        "key" => "user::4105",
        "value" => "json-user-data-goes-here",
        "cas" => "big-long-cas-token-taken-from-get-or-getMulti",
    )
)

I could have put $expiration in each of the items too, and maybe that’s a good idea? I figured keeping as close to Memcached’s existing methods was worthwhile, and how often are you going to need to have different expirations for everything?

Next, we get the old values of everything we’re changing, just in case we need to roll back. If any values have changed since before this function was called, SHUT DOWN EVERYTHING.

Then we try to cas each key individually. If a key is successfully set, we save it for later. If they’re all successfully set, no biggie. If any fail though, we go into recovery mode.

In recovery mode, we examine every previously-set item. If it hasn’t changed, we use cas() again to change it back to the way we found it before we tried anything. If it HAS changed, we just leave it – no sense in messing with anyone else’s work.

Yeah, so…

Now like I said, this is absolutely untested. And not in an “untested under load” kind of way – I mean I haven’t even run it yet. It might not even friggin’ compile. I’ll update this blog post once I have tested it sure, but for now, exercise caution.

And even if it does compile and seems to work, exercise caution. I wrote this late at night after a day of frustratedly poking at the Memcached docs and Google. There could be all kinds of situations where this Just Doesn’t Work.

And even if it does work in all situations as expected? It may not even be a good idea. There’s a LOT of cache reads going on here, and I really don’t know what that does to concurrency. There may very well be much more efficient ways of doing this.

So be warned. Have fun playing around with this, but try not to use it in production code.