New Adventures 2012 – Self Help for Web Designers

Posted on 23/1/2012 by Ian

Last week, like many others working on the web, I took the trip to Nottingham for New Adventures 2012. I’d seen the wonderful praise dished out for it last year and felt like I wanted to see what it was all about first hand.

I’ve learnt from experience that the most important part of any conference is not the talks or speakers involved, but the community around it and therefore I headed up the night before to ensure I didn’t miss a thing.

Colly and his team had done a lovely job of organising enough to keep us busy what with bowling, screenprinting, fr00tball, the after party and of course the main event.

Despite not actually branding myself as a web designer, it was great to feel welcomed and able to chat one on one with others about our day to day work. Dan Mall’s talked helped somewhat here, forcing introductions with everyone around me. I’m pretty much work in solitary confinement as a developer and to discuss the thoughts and ideas I have with other like minded creatives is always welcome.

During a dinner following the conference, another dev commented that the most of the presentations weren’t specific to web design. They were in fact so abstract, that it was more a “Self Help Class for Web Designers”. I pretty much agree with this. But that’s with a caveat – what else do you attend these presentations for? NA Conf is there to share, inspire and enable designers to be better designers. I want to leave giddy eyed with possibilities. Remember there are workshops too for specifics…

After the whole conference, I feel like there’s a whole lot more I could do to better myself at my craft. Hopefully over the course of the year I’ll do something about it. Most of the notes I made are to this effect too and I’m sure I’ll take away something different each time I look at them.

There’s only so much that can be covered in 45 mins, and for me I want to see a speaker paint me the big picture. After NAConf, I’ve got a whole bunch of big pictures to get through.

Posted in Design, Web Apps | Tagged , , , | Leave a comment

Goodbye 1 & 1

Posted on 12/1/2012 by Ian

I just sent the following email to 1 & 1′s billing department. I’ve been with them for coming up to 7 years now and they were, when I first used them, the only registraar I knew about and I was really happy.

As the number of domains I own has grown, their account admin has become really totally useless to manage them all. I recieved a payment request out of the blue today and it tipped me over the edge.

> We contacted you regarding a declined payment or an expired credit/debit card and have not yet heard back from you.
> Our records indicate that there is an unpaid balance in your 1&1 customer account no.: K####### as listed below.

Firstly – No you didn’t.

Secondly – It would be far more useful telling me which domain name in my account a payment demand is in reference to rather than my account number, as I have more than 1 domain with you.

Thirdly – The steps you’ve listed here to find my payment details miss steps and references links that don’t exist.

Fourthly – Threatening action when you haven’t actually sent a primary payment request email is rude, forceful and obnoxious.

> Please be aware that if this matter is not resolved, you may be sent to a debt collection agency. If the account is sent to a debt collection agency a £15.00 > charge will be added to the unpaid balance.

I’ve decided to take all of my business elsewhere at the earliest opportunity. I’d hope you’d take heed of these comments and pay your other longstanding customers the respect they deserve.

Posted in Computers and Internet, General | Tagged , | 2 Comments

Pagination in CouchDB Apps

Posted on 2/11/2011 by Ian

I’ve been working on some fun little node.js / couchdb projects of late. Given the fact I don’t use either as part of my work, I’ve spent some downtime experimenting and slowly iterating my approaches as I learn best practice.

I hit what I consider to be a fairly frustrating hurdle that couchdb threw up that I’ve been blissfully unaware of through all my couchdb dev. When it came to doing pagination it turns out I’ve always been doing it the “bad” way. Oh, well that’s upsetting.

The Wrong Way

My “slow” approach has always been to take the page no as a argument in the url, generating “skip” and “limit” variables to be used as parameters to my store. So for example, if I wanted to have the 2nd page of my app showing 10 items: 

var skip = (pageno==1) ? (0) : ((pageno-1) * 10);
curl -X GET http://127.0.0.1:5984/stuff/_design/stuff/_view/by-name?skip=10&limit=10

It turns out, that although you might think you’re starting at a particular result, CouchDB still starts at the first result, due to the way the view is created from the b-tree index, couchDB just surpresses the results you skip. This isn’t good news when you’re trying to skip say, 10000 results.

The Suggested Way

The suggested solution is to perform requests and instead of using a “skip” parameter, keep track of the startkey at which the next page begins. This is possible, by requesting a page 1 item longer than that of the number of items on a page and using the key of the result in any requests. So now, for a first page my query is: 

curl -X GET http://127.0.0.1:5984/stuff/_design/stuff/_view/by-name?limit=11

Returning something like: 

{"total_rows":17,"offset":0,"rows":[
  {"id":"8177bf155b952652129836a5d354b30e","key":"Ian Wootten","value":null},
  {"id":"bae2c490c70480aec7096d79e1e3bfc3","key":"Isambard Kingdom Brunel","value":null},
  {"id":"eaae74cfbe5cd13ea6b50dfd090827ca","key":"Christopher Columbus","value":null},
  {"id":"491e68b08d73256f060ebf4b8e063e1c","key":"Elizabeth Fry","value":null},
  {"id":"b45d8a7b9edee9ca66ac0860196f4504","key":"Edward Jenner","value":null},
  {"id":"8a4d3f46885701ffcc7532aeac7a5ae9","key":"Florence Nightingale","value":null},
  {"id":"71e6534c17429eca2cd9450cfc95c6bb","key":"Samuel Pepys","value":null},
  {"id":"6cbad847f0ae959b281b471a72d60587","key":"Pocahontas","value":null},
  {"id":"e5b026ec5c92c20f1575a2901defe14e","key":"Mary Seacole","value":null},
  {"id":"84a371a7b8414237fad1b6aaf68cd16a","key":"George Stephenson","value":null},
  {"id":"321aeb36e20d62660eb0d03c9fcd27b2","key":"Joe Bloggs","value":null}
]}

From the 11th returned result, I have the key “Joe Bloggs” – which can be used as a startkey arg to couch to obtain my second page. If we have duplicate keys, it is also neccessary to keep tabs on the last document’s id and supply as a startkey_docid arg in order to correctly page through everything.

What personally I dislike about the suggested approach, is the inability to create simple requests to arbritrary pages, even with low numbers. We always need to follow a path of links from the first page in order to view particular results. CouchDB’s response is “Not even Google is doing that!”, which is kind of weak to me. I want nice clean urls ala myapp.com/page/2 or myapp.com?page=2.

In fact, such a suggested approach only really allows us to have a single “more” type link in order to fetch results. Passing a startkey as part of a url param eg /page/321aeb36e20d62660eb0d03c9fcd27b2 just sounds (and looks) plain nasty and isn’t very good from a UX point of view for any users we may have.

At the moment, clean tangible page urls (the right way) are only possible using custom middleware. I’ve yet to find anything suitable for node.js. I intend to investigate how to cache document keys for low numbered pages as a separate db in order to produce a solution for my current project and I hope to write a later post detailing how I’ve got on.

Posted in Code, Web, Web Apps | Tagged , , | Leave a comment

Hosting an Octopress Blog on Amazon S3

Posted on 9/9/2011 by Ian

Octopress is a framework for blogging based on the static site generator jekyll. In short jekyll takes markdown and turns it into blog style html ready to be served straight away with Apache whilst Octopress dresses it up nicely with HTML5, responsive layout goodness and gives you a bunch of options for code formatting and the like. A perfect little blogging setup for hackers for those of us who typically won’t need all the bells and whistles on our blog I’m sure you’ll agree.

Using Octopress, you’d typically write your posts in markdown and process them locally. You’d then have a number of HTML pages to upload to your web server, which would appear no different than any other blog in terms of functionality.

The most interesting thing that coincincides with the creation of jekyll is that it is now possible to serve static websites straight from Amazon S3, meaning that it isn’t neccessary to have a web server to serve a Octopress blog. I’ve been playing a little and have been able to serve my own octopress blog which you can see over at www.ianwootten.com. Here’s how to do it:

Configure an S3 Bucket to Act as a Website

NB: If you want to use your own domain, then you’re going to need to create an S3 bucket from the AWS Console named the same as your domain (so for me I created a bucket called “www.ianwootten.com”.

To configure your bucket as a website, from the AWS Console, select the new bucket and click “Properties”. Choose the “Website” tab from the box that appears and check “enabled” and set the Index document as “index.html”. Click “Save”. Now move to the “permissions” tab and click “Add bucket policy”. Enter a policy as below, with “www.ianwootten.com” replaced with the name of the bucket you’re using.


{
"Version": "2008-10-17",
"Id": "",
"Statement": [
{
"Sid": "PublicReadForGetBucketObjects",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::www.ianwootten.com/*"
}
]
}

If you’ve opted not to use your own domain, then that’s it for S3. Your new bucket website will be available at a combination of the bucket name and the S3 storage location. In my case, this is http://www.ianwootten.com.s3-website-us-east-1.amazonaws.com/. There’s a list of endpoints on the “website endpoints” page in Amazons S3 website docs.

Configure DNS

If you’re wanting to use your own domain, then you’ll need to visit your domain registraar and edit some DNS entries. In my case, I added a CNAME entry “www” pointing to www.ianwootten.com.s3-website-us-east-1.amazonaws.com. You’ll need to use a tool like wwwizer in order to forward from the root of your domain to the www subdomain. e.g. “ianwootten.com” to “www.ianwootten.com”. To get that working, you’ll also need to modify the @ A record and point it to 174.129.25.170. Read this serverfault post if you’re interested to know why.

For more info on CNAME configuration and S3 see Amazons S3 website docs (Specifically the notes on virtual hosting).

Download Octopress and write some Markdown

Having successfully configured Amazon S3 to host your blog, you need to tackle the age old problem of writing some content for it. This was particularly hard for me, given my lack of knowledge about markdown! You may like to take a look at John Grubers explanation of markdown if you’re suffering like me. The Octopress blogging basics gives a good overview of how your new local blogging workflow will work. Essentially new posts are written in the ./_source/posts folder and when rake generate is executed, your entire websites content is output to the ./public folder. You can also run rake preview in order to preview on a local server.

TO publish your blog, it’s merely a matter of transferring your /public content to the S3 bucket you’ve generated.

Having done all that you’ve now no need to run your own server. You’re still dependent on those that operate at S3, wwwizer and your domain registraar, but you’ve removed your own from the equation. You may be interested in checking out Jerome Bernard’s tip on easily deploying Octopress blogs with s3cmd.

Posted in Computers and Internet, Geek | Tagged , , , , , , | 5 Comments

Lovefilm Filmstream Plugin v0.2

Posted on 20/7/2011 by Ian

Just a quick note to say I’ve updated the plugin that generates my filmstream pages. It’s become obvious that the Lovefilm API only exposes the last 10 features that you’ve rented in their “at home” list. This isn’t that great if you’ve been renting more than a few months. This is the same as the information you’d see in your account when logged in.

I’ve made a simple modification to use the actual ids that Lovefilm issue for films, rather than a autoincrement value. This way, there’s no need to empty what’s currently stored when updating and we capture the users history of rentals…

You can grab it over here

Posted in Code | Tagged , , | Leave a comment

Participate, Don’t Preach

Posted on 14/7/2011 by Ian

Last week, tickets went on sale for dConstruct 2011 and sold out within an impressive 7 hours. Having attended in 2010 and 2008 and judging it to be filled with the highest calibre web celebs from across the globe, I coughed up the entrance fee as part of the excited gold-rush. The cost of £150 includes access to the pre and after parties and the conference itself.

These type of events are stuffed to the brim with amazing, intelligent web professionals, many of whom I would consider to be colleagues. I’ve followed their progress, listened to them speak, bought their publications and hacked on code with them at many other previous events.

It’s often quoted within tech circles that the best parts of these events are not the talks themselves, but the networking parties that surround them. I really struggle in these environments, nervously trying to find a crowd I feel comfortable chatting with, introducing myself and trying to strike up some cutting edge debate on the topics of the day. I’m a developer who usually lurks behind a computer screen for 8 hours a day, so why wouldn’t I find it awkward? It feels like uncharted territory every time I find myself there. Although meeting some great, interesting people, I have ultimately never forged any new lasting relations in this way, where the interaction is somewhat forced and artificial. Maybe it’s just me, but given the com mon nature of everyones work, you’d think it might be easier. As a friend once observed – it’s like everyone turns up to sell their product, but there are no buyers.

It’s also great to be able to chat within the inner circle of the celebrities who attend. You’ve heard from them all year round and now have a brief opportunity to chat. I guess really I’d love to bounce questions and thoughts off of these guys all evening, but often I get the feeling I’m being evaluated against the rest of the attendees in the room in terms of worthiness. Until you’re edging
into such circles yourself, I’m not entirely sure what they would get from these conversations either.

In fact, what I have found is that my most fruitful and long lasting relations have been founded at events when in a participatory nature. Most of these have been Hack Days (where you’re invited to quickly hack together software as an exemplar of new apis from a number of vendors) and some of them have been at Bar Camps (where all attendees have to speak on a subject close to their heart). I feel closer to these people than I could ever do with someone I spoke with briefly in a noisy bar. I’ll also remember faces and names of people even if I never had the chance to interact with them.

This year, a new variety of geek meet has arisen, which flip these more traditional events on their heads and instead focus on providing environments that focus on “the best bits”. They allow everyone to participate on level pegging rather than follow a traditional conference structure. Earlier this month there was Geek Karting, The Insites Tour is kicking off next week and later this year there’s Activate. I really like the idea of these as a way of sparking interesting relationships and I’m hopefully going to head to Insites next week to see what Elliot and Keir have been able to cook up. I’d hope to see more of these unusual types of event and ultimately participate in more of them.

I still feel that conferences have their place, otherwise I wouldn’t have coughed up the fee for dConstruct. I enjoy hearing what these illustrious speakers have to say. That part of the event is still important for sparking debate and learning from for months and years afterward. However, I just don’t find surrounding parties as valuable to me as I first would have thought.

Posted in Events | Tagged , , , | 1 Comment

Maintaining References to Sockets with Express and Socket.io

Posted on 4/7/2011 by Ian

I hit a frustrating problem when trying to use expressjs alongside socket.io recently. The solution may seem somewhat trivial, but I struggled with it for a while, until finally asking for help on the socket.io irc channel. I’m not sure if a better solution exists for my case, but this is what I have for now.

In the main route of my express app, I needed to iterate across an array, performing a REST call using each element as a parameter and spit the results back for socket.io to send out to the browser. To get some idea of what that would look like, imagine the following.

app.get('/', function(req, res){
for(var i=0; i < someArray.length; i++){
 var someVariable = doSomething(function(err, results){
   socket.emit('msg', results);
});
}
});

My problem then became, how do I emit in this way, given my use case? All examples I could find on the socket.io homepages demonstrate emitting msgs on connection, like this:

io.sockets.on('connection', function (socket) {
  // send custom events to browser socket
  socket.emit('hi', { 'i can send': 'json!' });
});

But I wanted to emit messages as part of a loop, within an app route. How would I do that? I could wrap the loop with the connection logic:

app.get('/', function(req, res){
io.sockets.on('connection', function (socket) {
for(var i=0; i < someArray.length; i++){
var someVariable = doSomething(function(err, results){
   socket.emit('msg', results);
});
}
});
});

But if I do this, then my connection listener would be added on every refresh of the route. This would mean I'd end up with duplicate binding of event listeners and a load more results messages would be emitted than I'd expect. This would result in crazy behavior for any user who might choose to refresh your app.

Actually, the solution is rather simple although not entirely obvious given the nature of using node. If you maintain a reference to the socket as a global variable and then later emit upon that same socket, like so:

var global_socket;

io.sockets.on('connection', function (socket) {
  global_socket = socket;
});

app.get('/', function(req, res){
for(var i=0; i < someArray.length; i++){
 var someVariable = doSomething(function(err, results){
   global_socket.emit('msg', results);
});
}
});

This way, a connection will be established and maintained and we won't get duplicate bindings when refreshing the route of the app. Simple when you know how. I had a rather interesting discussion following this on irc about whether the use of express and socket.io together is warranted, given that socket.io supports use of variable passing and ultimately could entirely replace an express app.

Posted in General | Tagged , , , , | 6 Comments

Updating WordPress Automatically over SFTP

Posted on 7/5/2011 by Ian

Heres a quick how to on getting wordpress to update over SFTP. On a default install of php, wordpress only gives the option to update over FTP. I wanted to get it working as it’s more secure and means I don’t have to open up FTP on my virtual server and open another port up in the firewall.

I’m using ubuntu and the sequence of commands I followed were:

sudo apt-get install libssh2-1-dev libssh2-php

You should now find that if you list the modules installed in php and check if ssh2 is now there, you get the result “ssh2″:

php -m | grep ssh2

You’ll need to restart apache for the changes to begin being used.

sudo /etc/init.d/apache2 restart

If you head to updates in your dashboard, you should now see an extra option “SSH2″ and spaces for public and private authentication keys. I had to head back to the terminal and create these particular files.

cd ~
ssh-keygen
cat .ssh/id_rsa.pub >> .ssh/authorized_keys
chmod 755 .ssh
chmod 644 .ssh/*
sudo chown -R ian:www-data /var/www/pathtowordpress/

Now you can head into the updates page again, enter your username and password and the locations to your public and private keys (/home/ian/.ssh/id_rsa.pub and /home/ian/.ssh/id_rsa in my case). If you’re running ssh on a non standard port, be sure to append that to your hostname.

Thanks to the following helpful posts: http://kevin.vanzonneveld.net/techblog/article/make_ssh_connections_with_php/

Posted in Code, General, Linux | Tagged , , | Leave a comment

A Survey of Hype in Web Development Podcasts

Posted on 31/3/2011 by Ian

I listen to quite a few web dev podcasts. Something that irks me about a lot of them, is how much they tend to hype their own shows, products or the like. Sure, they’re allowed to, it is their own podcast after all, often from their own businesses etc. I’m fully aware this all possibly contributes toward the success of shows, but it annoys me nonetheless.

One of these things is the title of the show itself. I don’t really get why its necessary to repeat the title (Sure I do really, to drum it into my head), given it’s going to be within the title of the mp3. I know what I’m listening to after all, because my favourite media player tells me.

I’ve been wanting to count the number of times a show utters it’s name over its duration. Sure, I realise this is somewhat ocd and geeky, but hey. Here’s my results for a typical set of podcasts that I often listen to in a week. I’ve linked up the data I’ve used to create this chart on google spreadsheets, for anyone else who may be interested to contribute to and build upon further. I gathered the data all myself by listening to each podcast and counting when mentions of the show were made but it may be a good excuse for me to experiment with amazon turk.

The big web show comes out tops with one mention over 40 mins worth of show and boagworld comes last with 24 mentions of the show over its duration. (It’s unfortunate for boagworld that its theme tune includes a voice repeating the title in the theme. I did originally include echos, which would be 34 mentions, but my colleagues inform me this may be taking the experiment a little too far :) )

I plan to do the same for product mentions (adverts I guess) in a weeks worth of (different) episodes, but if anyone wants to go ahead and update with their own findings that would be great.

Posted in Computers and Internet, General, Web, Web Apps | Tagged , | 1 Comment

Blog rolling with CouchDB, Express and Node.js

Posted on 7/2/2011 by Ian

Over the last little while, I’ve been doing a lot of playing with Node.js, mostly to run data collection scripts. Last week, I started following Ciaran Jessup’s tutorial on getting started with node.js, Express and mongoDB. Express acts as a framework to node.js, allowing you to work in a familiar mvc format in a not so familar server side language. I hit a few problems along the way in the tutorial, so I thought I’ve list a few of my findings here. I also wanted to make use of my preferred flavour of nosql – couchdb with express, which proved extremely easy to port the mongo model over to it. I hope someone out there finds this useful as I’ve yet to find a vast community using couchdb/express.

First things first, you’ll want to install Node and npm (the node package manager) in order to be able to easily install node packages. You’ll also probably find it handy to have the original tutorial open alongside this one. I’ve been using the latest versions of node (0.3.7) and npm (0.2.17) at the time of writing.

Once that’s done, grab copies of the packages that we’ll be using:

npm install express

npm install jade

npm install sass

If you want to use couchdb, then make sure you have it installed – grab it over here and then install the package for talking to node.

npm install cradle

The first hurdle I found was that the way in which that express is called has changed a little.

Then by saving to a file called app.js and calling using:

node app.js

Once that’s done. You can then visit 127.0.0.1:3000 in your browser to see a rivetting message!

After creating folder beneath our original app.js in which to put views, you can use the original article provider file and the updated app.js below in order to have an app with a few articles shown.

Now when you re-run, you’ll see 3 separate articles. Oooh fancy!

Next I hit my first hurdle. Express no longer uses the HAML HTML template language and instead uses JADE by default. This requires converting the HAML templates across to their equivalent JADE counterparts. Basically, this is as simple as dropping the ‘%’ from the beginning of each line (I also replaced braces with brackets in later templates).

The app.js now becomes:

You’ll notice that here we enable the CSS compiler sass and HTML compiler jade. If you download the original sass CSS template into your views folder, you can now restart the app and inspect the fruits of your labour. The CSS shouldn’t actually sit in the views folder, according to the creator of Express, and should instead should be compiled with the sass package itself. I’ve yet to discover the correct way of doing this. To request a stylesheet in the view, we need to do the following:

If you reload now and visit 127.0.0.1:3000 you should see your posts with a little more style.

Creating a form for new posts looks like this:

And the new app.js routes are as follows:

In order to add persistence using mongodb, nothing changes in the original model file, so go ahead and use that. You’ll need to have installed the package ‘mongodb’ if you’d like to try out using it though and update your instantiation of the articleprovider class by supplying a port number to which mongo is installed.

Adding CouchDB Persistence

Here, I took my own angle on the tutorial and decided to give attempting to make my own persistence model using couchdb a go. It proved to be extremely easy, given the JSON representation and HTTP/GET method of access already built in to it.

I added the following view and route to my app.js too, to allow support for clicking upon articles.


And in order to support it, the main view now becomes as follows:

Anyway, so far I’ve not yet added comment support, but given the headway I made here, I’d imagine it would be extremely easy to integrate into my couchdb article model. I’ll update here if I ever get round to adding it!

Posted in Code, General | Tagged , , , , | 17 Comments