Beanstalk Growl Notifications
by leon on November 10, 2010
In this post, i’ll talk about how I managed to get updates from Beanstalk to my desktop, and why this was important.
My previous post has been about mashing 3rd party apps together in order to streamline and improve how we work. In this case, it was getting ServerDensity and custom stats to be displayed on our Geckoboard in the studio. This provided a central space to monitor the health and popularity of our sites. This was crucial during the launch process (putting 4 sites live that have served around 20 million page views on a Friday evening was a bit scary, so being able to view the load and memory was a god send).
This post focuses on how we actually maintained and deployed our code, first of all a little background on our server set up.
The four sites all shared a common core with the only difference being the theme and database, for this reason we decided to create a common core system within one repository and then use externals to pull in the core into each website. This meant that we only had to update the code base once, and this would automatically be reflected on all sites (I’ll do a separate post on Subversion externals in the not too distant future).
In order to manage of Subversion repository, we use Beanstalk. This is good for a number of reasons; it’s hosted, managed and backed up for us. The best reason is the deployments.
Beanstalk allows you to create environments or groups of servers and then either automatically or manually deploy your repo out to them. Because we have more than one server this was invaluable to us; we could easily push code to any combination of our three servers. The screen shot below shows the server set up for BRMB (the other sites, are identical).
The image above shows that we have three environments, Admin, Live and Dev.
The wp-admin sits on a separate server, with a separate host name.
The live sites sit behind a load balancer on two identical web servers. Creating one environment with two servers meant that when we deploy to Live the code is copied to both servers at the same time, meaning we are always in sync.
The third environment is Dev. Each time we make a change to the code, it is pushed to a dev domain that sits on only one of the two live boxes. This is so we can preview code in an environment that is as close to the live sites as possible, but away from prying eyes. We have chosen to deploy automatically, so that previewing code is as simple as committing a file – no ftp and no fuss!
Having all of these deploys going off is all very well, but it’s crucial to know when the code is actually on the server, especially as we have four sites – we need to know when the eagle has landed!
Beanstalk can help us out here, as it features deployment hooks; urls that are called when a deployment starts and ends. These can be used for anything, such as running clean-up scripts, backups etc. Getting notifications is as simple as hooking up a script to listen out for these hooks, and then sending the results somewhere.
There is more info on these hooks here - http://help.beanstalkapp.com/faqs/deployments/post-release-web-hooks-for-deployments
Beanstalk posts a JSON object containing data about your deployment. Using the rather fabulous Notify.io service, it’s just a matter of turning this JSON object into a notifcation.
Because i’m a nice guy, i’ve decided to write a CodeIgniter library that will allow you to easily send notifications to Notify.io and thus to Growl on your Mac.
https://github.com/leonbarrett/CodeIgniter-Notify.io-Api
Because, i’m even nicer i’ve decided to release the code that I use, so you can have Beanstalk notifcations when your code is deployed (and also on commits if you wish), i’ve even written a tutorial on how this all works below;
- First of all, go register an account over at Notify.io, install the Mac client and do a test (from the settings tab) to make sure notifications are coming through.
- If the email address you registered isn’t the same one you use for Beanstalk, add and confirm it in Notify.io (this is crucial!)
- Set up a CodeIgniter application on a server of your choice. If you’re new to CodeIgniter, go check out the tutorial.
- Get the latest version of my CodeIgniter Notify.io API from GitHub and drop the files in the correct places.
- Create a new controller (I called mine deploys) and drop this code into it.
- Log into your Beanstalk account and head to the repo you wish to have notifications for. Follow these instructions, adding the path to the url you just created. For example if you called your controller deploy and used the code above, this will be something like: http://example.com/deploys/pre or /post.
- Click save, deploy some code from the repo and sit back as you get notifications.
The script will get the email address of whoever deployed the code and use this as the Notify.io account (this is automatic, so this is why you need to ensure that the email address used for Beanstalk is set up in Notify.io). It will also display the server, revision and repo. Clicking on the notification will take you to the repo in question.
Adding an path to the icon, also adds a final bit of flair.
You can also get commit notifications, by adding the following code to a controller.
Simply add the relevant URL to the Web Hook option on the Integration settings for your repo.
Geckoboard and ServerDensity Take 2
by leon on November 9, 2010
So, last week I wrote a post about how we were using Geckoboard to monitor the launch of a set of sites – after a successful launch I can now reveal that those sites were for the Orion Media Group and their radio stations; BRMB, Beacon, Wyvern and Mercia.
The launch process started at 10am on Friday morning and ended at around 8.30pm! Throughout the whole process we had our Geckoboard displayed on a projector. Since I wrote the original post, I have modified the widgets to give us some more useful info (plus some more pretty graphs).
We can now see:
- the average 30 min load range
- the load range over the last 6 hours
- memory usage
- mysql status
- traffic in and out
This data is much more useful that we had, plus now that Geckoboard allows us to have more than one dashboard we can dedicate a dashboard solely for the use of server stats. We now also have some for user stats (number of signups today vs yersterday, sign up method – twitter/facebook/web and pageviews).
Luckily the servers have coped with a fair few thousand (and the rest) pageviews without the load going above 2.3, this is down to the excellent cacheing mechanisms put in place by @deplorableword.
I have also released the code that I used in order to make this all possible, which is available over on GitHub
ServerDensity API – https://github.com/leonbarrett/CodeIgniter-ServerDensity-API
GeckoBoard API – https://github.com/leonbarrett/CodeIgniter-Geckoboard-API
Geckoboard and ServerDensity getting it on!
by leon on October 24, 2010
This is a quick post to show you something that I knocked together in a few hours today – ServerDensity stats within Geckoboard.
At work we’re just about to push a set of sites live that combined have served some 20 million server requests this year. Needless to say, this is quite a stressfull time and one of the things keeping us up at night is ensuring that our servers can cope with the traffic.
When the sites go live (and hopefully don’t fall over), i’ll add a post detailing some of the hardware and set up that we have, but briefly we have two quad core servers for web traffic sitting behind a load balancer, and a separate quad core machine acting as a database server, with a combined 6GB of RAM for the three.
It’s crucial that we know the status of our servers at any time (even during the night), so we opted for using ServerDensity. This offers us email, iPhone push and SMS alerts to notify us, if any aspect of our set up is above pre-defined thresholds.
While ServerDensity has it’s own brilliant UI for displaying stats, I wanted to be able to have our server stats, and other related stats in one central place.
As we already using Geckoboard to monitor our internal stats, I decided that with some of the newly released features to role out a dashboard for these sites.
The main things I wanted to see were; server load, disk usage and memory usage. We also have Google Analytics plus the Twitter accounts (it’ll be interesting to see the public reaction when the new sites go live) for each site.
I’ll also be adding stats relating to user sign ups for each site, so we can gauge the uptake of some of the new features we have implemented (such as sign in with Twitter/Facebook).
The newly released Meter display is perfect for showing the current, min and max load for each server, plus the the Pie charts make viewing the disk/memory usage easy.
The process was actually quite simple due to the ease and simplicity of both API’s, and as such I wrote a CodeIgniter library. This essentially wraps the ServerDensity API in a number of calls so that it’s simple as easy to get the data you require. All data is returned in JSON format.
The next step was to write a simple (and dirty) wrapper for generating the output (in XML) required for Geckoboard to be able to display the data.
I’ll be releasing both libraries pretty soon (in the next week or so) so watch this space.
Have any other ideas for stats to display, or have your own usage examples, let me know in the comments!
Time for a new web app?
by leon on August 15, 2010
Sometimes it’s the simple ideas that prove to be the best.
A month or two ago, I got an invoice for a hosting package I set up two years ago for flCodr (which is no longer being developed). It came as a bit of a shock, as there wasn’t any notification from the hosting company to let me know.
While things like this are really up to me to know, it would have been nice to have a reminder. Many (good) hosting companies do this for you, but you sometimes have hosting and domain reg with two parties, plus for whatever reason you may have different providers for each client/site.
It can quickly become a bit of a nightmare, keeping track of renewal dates.
To solve this I propose to build a really simple web app (still to decide on a name) that will allow you to list all of your hosting and registration details and be notified via email and Twitter when they are due to expire. That’s it, nothing more. Oh and you can only sign up via Twitter (this does away with a full auth system) – no Twitter, no account!
In a nutshell the app will:
- Auto lookup the WhoIs info for each domain entered and try and auto fill details
- Allow you to sign up ONLY via Twitter oAuth. Avatar, display name and email (I think) will be pulled from Twitter.
- Notify you via email, (if you allow it) via Twitter (DM only – you probably don’t want the whole world knowing that http://www.kinkyladyboys is ready for renewal) and for iOS (prowl or Boxcar)
- Allow you to specify the notification period (month, week, day before)
- Allow you to notify (via email only) a third party (maybe an accounts dept)
- Assign an action next to each service (free text input that gets sent with the notification)
Hopefully this is something that I should be able to knock together fairly quickly using CodeIgniter.
If you have any thoughts, ideas or feedback (or a name for the service) then please sound off in the comments!
I’ll throw some wireframes together soon, and get them online!
iPad and Three MiFi
by leon on June 13, 2010
If you’re in two minds whether to get an iPad – don’t be!
The only question you really need to answer is 3G or not 3G.
I opted for the non 3G version – my logic was this:
The iPad whilst compact and portable is not truly portable – it’s not the kind of thing you can fit in your pocket and i’d feel less than comfortable using it on public transport. For that reason my use would mainly be restricted to home, work and family where Wi-Fi is available. For these reasons I opted for the cheaper WiFi only model.
Whilst this was more than fine at home, I decided to take the iPad out to the pub (there was quite a lot of family there, so thought it would be a good opportunity to go through some photos) and for the first time realised how dependant the iPad was on the internet. Apart from showing photos, a few videos and Pages most of the other apps I had installed required an internet connection.
Normally this wouldn’t be an issue as I would always have my iPhone with me and thus an internet connection, however I felt that I was missing out on something.
I had been contemplating getting a Three MiFi dongle for a while for use with my MacBook Air, and now seemed like the perfect time. £69.99 gets you a dongle on Pay as you go and 3GB or 3months of data, whichever gets used first.
Set up was a breeze and it worked out of the box – the credit was pre applied and no registration was needed. The device works in such a way that you can create a WiFi network, without being connected to the 3G network. Whilst this may seem an extra step, it means that you can have a local network without using any of your allowance in the background.
You may be thinking “for an extra £30 you could have had 3G built in”. Whilst true, this solution does mean that I have a connection for my MBA and iPad at the same time. I can also easily create a WiFi network between my iPhone and MBA which is ideal when using Keynote Remote (anybody that has tried to ad-hoc solution will agree that it is less than perfect).
I have found the connection to be good in the areas that I have tried to use it, and compared to the signal strength of o2′s network seems comparable.
Sure, it’s a little inconvenient to switch the dongle on first to connect, but the times when I need to quickly check something I will just use my iPhone, plus it isn’t like i’ll be taking the iPad out ‘on the road’ all of the time!
If you’d like more info in the Three MiFi hit the link below:
http://siz.la/hfk
First post from my iPad
by leon on May 29, 2010
If you already follow me on Twitter you will already know that I am now the proud father of Apple’s latest device.
Like most of the purchases I make from Apple, this was one of want rather than need. However saying that, this is worth all of the four hundred and twenty nine pounds that it was exchanged for.
This post is obviously being written on the wonderful WordPress for iPad app and is a joy to use.
There isn’t really much more to say really, any of you who own an iPad will understand just how amazing this thing really is.
Oh and if you haven’t already checked out Vimeo and iPlayer yet on an iPad, go and do so…. Now!
They are both shining examples of why flash isn’t needed for video online (that’s not to say flash doesn’t have its uses online – I’m a flash dev and there are times when using flash was not only the best choice but the only choice).
Testing a custom URL shortener
by leon on May 2, 2010
Testing a custom url shortener
Spotify profile widget
by leon on April 27, 2010
So @spotify have released a new version of their desktop based application that adds a stack of social features.
One of these is a user profile which displays your playlists, aswell as your top played artists and tracks.
I created a simple WordPress plugin that enables you to easily add a button in your sidebar that links to your Spotify profile.
The widget has two options – one for your username, and the other for the text you wish to be displayed on the button.
To install follow these steps:
- Download the plugin
- Upload to wp-content/plugins
- Activate
- Customise the widget by adding your username and the text you would like displayed in the link
This is a simple widget with a simple function. If you’d like more features, just ask and i’ll see what I can do!
Downloadr – jQuery file download plugin
by leon on April 8, 2010
Downloadr aims to solve the age old problem of downloading files from your website.
On countless occasions, I have been asked – “Why does the PDF open in my browser rather than download?” and my answer has always been – “Because you need to right click and choose Save As!”.
Rather than trying to do any fancy server side stuff Downloadr will display an overlay with instructions on how to do this. You can ofcourse just right click on the link and download as you normally would.
These instructions are tailored to the browser that the user is viewing your page in.
Not only that, but Downloadr will also style your link to look more like a button, so your user knows where to click.
To view a demo and download the plugin, head over to the microsite.
Display your FourSquare updates on your website
by leon on April 6, 2010
I recently made a few small changes to my personal website, including the addition of my most recent FourSquare check in’s. This post is a quick write up to show you how I did it for FourSquare, but this technique can be used for almost any social network or RSS feed.
You may be thinking – “Why not just use the widget they released?” – because it didn’t fit into the design or style of my site – plus with this I can pick and choose the information that is displayed.
Getting the updates onto my site was made easy by using the jFeed and timeago jQuery plugins.
First of all, go and grab the files from here and here.
The only files we really need from the jFeed download is the plugin itself located in build > dist > jquery.jfeed.pack.js and a file called proxy.php. Upload these and the jquery.timeago.js from the TimeAgo download to somewhere on your server and add a link to the head of your document.
Next, head over to the feeds section on Foursquare and copy the link to your RSS feed (it looks like the image below).
Next step is to create the Javascript to parse the feed. We need to pass this through the proxy.php file due to the fact that the file we are requesting is on a different server to the one we are calling it from. The read me documentation said that this shouldn’t be used – but what the hell I just ignored it!
Don’t forget to add your FourSquare feed in below!
$.getFeed({
url: 'proxy.php?url=YOUR FOURSQUARE FEED.rss',
success: function(feed) {
var html = '
- ';
for(var i = 0; i < feed.items.length && i < 5; i++) {
var item = feed.items[i];
html += '
- ' + '' + item.title + ' - ' + jQuery.timeago(item.updated); + ' '; } html += '
So to explain, the above Javascript passed the FourSquare RSS feed through the proxy.php and then creates a variable called html with an opening <ul> tag. Upon each iteration of the loop, upto a maximum of 5 the function then selects the title, link and when the checkin occurred and adds this to a <li> element. This newly created <li> containing the link and time is then added to the html variable. When the html variable ends, the closing </ul> tag is added.
The whole html variable is then appending via jQuery to a div with the ID of foursquare, pretty simple!
Oh, you’ll notice that the time stamp is passed through jQuery.timeago() – this is so we get a nicely formatted timestamp. You can read more about jQuery.timeago() here.
Take a look at a demo over at http://www.leonbarrett.com
