But, I don’t think this is a coincidence. Yes, I’m being skeptial here, but a few days ago the OECD predicted that UK growth would be negative in Q1/2012 by around 0.2-0.3%. The UK’s GDP for 2011 was around $2.48 trillion, and assuming a rate of ~$1.60/£1 for much of 2011, GDP can be calculated as £1.55 trillion in native currency. Therefore, for a contraction of 0.2% in Q1, we need to lose ((£1.55t/4)*0.002) or £775million.

The AA have said that there is a potential for 750million additional litres of fuel to be sold should everyone convert a half-full tank to a full one, while the normal sales of 90million litres per day has risen by 172% for petrol & 77% for diesel  (so approx 103.5millions extra litres per day based on the same split below). Given this started on March 28th, we have 5 days this quarter over which there could be panic buying, giving a maximum potential of, say, 517.5million litres.

With Petrol, as of yesterday, at around £1.409, and Diesel at £1.471, and using an old statistics that more diesel is now sold than petrol, I’ll assume here a 40/60% spilt, giving an average price of £1.446 per litre.

Therefore, 775million litres sold, at £1.446 per litre gives £748 million additional revenue for this quarter alone. That has the potential to increase GDP by up to 0.2% for this quarter, and save a double-dip recession that our government is so keep to avoid at all costs, lest the look like failures in managing our economy!

Whether this is the truth is a whole different matter, but it’s an interesting spin on the whole situation none-the-less!

As a quick breakdown…

GDP:
($2.48t @ £1.60/£1) ÷ 4 = £387.5b/Q
                ∴ 0.2% ~= £775m

Fuel Sales: 90m litres/day
  Petrol: 40% = 36m; +172% =  62.0m extra
  Diesel: 60% = 54m;  +77% =  41.5m extra +
                     Total = 103.5m extra
                   5x Days = 517.5m

Extra Sales: 517.5m @ £1.4462 = £748m
  Petrol: £1.409 × 40% = £0.5636
  Diesel: £1.471 × 60% = £0.8826 +
                 Total = £1.4462

GDP Increase: £0.748b ÷ £387.5b =~ 0.19%

Update: Didn’t notice that the 172% increase was for Petrol only, so I’ve added calculations for a 77% increase in diesel as well. Still works out that it would increase GDP by 0.2% rather than 0.3%!

This is mainly because the <modificationset /> <svn /> listener just runs an update against the server and allow a build to trigger build on changed. However, the one for <git /> checks only for changes within the repository – there is no ‘git origin pull‘ test.

For example, we use:

<!-- Project -->
<project name="svn-test" buildafterfailed="false">
  <property name="checkout" value="/path/to/checkout"/>
  <listeners>
    <currentbuildstatuslistener file="${logsdir}/status.txt"/>
  </listeners>
  <modificationset quietperiod="30">
    <svn localworkingcopy="${checkout}"/>
  </modificationset>
  <schedule>
    <ant buildfile="/cruise/conf/buildSvnTest.xml">
      <property name="working_dir" value="${checkout}"/>
    </ant>
  </schedule>

However, simply changing <svn localworkingcopy="${checkout}"/> to <git localworkingcopy="${checkout}"/> will not work. You need to create some <bootstrappers /> which will ensure a ‘git origin pull‘ is run first:

<!-- Project -->
<project name="get-test" buildafterfailed="false">
  <property name="checkout" value="/path/to/checkout"/>
  <plugin name="git" classname="net.sourceforge.cruisecontrol.sourcecontrols.Git"/>
  <bootstrappers>
    <gitbootstrapper LocalWorkingCopy="${checkout}"/>
  </bootstrappers>
  <listeners>
    <currentbuildstatuslistener file="${logsdir}/status.txt"/>
  </listeners>
  <modificationset quietperiod="30">
    <git localworkingcopy="${checkout}/"/>
  </modificationset>
  <schedule>
    <ant buildfile="/cruise/conf/buildSvnTest.xml">
      <property name="working_dir" value="${checkout}"/>
    </ant>
  </schedule>

You should now be able to successfully build your project on the next push to your origin.

However, the first attempt didn’t work (note: this assumes that mod_proxy and mod_header are enabled and AllowOverride All has been set in the httpd.conf):

# /.htaccess
RewriteEngine on
RewriteRule proxy/(.+)$ http://$1 [P,T=appliction/octet-stream,L]

While creating the proxy/ directory and adding a .htaccess file with:

# /proxy/.htaccess
RewriteEngine on
RewriteRule (.+)$ http://$1 [P,L]
Header set Content-Type application/octet-type

or even using

ForceType application/octet-type

also don’t work. Options to unset and then set or edit the header don’t apply (although in some cases you may be able to use append). In essence, it seams that you cannot alter headers received from the remote server when proxying request; only those created by Apache itself can be changed.

Therefore, using RFC2183‘s option for Content-Disposition you can get a browser to treat a response an attachment which in effect tells it to be downloaded while still retaining the correct Content-Type:

# /proxy/.htaccess
RewriteEngine on
RewriteRule (.+)$ http://$1 [P,L]
Header set Content-Disposition attachment

Just make sure that you put in place suitable conditions and/or rules so your web server doesn’t become an open pseudo-proxy for the entire web!

Watch this video on YouTube.

Please take the time to watch this video and spread the word. Seriously. Please. I came across it last night and he made me cry. For just a few short minutes, he opens up his heart and gives us a glimpse of his short life; his pain and his suffering.

He’s an inspiration, but he also just shows what’s wrong with the world at the moment: One of the silent majority who has decided to speak out about what he experiences every day.

If you do anything today, please do this and I dare you not to be moved by him!

Watch this video on YouTube.

I’ve been following this a lot since the news started breaking on Wednesday and it just moves me. I know in the USA there’s a lot of issue with LGBTQ youth in schools (to which almost anyone with power is largely ignoring), but I think this case is even more significant. Not least because it’s another in a long list of cries for help and/or release, but because for a short time, he had his brief, actual hope.

Despite his hardship for so long at the end of last year and the start of this, he started to see light at the end of the tunnel. Through family, friends, counselling and his music (with tribute to Lady GaGa there), saw hope, so much so that he recorded his own ‘It Gets Better’ video (this video). You can see the pain he’s been through when he talks and even though he is young he wants to pass a true message from someone going through it back out to those that need it.

Four months later, he killed himself.

All this hope, promise and belief that he managed to find in himself once again was taken way, beyond the point of no return, and one final sleep was the only way out he could now see.

All around this world, this must stop. The ignorance portrayed, and the hate give out, by Religion/the Religious or by those who see the world built around their morals, especially in those countries that consider themselves free and civilised, is really angering me. I know there are exceptions to the rule, and I know that there are those with faith that do truly follow the tenants of love, life and tolerance, but something must now give.

To those that use nothing more than belief and ignorance to perpetuate their hate, yet preach love, and want nothing more than to enforce their morals on everyone while at the same time extolling the virtues of freedom and choice – this boy’s blood is on your hands, along with all those before and any in the future that suffer this emptiness (although I hope with all my heart we are so close to the end).

I know my words can never mean much to him now and probably as a single person, my abilities to change the world, or even a small bit of it, are limited, but for as long as it takes, I will stand with those who are bullied or victimised for something they didn’t choose and is simple a part of who they are.

So kindly lay off those people you think are weird. If you put that energy you are using to put someone else down into figuring out your own true identity, I promise you that, even for you, your life and your feelings of self worth will get better. And you know what? Maybe the world will be a better place.

R.I.P. Jamey Rodemeyer. Despite people telling you that you horrible and disgusting and wrong for your love, you were, are, and forever will be fucking perfect. I hope you can see that now, wherever you are.

I won’t take credit for this; written by Love, Sex and Ramen.

This simply makes it an act of terrorism; akin to threatening to blow up something unless you get your way. If it was truly looking to accomplish what he set out, he’s failed. Spectacularly.

Go you ‘enlightened’ people of the ‘bible belt’!

• Slight Injuries:
  2009: 70; 2010: 55 (down since they were switched off);
• Serious Injuries:
  2009: 13; 2010: 13 (no change);
• Fatalities:
  2009: 0; 2010: 0 (no change); and
• Total Accidents:
  2009: 62; 2010: 60 (down since they were switched off).

Matching periods of 08/2009-01/2010 and 08/2010-01/2011, accidents and injuries at sites where the cameras have been placed have gone down, not up as is initially suggested.

To back up their figures, they need to cover the whole of Oxfordshire:

• Slight Injuries:
  2009: 982; 2010: 999 (slightly up since they were switched off);
• Serious Injuries:
  2009: 160; 2010: 179 (up since they were switched off);
• Fatalities:
  2009: 12; 2010: 18 (up since they were switched off); and
• Total Accidents:
  2009: 885; 2010: 867 (down since they were switched off).

This is not as clear cut as they’ve been making it out to be over the last two days and I don’t find it that surprising that it’s taken until now for the rest of these figures to be released. So, yes, while there is a significant increase in the amount of deaths (up 50%) and overall, injuries across the county are up 2.18%,injuries at camera sites are down 22.1%. This combined with the fact thataccidents are down 2.16% overall means that they have had to manipulate the statistics to back up their argument.

They have used a set of figures, loosely related to the ones they should have used, to justify a decision which I believe is in some way incorrect. £600,000 should be used to fund more police in cars to patrol the area, interact with the public and deter bad driving rather than just switching on camera.

After all: “Today the Police are switching back on Speed Cameras in the country after statistics show that accidents and injuries around camera sites have fallen” isn’t quite as catchy a headline!

We’re growing fast and although we haven’t reached the ceiling, some of the tools we used are tweaked versions of the ones developed when we had just a few servers. Knowing that there is this ceiling, and its approaching fast, I’ve been putting in some time recently researching on how to take it to the next level.

One of our senior developers suggested something he’d seen previously: Murder. Developed by Twitter last year to help them get code out to their thousands of servers, they were able to tap into the distributed peer-to-peer capabilities of Bittorrent as a powerful alternative to their original from-central-server pull configuration.

However, their tool is configured to drop into Capistrano. With us using Puppet to manage our server’s configuration, we have instead installed and configured MCollective instead, and therefore we needed to write a client and agent to integrate the two.

Making Murder

Murder is essentially in two parts: a BitTornado backend (with some slight reconfiguration to improve speed on a low-latency network but no changes to the actual libraries) and three custom scripts to make the torrents, provide the tracker and seed/peer the files themselves.

I took the configuration changes from Murder and forward-ported them to the latest version (the one in their Git responsibly is over a year old) and repacked them all into .deb packages (we run Debian on all our servers). Their changes disable UPnP (not needed outside of a home network), drop all the timeouts, decrease the time between updates from the tracker and imposing a limit on the upload and download speeds stopping transfers from swamping the network and server resources.

However, their choice was to set each peer to only download from one other, essentially cascading the updates through the system almost one server at a time. However my tests with this method was not particularly fast – upping the client limit to a maximum of two peers showed big improvements and I was able to transfer a 180Mb test file (an ISO image) between twelve servers simultaneously in 28 seconds.

With these changes added as a diff to the current BitTornado package, and another package created for the Murder programs, we added them to our internal repository and pushed them out with Puppet.

MCollective

MCollective is the spark here and I’m sure what your most interested in. I must however make a point right from the start: This is only the second program I’ve written in Ruby (the first being a multi-threaded latency monitor to analysis the different parts of our site and the different stages of each request to pinpoint issues in serving our site). Additionally, this is the first experience I’ve had in working with MCollective. Anyone who does use it, or takes a look at the code; feedback is welcome. Just don’t expect the most efficient code it could be right now!

In essence, the client program processes the arguments, does some sanity checking, then creates a .torrent file with murder_make_torrent before launching a separate thread which seeds the package with murder_client.

system("/usr/sbin/murder_make_torrent #{$opt[:package]} #{$opt[:tracker]} #{$opt[:torrent]}")
$opt[:seeder] = Thread.new($opt[:package], $opt[:torrent]) { |package,torrent|
  system("/usr/sbin/murder_client seed #{torrent} #{package} #{get_address($opt[:tracker].split(':',2).first)}")
}

The .torrent file is read, encoded into a base64 string and then sent to each agent via the main thread along with the name of the package, the location of the tracker and the application for which we’re deploying.

$opt[:package_hash] = Base64.encode64(Marshal.dump({
  :package => File.basename($opt[:package]),
  :tracker => $opt[:tracker],
  :torrent => IO.read($opt[:torrent])
})).split(/\n/).join

Each agent takes that data, launches a class which matches to the name of the application we’re deploying to (which in turn handles the specifics of deploying each application), and then runs a method corresponding to an action (for example, add() when we’re adding a new version or upgrade() for applications which supports only a single concurrent version at any one time).

class Deploy<RPC::Agent
  metadata :name => "deploy",
           :description => "Agent to deploy our website, including Assets and Tomcat files.",
           :author => "Jonathan Wright ",

  DeployPackage.valid_actions.each do |act|
    action act do
      # Get the details passed to us from the API
      application = request[:application]
      version = request[:version]
      package = request[:package]
      # Make sure that there's a class for this application's deployment
      reply.fail! "Application #{application} not available" \
        unless Agent.const_defined?(application.capitalize) \
            && Agent.const_get(application.capitalize).class.is_a?(Class)
      # If a class exists, create an instance of it so we can work with it
      worker = Agent.const_get(application.capitalize).new(reply, logger, version, package)
      # Now make sure that the action requests has a method for it, then run
      reply.fail! "Command not valid for target" \
        unless worker.do_validate(act) # <-- this is a custom method of worker's parent class
      reply[:output] = worker.send(act)
    end
  end
end

It will then launch murder_client as a peer, download the file and then extract it or process it as required. Although not currently enabled as part of our first round testing, it can then create a new thread which will run for a maximum of 30 seconds (using Timeout.timeout()) in which it will re-seed the package back to the network at the same time it’s processing the copy it has.

begin
  # This needs to be done in a separate thread as so not to interfere
  # with remainder of the deployment
  @thread = Thread.new {
    begin
      # We'll also wrap this in a Timeout class; fixed time run
      status = Timeout::timeout(@@timeout) {
        system("/usr/sbin/murder_client seed #{torrent} #{package} #{address}")
      }
    rescue Timeout::Error => e
      # We should always reach this section as the program as once the
      # timeout it reached Timeout::Error is raised
      nil
    end
  }
rescue => e
  # If we can't start the tread, we won't worry. Just make a note of it
  @logger.debug "Could not start thread for seeding #{@package}"
  @thread = nil
end

First Tests

Our first tests (completed yesterday) had distributed a 25Mb file containing around 2,200 images, css & javascript files among 41 servers in less than 30 seconds; an action that had previously taken between 3 and 10 minutes our original rsync method. It should be noted that the majority of servers completed in less than 15 seconds with the main seeder becoming increasingly important in serving the content as the nodes completed and disappeared from the tracker.

With the re-seeding thread enabled I expect this to drop as all the nodes stay available for longer.

I also tested with a smaller set of assets at 1.7Mb (about 350 files) and the program was able to distribute that file and extract it to the same 41 severs in 2.2 seconds.

The code isn’t yet read for full release, and I will do that in due course. A few things are needed first:

• Like the Agent part of the code, I would like to make the client part into a dynamic class-loading version which will separate logic and action. Much of the current code is heavily biased to our internal business processes and would need to be changed by any person wishing to use it before they could;
• Export the configuration to MCollective itself – far to much of it is within the program and classes as they’ve been written and added I’ve coded it all. This will also make it much easier to configure the program from within Puppet by altering the template for server.cfg;
• Rework BitTornado changing it’s namespace to Murder and combine the Libraries and Programs into one package to avoid conflicts with BitTornado and easier package management; plus
• Bug fixes (as I find them…) and more testing!

If you would like to see a copy of the current code, drop me a note on @twitter or leave a comment.

As a quick overview: In September last year, a fit of jealousy caused someone to set light to a mid-level flat in a multi-story house. The intended target was away, but his wife and three-year-old daughter were inside. They died.

The trial has concluded, and Barbara Zhanje has been jailed for their murder, but on the steps of the Court House after the judgement, the Uncle of the deceased and her daughter, Mr. Osobu, said:

“The fire brigade tried but it was not good enough. Lawyers will be looking at things to see if we can take further action.”

Although this article doesn’t quote the core incident here (and in fact appears to suggest that the blame for failing to save them lays with the Fire Brigade), this one does. Specifically:

Ms Elmi was taken out of her room with the assistance of a ladder. Firefighters entered the property to search the second floor but the landing collapsed – causing one of the officers to fall through the floor injuring himself.

They were unable to get to the back bedroom where Ms Ejifunmilayo and her daughter were and retreated.

“One of the firefighters fell through the landing.” In doing everything they can to search for these people, the building became unstable, a firefighter fell and was injured, so they “retreated.”

Now Mr. Osobu wants to sue the Fire Brigade for not doing enough to save those that died. At this point, I have to say, for fuck’s sake, the building was collapsing and a firefighter as well was almost killed; what more can they do?

It was very tragic what happened and should never have happened. But to lay the blame for the loss of life at the Fire Brigade as well as the murderer, and to then suggest that you will sue them for failing to save these two people is beyond belief.

I want to see him strap on a suit and run into a burning building without knowing what it’s like inside, where people are or whether you’re going to come back out alive, or undamaged, and see how he copes. Once he does that, maybe then he can consider his next step very carefully.