Skip to main content
Wedding Photo

Coming out… of the IT cupboard.

It’s been 335 days since I last wrote a blog post for this site, and it was looking like it might even be a bit longer.  That period has been nothing short of manic both personally and professionally – and professionally I have had to step out of the IT cupboard.

The most important thing which happened in that intervening period was of course marrying my best friend  –  a gorgeous day, surrounded by our friends and family; a day which made a lifetime of memories. I’m not going to gush too much about that here – this has always been a semi-professional blog and I don’t intend to change that (if you want the story, buy me a beer!).

The IT Cupboard.

I’ve always worked in IT, from manufacturing IT systems for multi-nationals to running smaller IT functions for companies in marketing services, logistics and warehousing. I’ve done everything from Email to Azure, Development to ERP – and latterly that has been as an IT Director for medium sized firms (~150 employees, ~£5m – £10m turnover), and was even involved in a management buyout of one of those firms.  However, despite sitting on boards and helping define company strategy and then deliver the digital tools to support that strategy, I’ve always been in “The IT Cupboard.”

You know “The IT Cupboard” – every company has one – it’s where you put the IT people, it’s where you go when your system doesn’t work, or you’ve an idea for changing a process, improving something and you need a system to help.  It’s been a place where many fear to tread in the past.   In reality – it’s a great place, it’s the engine room of successful businesses.  It’s one of the few places which should know how nearly everything works in a business – it’s unique in that it sees everything from accounting processes, manufacturing, customer service, phones to email.  IT departments have had to shed the overprotective,  “can’t do” attitude of the 1990s and become enablers and innovators of change – in no small part down to increasing digitisation and the influence of “disruptive” millennials coming into the work place.

I enjoy IT. There, I said it.  It’s great when it’s done well – it can help (non IT) businesses make more money and get things done.

Stepping Out.

Over the past 2 years I’ve been spending my evenings working on a project with a business partner which over the last year grew to a point where it generated enough profit that I could leave the “security” of employment, which I did in October 2016.  Whilst I have retained some IT consultancy, my primary focus is now that new venture.

Leaving full-time employment in IT, has also meant stepping out of the IT cupboard – whilst I (obviously) run the IT side of the new venture, I now have to run the operations of the business too; but importantly (for me) I’m starting to work on selling too.

My previous roles always included client facing work, but never working on new business directly. Fortunately my business partner is from a sales background and his experience and hard work has allowed us to get here; but now I want to use the time previously occupied by employment wisely and aggressively grow our new venture.

I’ve joined a business network and started going to various events to get “out and about.”  Building my name, brand and learning to soft sell..  2017 will be a great adventure!


Success: It’s Made From Swiss Cheese

I have a non-professional personal interest in aviation – I fly helicopters for a hobby. Part of that draws me to reading accident reports (mainly from the UK AAIB) so that I can try and make sure I don’t make the same mistakes that others have made.

Of course, there are lots of anecdotes and theories out there about safety in aviation and how accidents come to be; but one has become widely accepted and used in industries outside of aviation:  The Swiss Cheese Model.  Much to the annoyance of risk managers and accident investigators, I’ll over simplify it:

The end result of an accident rarely happens because of one single failure. Instead it happens because of a series of failures which alone are not enough to result in the accident, but when combined in the right order (the holes line up) you get the accident.

Risk managers and the like will describe each layer of the cheese (with a hole in) as a defence against an outcome you don’t want.

But What If You Do Want The Outcome?

I’ve been thinking a lot about success lately, what defines it and what gets you there (in no small part thanks to visiting Cmdr Chris Hadfield and listening to him talk).  We use the Swiss Cheese Model and we assume the outcome is negative and something we don’t want… but what if we turned that around and at the end it was the outcome we wanted; it was our success.

Once you’ve defined your idea of success you need to get there. Sadly very, very few people get there instantly (and even if you could, would there be the same sense of achievement?).  Instead we work at it.

What we really do is make a series of small decisions and eventually they should all line up like the proverbial Swiss Cheese.

Lining Up The Holes.

Making the decisions (the hole in each slice of cheese) is the easy bit – we can all make a decision.  What we need to do, to complete my analogy, is make sure we’re making the right decision and that it lines up with where we want to be… does it line up with our aim, our desired outcome?

Swap Hazards & Accident for Decision & Success.
Swap Hazards & Accident for Decision & Success.

When we’re making those decisions, choosing which hole in the cheese we want to take a path through, we have to do so knowing our ultimate aim:  make sure we chose the hole which fits with our strategy.

For example: If we’re implementing a Windows ERP solution and need to select a web server then we need to bear in mind our strategy is Windows and we’re building that skill base; an Apache server may not help the holes line up!

Each small decision lines up our Swiss Cheese slices – the culmination of those slices (decisions) is the result we want.


True Cost Of Commodity IT

Last week I got into a quick discussion on Twitter with Matt Ballantine (@ballantine70) and Mark Chillingworth (@mchillingworth) which ended up talking about the value of transparency being one of the largest factors changing the customer / supplier relationship as part of a move to a more digital marketplace.

It all started with a simple tweet from Matt around the amount of information a UK Courier, DPD, provide whilst your parcel is in their hands on its way to you.  I work as IT Director for a fulfilment firm and we send thousands of parcels a day – the performance of various carriers is never far from my mind; so I replied. DPD have a great reputation but aren’t cheap (from a sender’s point of view).


Matt sent me a link to an article he wrote back in November about how organisations can approach ‘digital’, development and IT.  The article is entitled “Digital Architecture Revisited,” and in it Matt describes a tool he likes to use to simplify where to make technology investments. I won’t recite it here (I encourage you to read the article), but essentially the opportunities for investment are divided into four sections based on how core to your business the activity is and then whether it is visible or invisible to your customer.

Digital Architecture - Matt Ballantine
Digital Architecture – Matt Ballantine


Invisible Support

I want to talk about the bottom left quadrant of this graph: Invisible Support.  This is where “traditional” IT departments operate, and where you can spend a lot of time for very little reward if you’re not careful.

In the article Matt makes the comment “Big ticket items like ERP systems, and IT infrastructure, sucked up all the cash…”  and this touched a nerve for me.  I’m in the throws of implementing a Microsoft Dynamics AX system, and it’s fair to say it’s a big ticket item and it’s sucking up cash.

The challenge from Matt’s model it is to try and get your IT operating in the top right corner – where your customer can see you working on your core activity.  This got me thinking about some of the decisions we’re making around our own ERP roll out.

You can create differentiation between your competitors and yourself by operating int he Visible/Core section of the model, but you I don’t believe you can be there until you’ve got the Invisible/Support section right. You have to have the ERP system and the infrastructure first to allow you to operate where you want to be for commercial reasons: DPD couldn’t be so transparent and well regarded if they didn’t have the infrastructure and systems to reliably deliver the information which started the conversation.

This sort of IT is what is referred to as “commodity IT” in many circles: it’s the desktop PCs, printers, network and systems which nearly every business has to do. If everyone has to do it (or similar to it) then why would you expend a lot of effort operating here? Why reinvent the wheel?  You wouldn’t…

… but then you need to make sure you’re buying the right size wheel.

The Cost Of Commodity IT

Providing this commodity IT is taken for granted, and as a good IT leader you’ll be wanting to get your resources operating in the top right as soon as you can.  If you “buy” the right sized wheel you can do this; but too often the success of this commodity IT is judged in terms like “on time,” “on budget” and “meets the specification.”

Of course you have to have control on cost, quality and time to deliver anything, even the commodity IT; but if you’re judging your day-to-day IT in these terms alone then you’re likely not getting the best from it.

We need to move to judging the commodity IT in terms of how it enables us to operate in a way which sets us apart from our competitors and delight our customers. Is your commodity IT an enabler?  If not then the true cost of it is much more than the £ and Hr you’ve spent on it… it’s a drag to getting the good stuff done.

As a real world example – how easily can you adapt your ERP system to provide a “surprise and delight” feature to your customers? If it is closed source, overly complex and/or requires a lot of consultant time to do so then no matter how on budget or on time it was it’s now having a very real negative cost to your organisation.

When we make decisions around day-to-day IT we need to do it with an eye on how it’ll get us into the top-right corner quicker…. because the top right corner is where the profit is.

I Was Wrong About Gamification: It Works.

What feels like way back in early 2012 I was still dabbling with a little bit of coding from time-to-time and scoffed when Microsoft announced that they were bringing the XBox to their Professional development environment, Visual Studio.


Visual Studio Achievements, as the plug in for Visual Studio was called, meant that developers could ‘Unlock Achievements’ and take part in leader boards to exhibit their prowess in particular areas of Visual Studio.  They could even share their achievements on social media, their blog and in a public leader board.

This was part of the emerging trend of Gamification – the act of taking game like characteristics such as point collecting, leader boards and the like, and applying them to other areas.  This was (and still is) spilling over from digital marketing, where it’s proved a very effective marketing tool for big and small brands alike.

My background has always been in Line-Of-Business application development, and predominantly for engineering or process-orientated companies: I worked in what I regarded as ‘the grown up world.‘  I didn’t like this approach and didn’t see how it would encourage quality code – although I could see how it would increase quantity.  It seemed like the ‘dumbing down’ of the business world, and where would it end?

Gamification – I Was Wrong & It’s Addictive.

I was wrong, and I may even be a convert. While I don’t use Visual Studio as a daily tool any more, so VS Achievements doesn’t apply to me, I do use a To Do List tool called ToDoIst to help me manage the tasks I have to do and remember those coming up.

I started using ToDoIst because it had some great reviews in some productivity blogs I happened to be reading at the time, it is easy to add tasks to, you can add them to ‘projects’ and it works easily across various devices.  I soon became addicted and it earned its place as a pinned tab in Firefox next to GoogleApps and Trello.

Then I noticed the ‘Karma‘ tab at the top.  It was gamifying my to-do list.  You get karma points for adding tasks to the list, more karma points for completing them and there are even achievement levels for karma. I found myself checking this often – it’s a really handy, easy measure of how productive I am being!


The real killer for me though, was the ‘Last 7 Days’ and the ‘Streak’ function.  You can set a target number of items to get done each day, tell ToDoIst what days you work and it’ll tell you whether you’re hitting that goal – and it’ll keep a track of how many days in a row you’ve done that.

Once you get on a roll and get a few days in a row when you’ve hit your target, you really don’t want to drop out, and you want a longer streak.  My record is 9 days – but I’m not doing so well this week! DailyStreak

So, I’ll have to admit: When presented well gamification can be great motivation.

It’s just evolution though…

That got me thinking about what was really going on… is gamification really any more than an evolution of the long established industrial / business tool of ‘visualisation’.   In factories and process businesses across the world, output and problem visualisation is a core component of getting the most out of the process – it dates back decades and is linked to The Toyota Production System’s concept of Jidoka.


Jidoka is the second of the two core principles of TPS and relates to the stopping of work as soon as a problem occurs (thus eliminating root cause as early as possible and increasing quality).  In manufacturing environments, Jidoka normally manifests itself as a production board which shows the problem when it occurs, but in normal running those boards (called Andon boards) display output… much like my ToDoIst Karma graphs.  I can see a problem immediately: my graph drops off and I’m not being productive – the root cause is harder to find though!

Could Gamification just be the application of Jidoka and Visualisation to non production / process work?

Parrot AR Drone In Flight

The Challenge In The Rising Use Of Drones

Over the past few weeks a number of the technology blogs and news sources I read regularly (particularly Harvey Nash Technology on Twitter) have been raising questions about the predicted rise in the use of ‘drones’ in the UK over the next 20 years, and inviting thoughts.

As both a technologist and a helicopter pilot, I’m in a good position to comment on that. However it’s not a topic which can be done justice in 140 characters on Twitter. I tried, failed, and so I thought I’d expand on it:


It’s not regulation.

I wasn’t quite accurate in my reply to Harvey Nash. It isn’t the regulatory framework which needs change; it’s the enforcement and awareness of it. The UK Civil Aviation Authority, who issue my pilot’s licence, are very clear on the rules for Unmanned Aircraft under 20kg in weight. Unless CAA permission has been granted, the pilot cannot:

  • Fly over or within 150m (492 ft) of a congested area
  • Fly over or within 150m (492 ft) of an organised open-air assembly of more than 1,000 persons
  • Fly within 50m (164 ft) of any vessel, vehicle or structure which is not under the control of the person in charge of the aircraft
  • Fly within 50m (164 ft) of any person.

These rules are, by CAA and Air Navigation Order standards, relatively simple and clear cut – if you don’t believe me, try and easily interpret Rule 5 (Low Flying) on Page 329 of CAP393.

The purpose of the rules is really quite clear – to keep small drones away from the risk of hitting people unless special permission has been granted by the CAA.  The reason is that 20kg falling from several hundred (or even tens) of feet will hurt. A lot.

Most commercially available drones don’t have the required redundancy to deal with a failure, don’t have the same certification as aircraft, and don’t even have guaranteed communications with the controller.. so the risk of something untoward happening is high and must be mitigated.

It should be noted we’re given much more flexibility to operate Unmanned Aerial Systems (UAS) ‘drones’ in the UK (and most of Europe as EASA takes over) than in other parts of the world. In the USA, for example, there is a complete ban on UAS for commercial purposes!

Awareness -vs- Consumerisation.

So, if the rules are pretty clear, why are we seeing headlines like this:

Drones flown in London and Liverpool despite CAA laws

UK’s first drone conviction will bankrupt me, says Cumbrian …

The answer, and the challenge, is awareness. The consumer can go and buy one of these drones relatively cheaply – under £300, and they’re cool. What gadget nerd wouldn’t want one? For a lot of us they are the work of the science fiction of our youths.

But they come with no warnings about about the legality of operating them. This advert for a Parrot AR Drone on contains only these safety warnings:


Amazon AR Drone Warnings.
The Only Warnings on Amazon AR Drone Advert

Where’s the big, bold, bright warning about legality of operating it?

We know that ignorance is no defence to breaking the law, but as unmanned aerial systems become more consumerised, and available, the manufacturers and retailers should surely draw to the attention of their customers that they run the risk of hurting people, and thus prosecution if they’re not careful about how they use their new toy!


I’m certainly no CAA-apologist (even they, under their new leadership, admit they need to improve in many areas). But they simply aren’t resourced to police these rules to any great extent. By virtue of cost and licensing they have managed to police and enforce manned flight rules to date.

They are under-resourced to police and enforce these rules across the whole of the UK, and unless somebody reports incidents they most likely won’t be detected and dangerous situations could become commonplace.

The UK Government requires that the CAA’s costs are met entirely from its charges on those whom it regulates. Unlike many other countries, there is no direct Government funding of the CAA’s work.

So, should the UAS / Drone manufacturers start to contribute to policing and regulating the safety of the devices they make money from selling? That seems fair to me, as someone who pays CAA fees!

Historically The CAA have brought few prosecutions, in line with the internationally established ‘Just Culture‘ which encourages reporting (so we can all learn) and seeks to only punish wilfully negligent acts. How will they manage this when unlicenced consumers are involved?

It’s my view that they need to educate, then prosecute, and then highlight prosecutions to raise awareness.

The Future.

I love aviation. I love technology & gadgets. I embrace consumerism. However, to harness the huge possibilities of drones / UAVs we need to find a way to make them work safely in society – before a single incident occurs which results in an outright ban.  

We could look to making them a licence-able aircraft, but those in aviation will attest that this will be costly, cumbersome and probably kill the industry and its undoubted benefits and enjoyment. We know that licenced commercial UAV operators exist, and use much more expensive, complicated, and safe aircraft – let them undertake the commercial work over populated areas.

But for domestic and hobby drone-flyers: the manufacturers need to step up to the mark with awareness. Very few people set out to deliberately break the law (and where they do the likelihood of being caught, and subsequent penalty should be a huge deterrent)… but they need to be aware of the rules.

Should we mandate the inclusion of warning notices in the boxes and on the packaging?

I think so… what do you think?

EU Map & Logo

Ready for GDPR? 5 Simple Steps!

According to Kroll Ontrack & Blancco, 4 in 5 IT managers are unaware of the upcoming changes to Data Protection regulations, The EU General Data Protection Regulation (GDPR).  This certainly means they can’t be preparing for it either… and unlike the Data Protection Act 1998 which it will supersede the penalties can be huge – 5% of global turnover or €100,000,000 (whichever is greater!).

What is GDPR?

The GDPR will be a common set of data protection rules across the European Union. Technically it will be an EU Regulation which will be implemented in law in all member states – the purpose being to harmonise data protection across the EU and (potentially) centralise policing for international companies.

At the moment it is still in draft form, but expectation is that although some changes have been made over its course through the European Parliament, it will be adopted in late 2014 or early 2015 and come into force in 2017 – allowing at least 2 years for companies to transition.

What’s different?

There are 5 main differences to the Data Protection Act in the UK

  1. Three New Rights For Data Subjects:  The Right To Be Forgotten, The Right To Data Portability, and The Right To Data Erasure.
  2. Mandatory Breach Notification. It becomes mandatory to report data breaches to the regulator within 72 hours, and data subjects must be notified if harm will occur.
  3. Explicit Consent. You must obtain explicit consent to hold the data from the data subject – opting in will become the norm.
  4. Penalties. As mentioned, the penalties rise significantly for negligent data breaches – rising to 5% of global turnover €100m for more serious breaches.
  5. The Data Protection Officer.  Public sector bodies and organisations processing over 5000 subjects data in 12 months must appoint a Data Protection Officer whose responsibility it will be to conduct risk assessments, analysis and to safeguard the data.

Don’t Panic – We’ve Got 2 Years!

Don’t Wait Either.

With such huge changes proposed there are the usual elements of the IT industry who either professing impending doom or who are suggesting that it won’t happen.  Neither seem likely to me – GDPR will happen in some form, likely to be close to the current drafts; and there is no impending doom.

There is at least two years until you have to be compliant, so there is plenty of time to start to integrate the new principles now so the change can be almost seamless by the time it becomes mandatory.  Data Security is becoming a large differentiating factor for consumers, so make sure you’re ahead of the game – I’ve got 5 simple steps you can start with now to make sure you’re ready come 2017:

5 Simple Steps to GDPR Compliance.

1.  Understand Where You Are.

Before you can start to think about how you’ll comply with GDPR you need to know what data you have now, where it is and how it is protected.  Undertake an audit of what you hold today, and what you’re likely to take on in the next few years.  Once you have this, assess how relevant it is – do you need to keep it?  Does it have value?   Of course you should be destroying data when it becomes unneeded under the DPA’s Principle 5… but check and delete.  If you don’t have it, you can’t lose it!

2.  Obtain Consent & Engage.

Now you know whose data you’ve got, remember that you’ll need to have their consent to hold it in a post-GDPR world.  You can use the next two years to obtain that consent when customers and data subjects contact you naturally.  Develop the processes for obtaining consent now – a simple ‘Do you mind if we hold your details?’ in a contact centre conversation will suffice.  Don’t try and obtain in surreptitiously, in small print or otherwise, people won’t take kindly and it’s not in the spirit of the regulations:  would you want your data held like this?

If you start asking now, by 2017 a huge proportion of the relevant data you hold (remember to destroy ageing data!) will be opted in and you won’t be one of the organisations embarrassingly seeking retrospective consent.

3.  Update Your Policies.

As the GDPR will enshrine a right for data subjects to see your privacy policy, this is a great opportunity to update it and make it obviously available – put it on your website.  You can use the results of your audit from step 1 to make sure it’s accurate too.  Ensure it is written in plain English and is easy to understand.

Update your Breach Policy & Detection.  With stiff penalties for failing to report breaches in a much shorter time scale than you might be used to working, it’s important that your process internally means that any breach, no matter how small, gets to your Data Protection Officer (or equivalent for smaller companies) promptly and you have a plan on how to react.  Ensure that everyone in the business knows the importance of reporting data loss promptly – include it in your recurrent data protection training from now onwards.

4.  Identify Responsible Person.

If you’re a public body, or process data on more than 5,000 subjects then you’ll need someone to fulfil the mandatory role of ‘Data Protection Officer,’ but even if you’re smaller you’ll want to identify one person in your organisation who can ensure you’re compliant.  Someone with experience of data protection principles and a good view of the company will be ideally place.  Resist the temptation to appoint someone from Finance or Legal teams unless they’ve genuinely got a good understanding of the data identified in point 1.

Use the time between now and implementation to make sure that they have conducted relevant Data Risk Assessments and mitigated risk wherever possible.

Then, importantly, empower this person – get senior management or the board involved to ensure that everyone knows about the changes and the importance of them (there’s €100m at stake here!).

5. ‘Change-In’ The GDPR now.

Come 2017, most companies won’t be operating exactly the same systems, in the same state, as they are today. This means the natural business changes which keep driving them forward can be used to ensure you’re designing in GDPR principles from today onwards.  Simple ideas and some best practice:

  • Select a data protection framework now (COBIT, NIST or ISO) and judge all new changes against it.
  • Build in monitoring and reporting tools which report loss.
  • Obfuscate and encrypt data wherever possible, especially on test systems.
  • Adopt ‘Opt In,’ ‘Data Portability’ and ‘Data Erasure’ functions in new software, now.
  • Audit your systems – consider getting external auditors to both probe your defences and your processes.

A series of small changes over the next two-to-three years will mean you’re ready for GDPR when it becomes law.

If you’re in doubt at any point, ask yourself:

Would I want my data held and processed like this?  Does this seem fair?

Internet Of Things Word Cloud

Manufacturing, The Internet Of Things, and Security.

During the SAP Radio ‘Future Of Business’ podcast this week a healthy debate emerged on how manufacturing is, will and should adopt Internet Of Things technologies.  The conclusion seems to have been drawn that European & North American Manufacturers aren’t ready for this leap, but that consumer adoption will drive it.  Computer Weekly best summarises the podcast in its article ‘Manufacturing Industry ‘Not ready for IoT’ says SAP’.

I agree – manufacturing isn’t ready for The Internet Of Things.

But not for the reasons highlighted in the podcast or the article – and I agree that the benefits of IoT adoption for manufacturers (especially larger scale) are potentially enormous, perhaps nothing short of a second industrial revolution if implemented properly.

Age of Plant.

The article and podcast highlight that the average age of industrial machinery has increased lately to the highest since 1938 (in itself a scary statistic), and with investment only running at 3% per annum it will take considerable time to replace this ageing kit.  The problem with this is predicated on older equipment not being compatible with the IoT – something which isn’t always true.

Older plant and machinery can generally be retrofitted with interfaces to allow it to be part of the IoT, and a good industrial integrator will do this relatively cheaply; so while old plant makes it more complicated it’s not the biggest barrier to manufacturing adoption of IoT, proving both benefit and security are.


Manufacturers, by virtue of what they do, tend to work in a very pragmatic, demonstrable way: they measure everything they can to look for improvements. Quick, easy, big wins are rare in manufacturing these days – it’s minor, incremental (often CI or Lean driven) changes which, when repeated, make gains for manufacturers.  Plant Managers want to see real benefit for deploying a technology, and plant maintenance teams are often skeptical of ‘high brow’ IT concepts; it’s simply not a world they operate comfortably in.

Talking about “the  interconnection of uniquely identifiable embedded computing devices within the existing internet infrastructure is likely to garner an awful lot of blank faces with manufacturing managers… but if you talk about the ability to monitor, control and report on plant from anywhere on the planet, then they are interested.  If you can talk of predicting plant failure by combining data from two devices across a plant, or of automatically or remotely controlling equipment based on trends and analysis then you’re demonstrating benefit. It comes down to money.

As an IT industry we need to stop talking our own language, and talk that of the target audience if we want to ‘sell’ our vision and allow business to benefit from the huge possibilities the IoT offers.


The biggest problem though will be similar to that faced by the now almost ubiquitous ‘cloud computing’ in its infancy: Security.  The cloud had to overcome both security and privacy concerns to gain trust before people would start allowing their data to be held and processed on servers they didn’t own and couldn’t physically touch. In many regards there is a similarity here to how SAP see IoT being driven into manufacturing: consumers did cloud first!

Manufacturers guard their plant as valuably as they do personal data.  It’s their bread and butter… without it nothing leaves the door and cash doesn’t come through the front door.  Worse still, if the production lines stop then you have mounting costs of a stagnant workforce and supply chain problems further down the line.  This is why manufacturers protect their plant (and probably a reason a lot of it is old – it works and the risks are known!).

I know of major manufacturers who separate all plant floor manufacturing equipment entirely, some physically and some logically, from their main office LAN to ensure that production continues, having been bitten by IT problems stopping production before. Convincing them to allow devices to talk directly to the internet is going to be a tough ask, especially with the recent vulnerability exploits in SCADA fresh in their minds.

What happens if an IoT connected PLC or device loses connectivity?
How do we ensure only authorised access to the data and control of an IoT-enabled plant?
Can we ‘pull the plug’ and continue operating if something bad happens?
Where is the redundancy?

These are all questions manufacturers will ask before they allow IoT devices into their day to day world, and as an industry we need to have convincing answers to all of them.

We’ll have to demonstrate that those devices are safe, secure and will deliver real benefit… we should concentrate on finding a way of doing this (perhaps something like the Cloud Security Alliance?) rather than berate the age of the kit manufacturers are using to make their living, because the benefits to manufacturing are huge.


HM Revenue & Customs Logo

HMRC Anonymous Data? Be Careful…

This weekend we awoke to hear of plans by Her Majesty’s Revenue & Customs (the UK tax authority, akin to the IRS in the USA, but with more power) to start selling anonymised tax data where doing so “would generate clear public benefits, and where there are robust safeguards in place.”

Although there is no formal announcement on the HMRC news section, you can see some of the press coverage on The BBCThe Guardian or The Telegraph.

You’ll see that one of the Government’s own MPs has described the plan as “borderline insane,” a tactic no doubt employed to garner some headlines and ensure that his opposition is well known; especially given the likely public reaction and HMRCs not-all-to-great record on data protection. But is it that insane?

Setting aside the plans to sell the data, and the slightly more nuanced debate that the sale of public data brings (and of course the OpenData / movement) I’d like to concentrate on the anonymisation of the data which HMRC might be proposing to use, and just how flawed that can be in the age of Big Data and Cloud Computing.

It is likely the proponents of the HMRC plan will assure the general public that their data won’t be identifiable and the principle of tax-payer confidentiality will be upheld… Well, it turns out that’s really hard to do!


Anonymous MaleRe-identification is the process of taking a dataset which is believed to have been anonymised of any personally identifiable information and by means of processing or data-matching re-establishing the personally identifiable information (PII) with some level of confidence.

In practice this generally means combining other publicly available information with the ‘anonymised’ information in a data-matching / ‘jigsawing’ exercise.  Historically this was hard, processor intensive work which could take days or weeks and thus was usually cost or time prohibitive – even with just one data set to combine.

However, the advances in ‘Big Data’ over recent years, combined with the scalable power of cloud computing, mean that multiple data sets could be combined in a matter of moments – making the re-identification of data not only possible but also practicable.

An often-quoted example of this process is when Netflix first released some anonymised usage data as part of the Netflix Prize was combined with IMDB reviews (and thus IMDB user names). It was possible to identify the user who had watched the Netflix movie, then link that to their IMDB review based on the time – a seemingly innocuous data point in the Netflix set. By then reversing this process it was possible to take the IMDB reviews and user names and come up with a complete listing of films watched by each user. More information on that here.

This was with two data sources – IMDB and Netflix Anonymised Data. Imagine if the researchers here had then added in social media data, perhaps by looking for similar user names, or perhaps looking for posts containing the films name around the correct time – something not that complicated to do with Big Data and Cloud tech. It would have been comparatively easy to go from anonymised film usage data to a picture, name and social media details of the person watching it, along with their recent film history.

Just think of the consequences if the same happened with your tax data!

What Do We Do?

Of course, we all want open data, don’t we? But if we get scared by the possibilities like those above, we’d never release any data. A similar recent debate in the UK formed around government plans to allow research based on NHS medical records – Fundamentally few people would disagree with using existing medical knowledge to try and improve care for the future, but medicine is complicated and you need a lot of data about an individual person to do that reliably. So, anonymised data would help, and surely we all want better health for our future generations (and maybe even us!).

Obviously we have to be careful HOW we anonymise data. The devil is in the detail. As data professionals we can take obvious steps to anonymise data effectively against the threats we know about at the time we anonymise it. We also look to anonymise data down to the lowest level needed to provide meaningful data for research & development, social good etc – perhaps by aggregating data into groups (for example postcode area SW1A rather than SW1A 2, or even SW1A 2AA – Downing Street).

The problem comes, as with most information security, that there will always be someone with more knowledge, more skills or a stronger, often nefarious, desire to break the defences put in place to protect that information. This is the “motivated intruder” attack. It is our job to protect against this as best we can when we anonymise data – it’s a higher standard than “can a reasonable person link data.”

Motivated Intruder Test

So, when anonymising our tax data, HMRC must think of the motivated intruder. In fact, The Information Comissioner’s Office details this exceptionally well in the Code Of Conduct for Anonymisation. HMRC will have to think about some, all, and hopefully more than the following:

  • What other information is out there?
  • What other information could be “jigsawed” with the tax information?
  • What information they release:
    • Can they aggregate without losing utility of the data?
    • What data points are in it which may help to identify a person?
    • What could the data be used for?
  • How difficult (and therefore likely) is it to use this data?

Some of these will be very hard to answer, or even unknown to HMRC. They are the realm of specialists who devote their whole professional life to this sort of question. It’s just like any other form of Information Security – you don’t know what you don’t yet… so best ask someone who does nothing else.  Actually, ask two people – or better still 20.

When we launch a new website, or service, or even maintain an existing one, the prudent amongst us employ the services of at least one (sometimes many) security consultancies to “penetration test” them. They use all the techniques they know how to try and break in / break the service. Anonymised data should be no different – HMRC must test their data sets with as many 3rd parties as possible and they should make those results public to instill confidence.

The publification of anonymised tax records could be very useful for so many aspects of life, some commercial, some social – but the potential harm of doing it incorrectly is huge and the risk of doing so is high. HMRC would be wise to tread very carefully and walk very slowly into this one.

Return After Hiatus.

It’s been over two years since I posted a blog entry on here and for the bulk of that time the blog itself was taken down from the internet (yes I’m aware nothing really disappears).

I took it down back in 2011 as I engaged in a lengthy legal battle with my former partner over custody of and access to my son.  Everything possible was being dragged up and thrown at me, in a spate of underhand and unnecessary tactics.  It wasn’t a happy period of time – no two reasonable people should resort to what we both did.  I won’t be dwelling too much on my little man, or my personal life, now the blog is back; but I thought I should explain why it went away.

Now I’m Back….

I should probably explain the inspiration for being back, seeing as I’ve explained the rationale for being away.  It is, as described:  Inspiration.  I do some work with the Duracell bunny that is Warwick Tweetup, and his partner in crime Jo’s Correctional Facility – and the energy from both of them is contagious.

If you take that and combine it with the fact I feel I have something useful to add these days, and take inspiration in content form from some of the blogs I enjoy reading (particularly Maria Langer’s ‘An Eclectic Mind’ – a great mix of Aviation and real life from the USA!) – I wanted to get back on with blogging.

As you’d expect, an awful lot has changed in the intervening two years; including my life and my little bit of the world.  I’m older, and hopefully a little wiser – I’m certainly a little more reflective before reacting to anything.  One little snippet I learnt along the way:

“Will it matter in a year?  If not, then it probably doesn’t matter now.”

With that mantra in mind, I intend to start blogging again probably on a more professional and business based basis than before; although still with a heavy hint of aviation!

For now I’ve just restored the old site, as was and upgraded WordPress – but over the next month or so I am going to tidy it up (looks and content), and ensure everything is linked properly to my social media bits and pieces, and then try and think of something interesting which you can all read!

It’s good to be back.

Be The Solution – Corporate Tweetings

I normally have a strong distaste for “motivational” expressions, witticisms, insights and the like; especially those sent down from above in the form of managerial nonsense.  I distrust them and often wonder who on Earth actually reads, believes and tries to live by them.

However, this week I have been reminded of the importance of one such little gem.  The gist of it is

Be The Solution, Not The Problem.

This maxim, this veritable pearl of wisdom applies especially to customer service environments, or when dealing with a customers problem.  It’s hit me twice this week; once including a good use of Twitter!


I recently ordered Sky HD for my apartment, but the engineer was unable to put a dish up on the gable end of my building (right next to an existing dish) because Sky have changed their rules on laying out long ladders on the ground and “flipping them” up against the wall. However I can get a single feed from a communal aerial socket.  This leaves me with half working Sky+, I can only watch & record one channel at a time.

I like Sky and prefer it to Virgin Media and am minded to keep it, but it feels a little unfair to be paying full price and only getting half the service.  I emailed Sky customer services and suggested that if they gave me my HD package for free, or at least reduced, I’d keep my subscription rather than cancelling it as I am entitled to do in my cooling off period.

To me this makes good sense for Sky.  They’ve spent some money sending out an engineer and providing equipment.  At full price they would get £35.25 from me a month.  If they gave me the HD pack for free this would be £25.  If I cancel they get £0 and have lost money installing it.

The response I got was clearly from a foreign call centre and simply restated my problems with additional explanation of how Sky+ worked.  No solution, or even a direct answer to question.  I emailed back and asked for an answer from someone who understood what I was saying.

I got a flat refusal.  It was “Sorry, no can do, cancel if you want.


This morning I saw a friend beating up Sky Customer Services (@skyhelpteam) on Twitter, so I thought I’d have a moan and ask if they wanted to retain my custom; expecting the same “No, cancel if you want; we’re not giving you discount.”

Not what I got.   They didn’t address the issue of discount, but they have offered to get another specialist engineer out to me to try and get a dish up.

It’s really refreshing when large companies with millions of customers take the time to use Twitter in such a way.  It must be more efficient for them too, as I haven’t tied up a call centre for 10/15 minutes explaining; the 140 character limit keeps it succinct.

Who knows if he or she will be able to get the dish up, but I am very impressed with Sky for at least trying to offer a solution.  I really hope it works, as I’d rather have it working than discount… but if it doesn’t and I can’t get discount I’m a little more likely to stay with them because of one little act of “can do.”

Apply some “can do” to your life today!