At the weekend, in an interview on Radio 4 the ONS’s chief economist Joe Grice said that the ‘did it/didn’t it?’ debate around the UK and double dip recessions was “counter productive” and that we’d all do better to worry less about shorter-term figures “whether one particular quarter was up a smidgen or down a smidgen” and instead stay focused on the big picture  and the general direction of trends in the economy over time.

New figures suggest that the so-called double dip recession may never have happened and hearing that figures have been revised, the public (encouraged by the media) can be quick to assume mistakes have been made. Not so. Statisticians regularly fine-tune figures as new statistics i.e. new information becomes available. This may result in a revised judgement. In this case, new construction sector data showed that it had contracted less than originally thought in early 2012. Next month’s statistics will make things clearer but the revision seems enough to believe that the overall economy has narrowly escaped a second recession.

It seems that the less confident we are with numbers, the more over-excited we can get about detail and the more likely we are to stop seeing the big picture. This can and does happen in management more generally – whether in business/industry. We have all been in meetings where faced with a series of graphs concerning performance, managers zoom in on the one figure which stands out as being out of kilter with expectation. Sales of one particular item down 30% this week/month rings alarm bells around the Board. Colleagues are commissioned  to investigate and they, in turn, involve their teams in the endeavour to fathom the unknown whys and wherefores. Before you know it the figures are up again but hours of  organisational time has been spent on that one figure. They would do much better to keep their eye on the big picture  – numbers go up, numbers go down – statisticians don’t rely on point estimates to make an inference. It’s good statistical practice to spot outliers, but, generally, the overall trend  is more important and what really matters.

The interview focused quite a bit on the distinction between predicting and forecasting. Nate encouraged us not to trust anybody who is too confident in their predictions, especially long-term predictions. Predicting the economy more than 3-6 months ahead is ”nearly impossible”.  Political pundits have been found to have no more than a 50-50 chance of turning out to be correct, most do no better at predicting election results, than if they had just flipped a coin.

On the whole, statisticians predict but they don’t mean predicting in the everyday meaning of the term. They deal in uncertainty and illogical though it may at first appear, there is more value and more accuracy in a forecast – of different possible futures – that says something about the underlying conditions of uncertainty in identified trends, than in attempting to predict exactly what specifically will happen, how and when.

Think about it when it comes to a political poll, you are not polling everybody just a sample of a population or group. Similarly with a survey, you will be looking at a small portion(s) of a larger population to see what the trend might be for that larger population.  It’s only to be expected that there will be some degree of error or uncertainty in your calculation.

Statisticians include a measure of uncertainty in their predictions –  typically captured by the ‘margin of error’ in a confidence interval. The more precise an estimate or prediction is, the smaller the margin of error. So an opinion poll based on a sample of 1000 people may have a margin of error of +/- 3%.  It would require a sample of 9,000 people to reduce the margin of error to +/-1%.

Confidence intervals and their associated margins of error are ways to mark the uncertainty attached to a statistical prediction. They make it clear that there is no such thing as a certain bet at least not until the race is over.

The world and society we live in are complex and very unpredictable. It what’s makes life exciting and grappling with – making sense of –  that unpredictability is what inspires and motivates statisticians.

A government, down in the polling dumps, gets anxious to extol its policies. It seizes eagerly on any sign they are working. Temptation looms, in the shape of exaggerating or, as some would say, actively misinterpreting data.

The Department of Work and Pensions is in the firing line over statements made about the flow of claimants off disability benefit, in anticipation of the new more rigorous testing regime that is coming in as Personal Independence Payments (PIP) replace the Disability Living Allowance.

Ministers want good news; civil servants, especially those in Whitehall press offices, feel obliged to varnish and sometimes hype up the language used in press releases. The result: what began and remains accurate data (the numbers themselves are sound) become the basis for a contentious claim about process and even, at its strongest, an assertion about causality.

People have stopped claiming benefits, says the minister. That’s evidence they fear the new test (and by implication were malingerers). But there’s a constant flow on and off benefit; are recent movements indicating some deeper change specifically to do with the arrival of PIP? Definitely not, say the government’s critics, who include the Department of Work and Pension’s own former chief economist, now director of the National Institute for Economic and Social Research, Jonathan Portes.

His post in the Guardian is scathing. Statements made, he asserts, are untrue.

That’s a serious charge. But who’s to adjudicate it? It takes stamina and a fair understanding of the benefits system to follow the argument. The UK Statistics Authority would be the obvious permanently-available custodian of good statistics.  In a statement they said “The Statistics Authority has received representations regarding the recent publication by the Department for Work and Pensions of ad hoc statistics about those identified as potentially impacted by the benefit cap. The Authority is reviewing the matter and will respond publicly in due course.”

But the danger is that, even when rebuttal takes place, there’s a residual stain, and the public’s disaffection with ‘the system’ grows.

Portes says ‘public cynicism about official statistics is often misplaced – restrictions on what government can do with official data are an unsung but essential element in modern democratic governance. When government seeks to get around these limitations by, in effect, simply making things up, this is not just an issue for geeks, wonks and pedants – it’s an issue for everyone’. He’s right.

A causal link can’t definitively be made with the Wakefield case in 1998 and the way, then and since certain media – notoriously the Daily Mail –  have campaigned against immunization. But rates of vaccination did slip, probably because parents had read and believed reports linking MMR to autism.  There’s more here about the disease.

It’s a sorry tale. Some people put more credence in anti-science commentators more interested in sensation than public enlightenment than they did in their own GPs or in trusted websites such as NHS Choices. The work of scientists that consistently found no link between the MMR vaccine and autism has been ignored.

What the episode proves, yet again, is how important basic statistical literacy is – meaning a broader public understanding of the fundamentals of risk appraisal, and a capacity to take significance from numbers. Whether it’s parents, editors or commentators ignorance matters: real harm results from a failure to see what the stats are saying. The trouble is, even if statistical education were better and more journalists given training in dealing with data and numbers, some people would ignore evidence. And some others would follow ideological agenda, even at the expense of children’s comfort and wellbeing.

Doctors will complain their training curriculum is already crowded – they don’t just have to conquer medicine but acquire personal, business and (if they are to play a part in the brave new world created by the Health and Social Care Act 2012) learn to be managers too. But without stats, how can they know whether a therapy or an intervention (or the performance of surgical colleagues) is effective and safe?

Not that medical statistics are easy, either in terms of data collection or interpretation. In an essay in the London Review of Books,  Paul Taylor of University College London shows how hard it is to get reliable data on performance, despite the claims made by commercial data companies. The latter do important work, for example Dr Foster on hospital standardized mortality ratios but their business is selling information and they don’t necessarily have a first-line interest in how it’s interpreted.

As for patients, relatives and those who run the NHS, they too need to acquire more statistical and data literacy, especially around comparing performance. Government ministers, and patients, sometimes think you can boil performance down as if clinics were football clubs, ranked according to their points score. Jeremy Hunt recently asked a compliant thinktank, the Nuffield Trust, to envision his version of the Premiership.

It’s much more difficult than that. But the way forward has to be more patients, their relatives, their nurses and doctors, and health managers understanding a little more about data and distributions. Stats can be a matter of life and death.

Budgets are ‘political’ and interpretation of the numbers they present will always be ‘pluralist’, the RSS getstats panel audience was told - the event taking place a day before Chancellor George Osborne did his best to prove the point.

But recognising political reality did not exonerate government, whether ministers, statisticians or accountants. Too often statistics offered the public are incomplete, misleading, impenetrable. The Treasury’s own annual report and accounts were  ‘obfuscating’, according to Margaret Hodge MP, chair of the Commons Public Accounts Committee, and a member of our panel.

The panel, chaired by Paul Lewis the BBC radio presenter and financial journalist, agreed that Budget documentation has been improving. A close reader can now extract useful comparisons between one time period and another. Several panel members praised the detail now available, albeit after some delay, in the Whole of Government Accounts published annually by the Treasury. The panel had words of praise, too, for the Office of Budgetary Responsibility, set up as an independent check on fiscal forecasts by the Treasury.

But more could be done to make the government’s financial paperwork intelligible and reliable. Panel member Tony Travers, the public finance expert from the London School of Economics, remained to be convinced that year-to-year comparisons could safely be made from much of the material put out by Whitehall departments. Too often they changed definitions or interrupted series, making it impossible to establish what might be changing over time.

Budget documents might show aggregates to be stable but what was happening underneath? Figures were being switched from current to capital spending; government work (Network Rail is an example) gets reclassified as private sector, and so on. ‘It’s often difficult to read tables consistently, he said.

From the journalist’s perspective Paul Lewis regretted the often ‘rapid-fire delivery’ of Budget speeches by chancellors, obscuring what was being presented. Inflation measurement had become confusing, with a set of competing measures: which is which, he asked.

Even the powerful National Audit Office had to labour to get at the true figures, Michael Kell, its chief economist said. Often the sheer detail of government accounting made it hard to get at the truth, to find out what it cost to deliver a service. ‘Getting reliable information can be difficult’ and the NAO had to delve deep. But the reason wasn’t deliberate attempts by civil servants to cover. Officials, including government statisticians, were often overwhelmed and distracted by having to respond to immediate demands from ministers.

The panel agreed the public needed to be better equipped to handle numbers – giving an implicit boost to our RSS getstats campaign to improve statistical literacy.

None the less, government could do more. Margaret Hodge said sometimes ministers thought they could just ‘dump’ statistics, and they failed to put enough effort into explanation and contextualisation. Government information had to be relevant. And of course government was often selective about the statistics it released. Panel member Paul Moxey of the Association of Chartered Certified Accountants noted government had its own priorities about which information to collect, and they did not always conform with the public interest

__________________________

This event was organised by the Royal Statistical Society in association with the Association of Chartered Certified Accountants (ACCA), the House of Commons Library and the All Party Parliamentary Group on Statistics.

For more information on getstats in parliament, please click here.

What a writer  calls ‘algorithmic detective work’  — the use of common Big Data techniques – could allow Facebook and similar operations to work out that if you like certain films or express certain views you are more likely to have this or that sexual orientation or religious beliefs.

The culprit, it turns out, is stats. That’s to say statistical techniques allowing inferences to be made from the clues  Facebook users offer through their likes. Look at enough examples of likes, combined with other fragments of data and reliable associations can be made to gender, orientation and belief and you can make reliable associations.

In a study researchers had been able to infer Facebook users’ race, IQ, sexuality, substance use, personality or political views – even where they had chosen not to reveal such information.

One of its authors, Michal Kosinski of the Cambridge University psychometric centre, professed himself spooked at the finding, which was published in the Proceedings of the National Academy of Sciences. ‘Everyone carries around their Facebook likes, their browsing history and their search history, trusting corporations that it will be used to predict their movies or music tastes.

‘But if  you ask about governments, I am not sure people would like them to predict things like religion or sexuality, especially in less peaceful or illiberal countries.’

They said they were able to predict whether men were homosexual with 88 per cent accuracy by their likes of Facebook pages on human rights and even musicals.

Looking forward to his new role, Robert said “The campaign is a great initiative. Improving public understanding of numbers, quantitative data and statistics is a huge challenge – for schools and colleges, journalists and bloggers, parliamentarians and commentators and civil servants and business people too. The campaign has made a great start”.

Prior to taking up his current post at the OBR in 2010, Robert was the Director of the Institute for Fiscal Studies for several years, before then an advisor to senior management at the IMF and a journalist on the Independent/Independent on Sunday. For more on Robert including his bio and statement of support for the aims of the campaign, see getstats Campaign Board.

Robert takes up the mantle from John Pullinger, Librarian at the House of Commons and, since January 2013, President of the Royal Statistical Society.

In passing the chairmanship to Robert and looking back on the campaign so far, John noted that the statistical literacy landscape is changing for the better. getstats has initiated some of these changes, some have been initiated by partners and getstats has helped to support their progress.

For example, good relationships have been built with MPs, peers and their staff through getstats in parliament’s seminars and training. Impact in the media has increased via training for many more journalists and the increased success of RSS awards for excellence in journalism. Campaign messages are reaching the education community and decision makers through lobbying for the recognition of statistical skills as part of a ‘whole’ education for every young person. More recently, new inroads have been made with employers, the police, the voluntary sector and others.

There is real momentum to the campaign and a sound platform from which to continue to build the statistical understanding, know-how and skills society needs (and is increasingly realising it needs.)

Oliver Letwin, the Cabinet Office minister said that “Government must become more rational”, hence the new ‘What Works’ centres which will draw on research to test whether policies on crime, local economic growth, ageing, health and schools are valid.

There will be:

- a Crime Reduction Centre, overseen by the College of Policing, which will tender for academic partners with the ESRC in Summer 2013

- a Centre for Local Economic Growth, which will receive £1m over 3 years, partly from the Economics and Social Research Council (ESRC)

- a Better Ageing Centre which will receive Big Lottery Funding (tbc)

- an Early Intervention Foundation which will receive £3.5m of government funding over the next 2 years.

The Cabinet Office will oversee a wider ‘What Works’ network comprising the four new centres and the National Institute for Health and Clinical Evidence, which already advises the NHS on the cost effectiveness and scientific standing of new drugs and treatments and the Sutton Trust’s Education Endowment Foundation, which assesses education policy.

The Network will be chaired by a national adviser (to be appointed) who will advise ministers and also explore the merits of creating a role of government chief social scientist.

At the launch Paul Boyle, Chief Executive of the ESRC, said the centres would put a premium on quantitative skills. RSS getstats has been helping the ESRC, Nuffield Foundation, British Academy and the Higher Education Funding Council launch initiatives to improve social science students’ grasp of statistics.

Geoff Mulgan, Chief Executive of Nesta (the National Endowment for Science, Technology and the Arts) which hosted the launch, said “Both public service professionals and policy makers are becoming more aware that the effectiveness of what they do depends on evidence.”.

Against a backdrop of the potential challenge evidence can pose to ministers and commentators -  if, for example, it shows that pet schemes or cherished ideas just don’t work - it’s good to see that decision makers in government are looking to ensure the validity of existing and future government policies and to measure their effectiveness.

For more on evidence and policy making, see ‘More Research, more numbers, more evidence-informed policy’, a recent article on this site.

Dirk Helbing was speaking at a session entitled “Predictability: from physical to data sciences”. This was an opportunity for participating scientists to share ways in which they have applied statistical methodologies they usually use in the physical sciences to issues which are more ‘societal’ in nature. Examples stretched from use of Twitter data to accurately predict where a person is at any moment of each day, to use of social network data in identifying the tipping point at which opinions held by a minority of committed individuals influence the majority view (essentially looking at how new social movements develop) through to reducing travel time across an entire road system by analysing mobile phone and GIS (Geographical Information Systems) data.

They are not alone in working in this way. With technological advances, greater interconnectivity and the availability of data in ever larger quantities, subjects across the physical and social sciences are becoming more quantitative and predictive in nature. Working across physical science and social science boundaries is increasingly common, new fields such as ’integrated systems biology’ and ‘computational social science’ reflect this.

The underlying thinking is that if we can model the weather, flight paths and factory production lines then we can model human networks and group behaviour. We can maybe model society itself.

With their eye on the big picture, Dr Helbing and multidisciplinary colleagues are collaborating on FuturICT, a 10-year, 1 billion EUR programme which, starting in 2013, is set to explore social and economic life on earth to create a huge computer simulation intended to simulate the interactions of all aspects of social and physical processes on the planet. This open resource will be available to us all and particularly targeted at policy and decision makers. The simulation will make clear the conditions and mechanisms underpinning systemic instabilities in areas as diverse as finance, security, health, the environment and crime. It is hoped that knowing why and being able to see how global crises and social breakdown happen, will mean that we will be able to prevent or mitigate them.

Modelling so many complex matters will take time but in the future, we should be able to use tools to predict collective social phenomena as confidently as we predict physical phenonema such as the weather now.

For more information on the AAAS session see The Economist article ‘Dr Seldon, I presume’ and the relevant pages of the AAAS annual (2013) meeting website.