1. AIA Vitality Life Insurance
2. Data Driven Healthcare Quality Markets
3. Abu Dhabi Health Authority – Weqaya
Vitality Institute Commission – Recommendation 3 http://thevitalityinstitute.org/commission/create-markets-for-health/
A very good article on the ins and outs of big data.
http://www.ft.com/intl/cms/s/2/21a6e7d8-b479-11e3-a09a-00144feabdc0.html#axzz3AVZ1Wv00
March 28, 2014 11:38 am
©Ed NacionalF
ive years ago, a team of researchers from Google announced a remarkable achievement in one of the world’s top scientific journals, Nature. Without needing the results of a single medical check-up, they were nevertheless able to track the spread of influenza across the US. What’s more, they could do it more quickly than the Centers for Disease Control and Prevention (CDC). Google’s tracking had only a day’s delay, compared with the week or more it took for the CDC to assemble a picture based on reports from doctors’ surgeries. Google was faster because it was tracking the outbreak by finding a correlation between what people searched for online and whether they had flu symptoms.
Not only was “Google Flu Trends” quick, accurate and cheap, it was theory-free. Google’s engineers didn’t bother to develop a hypothesis about what search terms – “flu symptoms” or “pharmacies near me” – might be correlated with the spread of the disease itself. The Google team just took their top 50 million search terms and let the algorithms do the work.
FirstFT is our new essential daily email briefing of the best stories from across the web
The success of Google Flu Trends became emblematic of the hot new trend in business, technology and science: “Big Data”. What, excited journalists asked, can science learn from Google?
As with so many buzzwords, “big data” is a vague term, often thrown around by people with something to sell. Some emphasise the sheer scale of the data sets that now exist – the Large Hadron Collider’s computers, for example, store 15 petabytes a year of data, equivalent to about 15,000 years’ worth of your favourite music.
But the “big data” that interests many companies is what we might call “found data”, the digital exhaust of web searches, credit card payments and mobiles pinging the nearest phone mast. Google Flu Trends was built on found data and it’s this sort of data that interests me here. Such data sets can be even bigger than the LHC data – Facebook’s is – but just as noteworthy is the fact that they are cheap to collect relative to their size, they are a messy collage of datapoints collected for disparate purposes and they can be updated in real time. As our communication, leisure and commerce have moved to the internet and the internet has moved into our phones, our cars and even our glasses, life can be recorded and quantified in a way that would have been hard to imagine just a decade ago.
Cheerleaders for big data have made four exciting claims, each one reflected in the success of Google Flu Trends: that data analysis produces uncannily accurate results; that every single data point can be captured, making old statistical sampling techniques obsolete; that it is passé to fret about what causes what, because statistical correlation tells us what we need to know; and that scientific or statistical models aren’t needed because, to quote “The End of Theory”, a provocative essay published in Wired in 2008, “with enough data, the numbers speak for themselves”.
©Ed NacionalUnfortunately, these four articles of faith are at best optimistic oversimplifications. At worst, according to David Spiegelhalter, Winton Professor of the Public Understanding of Risk at Cambridge university, they can be “complete bollocks. Absolute nonsense.”
Found data underpin the new internet economy as companies such as Google, Facebook and Amazon seek new ways to understand our lives through our data exhaust. Since Edward Snowden’s leaks about the scale and scope of US electronic surveillance it has become apparent that security services are just as fascinated with what they might learn from our data exhaust, too.
Consultants urge the data-naive to wise up to the potential of big data. A recent report from the McKinsey Global Institute reckoned that the US healthcare system could save $300bn a year – $1,000 per American – through better integration and analysis of the data produced by everything from clinical trials to health insurance transactions to smart running shoes.
But while big data promise much to scientists, entrepreneurs and governments, they are doomed to disappoint us if we ignore some very familiar statistical lessons.
“There are a lot of small data problems that occur in big data,” says Spiegelhalter. “They don’t disappear because you’ve got lots of the stuff. They get worse.”
. . .
Four years after the original Nature paper was published, Nature News had sad tidings to convey: the latest flu outbreak had claimed an unexpected victim: Google Flu Trends. After reliably providing a swift and accurate account of flu outbreaks for several winters, the theory-free, data-rich model had lost its nose for where flu was going. Google’s model pointed to a severe outbreak but when the slow-and-steady data from the CDC arrived, they showed that Google’s estimates of the spread of flu-like illnesses were overstated by almost a factor of two.
The problem was that Google did not know – could not begin to know – what linked the search terms with the spread of flu. Google’s engineers weren’t trying to figure out what caused what. They were merely finding statistical patterns in the data. They cared about correlation rather than causation. This is common in big data analysis. Figuring out what causes what is hard (impossible, some say). Figuring out what is correlated with what is much cheaper and easier. That is why, according to Viktor Mayer-Schönberger and Kenneth Cukier’s book, Big Data, “causality won’t be discarded, but it is being knocked off its pedestal as the primary fountain of meaning”.
But a theory-free analysis of mere correlations is inevitably fragile. If you have no idea what is behind a correlation, you have no idea what might cause that correlation to break down. One explanation of the Flu Trends failure is that the news was full of scary stories about flu in December 2012 and that these stories provoked internet searches by people who were healthy. Another possible explanation is that Google’s own search algorithm moved the goalposts when it began automatically suggesting diagnoses when people entered medical symptoms.
©Ed NacionalGoogle Flu Trends will bounce back, recalibrated with fresh data – and rightly so. There are many reasons to be excited about the broader opportunities offered to us by the ease with which we can gather and analyse vast data sets. But unless we learn the lessons of this episode, we will find ourselves repeating it.
Statisticians have spent the past 200 years figuring out what traps lie in wait when we try to understand the world through data. The data are bigger, faster and cheaper these days – but we must not pretend that the traps have all been made safe. They have not.
. . .
In 1936, the Republican Alfred Landon stood for election against President Franklin Delano Roosevelt. The respected magazine, The Literary Digest, shouldered the responsibility of forecasting the result. It conducted a postal opinion poll of astonishing ambition, with the aim of reaching 10 million people, a quarter of the electorate. The deluge of mailed-in replies can hardly be imagined but the Digest seemed to be relishing the scale of the task. In late August it reported, “Next week, the first answers from these ten million will begin the incoming tide of marked ballots, to be triple-checked, verified, five-times cross-classified and totalled.”
After tabulating an astonishing 2.4 million returns as they flowed in over two months, The Literary Digest announced its conclusions: Landon would win by a convincing 55 per cent to 41 per cent, with a few voters favouring a third candidate.
The election delivered a very different result: Roosevelt crushed Landon by 61 per cent to 37 per cent. To add to The Literary Digest’s agony, a far smaller survey conducted by the opinion poll pioneer George Gallup came much closer to the final vote, forecasting a comfortable victory for Roosevelt. Mr Gallup understood something that The Literary Digest did not. When it comes to data, size isn’t everything.
Opinion polls are based on samples of the voting population at large. This means that opinion pollsters need to deal with two issues: sample error and sample bias.
Sample error reflects the risk that, purely by chance, a randomly chosen sample of opinions does not reflect the true views of the population. The “margin of error” reported in opinion polls reflects this risk and the larger the sample, the smaller the margin of error. A thousand interviews is a large enough sample for many purposes and Mr Gallup is reported to have conducted 3,000 interviews.
But if 3,000 interviews were good, why weren’t 2.4 million far better? The answer is that sampling error has a far more dangerous friend: sampling bias. Sampling error is when a randomly chosen sample doesn’t reflect the underlying population purely by chance; sampling bias is when the sample isn’t randomly chosen at all. George Gallup took pains to find an unbiased sample because he knew that was far more important than finding a big one.
The Literary Digest, in its quest for a bigger data set, fumbled the question of a biased sample. It mailed out forms to people on a list it had compiled from automobile registrations and telephone directories – a sample that, at least in 1936, was disproportionately prosperous. To compound the problem, Landon supporters turned out to be more likely to mail back their answers. The combination of those two biases was enough to doom The Literary Digest’s poll. For each person George Gallup’s pollsters interviewed, The Literary Digest received 800 responses. All that gave them for their pains was a very precise estimate of the wrong answer.
The big data craze threatens to be The Literary Digest all over again. Because found data sets are so messy, it can be hard to figure out what biases lurk inside them – and because they are so large, some analysts seem to have decided the sampling problem isn’t worth worrying about. It is.
Professor Viktor Mayer-Schönberger of Oxford’s Internet Institute, co-author of Big Data, told me that his favoured definition of a big data set is one where “N = All” – where we no longer have to sample, but we have the entire background population. Returning officers do not estimate an election result with a representative tally: they count the votes – all the votes. And when “N = All” there is indeed no issue of sampling bias because the sample includes everyone.
But is “N = All” really a good description of most of the found data sets we are considering? Probably not. “I would challenge the notion that one could ever have all the data,” says Patrick Wolfe, a computer scientist and professor of statistics at University College London.
An example is Twitter. It is in principle possible to record and analyse every message on Twitter and use it to draw conclusions about the public mood. (In practice, most researchers use a subset of that vast “fire hose” of data.) But while we can look at all the tweets, Twitter users are not representative of the population as a whole. (According to the Pew Research Internet Project, in 2013, US-based Twitter users were disproportionately young, urban or suburban, and black.)
There must always be a question about who and what is missing, especially with a messy pile of found data. Kaiser Fung, a data analyst and author of Numbersense, warns against simply assuming we have everything that matters. “N = All is often an assumption rather than a fact about the data,” he says.
Consider Boston’s Street Bump smartphone app, which uses a phone’s accelerometer to detect potholes without the need for city workers to patrol the streets. As citizens of Boston download the app and drive around, their phones automatically notify City Hall of the need to repair the road surface. Solving the technical challenges involved has produced, rather beautifully, an informative data exhaust that addresses a problem in a way that would have been inconceivable a few years ago. The City of Boston proudly proclaims that the “data provides the City with real-time information it uses to fix problems and plan long term investments.”
Yet what Street Bump really produces, left to its own devices, is a map of potholes that systematically favours young, affluent areas where more people own smartphones. Street Bump offers us “N = All” in the sense that every bump from every enabled phone can be recorded. That is not the same thing as recording every pothole. As Microsoft researcher Kate Crawford points out, found data contain systematic biases and it takes careful thought to spot and correct for those biases. Big data sets can seem comprehensive but the “N = All” is often a seductive illusion.
. . .
Who cares about causation or sampling bias, though, when there is money to be made? Corporations around the world must be salivating as they contemplate the uncanny success of the US discount department store Target, as famously reported by Charles Duhigg in The New York Times in 2012. Duhigg explained that Target has collected so much data on its customers, and is so skilled at analysing that data, that its insight into consumers can seem like magic.
Duhigg’s killer anecdote was of the man who stormed into a Target near Minneapolis and complained to the manager that the company was sending coupons for baby clothes and maternity wear to his teenage daughter. The manager apologised profusely and later called to apologise again – only to be told that the teenager was indeed pregnant. Her father hadn’t realised. Target, after analysing her purchases of unscented wipes and magnesium supplements, had.
Statistical sorcery? There is a more mundane explanation.
“There’s a huge false positive issue,” says Kaiser Fung, who has spent years developing similar approaches for retailers and advertisers. What Fung means is that we didn’t get to hear the countless stories about all the women who received coupons for babywear but who weren’t pregnant.
©Ed NacionalHearing the anecdote, it’s easy to assume that Target’s algorithms are infallible – that everybody receiving coupons for onesies and wet wipes is pregnant. This is vanishingly unlikely. Indeed, it could be that pregnant women receive such offers merely because everybody on Target’s mailing list receives such offers. We should not buy the idea that Target employs mind-readers before considering how many misses attend each hit.
In Charles Duhigg’s account, Target mixes in random offers, such as coupons for wine glasses, because pregnant customers would feel spooked if they realised how intimately the company’s computers understood them.
Fung has another explanation: Target mixes up its offers not because it would be weird to send an all-baby coupon-book to a woman who was pregnant but because the company knows that many of those coupon books will be sent to women who aren’t pregnant after all.
None of this suggests that such data analysis is worthless: it may be highly profitable. Even a modest increase in the accuracy of targeted special offers would be a prize worth winning. But profitability should not be conflated with omniscience.
. . .
In 2005, John Ioannidis, an epidemiologist, published a research paper with the self-explanatory title, “Why Most Published Research Findings Are False”. The paper became famous as a provocative diagnosis of a serious issue. One of the key ideas behind Ioannidis’s work is what statisticians call the “multiple-comparisons problem”.
It is routine, when examining a pattern in data, to ask whether such a pattern might have emerged by chance. If it is unlikely that the observed pattern could have emerged at random, we call that pattern “statistically significant”.
The multiple-comparisons problem arises when a researcher looks at many possible patterns. Consider a randomised trial in which vitamins are given to some primary schoolchildren and placebos are given to others. Do the vitamins work? That all depends on what we mean by “work”. The researchers could look at the children’s height, weight, prevalence of tooth decay, classroom behaviour, test scores, even (after waiting) prison record or earnings at the age of 25. Then there are combinations to check: do the vitamins have an effect on the poorer kids, the richer kids, the boys, the girls? Test enough different correlations and fluke results will drown out the real discoveries.
There are various ways to deal with this but the problem is more serious in large data sets, because there are vastly more possible comparisons than there are data points to compare. Without careful analysis, the ratio of genuine patterns to spurious patterns – of signal to noise – quickly tends to zero.
Worse still, one of the antidotes to the multiple-comparisons problem is transparency, allowing other researchers to figure out how many hypotheses were tested and how many contrary results are languishing in desk drawers because they just didn’t seem interesting enough to publish. Yet found data sets are rarely transparent. Amazon and Google, Facebook and Twitter, Target and Tesco – these companies aren’t about to share their data with you or anyone else.
New, large, cheap data sets and powerful analytical tools will pay dividends – nobody doubts that. And there are a few cases in which analysis of very large data sets has worked miracles. David Spiegelhalter of Cambridge points to Google Translate, which operates by statistically analysing hundreds of millions of documents that have been translated by humans and looking for patterns it can copy. This is an example of what computer scientists call “machine learning”, and it can deliver astonishing results with no preprogrammed grammatical rules. Google Translate is as close to theory-free, data-driven algorithmic black box as we have – and it is, says Spiegelhalter, “an amazing achievement”. That achievement is built on the clever processing of enormous data sets.
But big data do not solve the problem that has obsessed statisticians and scientists for centuries: the problem of insight, of inferring what is going on, and figuring out how we might intervene to change a system for the better.
“We have a new resource here,” says Professor David Hand of Imperial College London. “But nobody wants ‘data’. What they want are the answers.”
To use big data to produce such answers will require large strides in statistical methods.
“It’s the wild west right now,” says Patrick Wolfe of UCL. “People who are clever and driven will twist and turn and use every tool to get sense out of these data sets, and that’s cool. But we’re flying a little bit blind at the moment.”
Statisticians are scrambling to develop new methods to seize the opportunity of big data. Such new methods are essential but they will work by building on the old statistical lessons, not by ignoring them.
Recall big data’s four articles of faith. Uncanny accuracy is easy to overrate if we simply ignore false positives, as with Target’s pregnancy predictor. The claim that causation has been “knocked off its pedestal” is fine if we are making predictions in a stable environment but not if the world is changing (as with Flu Trends) or if we ourselves hope to change it. The promise that “N = All”, and therefore that sampling bias does not matter, is simply not true in most cases that count. As for the idea that “with enough data, the numbers speak for themselves” – that seems hopelessly naive in data sets where spurious patterns vastly outnumber genuine discoveries.
“Big data” has arrived, but big insights have not. The challenge now is to solve new problems and gain new answers – without making the same old statistical mistakes on a grander scale than ever.
——————————————-
Tim Harford’s latest book is ‘The Undercover Economist Strikes Back’. To comment on this article please post below, or email magazineletters@ft.com
Punchy.
http://www.jeffreybraithwaite.com/new-blog/2014/11/20/youll-be-dying-to-hear-about-this
There’s lots of death in the world. Transport is risky, for instance—planes, automobiles, trains and ships can crash, maiming or killing passengers. You don’t have to go much further than seeing the road toll, or hearing about Malaysian Airlines Flight MH17 shot down over the Ukraine, or watching the TV scenes of the Costa Concordia, run aground just off Isola del Giglio near the coast of Italy, to appreciate that death is never far away.
Then there’s infectious diseases. You can all-too-readily catch a cold, or the flu, or TB, or lately, the Ebola virus. And there seem to be never-ending wars and skirmishes in the Middle East; and terror, spread by fundamentalists.
Each of these, depending on fate, can hasten someone’s demise. Wrong place, wrong time, wrong circumstances.
Lifestyle issues can cause problems for your risk profile too—but these are slower, and more stealthy. Think of smoking, drinking too much, eating yourself into a coma or just gross obesity, or the more insidious dangers of sitting at a computer for years on end with little exercise. These can translate over time into heart or lung disease, diabetes, and cancer.
Whether you are active or passive, things you do or don’t do can shorten your lifespan, or kill you a little or a lot faster than you would otherwise last. So what levels of risk do you actually, quantitatively, face in your own life?
*****
Stanford University decision scientist Ron Howard in the 1970s presented a novel way to calculate this risk. He introduced the idea of the micromort, defined as a one-in-a-million likelihood of death. This is such an evocative unit of measurement that it deserves a little further attention.
If you live in the US or another relatively rich, OECD-style country, with good law and order, legislation that keeps society relatively risk free (such as with environmental and public health issues sorted out, effective building codes, and so forth), a well-educated population, access to health care, and a buoyant GDP, you can expect a micromort of one on any particular day. Another way of saying this is that’s the standard expected death rate for any individual today in any one 24 hour period: a microprobability of one in a million is your index of baseline risk.
These are great odds for you, today, as you read this; you are very likely to get through it. Congratulations if you do.
What circumstances lead to an elevated risk? Say if you do dangerous things or even just live life to the full? How does your micromort level get upgraded?
In the United States, you accumulate an extra 16 micromorts each time you ride a motorcycle 100 miles, for instance. Or 0.7 micromorts are added for each day you go skiing; so go for a week and you’ve added five more.
Or you might decide to do something a little more strenuous. With hangliding, the additional risk of dying equates to eight micromorts per flight; or skydiving, nine per freefall.
They are relatively benign compared to moving up to base-jumping. Do so, and you rapidly earn many more risk points: 430 micromorts per jump, in fact.
Marathon running, anyone? That will be seven micromorts to your debit account for each run. Even walking 17 miles adds one micromort, as does a 230 mile car trip, and add another one for every 6,000 mile train trip. But the puzzle is, it’s not always clear how to treat these: the walking introduces an element of risk (you could be out and about and get run over, or be struck by lightning) but it’s also beneficial (it contributes to improved health).
Perhaps even more interesting, there are microprobabilities associated with accumulated chronic risks in contrast to these other single-shot event risks. These are lifestyle choices and behaviors that incrementally add a little more risk through exposure. They won’t kill you if you have bad luck on a given day, but will slowly have an effect—and may claim you in the end.
Every half a liter of wine exposes you to a micromort because it can accrue into cirrhosis of the liver. Each one and a half cigarettes does the same, but the menace here is cancer or heart disease. Even eating 100 char-broiled steaks, 40 tablespoons of peanut butter or 1,000 bananas sneaks up on you in the form, respectively, of cancer risk from benzopyrene, liver cancer risk from aflatoxin B or cancer risk from radioactive potassium-40.
*****
Hang on though. I doubt I’ve done much to help anyone.
Because a clear problem is that people aren’t very good at doing these kinds of statistics, or applying them to their own lives—and are even less capable of acting on them. We can readily appreciate that skiing or motorcycling add some risk for the time you are doing them compared to the everyday activities of being at work or hanging out at home, yet many people are undeterred. People even cheerfully find ways of taking on more risk, such as by climbing Everest, driving fast cars, or having unsafe sex.
Everyone knows about that steadily accumulated risk, too: not too many of us are blind to the fact that drinking too much alcohol can lead to liver disease or smoking to lung cancer over time. And although both have been falling for decades, this hasn’t stopped millions of people indulging. There’s 42.1 million US smokers at last count, or 18.1% of the population, and on average each adult US citizen consumes 8.6 liters of alcohol annually.
This is not the best performance internationally but is by no means high by international standards, and Eastern Europeans smoke more heavily, and really give hard booze like vodka a nudge. Nevertheless, both activities contribute to what public health people quaintly call excess deaths and the rest of us know by “their drinking or smoking (or both) killed them eventually.”
But what does it actually mean that you expose yourself to increased risk if you go out walking regularly or eat bananas? We need another way of looking at this, because it’s too hard to do the sums.
*****
Enter the University of Cambridge medical statistician David Spiegelhalter and his colleague Alejandro Leiva who invented the idea of a microlife. This is another unit of risk which has the calculation built in for you. It is half an hour of your life.
If you increase your risk by one micromort, then this shortens your life by half an hour. These calculations apply to people on average, and work out for entire populations, but any one of us might be lucky or unlucky, depending on our individual characteristics. Any particular risk doesn’t convert exactly to the specific individual. But with enough people in the US (beyond 316 million now) and on the planet (7 billion and rising), there’s a relentlessness accuracy about the statistics.
So now let’s do some life expectancy math with Spiegelhalter. Smoke a pack a day? You lose up to five hours a day. Accumulated, that’s up to eight years off your life. Have six drinks a day and that binge costs you one half hour allocation—a shortened life by ten months or so. Stay eleven pounds overweight and you sacrifice half an hour every day you do so (another ten months across your lifespan), as you do if you watch TV for two hours. Your coffee habit at 2-3 cups daily takes away another half hour lot. So does every portion of red meat each day. Another ten months each time.
It’s not all negative. There’s good news. Eat five serves of fruit and vegetables every day and you gain up to a couple of hours each time. You get three years back. Exercise and the first 20 minutes per day earns you a surprising hour (there’s a good investment—a year and a half), and each subsequent 40 minutes adds up to one more half hour bonus to your credit (a bit more work but that seems a pretty good deal, too, to get a ten month return).
If you have a hobby, activity or diet and it’s not been dealt with so far, you can fill in some of the gaps with some good guesstimates. Do you have passive pursuits, akin to watching TV? This is a net deficit. Do you do active, exercise-oriented activities, such as weekly amateur netball, soccer, bowling or basketball—or just walking regularly? Add some lifespan.
These half hour allocations alter somewhat depending on your genetics of course (you can have lucky or unlucky genes) or your socioeconomic status (wealthy people typically live longer than poorer folks) or your gender (women on the whole live longer than men). That said, with this idea you are now able to alter your risk profile by changing your behavior with a tangible, calculable return.
*****
There’s a punchline to this, and it may be already occurring to you as you reflect on your own lifestyle and lifespan. There are a million microlives in fifty seven years of existence. That, for many of us, is roughly the adult allocation.
Let’s call that your life expectancy baseline. We can assume that you have had a reasonably healthy childhood (not so for everyone, of course, but true for many US children, and true for most readers). Then, from that point on, a large part of your healthy adult life is now measureable.
So: come out of your teens, reach your 21st birthday, and as the “jolly good fellow” and “happy birthday to you” songs subside, imagine you then have 57 years to go. That is, you have an allocation of 78 years in total, maybe a little longer, maybe a little shorter.
Yes, all sorts of unexpected things might happen along the way, but to some degree your lifespan is now no longer vague, but quantifiable. The actual life expectancy in the US indeed hovers around this: it’s 79.8 years overall, 77.4 for males and 82.2 for females. (It’s higher in some northern European countries and Japan, but that’s a story for another day).
However, you might be reading this thinking: Yikes. I’m not 21: I’m a bit older than that. In this case, you’ve already used up a proportion of your time left. Console yourself. At least you got through the riskiest stage of all: being a baby, up to one year of age, and childhood, up to six or so, when many things can go wrong.
But have you used what you were given so far, well? Or do you have a fair bit of regret?
To make an obvious point, however, this isn’t Doctor Who. You don’t have a Tardis to go back in time and fix the past. So stop any lamentations. Look forward.
By now, if you’ve come to value more readily each half hour and especially the cumulative effect of your lifestyle choices to date, don’t listen to me preaching. Feel completely empowered. You know what to do and how to alter your own numbers.
Now, all that’s left is to do the math. You’ll have a much clearer picture of your life and potential death than ever before. It’s your move: what’s next?
Further reading
Blastland, Michael and Spiegelhalter, David (2014). The Norm Chronicles: Stories and Numbers About Danger and Death. New York: Basic Books.
Howard, Ronald (1984). On fates comparable to death. Management Science 30 (4): 407–422.
Spiegelhalter, David (2012). Using speed of ageing and “microlives” to communicate the effects of lifetime habits and environment. British Medical Journal 345: e8223.
Spiegelhalter, David (2014). The power of the MicroMort. BJOG: An International Journal of Obstetrics & Gynaecology 121 (6): 662–663.
On the things that computers can’t do but humans can, and vice versa…
http://bigthink.com/think-tank/luis-von-ahn-on-recaptcha
Back at the beginning of the century, a 21-year-old Luis von Ahn helped invent CAPTCHA, which is that familiar internet thing you see above this post. Commonly used as a security mechanism, CAPTCHA is a way for a website to determine if someone trying to obtain access is actually human and not a computer. In his recent Big Think interview, von Ahn describes how the idea behind CAPTCHA formed the essence of reCAPTCHA, which he invented in 2007. ReCAPTCHA relies on what is known as human computation, which harnesses the unique abilities of both humans and computers to accomplish difficult tasks:
Video Link: http://bcove.me/ghm5j3n2
In describing human computation, von Ahn explains that both computers and humans have their own sets of advantages and disadvantages when it comes to problem solving:
“There are problems that computers cannot yet solve. It’s funny because some of these problems are very simple problems seemingly. For example, a computer cannot tell you what’s inside an image. They can tell you somethings but it can’t really quite tell you there’s a cat next to a dog and they’re both running. A computer can’t do that. Well humans, we can do it super easily.”
Simple enough in concept, right? There may soon come a day when computer cognition takes a huge step forward and current limitations vanish. But until then, image identification and thematic analysis are the stuff of human expertise. Way to go, fellow humans.
On the flip side though…
“There are also things that computers can do that humans can’t do. I mean computers can multiply humongous numbers, humans may be able to do it but very slowly and we’re error-prone.”
Alas, we dumb humans exhibit our own limitations, particularly when it comes to the scale of a certain task. Any one person could memorize a poem yet no human being could memorize every piece of poetry written since Antiquity. Computers can and do. In a way, we and computers form a Yin to each other’s Yang. Our abilities match up with computers’ weaknesses like corresponding puzzle pieces.
This is where human computation comes in.
So the essential idea is that there are certain tasks that require both a human’s attention to detail and a computer’s ability to store vast quantities of information. These are problems neither side can solve alone. Human computation therefore harnesses the talents of both. This is how reCAPTCHA works:
“The Idea with reCAPTCHA is that we take a problem that neither humans nor computers can solve by themselves, which is fully digitizing books. The idea there is we would like to digitize books. And the way this process works is you start with a book and then you scan it. The next step in the process is that the computer needs to be able to decipher all of the words in this picture. It’s a picture of words. The computer needs to be able to decipher all of those words. The problem is that sometimes the computer cannot decipher these words because for older books the ink has faded a little or the pages have turned yellow so the computer cannot decipher all of the words. But, humans can.”
You may, at this point, be able to identify where von Ahn is heading here. Just like he explains in his interview about Duolingo, von Ahn has created a piece of technology that serves multiple purposes. ReCAPTCHA is partly a security device and partly a tool of crowdsourcing brilliance. It’s still the same idea as CAPTCHA, except with one added component:
“So what we’re doing with reCAPTCHA… the idea is that some of these [squiggl CAPTCHA] words, nowadays some of these words are words that are actually coming from books that the computer could not recognize in this process and we’re using what people enter to help us digitize the books.”
Von Ahn sold reCAPTCHA to Google in 2009. Since its inception, over 1.1 billion people worldwide have contributed by way of reCAPTCHA to the digitization of old books. Google is now digitizing 2 million per years utilizing the respective powers of humans and computers.
And that’s how human computation works.
So pay for performance doesn’t work. This is hardly surprising when you see the compromise and mediocrity forced upon policy makers to get ideas through. There have been instances of success in health care. Indeed, one could argue that the exemplary success of big pharma in changing physician behaviour has provided a rod for its own back. Why not harness this expertise in getting under the skin of doctors, and pay big pharma sales outfits to guide physician practice in constructive directions, rather than being distracted by flogging pills that don’t really work that well anyway, and potentially harm? Might have a chat with Christian.
http://www.nytimes.com/2014/07/29/upshot/the-problem-with-pay-for-performance-in-medicine.html
“Pay for performance” is one of those slogans that seem to upset no one. To most people it’s a no-brainer that we should pay for quality and not quantity. We all know that paying doctors based on the amount of care they provide, as we do with a traditional fee-for-service setup, creates incentives for them to give more care. It leads to increased health care spending. Changing the payment structure to pay them for achieving goals instead should reduce wasteful spending.
So it’s no surprise that pay for performance has been an important part of recent reform efforts. But in reality we’re seeing disappointingly mixed results. Sometimes it’s because providers don’t change the way they practice medicine; sometimes it’s because even when they do, outcomes don’t really improve.
The idea behind pay for performance is simple. We will give providers more money for achieving a goal. The goal can be defined in various ways, but at its heart, we want to see the system hit some target. This could be a certain number of patients receiving preventive care, a certain percentage of people whose chronic disease is being properly managed or even a certain number of people avoiding a bad outcome. Providers who reach these targets earn more money.
The problem, one I’ve noted before, is that changing physician behavior is hard. Sure, it’s possible to find a study in the medical literature that shows that pay for performance worked in some small way here or there. For instance, a study published last fall found that paying doctors $200 more per patient for hitting certain performance criteria resulted in improvements in care. It found that the rate of recommendations for aspirin or for prescriptions for medications to prevent clotting for people who needed it increased 6 percent in clinics without pay for performance but 12 percent in clinics with it.
Good blood pressure control increased 4.3 percent in clinics without pay for performance but 9.7 percent in clinics with it. But even in the pay-for-performance clinics, 35 percent of patients still didn’t have the appropriate anti-clotting advice or prescriptions, and 38 percent of patients didn’t have proper hypertensive care. And that’s success!
It’s also worth noting that the study was only for one year, and many improvements in actual outcomes would need to be sustained for much longer to matter. It’s not clear whether that will happen. A study published in Health Affairs examined the effects of a government partnership with Premier Inc., a national hospital system, and found that while the improvements seen in 260 hospitals in a pay-for-performance project outpaced those of 780 not in the project, five years later all those differences were gone.
The studies showing failure are also compelling. A study in The New England Journal of Medicine looked at 30-day mortality in the hospitals in the Premier pay-for-performance program compared with 3,363 hospitals that weren’t part of a pay-per-performance intervention. We’re talking about a study of millions of patients taking place over a six-year period in 12 states. Researchers found that 30-day mortality, or the rate at which people died within a month after receiving certain procedures or care, was similar at the start of the study between the two groups, and that the decline in mortality over the next six years was also similar.
Moreover, they found that even among the conditions that were explicitly linked to incentives, like heart attacks and coronary artery bypass grafts, pay for performance resulted in no improvements compared with conditions without financial incentives.
In Britain, a program was begun over a decade ago that would pay general practitioners up to 25 percent of their income in bonuses if they met certain benchmarks in the management of chronic diseases. The program made no difference at all in physician practice or patient outcomes, and this was with a much larger financial incentive than most programs in the United States offer.
Even refusing to pay for bad outcomes doesn’t appear to work as well as you might think. A 2012 study published in The New England Journal of Medicine looked at how the 2008 Medicare policy to refuse to pay for certain hospital-acquired conditions affected the rates of such infections. Those who devised the policy imagined that it would lead hospitals to improve their care of patients to prevent these infections. That didn’t happen. The policy had almost no measurable effect.
There have even been two systematic reviews in this area. The first of them suggested that there is some evidence that pay for performance could change physicians’ behavior. It acknowledged, though, that the studies were limited in how they could be generalized and might not be able to be replicated. It also noted there was no evidence that pay for performance improved patient outcomes, which is what we really care about. The secondreview found that with respect to primary care physicians, there was no evidence that pay for performance could even change physician behavior, let alone patient outcomes.
One of the reasons that paying for quality is hard is that we don’t even really know how to define “quality.” What is it, really? Far too often we approach quality like a drunkard’s search, looking where it’s easy rather than where it’s necessary. But it’s very hard to measure the things we really care about, like quality of life and improvements in functioning.
In fact, the way we keep setting up pay for performance demands easy-to-obtain metrics. Otherwise, the cost of data gathering could overwhelm any incentives. Unfortunately, as a recent New York Times article described, this has drawbacks.
The National Quality Forum, described in the article as an influential nonprofit, nonpartisan organization that endorses health care standards, reported that the metrics chosen by Medicare for their programs included measurements that were outside the control of a provider. In other words, factors like income, housing and education can affect the metrics more than what doctors and hospitals do.
This means that hospitals in resource-starved settings, caring for the poor, might be penalized because what we measure is out of their hands. A panel commissioned by the Obama administration recommended that the Department of Health and Human Services change the program to acknowledge the flaw. To date, it hasn’t agreed to do so.
Some fear that pay for performance could even backfire. Studies in other fields show that offering extrinsic rewards (like financial incentives) can undermine intrinsic motivations (like a desire to help people). Many physicians choose to do what they do because of the latter. It would be a tragedy if pay for performance wound up doing more harm than good.
https://www.youtube.com/watch?v=qHQWlFtSptU
Vitality absolutely smash it across the board…
Must get on to these guys…..
PDF: Vitality_Recommendations2014_Report
PDF: InvestingInPrevention_Slides
Presentation: https://goto.webcasts.com/viewer/event.jsp?ei=1034543 (email: blackfriar@gmail.com)
From Forbes: http://www.forbes.com/sites/brucejapsen/2014/06/18/how-corporate-america-could-save-300-billion-by-measuring-health-like-financial-performance/
Bruce Japsen, Contributor
I write about health care and policies from the president’s hometown
The U.S. could save more than $300 billion annually if employers adopted strategies that promoted health, prevention of chronic disease and measured progress of “working-age” individuals like they did their financial performance, according to a new report.
The analysis, developed by some well-known public health advocates brought together and funded by The Vitality Institute, said employers could save $217 billion to $303 billion annually, or 5 to 7 percent of total U.S. annual health spending by 2023, by adopting strategies to help Americans head off “non-communicable” diseases like cancer, diabetes, cardiovascular and respiratory issues as well as mental health.
To improve, the report’s authors say companies should be reporting health metrics like BMI and other employee health statuses just like they regularly report earnings and how an increasing number of companies report sustainability. Corporations should be required to integrate health metrics into their annual reporting by 2025, the Vitality Institute said. A link to the entire report and its recommendations is here.
“Companies should consider the health of their employees as one of their greatest assets,” said Derek Yach, executive director of the Vitality Institute, a New York-based organization funded by South Africa’s largest health insurance company, Discovery Limited.
Those involved in the report say its recommendations come at a time the Affordable Care Act and employers emphasize wellness as a way to improve quality and reduce costs.
“Healthy workers are more productive, resulting in improved financial performance,” Yach said. “We’re calling on corporations to take accountability and start reporting health metrics in their financial and sustainability reports. We believe this will positively impact the health of both employees and the corporate bottom line.”
The Institute brought together a commission linked here that includes some executives from the health care industry and others who work in academia and business. Commissioners came from Microsoft (MSFT); the Robert Wood Johnson Foundation; drug and medical device giant Johnson & Johnson (JNJ); health insurer Humana (HUM); and the U.S. Department of Health and Humana Services.
The Vitality Institute said up to 80 percent of non-communicable diseases can be prevented through existing “evidence-based methods” and its report encourages the nation’s policymakers and legislative leaders to increase federal spending on prevention science at least 10 percent by 2017.
“Preventable chronic diseases such as lung cancer, diabetes and heart disease are forcing large numbers of people to exit the workforce prematurely due to their own poor health or to care for sick relatives,” said William Rosenzweig, chair of the Vitality Institute Commission and an executive at Physic Ventures, which invests in health and sustainability projects. “Yet private employers spend less than two percent of their total health budgets on prevention. This trend will stifle America’s economic growth for decades to come unless health is embraced as a core value in society.”
http://www.economist.com/news/books-and-arts/21602984-why-everything-you-heard-about-fat-wrong-case-eating-steak-and-cream
Economist Book Review
Shifting the argumentThe Big Fat Surprise: Why Butter, Meat and Cheese Belong in a Healthy Diet. By Nina Teicholz. Simon & Schuster; 479 pages; $27.99. Buy from Amazon.com,Amazon.co.uk
“EATING foods that contain saturated fats raises the level of cholesterol in your blood,” according to the American Heart Association (AHA). “High levels of blood cholesterol increase your risk of heart disease and stroke.” So goes the warning from the AHA, the supposed authority on the subject. Governments and doctors wag their fingers to this tune the world over. Gobble too much bacon and butter and you may well die young. But what if that were wrong?
Nina Teicholz, an American journalist, makes just that argument in her compelling new book, “The Big Fat Surprise”. The debate is not confined to nutritionists. Warnings about fat have changed how food companies do business, what people eat, and how and how long they live. Heart disease is the top cause of death not just in America, but around the world. The question is whether saturated fat is truly to blame. Ms Teicholz’s book is a gripping read for anyone who has ever tried to eat healthily.
The case against fat would seem simple. Fat contains more calories, per gram, than do carbohydrates. Eating saturated fat raises cholesterol levels, which in turn is thought to bring on cardiovascular problems. Ms Teicholz dissects this argument slowly. Her book, which includes well over 100 pages of notes and citations, covers decades of nutrition research, including careful explorations of academics’ methodology. This is not an obvious page-turner. But it is.
Ms Teicholz describes the early academics who demonised fat and those who have kept up the crusade. Top among them was Ancel Keys, a professor at the University of Minnesota, whose work landed him on the cover of Time magazine in 1961. He provided an answer to why middle-aged men were dropping dead from heart attacks, as well as a solution: eat less fat. Work by Keys and others propelled the American government’s first set of dietary guidelines, in 1980. Cut back on red meat, whole milk and other sources of saturated fat. The few sceptics of this theory were, for decades, marginalised.
But the vilification of fat, argues Ms Teicholz, does not stand up to closer examination. She pokes holes in famous pieces of research—the Framingham heart study, the Seven Countries study, the Los Angeles Veterans Trial, to name a few—describing methodological problems or overlooked results, until the foundations of this nutritional advice look increasingly shaky.
The opinions of academics and governments, as presented, led to real change. Food companies were happy to replace animal fats with less expensive vegetable oils. They have now begun abolishing trans fats from their food products and replacing them with polyunsaturated vegetable oils that, when heated, may be as harmful. Advice for keeping to a low-fat diet also played directly into food companies’ sweet spot of biscuits, cereals and confectionery; when people eat less fat, they are hungry for something else. Indeed, as recently as 1995 the AHA itself recommended snacks of “low-fat cookies, low-fat crackers…hard candy, gum drops, sugar, syrup, honey” and other carbohydrate-laden foods. Americans consumed nearly 25% more carbohydrates in 2000 than they had in 1971.
In the past decade a growing number of studies have questioned the anti-fat orthodoxy. Ms Teicholz’s book follows the work of Gary Taubes, a science journalist who has cast doubts on the link between saturated fat and health for well over a decade—and been much disparaged for his pains. There is increasing evidence that a bigger culprit is most likely insulin, a hormone; insulin levels rise when one eats carbohydrates. Yet even now, with more attention devoted to the dangers posed by sugar, saturated fat remains maligned. “It seems now that what sustains it,” argues Ms Teicholz, “is not so much science as generations of bias and habit.”
Automated Life Logging. Go Figure.
http://www.engadget.com/2014/05/20/ifttt-fitbit-channel/
If you’re sporting one of Fitbit’s activity trackers, you can now automate tasks and reminders with the help of IFTTT (If This Then That). The recipe-based software announced a dedicated channel for the sporty gadgets today, handling duties based on goals, activity, sleep, weight and more. For example, you can now log a weigh-in via text message or automatically beam sleep stats to a Google Spreadsheet each morning. Of course, those are just a couple of the possibilities, and users can construct their own formulas as well. Those who prefer Jawbone’s wearables have already been privy to the automated life logging, with other wrist-worn devices like the Pebble smartwatch supported too.
Solid report on personal health data. Interesting observation re. (lack of) alignment between research and business objectives… i.e. public vs private goods?
http://www.rwjf.org/en/research-publications/find-rwjf-research/2014/03/personal-data-for-the-public-good.html
Report: http://www.rwjf.org/content/dam/farm/reports/reports/2014/rwjf411080
PDF:
1. Executive Summary
Individuals are tracking a variety of health-related data via a growing number of wearable devices and smartphone apps. More and more data relevant to health are also being captured passively as people communicate with one another on social networks, shop, work, or do any number of activities that leave “digital footprints.”
Almost all of these forms of “personal health data” (PHD) are outside of the mainstream of traditional health care, public health or health research. Medical, behavioral, social and public health research still largely rely on traditional sources of health data such as those collected in clinical trials, sifting through electronic medical records, or conducting periodic surveys.
Self-tracking data can provide better measures of everyday behavior and lifestyle and can fill in gaps in more traditional clinical data collection, giving us a more complete picture of health. With support from the Robert Wood Johnson Foundation, the Health Data Exploration (HDE) project conducted a study to better understand the barriers to using personal health data in research from the individuals who track the data about their own personal health, the companies that market self-tracking devices, apps or services and aggregate and manage that data, and the researchers who might use the data as part of their research.
Perspectives
Through a series of interviews and surveys, we discovered strong interest in contributing and using PHD for research. It should be noted that, because our goal was to access individuals and researchers who are already generating or using digital self-tracking data, there was some bias in our survey findings—participants tended to have more education and higher household incomes than the general population. Our survey also drew slightly more white and Asian participants and more female participants than in the general population.
Individuals were very willing to share their self-tracking data for research, in particular if they knew the data would advance knowledge in the fields related to PHD such as public health, health care, computer science and social and behavioral science. Most expressed an explicit desire to have their information shared anonymously and we discovered a wide range of thoughts and concerns regarding thoughts over privacy.
Equally, researchers were generally enthusiastic about the potential for using self-tracking data in their research. Researchers see value in these kinds of data and think these data can answer important research questions. Many consider it to be of equal quality and importance to data from existing high quality clinical or public health data sources.
Companies operating in this space noted that advancing research was a worthy goal but not their primary business concern. Many companies expressed interest in research conducted outside of their company that would validate the utility of their device or application but noted the critical importance of maintaining their customer relationships. A number were open to data sharing with academics but noted the slow pace and administrative burden of working with universities as a challenge.
In addition to this considerable enthusiasm, it seems a new PHD research ecosystem may well be emerging. Forty-six percent of the researchers who participated in the study have already used self-tracking data in their research, and 23 percent of the researchers have already collaborated with application, device, or social media companies.
The Personal Health Data Research Ecosystem
A great deal of experimentation with PHD is taking place. Some individuals are experimenting with personal data stores or sharing their data directly with researchers in a small set of clinical experiments. Some researchers have secured one-off access to unique data sets for analysis. A small number of companies, primarily those with more of a health research focus, are working with others to develop data commons to regularize data sharing with the public and researchers.
SmallStepsLab serves as an intermediary between Fitbit, a data rich company, and academic researchers via a “preferred status” API held by the company. Researchers pay SmallStepsLab for this access as well as other enhancements that they might want.
These promising early examples foreshadow a much larger set of activities with the potential to transform how research is conducted in medicine, public health and the social and behavioral sciences.
Opportunities and Obstacles
There is still work to be done to enhance the potential to generate knowledge out of personal health data:
•
Privacy and Data Ownership: Among individuals surveyed, the dominant condition (57%) for making their PHD available for research was an assurance of privacy for their data, and over 90% of respondents said that it was important that the data be anonymous. Further, while some didn’t care who owned the data they generate, a clear majority wanted to own or at least share ownership of the data with the company that collected it.
•
Informed Consent: Researchers are concerned about the privacy of PHD as well as respecting the rights of those who provide it. For most of our researchers, this came down to a straightforward question of whether there is informed consent. Our research found that current methods of informed consent are challenged by the ways PHD are being used and reused in research. A variety of new approaches to informed consent are being evaluated and this area is ripe for guidance to assure optimal outcomes for all stakeholders.
•
Data Sharing and Access: Among individuals, there is growing interest in, as well as willingness and opportunity to, share personal health data with others. People now share these data with others with similar medical conditions in online groups like PatientsLikeMe or Crohnology, with the intention to learn as much as possible about mutual health concerns. Looking across our data, we find that individuals’ willingness to share is dependent on what data is shared, how the data will be used, who will have access to the data and when, what regulations and legal protections are in place, and the level of compensation or benefit (both personal and public).
•
Data Quality: Researchers highlighted concerns about the validity of PHD and lack of standardization of devices. While some of this may be addressed as the consumer health device, apps and services market matures, reaching the optimal outcome for researchers might benefit from strategic engagement of important stakeholder groups.
We are reaching a tipping point. More and more people are tracking their health, and there is a growing number of tracking apps and devices on the market with many more in development. There is overwhelming enthusiasm from individuals and researchers to use this data to better understand health. To maximize personal data for the public good, we must develop creative solutions that allow individual rights to be respected while providing access to high-quality and relevant PHD for research, that balance open science with intellectual property, and that enable productive and mutually beneficial collaborations between the private sector and the academic research community.