SLOW Movement

WSJ: Making Data Beautiful

MARVELS
April 6, 2012, 6:36 p.m. ET

Making Data Beautiful

The most inspiring new art is visualized information

By HOLLY FINN

Aaron Koblin

Aaron Koblin's New York Talk Exchange shows, in real time, the volume of telephone and Internet data flowing to other cities.

Dateline: Dullsville. This week the Tate Modern museum in London unveiled a Damien Hirst retrospective that's about as fresh as one of its featured pieces: "A Thousand Years" is an actual rotting cow's head. Why are we picking at these carcasses of creativity? We should instead be celebrating the really new and relevant: the rise of the data visualizers. Their medium is the one with momentum, the one genuinely changing how we think and feel. And it's about to boom.

At companies and universities, and far beyond, the goal of data-driven digital artists is clear, not cynical: convey complex concepts quickly and crisply. They want to generate not Art-with-a-capital-A, necessarily, but understanding. They take stone-cold data—units of information—and turn them into something warmly communicative. Beautiful, too. So they become a pleasure for us to absorb.

Humans process information 17 times faster using sight than other senses, according to one Danish physicist. Take Gareth Lloyd's Web-based "A History of the World in 100 Seconds": over 14,000 geo-tagged Wikipedia articles, digitally mapped from 499 B.C. to the present. You can watch the data points pop like fireworks as they gradually form a glowing map of the continents. Hans Rosling's Gapminder website tracks data on the wealth and health of nations from 1800 on. You watch as bubbles representing each country expand, contract and bounce along over time—almost hopefully.

Data visualization has a history, of course. William Playfair invented four key visualization types in 1786, influencing Florence Nightingale, among others. She used a graphic polar chart in 1859 to show that soldiers were dying from infections, not wounds. But big data is today's specific bane. We all battle data obesity—too much information, not enough of it nutritious—and crave experts to help us sort and savor it. Happily, technology has handed us, and them, the tools. Faster computers, new programming languages.

"It's not unlike a microscope—taking something you can't see and bringing it into the scale of perception," Aaron Koblin, 30, told me at lunch in Google's San Francisco office. He's head of the company's Data Arts Team.

Mr. Koblin has temporally mapped text messages in New York. 'You really understand a lot about cities from flows,' he says.

Mr. Koblin's work sits right on the line between art and information. The shimmery tiles in eCloud, his installation at San Jose International Airport, change from opaque to transparent depending on the global weather data they're receiving. His New York Talk Exchange project visualizes the volume of long-distance telephone and Internet data between New York and other cities, revealing New Yorkers' relationships with the world.

He has temporally mapped text messages, too, in Amsterdam and New York. "You really understand a lot about cities from flows, when people are awake and doing what things at what locations," says Mr. Koblin. "And you can say, people in Brooklyn tend to get up later than people in Manhattan."

Add to that sonification: sound reiterating what you see, helping to "storify" information, especially for those who are pitch-sensitive. "You can turn data into rhythms," says Mr. Koblin, such as cable-box data. "CNBC has a constant rhythm, really local. But CNN is really event-driven—and you get these crazy spikes."

Tomorrow's prime exhibition space is online, not in a gallery. Still, some of Mr. Koblin's visualizations are part of the permanent collections at the Museum of Modern Art in New York and the Centre Georges Pompidou in Paris. He was nominated for a Grammy for his Radiohead video. And he co-created the first made-for-Web music video for Arcade Fire's "The Wilderness Downtown." Plug in the address where you grew up and the video, using Google Street View, transports you back there. You can almost taste the chocolate milk.

Delivering information and entertainment in such delightfully shocking ways is what the coming age of data arts is all about: a flat screen reaching out to smack, or sway, you. Talk about fresh. There are no putrefying cow heads here.

A version of this article appeared April 7, 2012, on page C12 in some U.S. editions of The Wall Street Journal, with the headline: Making Data BeautifulMaking Data Beautiful.

NY Times: The Yin and the Yang of Corporate

 

---

January 26, 2012

The Yin and the Yang of Corporate Innovation

By STEVE LOHR

IN the hunt for innovation, that elusive path to economic growth and corporate prosperity, try a little jazz as an inspirational metaphor.

That’s the message that John Kao, an innovation adviser to corporations and governments — who is also a jazz pianist — was to deliver in a performance and talk on Saturday at the World Economic Forum in Davos, Switzerland. Jazz, Mr. Kao says, demonstrates some of the tensions in innovation, between training and discipline on one side and improvised creativity on the other.

In business, as in jazz, the interaction of those two sides, the yin and the yang of innovation, fuels new ideas and products. The mixture varies by company.

Mr. Kao points to the very different models of innovation represented by Google and Apple, two powerhouses of Silicon Valley, the world’s epicenter of corporate creativity.

The Google model relies on rapid experimentation and data. The company constantly refines its search, advertising marketplace, e-mail and other services, depending on how people use its online offerings. It takes a bottom-up approach: customers are participants, essentially becoming partners in product design.

The Apple model is more edited, intuitive and top-down. When asked what market research went into the company’s elegant product designs, Steve Jobs had a standard answer: none. “It’s not the consumers’ job to know what they want,” he would add.

The Google-Apple comparison, Mr. Kao says, highlights the “archetypical tension in the creative process.”

Google speaks to the power of data-driven decision-making, and of online experimentation and networked communication. The same Internet-era tools enable crowd-sourced collaboration as well as the rapid testing of product ideas — the essence of the lean start-up method so popular in Silicon Valley and elsewhere.

“These are business and management innovations lubricated by technology,” says Thomas R. Eisenmann, a professor at the Harvard Business School.

The benefits, experts say, are most apparent in markets like Internet software, online commerce and mobile applications for smartphones and tablets. “The cost of creation, distribution and failure is low, so it takes relatively little time, money and effort to float trial balloons,” says Randy Komisar, a partner in Kleiner Perkins Caufield & Byers, the venture capital firm, and a lecturer on entrepreneurship at Stanford.

That style of innovation is being applied well beyond Google’s products and Internet start-ups. The National Science Foundation, for example, is embracing the formula to try to increase commercialization of the university research it finances. Last fall, the foundation announced the first of a series of grants for what it calls the N.S.F. Innovation Corps. The 21 three-member teams received a crash course at Stanford in lean start-up techniques, and have been given $50,000 each and six months to test whether their inventions are marketable.

The lean formula, with its emphasis on constantly testing ideas and products with customers, amounts to applying “the scientific method to market-opportunity identification,” says Errol B. Arkilic, program director at the foundation.

Yet while networked communications and marketplace experiments add useful information, breakthrough ideas still come from individuals, not committees. “There is nothing democratic about innovation,” says Paul Saffo, a veteran technology forecaster in Silicon Valley. “It is always an elite activity, whether by a recognized or unrecognized elite.”

Successful innovation, Mr. Saffo observes, requires “an odd blend of certainty and openness to new information.” In other words, it is a blend of top-down and bottom-up discovery.

OPEN innovation isn’t a new idea. It flourished, in its way, even in the late 19th and early 20th centuries, notes Tom Nicholas, a historian at the Harvard Business School. In fields like electricity, pharmaceuticals and communications, big corporations including General Electric and Dow Chemical routinely monitored the research beyond their walls, and bought or licensed promising work, especially the inventions of university scientists. The result, Mr. Nicholas says, was a thriving “ecosystem of private and corporate innovation.”

A century later, the corporate labs at G.E. are trying to quicken the pace of innovation — but this is long-cycle innovation, since G.E.’s power generators, jet engines and medical-imaging equipment last for decades. The company is opening a software center in Northern California to make its machines more intelligent with data-gathering sensors, wireless communications and predictive algorithms. The goal is to develop machines, such as jet engines or power turbines, that can alert their human minders when they need repairs, before equipment failures occur. Such smarter machines, the company says, are early arrivals in what it calls the Industrial Internet.

To tap outsider ideas, G.E.’s research arm has made investments with venture capital funds in clean-energy technology and health care, and it works with corporations, government labs and universities on hundreds of collaborative projects. “We’re much more externally focused and connected to the outside world than we were several years ago,” says Michael Idelchik, G.E.’s vice president of advanced technologies.

Apple’s smartphones, tablets and computers typically have life spans measured in a few years instead of decades, with new models introduced regularly. But like G.E., Apple is in the hardware business, where innovation cycles are beholden to the limits of materials science and manufacturing.

Apple’s physical world is far different from Google’s realm of Internet software, where writing a few lines of new code can change a product instantly. The careful melding of hardware with software in Apple’s popular products is a challenge in multidisciplinary systems design that must be orchestrated by a guiding hand — though it will no longer be the hand of Mr. Jobs, who died last October.

Yet Apple has also repeatedly displayed its openness to new ideas and influences, as exemplified by the visit that Mr. Jobs made to the Palo Alto research center of Xerox in 1979. He saw an experimental computer with a point-and-click mouse and graphical on-screen icons, which he adopted at Apple. It later became the standard for the personal computer industry.

In 2010, Apple bought Siri, a personal assistant application for smartphones. At the time, it was a small start-up in Silicon Valley that originated as a program funded by theDefense Advanced Research Projects Agency of the Pentagon. Last year, Siri became the talking question-answering application on iPhones.

Apple product designs may not be determined by traditional market research, focus groups or online experiments. But its top leaders, recruited by Mr. Jobs, are tireless seekers in an information-gathering network on subjects ranging from microchip technology to popular culture. “It’s a lot of data crunched in a nonlinear way in the right brain,” says Erik Brynjolfsson, director of the M.I.T. Center for Digital Business.

Apple and Google pursue very different paths to innovation, but the gap between their two models may be closing a bit. In the months after Larry Page, the Google co-founder, took over as chief executive last April, the company eliminated a diverse collection of more than two dozen projects, a nudge toward top-down leadership. And Timothy D. Cook, Apple’s C.E.O., will almost surely be a more bottom-up leader than Mr. Jobs.

“What we’re likely to see,” Mr. Kao says, “is Google and Apple each borrowing from the playbook of the other.”

NY Times: The Age of Big Data

February 11, 2012

The Age of Big Data

By STEVE LOHR

GOOD with numbers? Fascinated by data? The sound you hear is opportunity knocking.

Mo Zhou was snapped up by I.B.M. last summer, as a freshly minted Yale M.B.A., to join the technology company’s fast-growing ranks of data consultants. They help businesses make sense of an explosion of data — Web traffic and social network comments, as well as software and sensors that monitor shipments, suppliers and customers — to guide decisions, trim costs and lift sales. “I’ve always had a love of numbers,” says Ms. Zhou, whose job as a data analyst suits her skills.

To exploit the data flood, America will need many more like her. A report last year by the McKinsey Global Institute, the research arm of the consulting firm, projected that the United States needs 140,000 to 190,000 more workers with “deep analytical” expertise and 1.5 million more data-literate managers, whether retrained or hired.

The impact of data abundance extends well beyond business. Justin Grimmer, for example, is one of the new breed of political scientists. A 28-year-old assistant professor at Stanford, he combined math with political science in his undergraduate and graduate studies, seeing “an opportunity because the discipline is becoming increasingly data-intensive.” His research involves the computer-automated analysis of blog postings, Congressional speeches and press releases, and news articles, looking for insights into how political ideas spread.

The story is similar in fields as varied as science and sports, advertising and public health — a drift toward data-driven discovery and decision-making. “It’s a revolution,” says Gary King, director of Harvard’s Institute for Quantitative Social Science. “We’re really just getting under way. But the march of quantification, made possible by enormous new sources of data, will sweep through academia, business and government. There is no area that is going to be untouched.”

Welcome to the Age of Big Data. The new megarich of Silicon Valley, first at Google and now Facebook, are masters at harnessing the data of the Web — online searches, posts and messages — with Internet advertising. At the World Economic Forum last month in Davos, Switzerland, Big Data was a marquee topic. A report by the forum, “Big Data, Big Impact,” declared data a new class of economic asset, like currency or gold.

Rick Smolan, creator of the “Day in the Life” photography series, is planning a project later this year, “The Human Face of Big Data,” documenting the collection and uses of data. Mr. Smolan is an enthusiast, saying that Big Data has the potential to be “humanity’s dashboard,” an intelligent tool that can help combat poverty, crime and pollution. Privacy advocates take a dim view, warning that Big Data is Big Brother, in corporate clothing.

What is Big Data? A meme and a marketing term, for sure, but also shorthand for advancing trends in technology that open the door to a new approach to understanding the world and making decisions. There is a lot more data, all the time, growing at 50 percent a year, or more than doubling every two years, estimates IDC, a technology research firm. It’s not just more streams of data, but entirely new ones. For example, there are now countless digital sensors worldwide in industrial equipment, automobiles, electrical meters and shipping crates. They can measure and communicate location, movement, vibration, temperature, humidity, even chemical changes in the air.

Link these communicating sensors to computing intelligence and you see the rise of what is called the Internet of Things or the Industrial Internet. Improved access to information is also fueling the Big Data trend. For example, government data — employment figures and other information — has been steadily migrating onto the Web. In 2009, Washington opened the data doors further by starting Data.gov, a Web site that makes all kinds of government data accessible to the public.

Data is not only becoming more available but also more understandable to computers. Most of the Big Data surge is data in the wild — unruly stuff like words, images and video on the Web and those streams of sensor data. It is called unstructured data and is not typically grist for traditional databases.

But the computer tools for gleaning knowledge and insights from the Internet era’s vast trove of unstructured data are fast gaining ground. At the forefront are the rapidly advancing techniques of artificial intelligence like natural-language processing, pattern recognition and machine learning.

Those artificial-intelligence technologies can be applied in many fields. For example, Google’s search and ad business and its experimental robot cars, which have navigated thousands of miles of California roads, both use a bundle of artificial-intelligence tricks. Both are daunting Big Data challenges, parsing vast quantities of data and making decisions instantaneously.

The wealth of new data, in turn, accelerates advances in computing — a virtuous circle of Big Data. Machine-learning algorithms, for example, learn on data, and the more data, the more the machines learn. Take Siri, the talking, question-answering application in iPhones, which Apple introduced last fall. Its origins go back to a Pentagon research project that was then spun off as a Silicon Valley start-up. Apple bought Siri in 2010, and kept feeding it more data. Now, with people supplying millions of questions, Siri is becoming an increasingly adept personal assistant, offering reminders, weather reports, restaurant suggestions and answers to an expanding universe of questions.

To grasp the potential impact of Big Data, look to the microscope, says Erik Brynjolfsson, an economist at Massachusetts Institute of Technology’s Sloan School of Management. The microscope, invented four centuries ago, allowed people to see and measure things as never before — at the cellular level. It was a revolution in measurement.

Data measurement, Professor Brynjolfsson explains, is the modern equivalent of the microscope. Google searches, Facebook posts and Twitter messages, for example, make it possible to measure behavior and sentiment in fine detail and as it happens.

In business, economics and other fields, Professor Brynjolfsson says, decisions will increasingly be based on data and analysis rather than on experience and intuition. “We can start being a lot more scientific,” he observes.

There is plenty of anecdotal evidence of the payoff from data-first thinking. The best-known is still “Moneyball,” the 2003 book by Michael Lewis, chronicling how the low-budget Oakland A’s massaged data and arcane baseball statistics to spot undervalued players. Heavy data analysis had become standard not only in baseball but also in other sports, including English soccer, well before last year’s movie version of “Moneyball,” starring Brad Pitt.

Retailers, like Walmart and Kohl’s, analyze sales, pricing and economic, demographic and weather data to tailor product selections at particular stores and determine the timing of price markdowns. Shipping companies, like U.P.S., mine data on truck delivery times and traffic patterns to fine-tune routing.

Online dating services, like Match.com, constantly sift through their Web listings of personal characteristics, reactions and communications to improve the algorithms for matching men and women on dates. Police departments across the country, led by New York’s, use computerized mapping and analysis of variables like historical arrest patterns, paydays, sporting events, rainfall and holidays to try to predict likely crime “hot spots” and deploy officers there in advance.

Research by Professor Brynjolfsson and two other colleagues, published last year, suggests that data-guided management is spreading across corporate America and starting to pay off. They studied 179 large companies and found that those adopting “data-driven decision making” achieved productivity gains that were 5 percent to 6 percent higher than other factors could explain.

The predictive power of Big Data is being explored — and shows promise — in fields like public health, economic development and economic forecasting. Researchers have found a spike in Google search requests for terms like “flu symptoms” and “flu treatments” a couple of weeks before there is an increase in flu patients coming to hospital emergency rooms in a region (and emergency room reports usually lag behind visits by two weeks or so).

Global Pulse, a new initiative by the United Nations, wants to leverage Big Data for global development. The group will conduct so-called sentiment analysis of messages in social networks and text messages — using natural-language deciphering software — to help predict job losses, spending reductions or disease outbreaks in a given region. The goal is to use digital early-warning signals to guide assistance programs in advance to, for example, prevent a region from slipping back into poverty.

In economic forecasting, research has shown that trends in increasing or decreasing volumes of housing-related search queries in Google are a more accurate predictor of house sales in the next quarter than the forecasts of real estate economists. The Federal Reserve, among others, has taken notice. In July, the National Bureau of Economic Research is holding a workshop on “Opportunities in Big Data” and its implications for the economics profession.

Big Data is already transforming the study of how social networks function. In the 1960s, Stanley Milgram of Harvard used packages as his research medium in a famous experiment in social connections. He sent packages to volunteers in the Midwest, instructing them to get the packages to strangers in Boston, but not directly; participants could mail a package only to someone they knew. The average number of times a package changed hands was remarkably few, about six. It was a classic demonstration of the “small-world phenomenon,” captured in the popular phrase “six degrees of separation.”

Today, social-network research involves mining huge digital data sets of collective behavior online. Among the findings: people whom you know but don’t communicate with often — “weak ties,” in sociology — are the best sources of tips about job openings. They travel in slightly different social worlds than close friends, so they see opportunities you and your best friends do not.

Researchers can see patterns of influence and peaks in communication on a subject — by following trending hashtags on Twitter, for example. The online fishbowl is a window into the real-time behavior of huge numbers of people. “I look for hot spots in the data, an outbreak of activity that I need to understand,” says Jon Kleinberg, a professor at Cornell. “It’s something you can only do with Big Data.”

Big Data has its perils, to be sure. With huge data sets and fine-grained measurement, statisticians and computer scientists note, there is increased risk of “false discoveries.” The trouble with seeking a meaningful needle in massive haystacks of data, says Trevor Hastie, a statistics professor at Stanford, is that “many bits of straw look like needles.”

Big Data also supplies more raw material for statistical shenanigans and biased fact-finding excursions. It offers a high-tech twist on an old trick: I know the facts, now let’s find ’em. That is, says Rebecca Goldin, a mathematician at George Mason University, “one of the most pernicious uses of data.”

Data is tamed and understood using computer and mathematical models. These models, like metaphors in literature, are explanatory simplifications. They are useful for understanding, but they have their limits. A model might spot a correlation and draw a statistical inference that is unfair or discriminatory, based on online searches, affecting the products, bank loans and health insurance a person is offered, privacy advocates warn.

Despite the caveats, there seems to be no turning back. Data is in the driver’s seat. It’s there, it’s useful and it’s valuable, even hip.

Veteran data analysts tell of friends who were long bored by discussions of their work but now are suddenly curious. “Moneyball” helped, they say, but things have gone way beyond that. “The culture has changed,” says Andrew Gelman, a statistician and political scientist at Columbia University. “There is this idea that numbers and statistics are interesting and fun. It’s cool now.”

Steve Lohr is a technology reporter for The New York Times.

NY Times: Using Google's Data to Reach Consumers

---

December 22, 2011

Using Google’s Data to Reach Consumers

By ANDREW ADAM NEWMAN

THE viruses that Google researchers usually focus on are those that strike computers, but in recent years, they have studied those that infect people, too.

Google correlated billions of flu-related Web searches from 2003 to 2008 with actual Centers for Disease Control and Prevention data over the same period. Then, because Web searchers’ Internet addresses indicated location, Google devised a formula to estimate regional flu activity based solely on searches, with a reporting lag of only about a day, outdoing C.D.C. flu reports, which typically are published a week or two after outbreaks.

In 2009, researchers from Google and the C.D.C. wrote an article summarizing their findings in the journal Nature, and stated that the new predictive model — now called Google Flu Trends and accessible in an interactive format online — could be a boon to public health. “Up-to-date influenza estimates may enable public health officials and health professionals to better respond to seasonal epidemics,” they wrote.

What the researchers probably did not predict was that the Google flu data would end up being the cornerstone of an advertising campaign.

For the Vicks Behind Ear Thermometer, a new product that determines temperature when placed in the soft area behind the ear, marketers wanted to reach mothers, the primary purchasers of thermometers.

•

A mobile campaign by Blue Chip Marketing Worldwide, which is based in Chicago, places the ads for the thermometer within popular apps like Pandora that collect basic details about users, including their sex and whether they are parents, and can pinpoint specific demographics to receive ads.

But not all mothers will see the ad on their smartphones. Rather, the ads will be sent only to devices that, according to Google, are in regions experiencing a high incidence of flu. Also, the ads will only be delivered to mothers within two miles of retailers that carry the thermometer, including Walmart, Target and Babies “R” Us.

“Flu levels in your area are high,” says the banner ad within an app. “Be prepared with Vicks revolutionary Behind Ear Thermometer.”

Tapping the ad, which also notes the nearest store that sells the thermometer (“Buy at Rite Aid .3 miles away.”), brings users to a product page with items including an informational video and a list of nearby retailers. Tapping a retailer reveals directions there.

Also handling the campaign, which was introduced in a limited way early in December and will be at full throttle when flu season peaks in January and February, is Where, the location-based mobile advertising network that is a division of PayPal, a unit of eBay.

“It understands how this mom lives and shops, and she receives the message in the most relevant manner,” said Stanton Kawer, chief executive of Blue Chip, about the campaign. “It is so fantastically targeted that it’s really amazing.”

The new thermometer is made by Kaz, which through licensing agreements markets thermometers under the Vicks and Braun brands (both owned by Procter & Gamble), and which also makes Vicks humidifiers.

Sales for thermometers follow flu season, picking up in October, peaking in January and February, and tapering off in March and April, according to Lara Peterson, a vice president for marketing at Kaz, a subsidiary of Helen of Troy Limited.

Smartphones are used by 53 percent of Americans ages 18 to 24 and 64 percent of those 25 to 34, according to Nielsen. Google reports that 79 percent of owners use them for shopping purposes like comparing prices and locating a retailer.

Organizations including the American Academy of Pediatrics now recommend against mercury thermometers (citing the danger of both mercury and glass). Sales of digital thermometers are up, growing about 17 percent from 2005 to 2010, according to a report from Mintel, a market research firm.

The American Academy of Pediatrics still recommends taking temperature rectally for newborns under 3 months, and not taking it orally until children are at least 4 years old. The newest generation of digital thermometers measure parts of the body including the underarm, forehead and inside the ear.

The new Vicks thermometer, with a suggested retail price of $40 to $50, is the first in the category made specifically to measure behind the ear, according to Ms. Peterson, of Kaz.

“It measures fever in a very noninvasive way,” said Ms. Peterson, adding that the location’s proximity to the carotid artery, which carries blood to the brain, ensures accurate readings.

•

Because feverish children can be particularly fidgety, the ease of use strikes a chord with parents. Kaz recently commissioned a study where the device was used to take the temperature of napping babies at a day care center, and 95 percent slept through it.

In addition to print and online advertising, also by Blue Chip, commercials for the thermometer will be shown in more than 2,600 pediatricians’ offices for two months beginning Jan. 15, through a deal with KidCare TV, which provides informational programming to waiting rooms.

As for the mobile campaign aimed at mothers in high-flu areas, Brian Morrissey, editor in chief of Digiday, an online publication that covers digital marketing and media, said being tracked by advertisers had a “creepy factor” for some consumers.

Many Facebook users, for example, balked when in 2007 it introduced Beacon, a program that shared their online purchases with their social network. “Creepiness comes in when consumers are surprised,” Mr. Morrissey said.

But he predicted consumers would actually like the thermometer campaign.

“It seems like they’ll pull it off, because they’re using the data in a smart way, for advertising that’s more relevant and useful,” Mr. Morrissey said.

Strategy + Business: A New Way to Gain Customer Insights

Published: November 7, 2011   Marketing, Media & Sales   A New Way to Gain Customer Insights How conjoint analysis, a tried-and-true market research tool, can be used to support organic growth. by David Meer When Dow Jones decided to revamp the Wall Street Journal in the mid-2000s, the newspaper had just endured five years of flat circulation and advertising revenue, and the whole industry was ailing. Although the Journal didn’t want to alienate its core readership, it wanted to attract new readers — in particular a younger demographic that advertisers would value. Dow Jones knew it had to make changes to its then 125-year-old newspaper. But the company’s bigger purpose was to understand the needs of an emerging segment of business news consumers that the Journal was not successfully reaching. To help develop its strategy, Dow Jones employed a variant of conjoint analysis, a technique that has been widely used in market research for 30 years. In a traditional conjoint analysis, survey respondents are asked which products or product attributes they value as a trade-off between two or more options, repeated in enough combinations to yield a reliable ranking of each attribute’s importance. Dow Jones used this type of analysis in a new way, to identify prospective readers and reveal their preferences. After its redesign to attract this new customer segment, the Journal (now part of the News Corporation) saw a 35 percent improvement in its efforts to add new subscribers through direct marketing, reversed a three-year slump in ad sales, and experienced an annual revenue improvement of US$25 million from new programs and pricing initiatives. In fact, for companies in industries as varied as luxury goods and retail banking, conjoint analysis is emerging as a strategic tool, providing actionable intelligence businesses can use to go beyond product optimization to support organic growth. The methodology needed to use conjoint analysis in this new way is very similar to the methodology that traditional practitioners of this type of analysis have used, but has a key variation: The marketing team uses the results to organize customer groups (and prospects) with similar preferences, providing a more detailed view of the categories they fall into, the needs they have, and the likelihood that they might become bigger (or smaller) sources of revenue. In other words, conjoint analysis has become a new source of insight into customer segments. Of course, it would be hard to find a company that hasn’t done some kind of customer segmentation, and using conjoint analysis is certainly not the only way to achieve it. Companies usually have a sense of who their real and prospective customers are, and have an idea of what each segment considers important. But by segmenting customers with the help of conjoint analysis, companies can develop a more layered form of intelligence, with implications for which segments to prioritize, which value propositions to offer them, and how to market to them. Looking for Luxury Shoppers In recent years, a manufacturer of luxury gifts had become dissatisfied with the pace of growth in one of its largest geographic markets. Was the company targeting the wrong customers? Using the wrong materials? Supporting a brand with an undifferentiated value proposition? Advertising ineffectively? The company thought that if it could answer these questions, it would gain some of the insights needed to transform its organic growth strategy. Using traditional conjoint analysis techniques, the manufacturer surveyed 2,000 luxury gift–buying consumers to find out the extent to which they were price- and brand-conscious; valued materials such as fine leather, fabrics, and metals; and wanted their gifts to elicit oohs and aahs from friends and family. The manufacturer then combined the data from the conjoint analysis with the results from other survey questions to define five customer segments and decided it had headroom — an opportunity to pick up significant market share — in several of those segments, including a group it called “classic shoppers.” Thanks to the conjoint analysis survey, the manufacturer knew that in the “classic shopper” segment, customers ascribed great importance to prestige, cared a lot about high-quality materials, and preferred designs that made bold statements. The least important attribute to this customer segment was price — these customers didn’t mind paying a premium to get what they wanted. The company might have already had an intuitive sense of these findings. However, the intelligence from the conjoint analysis was definitive. The results of the analysis have played a role in changing the company’s product line, changing what happens within the company’s distribution channels, and changing how and where the company spends its marketing dollars. Protecting Profits at a Bank In another recent example, a European bank was picking up signals that regulators were going to force it to become more transparent about the costs of loan protection, a product the bank made available to consumers who held unsecured loans. The bank didn’t make money selling unsecured loans, but it made a considerable profit selling insurance that guaranteed payment if a borrower lost his or her job or otherwise suffered an interruption of income. What would happen to the business model if regulators insisted on changes? Would there be a way to keep making money in the business of unsecured loans and loan protection? The bank used a conjoint analysis survey of 1,600 people who had unsecured loans to estimate price elasticity for the loans themselves and for loan protection insurance. This was a way of anticipating the options it would have in the event that the regulatory environment changed, and banks were forced to raise (or lower) prices on either loan or loan-protection products. The conjoint analysis answered the price elasticity question in the aggregate. After the bank clustered the panelists into five segments, it was also able to answer this question in a more granular way. For instance, customers in a segment the bank called “bargain hunters” were very sensitive to pricing — this group would not pay more to take out a loan or to insure it. By contrast, customers in a segment the bank designated as “personal bankers” (those who liked the high-touch approach, were willing to hear advice, and were open to special offers) were not particularly price sensitive. There would be ways, even in the event of a regulatory change, of selling this segment higher-priced unsecured loans and loan protection and profiting from it. Indeed, one of the intriguing things about this bank’s use of conjoint analysis was the broad utility of the results. Although the analysis started off as a way to test price elasticity and prepare for external changes, the information the conjoint analysis generated — not only about how customers would respond in the event of a price increase, but also about more basic findings such as how people make borrowing decisions and how they think about financial providers — allowed the bank to identify tailored product strategies that would appeal to all its customer segments. The company decided its existing product would work for some of those segments, but that it should probably develop a no-frills product for the “bargain hunters” among its customers and a premium product for its “personal bankers.” Segmenting for Growth In an era of cautious consumer spending, many companies are looking for new ways to identify growth opportunities through improved customer insight. Conjoint analysis is at the forefront of this effort. The analytic rigor it brings is helping creative companies move forward with promising initiatives that they may have thought sounded good but couldn’t agree to implement without the data to back them up. Other companies find that it is generating avenues for organic growth they might not have come up with on their own. In this way, conjoint analysis, which has historically informed relatively narrow product decisions (enhance this feature, remove that one) is turning out to have bigger strategic implications. It is a powerful tool that can fundamentally change companies’ perceptions about where opportunity lies and how to pursue it.

 

Media: Why you can't lie to me

Why You Can't Lie to Me

Douglas Quenqua, Apr 13, 2011 05:24 PM

You've seen the TV show but there's some real science behind facial coding It's true: Advertisers lie. But here's another fact: so do consumers. Anyone who's ever monitored a focus group can tell you that. Maybe it's not lying of the shameless "It was like that when I got here" variety. It's more the "I base my grocery-buying decisions on nutrition, not packaging" kind of fibs. But either way, it's misleading and advertisers pay the price for it. Luckily, faces don't lie. Someone watching an offensive commercial is going to look disgusted, just as the lips of someone watching a poignant ad will quiver as he fights back tears. All the questionnaires in the world can't match what a trained professional (or now, software) can learn from watching someone's face. If only there were a way to harvest the facial expressions of consumers of all demographics from all over the world, then translate them into actionable marketing data. To be clear, there isn't...yet. But there are a handful of companies doing some clever things with facial coding and eye tracking and other whizbang technologies to help advertisers better gauge how their customers really feel about them. Or at least how they feel about their products, packaging, ads, store designs and Web sites. "Darwin realized that the face was the best place in the body where we reflect and communicate our emotions," says Dan Hill, president and founder of Sensory Logic, a market-research firm based in Minneapolis, Minn. "It's the only place in the body where muscles attach right to the skin." Darwin's insight was based on the observation that blind people make the same facial expressions, corresponding to the same emotions, as sighted people, suggesting that facial expressions are not learned but hardwired into our brains. Facial coding may owe a philosophical debt to Darwin. But the technology traces back to a psychologist named Paul Ekman, who in the late 1970s developed what's known as the Facial Access Coding System, or FACS. The system, which has so far been of greater use to animators than advertisers, reduces all facial movements to mathematical units. By measuring the movements of a person's face, one can interpret that person's emotions. Thirty years after the development of FACS, companies like Sensory Logic are helping advertisers apply that technology to market research. The primary way Sensory Logic does this is exposing test subjects to a commercial or a radio ad while a Logitech camera records the subject's face (the process is called "exposure"). Experts then decode the facial expressions and line them up with the content the subject was watching to determine, down to the second, how that person really felt about what he was watching. In the past few years, the company has even begun offering this service remotely. As long as a subject has a camera installed in his home computer, the company can record his facial expressions as he watches the content from home. Other companies are now working on bringing the technology to smartphones. The invention of the Flip camera has also allowed Sensory Logic to combine facial coding with traditional focus groups. Now market researchers can record the faces of focus-group subjects to see when their expressions betray their words. The company can also record the faces of subjects answering questionnaires to look for "any gaps or illumination between what they see and feel," says Hill. To appreciate the value of such tests, consider the example of a pharmaceutical company trying to determine what mattered most to doctors when prescribing medicines. When asked why they choose one drug over another, "of course the first thing the doctors say is efficacy of the drug," says Hill, "but when we looked at the emotional data they hardly emoted about that at all." What did capture the doctors' hearts, according to their faces? "Ease of filing for reimbursement and the payment schedules and so forth," says Hill. In other words, "things that were much more central to their own pocketbook. Those things were much more of interest to the doctors." This kind of research has also helped quantify the shortcomings of traditional focus groups. Most advertisers already know that people in a focus-group setting will sometimes adjust their answers to make themselves look smart, or to cozy up to an alpha dog in the room. But Sensory Logic has taken this further by looking at how people of different ethnicities and demographic backgrounds behave differently. "Hispanics tend to respect authority, which means they might be a little less forthcoming to make a criticism," Hill says. "African-Americans are pretty straightforward. In our research they come closest to just saying how they feel. But whites and Hispanics and Asian-Americans - you're talking about a gap that's double digits between how much they're willing to admit they like something" and how much they really do. In an age when advertisers spend a lot of time and money crafting discrete messages for different ethnic groups, such lapses in insight can add up to significant waste. Sensory Logic is also eager to take facial coding outside the research room and is looking to partner with companies that provide eye-tracking technology. Such companies - EyeTracking, located in San Diego, and Tobii, in Sweden, are two - use special glasses worn by the consumer to determine where his eyes settle as he walks through a retail environment. Combine that with facial-coding technology and you can tell not just where people are looking, but how they feel about what they're seeing. The application has exciting implications for retailers, who could use it when designing their store layouts. Of course, it wouldn't be an emerging technology if there weren't disagreements over how best to apply it. Gallup & Robinson, a market research firm in Pennington, N.J., practices a different brand of consumer face reading. "Our focus is on what we call emotional valence," says G&R president Scott Purvis. "Most of the other systems are on what we would call arousal." To the layperson, the difference between what Sensory Logic and Gallup & Robinson do may seem tiny. But it's significant. Put simply, emotional valence is the measurement of all human reactions on a scale of negative to positive. So Gallup & Robinson is less concerned with whether you're smiling or frowning than how intensely you're doing one or the other. "The idea is that people have a positive and a negative rewards-type system in their brain," says Purvis, "and these things operate both at the same time, so you can feel both very positive and negative toward a stimulus at the same time. It's not ends of a scale, it's two different systems, so that's why people are often ambivalent about many of the things they're seeing. We're trying to get at that relationship, rather than whether somebody is disgusted or happy." Purvis uses the example of a typical pharmaceutical commercial, the kind that begins with patients suffering from an ailment, then features a (frequently animated) illustration of the treatment working, followed by scenes of a healthy, happy patient. The intensity with which a viewer follows those phases of emotion "can be very important in terms of overall effectiveness of the commercial," he says. "And I don't know that we understand it as well when looking at it from the point of view of, 'Did you like it?'" Not surprisingly, Hill doesn't put much stock in emotional valence. He points out that there is a wide range of emotions that can be considered "negative," not all of which are the ones you may want your commercial to inspire. "Sadness versus anger versus fear are all quote-unquote negative emotions," he says. "But those emotions can not only be vastly different in their meanings, but also in their implications for your creative." Hill, too, uses the example of a pharmaceutical commercial to illustrate his point. "You're trying to create a problem-solution commercial, so you're supposed to make your viewer feel sadness at first because they're disappointed with the status quo," he says. "But what if you're actually making them angry because they're confused? Then you've lost them." Hill even has a term for it. He calls it being "off-emotion," as in being off-message. Measuring emotional valence comes with some other trade-offs. Rather than making video of a subject, Gallup & Robinson attaches electrodes to his or her face, specifically at the frown and smile muscles. The process, known as electromyography, or EMG, may do a more accurate job of measuring the intensity of muscular reaction - but it doesn't travel well. "There's some work being done now to use the camera to distribute it over the Internet so Webcams would be able to get an EMG-type response," says Purvis. "But so far it hasn't gotten to the point where I think people have a lot of confidence in it." Regardless of which method you put your faith in, there's a good reason that both remain little more than a theoretical threat to traditional market research: They have a serious scalability problem. Both approaches require trained professionals to translate the results. Training such professionals is said to take up to 100 hours; of course the process of watching the videos and drawing conclusions takes many hours more. So while face reading makes a nice complement to focus groups or questionnaires, they've never really been scalable enough to replace them. Enter ThirdSight. This 11-person firm in the Netherlands may be the newest thing in face-reading market research, having been spun off from the University of Amsterdam on January 1, 2011. ThirdSight's signature technology is software that reads and interprets video of test subjects' facial expressions, eliminating the need for a pricey, highly trained researcher to wade through hours of slow-motion video. "The software detects the face of the subject, then localizes landmarks like mouth corners, cheeks and eyebrows and tracks these facial features over time with a grid like a spire," says Theo Gevers, scientific advisor at ThirdSight and an associate professor of computer science at the University of Amsterdam. The software is taught to recognize facial configurations that correspond to six different emotions, so it can tell instantly how a test subject is reacting to content. ThirdSight has even created a smartphone app that can read and interpret a user's facial expressions. "It's not yet real time," says Gevers, "so it will record your face and then process it." But that will change quickly as smartphone CPUs become faster and more powerful. The drawback to ThirdSight's process is an admitted lack of accuracy. "We might have some disadvantages in the sense that people who have been trained for 100 hours might be a little bit better" than the software is at reading the subtleties of facial expressions, Gevers notes. "We have a fairly accurate system, but it's not as good as a human operator." Still, the potential effect of scalable facial coding on the world of market research is huge. Imagine a world in which consumers anywhere could download an iPhone app that records their faces as they watch a commercial - which also plays on their phones, incidentally - then instantly beams back that person's emotions to a waiting database. Today, companies like Sensory Logic and Gallup & Robinson are limited to running sessions with about 40 people; more than that would simply be untenable given the man-hours that go into interpreting the results. But a smartphone app equipped with software that can read faces in real-time has the potential to make facial coding an easy, inexpensive mass-market research tool with a higher level of reliability than anything available today. "The conservative estimation is that 95 percent of people's thought activity isn't fully conscious - that we're mostly intuitive, subconscious decision makers," says Hill. That, more than any other reason, is why simply asking consumers how they feel about a product or campaign is bound to come up short. No matter how much they want to tell the truth, people just might not be able to. Bypassing the consumer to go directly to his brain, courtesy of his expressions, may be the market research industry's best chance to save face.

WSJ: The Really Smart Phone

WHAT THEY KNOW
APRIL 23, 2011

The Really Smart Phone

Researchers are harvesting a wealth of intimate detail from our cellphone data, uncovering the hidden patterns of our social lives, travels, risk of disease—even our political views.

By ROBERT LEE HOTZ

Photo-illustration by Adam Magyar

'Phones can know,' says an MIT researcher. 'People can get this god's-eye view of human behavior.'

Apple and Google may be intensifying privacy concerns by tracking where and when people use their mobile phones—but the true future of consumer surveillance is taking shape inside the cellphones at a weather-stained apartment complex in Cambridge, Mass.

For almost two years, Alex Pentland at the Massachusetts Institute of Technology has tracked 60 families living in campus quarters via sensors and software on their smartphones—recording their movements, relationships, moods, health, calling habits and spending. In this wealth of intimate detail, he is finding patterns of human behavior that could reveal how millions of people interact at home, work and play.

Through these and other cellphone research projects, scientists are able to pinpoint "influencers," the people most likely to make others change their minds. The data can predict with uncanny accuracy where people are likely to be at any given time in the future. Cellphone companies are already using these techniques to predict—based on a customer's social circle of friends—which people are most likely to defect to other carriers.

A wave of ambitious social-network experiments is underway in the U.S. and Europe to track our movements, probe our relationships and, ultimately, affect the individual choices we all make. WSJ's Robert Lee Hotz reports.

The data can reveal subtle symptoms of mental illness, foretell movements in the Dow Jones Industrial Average, and chart the spread of political ideas as they move through a community much like a contagious virus, research shows. In Belgium, researchers say, cellphone data exposed a cultural split that is driving a historic political crisis there.

And back at MIT, scientists who tracked student cellphones during the latest presidential election were able to deduce that two people were talking about politics, even though the researchers didn't know the content of the conversation. By analyzing changes in movement and communication patterns, researchers could also detect flu symptoms before the students themselves realized they were getting sick.

"Phones can know," said Dr. Pentland, director of MIT's Human Dynamics Laboratory, who helped pioneer the research. "People can get this god's-eye view of human behavior."

So far, these studies only scratch the surface of human complexity. Researchers are already exploring ways that the information gleaned from mobile phones can improve public health, urban planning and marketing. At the same time, researchers believe their findings hint at basic rules of human interaction, and that poses new challenges to notions of privacy.

"We have always thought of individuals as being unpredictable," said Johan Bollen, an expert in complex networks at Indiana University. "These regularities [in behavior] allow systems to learn much more about us as individuals than we would care for."

Today, almost three-quarters of the world's people carry a wireless phone. That activity generates immense commercial databases that reveal the ways we arrange ourselves into networks of power, money, love and trust. The patterns allow researchers to see past our individual differences to forms of behavior that shape us in common.

As a tool for field research, the cellphone is unique. Unlike a conventional land-line telephone, a mobile phone usually is used by only one person, and it stays with that person everywhere, throughout the day. Phone companies routinely track a handset's location (in part to connect it to the nearest cellphone tower) along with the timing and duration of phone calls and the user's billing address.

Typically, the handset logs calling data, messaging activity, search requests and online activities. Many smartphones also come equipped with sensors to record movements, sense its proximity to other people with phones, detect light levels, and take pictures or video. It usually also has a compass, a gyroscope and an accelerometer to sense rotation and direction.

What They Know

A Wall Street Journal investigation into the world of digital privacy.

• The Web's New Gold Mine: Your Secrets
• Sites Feed Personal Details To New Tracking Industry
• On Web's Cutting Edge, Anonymity in Name Only

Advances in statistics, psychology and the science of social networks are giving researchers the tools to find patterns of human dynamics too subtle to detect by other means. At Northeastern University in Boston, network physicists discovered just how predictable people could be by studying the travel routines of 100,000 European mobile-phone users.

After analyzing more than 16 million records of call date, time and position, the researchers determined that, taken together, people's movements appeared to follow a mathematical pattern. The scientists said that, with enough information about past movements, they could forecast someone's future whereabouts with 93.6% accuracy.

The pattern held true whether people stayed close to home or traveled widely, and wasn't affected by the phone user's age or gender.

"For us, people look like little particles that move in space and that occasionally communicate with each other," said Northeastern physicist Albert-Laszlo Barabasi, who led the experiment. "We have turned society into a laboratory where behavior can be objectively followed."

Only recently have academics had the opportunity to study commercial cellphone data. Until recently, most cellphone providers saw little value in mining their own data for social relationships, researchers say. That's now changing, although privacy laws restrict how the companies can share their records.

Several cellphone companies in Europe and Africa lately have donated large blocks of calling records for research use, with people's names and personal details stripped out.

"For the scientific purpose, we don't care who the people are," said medical sociologist Nicholas Christakis at Harvard University, who is using phone data to study how diseases, behavior and ideas spread through social networks, and how companies can use these webs of relationships to influence drug marketing and health-care decisions.

His work focuses on "social contagion"—the idea that our relationships with people around us, which are readily mapped through cellphone usage, shape our behavior in sometimes unexpected ways. By his calculation, for instance, obesity is contagious. So is loneliness.

Even though the cellphone databases are described as anonymous, they can contain revealing personal details when paired with other data. A recent lawsuit in Germany offered a rare glimpse of routine phone tracking. Malte Spitz, a Green party politician, sued Deutsche Telekom to see his own records as part of an effort by Mr. Spitz to highlight privacy issues.

In a six-month period, the phone company had recorded Mr. Spitz's location more than 35,000 times, according to data Mr. Spitz released in March. By combining the phone data with public records, the news site Zeit Online reconstructed his daily travels for months.

Read More

• Our Social Networks, Ourselves
• Google Defends Way It Gets Phone Data
• Avoiding Mobile Trackers

In recent days, Apple Inc. triggered privacy alarms with the news that its iPhones automatically keep a database of the phone's location stretching back for months. On Friday, The Wall Street Journal reported that both Apple and Google Inc. (maker of the Android phone operating system) go further than that and in fact collect location information from their smartphones. A test of one Android phone showed that it recorded location data every few seconds and transmitted it back to Google several times an hour.

Google and Apple have said the data transmitted by their phones is anonymous and users can turn off location sharing.

"We can quantify human movement on a scale that wasn't possible before," said Nathan Eagle, a research fellow at the Santa Fe Institute in New Mexico who works with 220 mobile-phone companies in 80 countries. "I don't think anyone has a handle on all the ramifications." His largest single research data set encompasses 500 million people in Latin America, Africa and Europe.

Among other things, Mr. Eagle has used the data to determine how slums can be a catalyst for a city's economic vitality. In short, slums provide more opportunities for entrepreneurial activity than previously thought. Slums "are economic springboards," he said.

Cellphone providers are openly exploring other possibilities. By mining their calling records for social relationships among customers, several European telephone companies discovered that people were five times more likely to switch carriers if a friend had already switched, said Mr. Eagle, who works with the firms. The companies now selectively target people for special advertising based on friendships with people who dropped the service.

At AT&T, a research team led by Ramon Caceres recently amassed millions of anonymous call records from hundreds of thousands of mobile-phone subscribers in New York and Los Angeles to compare commuting habits in the two metropolitan areas.

Dr. Caceres, a lead scientist at AT&T Labs in Florham Park, N.J., wanted to gauge the potential for energy conservation and urban planning. "If we can prove the worth of this work, you can think of doing it for all the world's billions of phones," he said.

Thousands of smartphone applications, or "apps," already take advantage of a user's location data to forecast traffic congestion, rate restaurants, share experiences and pictures, or localize radio channels. Atlanta-based AirSage Inc. routinely tracks the movements of millions of cellphones to generate live traffic reports in 127 U.S. cities, processing billions of anonymous data points about location every day.

View Full Image

M. Scott Brauer for The Wall Street Journal

Alex Pentland, director of MIT's Human Dynamics Laboratory, above. Using location data, he said, 'I can say a lot about the music you like...your financial risks.'

What Phones Know

• One study found that the U.K.'s happiest time is 8 p.m. Saturday; its unhappiest day is Tuesday.
• European phone companies discovered their customers were five times more likely to switch carriers if friends had switched, allowing the companies to target their ads.
• Another study was able to determine that two people were talking about politics—without the researchers hearing the call

As more people access the Internet through their phones, the digital universe of personal detail funneled through these handsets is expanding rapidly, and so are ways researchers can use the information to gauge behavior. Dr. Bollen and his colleagues, for example, found that the millions of Twitter messages sent via mobile phones and computers every day captured swings in national mood that presaged changes in the Dow Jones index up to six days in advance with 87.6% accuracy.

The researchers analyzed the emotional content of words used in 9.7 million of the terse 140-character text messages posted by 2.7 million tweeters between March and December 2008. As Twitter goes, so goes the stock market, the scientists found.

"It is not just about observing what is happening; it is about shaping what is happening," said Dr. Bollen. "The patterns are allowing us to learn how to better manipulate trends, opinions and mass psychology."

Some scientists are taking advantage of the smartphone's expanding capabilities to design Android and iPhone apps, which they give away, to gather personal data. In this way, environmental economist George MacKerron at the London School of Economics recruited 40,000 volunteers through an iPhone app he designed, called Mappiness, to measure emotions in the U.K.

At random moments every day, his iPhone app prompts the users to report their moods, activities, and surroundings. The phone also automatically relays the GPS coordinates of the user's location and rates nearby noise levels by using the unit's microphone. It asks permission to photograph the locale.

By early April, volunteers had filed over two million mood reports and 200,000 photographs.

Publicly, Mr. MacKerron uses their data to chart the hour-by-hour happiness level of London and other U.K. cities on his website. By his measure, the U.K.'s happiest time is 8 p.m. Saturday; its unhappiest day is Tuesday.

Perhaps less surprisingly, people are happiest when they are making love and most miserable when sick in bed. The most despondent place in the U.K. is an hour or so west of London, in a town called Slough.

On a more scholarly level, Mr. MacKerron is collecting the information to study the relationship between moods, communities and the places people spend time. To that end, Mr. MacKerron expects to link the information to weather reports, online mapping systems and demographics databases.

Several marketing companies have contacted him to learn whether his cellphone software could help them find out how people feel when they are, for instance, near advertising billboards or listening to commercial radio, he said.

Mr. MacKerron said he's tempted—but has promised his users that their personal information will be used only for scholarly research. "There is a phenomenal amount of data we can collect with very little effort," he said.

Some university researchers have begun trolling anonymous billing records encompassing entire countries. When mathematician Vincent Blondel studied the location and billing data from one billion cellphone calls in Belgium, he found himself documenting a divide that has threatened his country's ability to govern itself.

Split by linguistic differences between a Flemish-speaking north and a French-speaking south, voters in Belgium set a world record this year, by being unable to agree on a formal government since holding elections last June. Belgium's political deadlock broke a record previously held by Iraq.

The calling patterns from 600 towns revealed that the two groups almost never talked to each other, even when they were neighbors.

This social impasse, as reflected in relationships documented by calling records, "had an impact on the political life and the discussions about forming a government," said Dr. Blondel at the Catholic University of Louvain near Brussels, who led the research effort.

The MIT smartphone experiment is designed to delve as deeply as possible into daily life. For his work, Dr. Pentland gave volunteers free Android smartphones equipped with software that automatically logged their activities and their proximity to other people. The participants also filed reports on their health, weight, eating habits, opinions, purchases and other personal information, so the researchers could match the phone data to relationships and behavior.

The current work builds on his earlier experiments, beginning in 2004, conducted in an MIT dormitory that explored how relationships influence behavior, health, eating habits and political views. Dr. Pentland and his colleagues used smartphones equipped with research software and sensors to track face-to-face encounters among 78 college students in a dorm during the final three months of the 2008 presidential election.

Every six minutes, each student's phone scanned for any other phone within 10 feet, as a way to identify face-to-face meetings. Among other things, each phone also reported its location and compiled an anonymous log of calls and text messages every 20 minutes. All told, the researchers compiled 320,000 hours of data about the students' behavior and relationships, buttressed by detailed surveys.

"Just by watching where you spend time, I can say a lot about the music you like, the car you drive, your financial risk, your risk for diabetes. If you add financial data, you get an even greater insight," said Dr. Pentland. "We are trying to understand the molecules of behavior in this really complete way."

Almost a third of the students changed their political opinions during the three months. Their changing political ideas were related to face-to-face contact with project participants of differing views, rather than to friends or traditional campaign advertising, the analysis showed.

"We can measure their daily exposure to political opinions," said project scientist Anmol Madan at MIT's Media Lab. "Maybe one day, you would be able to download a phone app to measure how much Republican or Democratic exposure you are getting and, depending on what side you're on, give you a warning."

As a reward when the experiment was done, the students were allowed to keep the smartphones used to monitor them.

Write to Robert Lee Hotz at sciencejournal@wsj.com

NY Times: In a Data-Heavy Society, Being Defined by the numbers

 

---

April 22, 2011

In a Data-Heavy Society, Being Defined by the Numbers

By ALINA TUGEND

I HAVE a confession to make. I started using Twitter about six months ago and eagerly watched my “followers” rise — 20 to 30 to 40. I made it to 60 and suddenly plateaued — a few would follow and then (heartbreak) “unfollow.”

At one point, I signed up my sons, who didn’t even use Twitter, to follow me. While part of me was laughing at myself — how senseless was this? — I also took some pleasure in seeing my numbers rise.

Numbers and rankings are everywhere. And I’m not just talking about Twitter followers and Facebook friends. In the journalism world, there’s how many people “like” an article or blog. How many retweeted or e-mailed it? I’ll know, for example, if this column made the “most e-mailed” of the business section. Or of the entire paper. And however briefly, it will matter to me.

Offline, too, we are turning more and more to numbers and rankings. We use standardized test scores to evaluate teachers and students. The polling companies have already begun to tell us who’s up and who’s down in the 2012 presidential election. Companies have credit ratings. We have credit scores.

And although most people acknowledge that there are a million different ways to judge colleges and universities, the annual rankings by U.S. News & World Report of institutions of higher education have gained almost biblical importance.

“Numbers make intangibles tangible,” said Jonah Lehrer, a journalist and author of “How We Decide,” (Houghton Mifflin Harcourt, 2009). “They give the illusion of control.”

Too many people shopping for cars, for example, get fixated on how much horsepower the engine has, even though in most cases it really doesn’t matter, Mr. Lehrer said.

“We want to quantify everything,” he went on, “to ground a decision in fact, instead of asking whether that variable matters.”

Even before we could measure — and flaunt — numbers online, some had long been a reference point. Consider the many years that fans have meticulously followed baseball statistics.

And we often do need to find ways to measure and evaluate people and products in as objective a way as possible.

The trouble, though, is when we mindlessly and blindly rely on those numbers to tell us everything, said Sherry Turkle, a professor of social studies of science and technology and director of the Massachusetts Institute of Technology’s Initiative on Technology and Self.

Numbers become not just part of the way we judge and assess, but the only way.

“One of the fantasies of numerical ranking is that you know how you got there,” said Professor Turkle, who is the author of “Alone Together: Why We Expect More From Technology and Less From Each Other” (Basic Books, 2011). “But the problem is if the numbers are arrived at in an irrational way, or black-boxed, so we don’t understand how we got there, then what use are they?”

My colleague Michael Winerip recently wrote an article about an excellent and exceptionally dedicated middle-school teacher, with terrific performance evaluations. But a formula used by the New York Department of Education put the teacher in the seventh percentile of her teaching peers.

That formula used 32 variables plugged into a statistical model that “appears transparent, but is clear as mud,” Mr. Winerip wrote.

And even if we understand the numbers — something as apparently clear-cut as how many books an author sells — they aren’t always helpful.

Robin Black, author of the short story collection, “If I Loved You, I Would Tell You This,” (Random House, 2010) wrote a blog post on how fretting about the different ways to measure her book’s success has overshadowed why she wrote it in the first place.

“I go to a place where everything has a number,” Ms. Black told me. “How many advance copies, how many reviews, how many sales.”

Amazon recently made it possible for authors to check how many books they’ve sold and, using interactive maps, it even zeroes in on how many sales occurred in which cities.

“Twenty years ago, maybe every Sunday you looked at The New York Times best-seller list,” she said. “Now you can torture yourself 24 hours a day, seven days a week. It becomes an exercise in scab-picking.”

And those black-and-white statistics, while arguably irrefutable in one way, really tell us almost nothing. Amazon’s rankings of book sales, for instance — which anyone can view — can vary wildly based on the sale of very few books.

All those numbers help us lose sight of why we’re really doing what we’re doing. Ms. Black, for instance, said her books were largely about loss.

“I’ll get a letter from someone who says, ‘My daughter died, and reading your book really helped,’ ” Ms. Black said. “That’s so meaningful. How do I measure that against 500 Twitter followers?”

Eric Frankel is founder of a company called 10 Minutes to Change, which works on human resources strategies and talent management, which means figuring out how to improve workers’ performance.

He’s also a certified public accountant, so he knows the importance of numbers. But, he said: “Just because we have the skills and ability to put metrics on everything doesn’t mean we should. People are ever-changing, fascinating and incredibly frustrating.”

Metrics to evaluate employee potential, teamwork and salesmanship can’t and shouldn’t replace hands-on interpersonal skills and instinct, Mr. Frankel said. Even professional baseball scouts, who have access to every conceivable statistic, know true worth is “steeped in the intangibles garnered by close observation, team orientation, makeup, work habits, leadership skills and professional presence,” he said.

This reliance and overweening trust in numbers is to some extent generational, said Howard Gardner, a professor of cognition and education at Harvard Graduate School of Education.

“For almost anybody in the United States under the age of 25, the only models are quantifiable rankings,” he said.

So when students are researching a paper, how do they decide where to turn for the greatest expertise? Often, he said, by looking at what articles or papers online have the most hits.

“Let’s take Britannica versus Wikipedia,” said Professor Gardner, who is the author of “Truth, Beauty and Goodness Reframed” (Basic Books, 2011). “Should it be that whatever has the most hits or the most editors makes it better than someone who spent his life studying Kant?”

The obsession with numbers, he said, means we don’t trust or even look for the intangibles that can’t be measured, like wisdom, judgment and expertise.

We also lose a sense of ourselves as anything but a number and a rank, and start feeling bad if our numbers don’t measure up to others.

Mr. Lehrer, in a blog post titled Online Status Anxiety on his Web site, wrote, “What I’m most troubled by is the desire of individuals (especially myself) to constantly check up on these numbers, and to accept these measurements as a measure of something meaningful.”

He went on, “That’s why I wish there was a popular social platform that didn’t measure anything. I doubt such a platform will ever exist — we clearly want the explicit hierarchies, even when they drive us crazy — but it sure would be a relief.”

By the way, 320 people “liked” that blog post.

The most frustrating thing for those of us who have a tendency to obsess over rankings is that we know we can simply refuse to keep checking. And if we must see how our blog or book or number of online friends measure up, we can also remind ourselves that those statistics have only as much importance as we’re willing to invest in them.

Or as Ms. Black put it: “I have to stop worrying about numbers. I have to reclaim the ambiguous part of my own intelligence.”

E-mail: shortcuts@nytimes.com

WSJ: Why Are Fitting Rooms So Awful?

FASHION
APRIL 6, 2011

Why Are Fitting Rooms So Awful?

Stores Try To Beautify Spaces, 'Seduce' Shoppers to Buy More With Better Lighting, Mirrors, Design

By ELIZABETH HOLMES and RAY A. SMITH

You spot a gorgeous dress on a hanger and, hopes high, make your way to the fitting room.

The Dream Dressing Room

Stores are making over these spaces with a variety of features.

View Interactive

It's downhill from there. The door doesn't fully shut, the lighting makes you look ill and the frock looks frumpy in a mirror fit for a fun house.

The retail dressing room can be a total sales killer.

Now stores are making a big push to transform the dingy fitting room into a more flattering space. It's part of an ongoing struggle by retailers to coax consumers to spend more. Having saturated the market with stores over the last decade, they can't expand to increase sales. Instead, companies are trying to squeeze more profits out of their current fleet.

Retailers are taking steps to battle a sales killer: the dressing room. As Elizabeth Holmes explains, by fixing up the rooms, stores can increase the odds customers will buy clothing.

Customers who try on clothes in fitting rooms have a conversion rate—meaning they ultimately buy something they tried on—of 67%, according to retail consultant Envision Retail Ltd., of Surrey, England, based on observations of more than 8,000 shoppers. Customers who don't use the fitting rooms have only a 10% conversion rate. Shoppers who use the fitting rooms spend a third of their in-store time there.

Aesthetically, the dressing-room makeovers are an attempt to go from bland and barren to comfortable and inviting, as if the shopper is trying on clothing in her own home. Chandeliers are cropping up in luxurious anterooms, where racks of camisoles hang waiting to smooth the unexpected bulge. Inside the fitting rooms, wallpaper and back-lit mirrors aim to flatter, not frighten.

Before

To revamp its fitting rooms, Ann Taylor changed a bland space, above, into a lounging area for shoppers with clothing, accessories and seating, below.

After

Stores are focusing their design efforts on dressing rooms in the women's clothing sections. Men tend to buy without trying on in the store, says Paco Underhill, founding president of Envirosell, a retail behavioral research and consultancy firm, who has timed men's visits in stores.

The new design at Ann Taylor, a division of Ann Inc., is intended to replicate a shopper's walk-in closet, says Samantha Dorfman, senior vice president of store development, design and facilities. A huge chandelier and larger-than-life marketing poster fill the entrance to "seduce" customers back to the space, Ms. Dorfman says. It's an about-face from the barren, beige entrance to the fitting rooms in the current layout.

Old Navy, the bargain-priced division from Gap Inc., moved fitting rooms to the center of its new store design. The relocation came after customers referred to the old tucked-away location as the "dungeon," says Tom Wyatt, Old Navy's president. To further cater to its target customer—a time-starved, young mother—the chain added what it calls "quick change" areas. The half-circle spaces are enclosed by a curtain.

Noticing that many women like to go into a fitting room together, Anthropologie, a division of Urban Outfitters Inc., makes sure each room can accommodate more than one person. "They consider it a little bit of a party," says Co-President Wendy B. McDevitt.

This Bloomingdale's concept store has created a dressing 'pod' made of fabric that can be hoisted to the ceiling to make room for events. Bloomingdale's plans to use these in other stores as well.

For women shopping with children or a boyfriend or husband, department store chain Macy's Inc. has been gradually updating its stores to add communal waiting spots with flat-screen TVs tuned to either sports or cartoons and upholstered seating. The theory: Parking a husband, boyfriend or kids in that spot helps a shopper feel "less rushed and stressed," a Macy's spokeswoman said in an email.

Clothing stores typically allot about 20% of the square footage towards fitting rooms and storage and 80% towards displaying the merchandise, says Robin Kramer, head of Kramer Design Group, a retail branding and design firm which works with clients from Talbots to Alexander Wang. She has lobbied stores to dedicate more space to their fitting rooms. "It's the moment where you put the brand on your body and decide if that's what you want," Ms. Kramer says.

Anthropologie wants shoppers to feel 'relaxed and out of the fray,' says Co-President Wendy B. McDevitt.

Some retailers are filling the fitting room area with merchandise. At Ann Taylor, racks and drawers in the lounge area display undergarments such as shapewear and camisoles. Its sister division, Loft, also includes a bar of belts and jewelry. Sales associates manning the fitting room are able to sell more items to a customer without leaving the space.

Lighting is a critical component. "Any woman who goes into a fitting room that has really bad lighting, you look at yourself, you look at your skin and you're completely distracted," says Anthropologie's Ms. McDevitt. "Then you lose the real reason why you were in there."

The new Ann Taylor rooms have six sources of lighting and three types of bulbs, compared to one source and type of lighting in the old design. The mixture of ceramic metal halide, compact fluorescent and low-voltage bulbs is more flattering, Ms. Dorfman says.

Bloomingdale's has installed rear-lit three-way mirrors, which allow customers to see themselves from a variety of angles. There is also a ceiling-mounted light three feet back from the mirror to eliminate shadows on the shopper's face, says Jack Hruska, executive vice president of creative services at Bloomingdale's.

Old Navy added quick-change areas.

Personal touches at Old Navy include labels placed over a trio of bright yellow hooks, helping shoppers organize their haul. "I love it!" and "Not for me," the signs read. Anthropologie writes shoppers' first names on the fitting room door, making it easier for sales staff to refer to them by name. That one-on-one time with a customer helps the store to engage shoppers on an intimate level, Ms. McDevitt says.

Because fitting rooms are highly trafficked areas, stores must make the design durable as well aesthetically pleasing. At Ann Taylor, the gray tones of the custom-designed floral wallpaper mask scuffs. The leather benches were chosen for the material's resistance to stains, like lipstick.

To help shoppers get the attention of salespeople, some fitting rooms at Bloomingdale's, such as those in the intimates departments, have phones. Customers are automatically connected to a sales associate to obtain another size, or ask questions regarding fit. Bloomingdale's says the phones are used frequently.

Victoria's Secret, a lingerie chain from Limited Brands Inc., is testing buttons in a small number of its fitting rooms which, when pushed, alert a sales associate via headset. The system also keeps track of how many people use the fitting rooms, how long they stay and how many times they ask for help, says Marge Laney, president of Houston-based manufacturer Alert Technologies, which installed the Victoria's Secret system.

Write to Elizabeth Holmes at elizabeth.holmes@wsj.com and Ray A. Smith at ray.smith@wsj.com

Copyright 2011 Dow Jones & Company, Inc. All Rights Reserved

Performance Insider: Putting It All Into Context

Putting It All Into Context

Jesse Chenard, Mar 31, 2011 01:15 PM

As each phase of online offerings prove, there are many reasons we spend an increasing amount of time searching, emailing, connecting with friends, colleagues and family, and entertaining ourselves with the amazing array of options available via a browser or mobile app. What state of mind are you in when you read your favorite news source, check sports scores, or gossip sites? What about when you are in the market for something (could be a car, could be a new recipe for dinner, information on health matters, whatever) or when you just need to connect, to feel like the people, places and opinions that matter most to you are close and accessible? Despite all the digital analytics, it's surmising your state of mind that is the ultimate goal: are you searching car sites because you are in market or because you just love to see new cars, or maybe you need to know what is coming down the pike in automotive offerings because your job requires it? What emotions are triggered by content is why endemic advertising is so highly prized. Knowing someone is actively engaged reading about new car models is one indicator that they may be in market for purchasing a car soon. Medical information about specific conditions is valuable to marketers of medicine for that condition because people who have the condition, or those that treat them, read it closely. That's why one of the most time-honored ways advertisers choose to engage with their intended customers is via association with content that triggers emotions, matching what the marketer hopes to activate - association and action because the state of mind is ripe for what the marketer is offering. When considering context, online widens the options for marketers in surprising ways. When your message is associated with content that is aligned with your message, the 'halo' effect heightens the message's impact. For example, most of us care about safety features in the cars we drive, but it's not necessarily the only reason we choose to buy one model over another. While reading an article on safety measures for children, ads highlighting features specifically about car models and brands in the context of safety can greatly increase engagement and improve brand metrics for the advertisers. Since few can afford to place their message indiscriminately to the fragmented masses, consider all options of filtering to find your ideal consumer at the ideal time. This has been the promise that's fueled mobile marketing's growth for years now. It's also why audience segment targeting is widely used. However, in the rush to target exactly the right people at the exact right time, we sometimes overlook the most powerful association available: contextually relevant content. Bottom line: if you put your brand in context, within content that is aligned with your product or service, and associated with the brand promise, you will gain valuable impact. Just because placing ads in context within specific content is an age-old technique for heightening advertiser results, doesn't mean it's no longer valuable. Indeed, today's fragmented audience and marketplace make context even more important than ever.

NY Times: When the Data Struts Its Stuff

 

---

April 2, 2011

When the Data Struts Its Stuff

By NATASHA SINGER

IN an uncharted world of boundless data, information designers are our new navigators.

They are computer scientists, statisticians, graphic designers, producers and cartographers who map entire oceans of data and turn them into innovative visual displays, like rich graphs and charts, that help both companies and consumers cut through the clutter. These gurus of visual analytics are making interactive data synonymous with attractive data.

“Statistics,” says Dr. Hans Rosling, a professor of international health at the Karolinska Institute in Sweden, “is now the sexiest subject around.”

Dr. Rosling is a founder of Gapminder, a nonprofit group based in Stockholm that works to educate the public about disparities in health and wealth around the world — by offering animated interactive statistics online that help visitors spot trends on their own.

Hit the play button and an animated graphic, called Gapminder World, shows a constellation of brightly colored bubbles, each representing a different country, bouncing along over two centuries. Without ever having to view yawn-inducing numbers on gross domestic product per capita, you can watch some countries, like the United States, rapidly growing healthier and wealthier before your eyes while smaller bubbles, for countries like Congo, rise on the life expectancy axis even as they dip on the income line.

 
The advanced animation has let Dr. Rosling make wonky statistics about poverty as intuitive and potentially fascinating for viewers as a nature program about the Serengeti on TV. “If we show a herd of zebras, and one zebra has a bad leg and lags behind, you can see that immediately,” says Dr. Rosling, whose video clip from the BBC on health and wealth statistics has been viewed more than four million times on YouTube. “If one country gets left behind, you can see that, too.”

Visual analytics play off the idea that the brain is more attracted to and able to process dynamic images than long lists of numbers. But the goal of information visualization is not simply to represent millions of bits of data as illustrations. It is to prompt visceral comprehension, moments of insight that make viewers want to learn more.

“The purpose of visualization,” says Ben Shneiderman, founding director of the Human-Computer Interaction Laboratory at the University of Maryland, “is insight, not pictures.”

The growing field has implications for companies, governments, academic institutions, nonprofit groups, news organizations and marketers — just about anybody who tries to convey huge amounts of information in visual, interactive forms. But advances, he says, come with both benefits and risks.

On the benefit side, people become more engaged when they can filter information that is presented visually and make discoveries on their own.

On the risk side, Professor Shneiderman says, tools as powerful as visualizations have the potential to mislead or confuse consumers. And privacy implications arise, he says, as increasing amounts of personal, housing, medical and financial data become widely accessible, searchable and viewable.

“The visual analytics research community works on these issues,” he says, “but more needs to be done.”

In the 1990s, Professor Shneiderman developed tree mapping, which uses interlocking rectangles to represent complicated data sets. The rectangles are sized and colored to convey different kinds of information, like revenue or geographic region, says Jim Bartoo, the chief executive of the Hive Group, a software company that uses tree mapping to help companies and government agencies monitor operational data. When executives or plant managers see the nested rectangles grouped together, he adds, they should be able to immediately spot anomalies or trends.

In one tree-map visualization of a sales department on the Hive Group site, red tiles represent underperforming sales representatives while green tiles represent people who exceeded their sales quotas. So it’s easy to identify the best sales rep in the company: the biggest green tile. But viewers can also reorganize the display — by region, say, or by sales manager — to see whether patterns exist that explain why some employees are falling behind.

“It’s the ability of the human brain to pick out size and color” that makes tree mapping so intuitive, Mr. Bartoo says. Information visualization, he adds, “suddenly starts answering questions that you didn’t know you had.”

For entertainment value, the Hive Group has also posted a tree map of the 100 most popular songs on iTunes, updated every 24 hours.

The fact that serious software companies are now tree mapping the pop charts is a sign that data visualization is no longer just a useful tool for researchers and corporations. It’s also an entertainment and marketing vehicle.

In 2009, for example, Stamen Design, a technology and design studio in San Francisco, created a live visualization of Twitter traffic during the MTV Video Music awards. In the animated graphic, floating bubbles, each displaying a photograph of a celebrity, expanded or contracted depending on the volume of Twitter activity about each star. The project provided a visceral way for viewers to understand which celebrities dominated Twitter talk in real time, says Eric Rodenbeck, the founder and creative director of Stamen Design.

Information visualization has changed substantially in the 10 years since the studio has been in business, Mr. Rodenbeck says. Designers once created visual representations of data that would steer viewers to information that seemed the most important or newsworthy, he says; now they create visualizations that contain attractive overview images and then let users direct their own interactive experience — wherever it may take them.

“It’s not about leading with a certain view anymore,” he says. “It’s about delivering the view that gets the most participation and engagement.”

 

TO that end, the company has just introduced an initial version of mondowindow.com, a site that shows airline passengers a detailed satellite map of the landscape they are flying over — and lets them direct the view.

For passengers with Wi-Fi access who enter their airline and flight number on the site, mondowindow.com displays more than just the terrain below. It also offers information bubbles highlighting different place names, local landmarks and tourist attractions like schools and botanical gardens, and photos of native fauna, like a blue jay.

On the ground, we may live in a world of T.M.I. — too much information. But Stamen Design is betting that we will relish rich images of ground data when we are flying several miles high.

FT: Happiness - A measure of cheer

Source - http://www.ft.com/cms/s/0/990cbbde-11e5-11e0-92d0-00144feabdc0.html#ixzz19Q9canHr

Happiness: A measure of cheer

By Tim Harford

Published: December 27 2010 18:44 | Last updated: December 27 2010 18:44

‘The welfare of a nation can ... scarcely be inferred from a measure of national income.’

So the US Congress was warned in 1934 by Simon Kuznets, who thus continued a long tradition of pointing out that there is more to life than money. But the economist’s comments broke particular ground: they were attached to the first serious attempts to produce national income accounts – the tally of all that a country produces and earns – for the US. Kuznets and his small research team had built them, and he knew their limits.

Where Kuznets led, others have followed. From the upper echelons of the administration of President Barack Obama to the offices of Nicolas Sarkozy, his French counterpart, to David Cameron, UK prime minister, the goal of gauging a nation’s well­being has captured the imagination of policymakers. They join less likely countries such as Bhutan, whose mission to measure “gross national happiness” has made the Himalayan mountain kingdom a trendsetter.

Mr Cameron was the most recent to take up the cause, saying Britain needed to look for alternative measures that would show national progress “not just by how our economy is growing, but by how our lives are improving; not just by our standard of living, but by our quality of life”. While some analysts suspect each politician has his own motives – appearing nice to electors, flattering the economy and so on – the result has been to create a sense of momentum behind happiness economics.

EDITOR’S CHOICE

What do happiness yardsticks add up to? - Dec-27

Tim Harford: Call no man happy until he is a government statistic - Nov-19

Editorial Comment: A happy medium - Nov-26

Happiness rethink - Oct-22

Oxford economists draw up poverty meter - Jul-02

Samuel Brittan: We do not prosper by income or happiness alone - Aug-03

Embracing happiness is one thing; measuring it another. There are, broadly speaking, three approaches to measuring a country’s wellbeing. One is to use the long-established framework of national income accounts and adjust it to better reflect national welfare. The second is to collect data on objective measures that are plausibly related to wellbeing: anything from life expectancy to crime, suicide rates to income inequality. The third – and the three are not mutually exclusive – is to try to measure national welfare directly by asking people how they feel: the equivalent of measuring wealth by asking: “On a scale of one to 10, how rich are you?”

Intellectually the project of measuring national welfare seems to belong on the centre-left. National income accounts themselves – like unemployment statistics – rose to prominence in the 930s as Franklin Roosevelt tried to lead the US out of the Great Depression and the government recognised it understood very little about how the economy was really doing – beyond very badly. The second world war and the central planning mentality that came with it merely intensified the focus on the government being able to understand how the economy was functioning.

The countervailing view was most pithily expressed by John Cowperthwaite, the laisser-faire financial secretary of Hong Kong in the 1960s, who claimed he refused to collect economic statistics because it would only provide ammunition to the planners. His views have not prevailed. Kuznets won the Nobel Prize in economics in 1971 – only the third year in which it was awarded – and it is hard to find serious economists who think that collecting national income accounts is pointless. The question is whether there is a benefit in trying to supplement those with accounts of national welfare – or, to put it cutely, an index of national happiness.

Mr Cameron says that there is. “You cannot capture happiness on a spreadsheet any more than you can bottle it,” he declared last month. The happiness index project, he added, had a practical purpose: to help the government understand, “with evidence”, the best way of improving people’s wellbeing. In short, Mr Cameron believes happiness indices can help his Conservative-Liberal Democrat coalition government plan its way to a happier nation in the same way Kuznets’ national accounts helped policymakers respond to the Depression and the outbreak of war.

. . .

Before understanding what governments might do with measures of happiness, it is worth asking how to measure it. Although “neuro-economists” are not averse to sticking people into brain scanners with a view to gleaning their inner thoughts, the easiest way to find out whether somebody is happy is to ask them. This might seem uncontroversial; after all, it seems almost definitional that if you’re happy then you know it.

Yet even here there are two quite distinct approaches. The first, and best known, is to collect information about “life satisfaction”, asking people how satisfied they are with their lives, taken as a whole, on (for instance) a scale of one to 10, eliciting a once-and-for-all answer to the question of how life is working out.

The alternative is to zoom in on a particular set of events and ask what feelings they produced. One example is the “day reconstruction method” produced by researchers including Norbert Schwarz, a psychologist, the economist Alan Krueger (until recently chief economist of the US Treasury), and Daniel Kahneman, a psychologist and another winner of the Nobel memorial prize in economics. The DRM asks people to recall, episode by episode, the previous day’s events and the most prevalent accompanying feeling – stress, peace, exhaustion, elation.

In short, one approach measures life satisfaction and the other mood – and the two concepts are quite different ways to think about happiness. One survey comparing women in Rennes, France, and Columbus, Ohio, found that the American women were twice as likely to say they were very satisfied with their lives, but the Frenchwomen spent more of their day in a good mood. “We have tended for too long to use a single word to refer to a wide variety of things,” says Prof Kahneman. “In particular there’s a real need to distinguish between life satisfaction and mood or experienced happiness. They are quite distinct and they have different causes and consequences.” If politicians are to incorporate the measurement of happiness into the fabric of national statistical frameworks, this is a distinction they will have to start taking seriously.

One obvious objection to the new-found emphasis on indicators of national welfare is that it is not, in fact, new-found at all. The most famous (albeit contested) finding in happiness economics is the Easterlin Paradox – that money buys happiness in any given society but richer societies seem to be no happier than middle-income ones – which dates back to the 1970s.

Still, data on subjective wellbeing have been improving ever since. The UK has published Social Trends for four decades, an almanac for modern times that includes information on crime and the fear of it, attitudes to marriage, satisfaction with the National Health Service, calls to child protection numbers, air pollution, North Sea fish stocks and much else. Other countries do likewise.

. . .

This puts the wellbeing agenda in a strange place. Those who believe in it know that the data already exist: Andrew Oswald of Warwick University and a leading advocate of the use of well­being data points to the UK’s Labour Force Survey, which already includes questions on depression and anxiety. Prof Oswald wants to see more such questions and have more attention paid to them, but this is a request to do more rather than to do something profoundly different.

But professors Kahneman and Krueger, and others, do have plans for a more radical departure. They propose the publication of “time accounting” measures alongside regular national accounts. The US Bureau of Labour Statistics has, funded by the National Institute on Aging, begun to implement some of their proposals.

Time-use surveys use representative samples to see how a nation spends its time – whether cooking, commuting or watching television. They are a well-established part of the US statistical apparatus. The happiness element comes in when the surveys are combined with the DRM. Because this asks survey respondents to rank the emotions they felt while spending time in various activities, it can be used to produce a measure of how long people spend in a predominantly unpleasant state of mind – as well as the kinds of things they are doing at the time. (For the curious, the particularly miserable activities include commuting and working. Sex, lunch and dinner are rarely unpleasant.)

The Kahneman-Krueger team have developed a stripped-down version of the DRM, which can be conducted as a regular telephone survey, and the Bureau of Labour Statistics has already conducted its first survey with the new method. Pilot studies seem to produce results broadly faithful to the DRM; the first data from the BLS work are due early in 2011.

Will national time accounts – or some alternative measure of welfare – give politicians a tool of practical importance? Or will Cowperthwaite be proved correct: that the data will only encourage politicians to meddle? Prof Krueger thinks not. “This approach is particularly amenable to evaluating public policy interventions, as investing in roads and high-speed rail can affect commuting time, overtime laws affect work hours, and playgrounds affect leisure activities.”

National time accounts will do more than that, because they sidestep an insuperable problem of the life satisfaction approach: that one person’s five out of 10 might be another person’s eight out of 10. Up to a point this problem can be shrugged off, especially when looking at large sample sizes, but if there are systematic differences between the way people express these judgments – differences between urban and rural, white and black, men and women, Danish and French – the data cannot be used to make comparisons across these groups. (In one Eurobarometer survey, 64 per cent of Danes described themselves as “very satisfied” and only 16 per cent of French. It is tempting to question how much the survey really tells us about the relative wellbeing of France and Denmark.)

National time accounts measure how much time people spend engaged in activities where the strongest of many emotions is a negative one. This greatly lessens the interpersonal comparability problem (although it cannot remove it entirely). In the end it may produce controversial findings – imagine, for instance, if it were shown that women spent far more time doing things they disliked than men did, or that the elderly lived blessed lives compared with the young.

The UK Office for National Statistics will continue its consultation until mid-April. By then the first American time-use accounts should be available. If Mr Cameron is serious about happiness, he will be casting an optimistic glance across the Atlantic.

Copyright The Financial Times Limited 2010.

NY Times: Disney Tackles Major Theme Park Problem - Lines

 

December 27, 2010

Disney Tackles Major Theme Park Problem: Lines

By BROOKS BARNES

ORLANDO, Fla. — Deep in the bowels of Walt Disney World, inside an underground bunker called the Disney Operational Command Center, technicians know that you are standing in line and that you are most likely annoyed about it. Their clandestine mission: to get you to the fun faster.

To handle over 30 million annual visitors — many of them during this busiest time of year for the megaresort — Disney World long ago turned the art of crowd control into a science. But the putative Happiest Place on Earth has decided it must figure out how to quicken the pace even more. A cultural shift toward impatience — fed by video games and smartphones — is demanding it, park managers say. To stay relevant to the entertain-me-right-this-second generation, Disney must evolve.

And so it has spent the last year outfitting an underground, nerve center to address that most low-tech of problems, the wait. Located under Cinderella Castle, the new center uses video cameras, computer programs, digital park maps and other whiz-bang tools to spot gridlock before it forms and deploy countermeasures in real time.

In one corner, employees watch flat-screen televisions that depict various attractions in green, yellow and red outlines, with the colors representing wait-time gradations.

If Pirates of the Caribbean, the ride that sends people on a spirited voyage through the Spanish Main, suddenly blinks from green to yellow, the center might respond by alerting managers to launch more boats.

Another option involves dispatching Captain Jack Sparrow or Goofy or one of their pals to the queue to entertain people as they wait. “It’s about being nimble and quickly noticing that, ‘Hey, let’s make sure there is some relief out there for those people,’ ” said Phil Holmes, vice president of the Magic Kingdom, the flagship Disney World park.

What if Fantasyland is swamped with people but adjacent Tomorrowland has plenty of elbow room? The operations center can route a miniparade called “Move it! Shake it! Celebrate It!” into the less-populated pocket to siphon guests in that direction. Other technicians in the command center monitor restaurants, perhaps spotting that additional registers need to be opened or dispatching greeters to hand out menus to people waiting to order.

“These moments add up until they collectively help the entire park,” Mr. Holmes said.

In recent years, according to Disney research, the average Magic Kingdom visitor has had time for only nine rides — out of more than 40 — because of lengthy waits and crowded walkways and restaurants. In the last few months, however, the operations center has managed to make enough nips and tucks to lift that average to 10.

“Control is Disney’s middle name, so they have always been on the cutting edge of this kind of thing,” said Bob Sehlinger, co-author of “The Unofficial Guide: Walt Disney World 2011” and a writer on Disney for Frommers.com. Mr. Sehlinger added, “The challenge is that you only have so many options once the bathtub is full.”

Disney, which is periodically criticized for overreaching in the name of cultural dominance (and profits), does not see any of this monitoring as the slightest bit invasive. Rather, the company regards it as just another part of its efforts to pull every possible lever in the name of a better guest experience.

The primary goal of the command center, as stated by Disney, is to make guests happier — because to increase revenue in its $10.7 billion theme park business, which includes resorts in Paris and Hong Kong, Disney needs its current customers to return more often. “Giving our guests faster and better access to the fun,” said Thomas O. Staggs, chairman of Walt Disney Parks and Resorts, “is at the heart of our investment in technology.”

Disney also wants to raise per-capita spending. “If we can also increase the average number of shop or restaurant visits, that’s a huge win for us,” Mr. Holmes said.

Disney has long been a leader in technological innovation, whether that means inventing cameras to make animated films or creating the audio animatronic robots for the attraction It’s a Small World.

Behind-the-scenes systems — typically kept top secret by the company as it strives to create an environment where things happen as if by magic — are also highly computerized. Ride capacity is determined in part by analyzing hotel reservations, flight bookings and historic attendance data. Satellites provide minute-by-minute weather analysis. A system called FastPass allows people to skip lines for popular rides like the Jungle Cruise.

But the command center reflects how Disney is deepening its reliance on technology as it thinks about adapting decades-old parks, which are primarily built around nostalgia for an America gone by, for 21st century expectations. “It’s not about us needing to keep pace with technological change,” Mr. Staggs said. “We need to set the pace for that kind of change.”

For instance, Disney has been experimenting with smartphones to help guide people more efficiently. Mobile Magic, a $1.99 app, allows visitors to type in “Sleeping Beauty” and receive directions to where that princess (or at least a costumed stand-in) is signing autographs. In the future, typing in “hamburger” might reveal the nearest restaurant with the shortest wait.

Disney has also been adding video games to wait areas. At Space Mountain, 87 game stations now line the queue to keep visitors entertained. (Games, about 90 seconds in length, involve simple things like clearing runways of asteroids). Gaming has also been added to the queue for Soarin’, an Epcot ride that simulates a hang glider flight.

Blogs that watch Disney’s parks have speculated that engineers (“imagineers,” in the company’s parlance) are also looking at bigger ideas, like wristbands that contain information like your name, credit card number and favorite Disney characters. While Disney is keeping a tight lid on specifics, these devices would enable simple transactions like the purchase of souvenirs — just pay by swiping your wristband — as well as more complicated attractions that interact with guests.

“Picture a day where there is memory built into these characters — they will know that they’ve seen you four or five times before and that your name is Bobby,” said Bruce E. Vaughn, chief creative executive at Walt Disney Imagineering. “Those are the kinds of limits that are dissolving so quickly that we can see being able to implement them in the meaningfully near future.”

Dreaming about the future was not something on Mr. Holmes’s mind as he gave a reporter a rare peek behind the Disney operations veil. He had a park to run, and the command center had spotted trouble at the tea cups.

After running smoothly all morning, the spinning Mad Tea Party abruptly stopped meeting precalculated ridership goals. A few minutes later, Mr. Holmes had his answer: a new employee had taken over the ride and was leaving tea cups unloaded.

“In the theme park business these days,” he said, “patience is not always a virtue.”

FT: Brands get up close and personal

Brands get up close and personal

By Louise Lucas

Published: October 13 2010 20:49 | Last updated: October 13 2010 20:49

A Chinese woman squats on a low plastic stool, reaches for the shampoo and begins massaging it into her head, fingers working deep into her scalp in a kind of ritual. She is not the only Chinese woman to use conditioner like this – as testified by hundreds of hours of videotape, all filmed in Chinese bathrooms.

Perverse? Pornography? Neither: this is the Holy Grail of fast-moving consumer goods companies seeking to make products that consumers want. To that end, FMCG groups are earnestly studying human behaviour through focus groups, surveys and, more frequently in recent years, ways that are distinctly up close and personal.

The business of consumer research is intensifying on virtually every level. FMCG companies, which typically spend 2-3 per cent of sales on research and development, are going further afield geographically and conducting ever deeper research. New media are being harnessed, be it through social networks or blogs.

“Traditional research concentrated on the ‘what’. Now we are trying to establish the ‘why’,” says Simon Stewart, marketing director at Britvic, the beverages company. “We are not asking what they think about products and ideas but focusing on what makes them tick.”

The spread of digital social networks is one reason for the more sophisticated approach. But changing market dynamics are also behind the new focus. Competition and a weak economy make it ever more important to win customers, particularly in emerging markets. Indeed, so lucrative is the business of understanding what makes the Asian consumer tick that Singapore, never a place to miss a business opportunity, plans next year to set up a government-funded Institute for Asia Consumer Insights.

The same dynamics inform L’Oréal’s experimentation in bathroom photography. In similar experiments conducted in other markets, the French cosmetics group discovered that Korean women apply more potions and cosmetics to their faces than anyone else – a total of more than 25 creams and cosmetics at any one time, compared with 20-25 in Japan and more than double the amount used by American or European women. Japanese women may apply more than 50 coatings of mascara at one time, making European women – five to 10 coatings – look mere amateurs.

New ways to generate feedback

Fast-moving consumer goods companies are devoting more resources to digital media. The increasing spread of social media gives them the perfect tool to understand consumers. Starbucks has nearly 15m fans on Facebook, more than Barack Obama or South Park; Coca-Cola has 13.5m fans and Oreo cookies 11m.

In addition to sheer numbers of consumers, the social networks can provide quick feedback. Britvic, for instance, points out that teenage boys are both big Tango drinkers and heavy internet users, providing a constant source of information and feedback.

These companies are also replacing old-style surveys and focus groups with field work. Experts from a range of disciplines – historians, anthropologists, semioticians – are increasingly called on to study consumers at close hand in order to analyse trends and build databases.

“It all starts with observation,” says Patricia Pineau, who oversees L’Oréal’s consumer insights team, talking about the company’s “evaluation centres”, which involve “labs” decked out as bathrooms as well as cameras in people’s homes.

“Observing is necessary to decode exactly what [women] are trying to get and what they are attracted to. Sometimes it is the gesture that will reveal something that they really want to gain,” says Ms Pineau.

And what gestures. Japanese women spend a full minute massaging in lotions, patting their faces and eyelids. In Brazil, women change their nail polish every day to match their dress – and are wanton with the brush, painting their fingers along with their nails and relying on a cotton bud to mop up afterwards.

Back in the labs, scientists respond in turn. Thus Lancôme’s Génifique Youth Activating Concentrate has a stickier consistency in Japan than in Europe or the US, the better to pat in. Lip gloss is lighter in Japan, the better to allow the constant reapplication beloved of Japanese women.

Sometimes, however, gestures are not enough. Hence Nestlé’s strategy of embedding researchers in family homes, taking tea with a multigenerational Indian family or sitting cross-legged on the floor pounding pulses with a group of scarved women and their jeans-wearing daughters in Syria.

Chandan Mukherji is a veteran of embedding. The New Delhi-based head of consumer insights at Nestlé India has in recent years watched the lifestyle of the middle classes in towns and cities seep out to the villages and countryside. Seated at the family table, he saw housewives were turning to instant noodles but that they still supplemented them with extra vegetables and garnishes. Nestlé responded by supplementing its own instant noodles with vegetables.

“It really is an eye-opener for most of the teams, because they get a real life understanding,” says Mr Mukherji.

Also, after witnessing at first hand India’s small kitchens and vulnerability to rodent infestations, the Swiss group reduced pack sizes.

Drinks companies take a similarly up close and broad remit – looking not just at what people like to drink but where and how. Take SABMiller, the brewer. In the important market of Latin America, the urge to go out to where its drinks were actually drunk – and especially to mix that favourite cocktail of beer and football – was strong but the availability of venues less so.

Rob Priday, managing director of SABMiller’s Peruvian operations, says: “There’s a dearth of pubs in the country. In the terrorist years, people did not go out and now it is over, there are not enough pubs for on-premises drinking.”

Research showed that consumers want­ed more choice of places to drink and the results can be seen on the streets of Colombia and Peru. The company worked with local businesses and entrepreneurs to open café-style joints attached to soccer fields, where fathers can down a glass while their children play – or even have a kick-about themselves before sinking a glass or two. There are also bars where customers can play simulated golf or a few rounds of cards.

When Nestlé in Peru was debating which new flavour to introduce for its popular Besos de Moza – a marshmallow and cookie confection – it asked sweet-toothed consumers themselves to choose between the lucuma, a local fruit, or strawberry. “The feedback was amazing,” recalls Carlos Velasco, who heads the Peruvian operation. They chose lucuma, and Nestlé went on to sell 56m lucuma-flavoured units in 2009.

Asking consumers directly is valuable but FMCG companies also call on a range of experts: historians for historical context (say, the caste system in India), psychologists, anthropologists and ethnologists and even semioticians.

Greg Rowland, a semiotician based in London, advises FMCG companies on the messages they need to send to woo customers through their packaging and placing on supermarket shelves.

Supermarkets, Mr Rowland says, are a battleground between puritan and hedonistic pleasures – plain fruit and veg stalls as customers first walk in, with the indulgent pleasures of alcohol and chocolate waiting to assail them further down the aisles.

We instinctively want to stay on the sober side, says Mr Rowland, so packaging of indulgent items is classily low-key. Hence the naive smiley face on Innocent’s drink cartons or the use of white space on packaging for top-notch biscuits. Or take the heraldic devices on bottles of lager, designed to establish the brewer’s authority in the minds of drinkers. This is serious stuff, the emblems say, none of your “hooch”.

For all the time, money and expert investigation FMCG companies throw into understanding consumers, mistakes are made. During research into Japanese skincare, a L’Oréal team was surprised to see a woman supplementing her beauty routine with a tiny razor.

She slid it around her nose, below her eyebrows and on the nape of her neck. The reason: she was removing almost imperceptible hairs to get a better effect when powder was applied. As a non-daily ritual, and one that required the woman to bring her own razors (which were not on offer in L’Oréal’s labs), it had until then completely bypassed the ranks of researchers.

Delivery can miss a step too: SABMiller, identifying a desire for a sweeter drink for women in Peru, produced the cider-like Red – which turned out to be extremely popular with the men.

But for consumers, the question is surely whether they are just victims of clever advertising.

As Mr Rowland says: “You may think you are buying and eating this tin of biscuits. But really, the tin of biscuits is buying and eating you.”

Copyright The Financial Times Limited 2010.

WSJ: Research Into Human Sexuality Leaves a Lot to Be Desired

THE NUMBERS GUY
OCTOBER 9, 2010

Research Into Human Sexuality Leaves a Lot to Be Desired

• By CARL BIALIK

Let's talk about sex.

Nearly 6,000 people between the age of 14 and 94 did so at the behest of Indiana University researchers, who this week published results from their national survey. These respondents said, among other things, how often they have sex with people of the same or opposite genders; which sexual practices they engage in; and whether they used condoms. Sex researchers say it is the first study of this scope and scientific rigor since 1992, when the University of Chicago's National Opinion Research Center surveyed nearly 3,500 people.

The survey's results are fascinating, but not the final word on sex in America. While researchers took care to find a representative sample of people, the survey population limits or omits a few key groups that could have skewed the results, some health researchers say. And because of the inevitable challenge studies on sex face in persuading people to respond honestly, the findings should be treated with some skepticism.

One area of inquiry that tends to draw a lot of interest can be especially problematic, statistically speaking: comparing how each gender reports on sex. That came into play in the latest study, which found that boys age 14 to 17 say they use condoms 79% of the time during intercourse, compared to just 58% of girls in that age group who said their male partners used condoms.

Numbers Guy Blog

• Sex, and Studying It, Is Complicated

Those numbers might suggest that boys are exaggerating their condom use, or that girls are underreporting it. But the potential mismatch between the sexual encounters each group described also might account for some of the difference. Some respondents might have been having sex with people outside the 14-17 age range. This could affect the results if age influences whether someone uses a condom.

"Sometimes people portray this as if men and women in surveys are closed populations," says William Mosher, a statistician at the Centers for Disease Control and Prevention's National Center for Health Statistics, about the comparisons between male and female sexual behavior. "They are not closed populations."

Then there is the challenge of ensuring that people answer honestly about deeply private matters. Researchers who led the Indiana study acknowledge the difficulty of obtaining truthful information. "There is no perfect method," says Debby Herbenick, research scientist at Indiana University and co-author of the study.

Nonetheless, she says the Indiana group's approach had certain advantages. Researchers hired the survey company Knowledge Networks to interview people electronically. Many pollsters are skeptical of Web surveys because not everyone in the U.S. is online—or online often enough to see ads for Internet-based panels. But Knowledge Networks takes a different approach, contacting people via mail by randomly selecting addresses, and persuading about one in four or five to join a panel. If panelists lack the means to answer online polls, the company equips them.

Another problem with the study: Soldiers, prostitutes and Americans living abroad are unlikely to participate in sex surveys, either because they are outside the geographical range, or because they are reluctant to talk about illicit activities. Their absence can affect results, researchers say.

The survey also had limited statistical power to describe the sexual habits of subgroups of Americans, because of small sample sizes. This prevented researchers from learning much about same-gender sex. And, for instance, an analysis of whether black women who used marijuana before sex also used condoms was hampered by the presence of just eight such women in the sample.

While the anonymity of online surveys might make respondents more comfortable responding honestly, it also can be harder to verify their identity—perhaps the person who claims to be a 45-year-old woman is actually a 20-year-old man. For the last comprehensive survey on Americans' sexual behavior in 1992, researchers faced a different set of issues. Online polling wasn't an option, so interviewers went to people's homes, explains Robert T. Michael, an economist at the University of Chicago and a principal investigator on the study. To persuade respondents to divulge highly personal information, pollsters appealed to people's desire to help supply information and fight the spread of AIDS. "We had a compelling rationale, and our field people were good," Prof. Michael says.

"It's exciting, intellectually challenging research," Prof. Michael says of sex studies. "It's important to get it right."

Copyright 2009 Dow Jones & Company, Inc. All Rights Reserved

===

October 8, 2010, 7:16 PM ET

Sex, and Studying It, Are Complicated

My print column this week examines the fascinating numbers and tricky statistical issues that accompany sex surveys, such as a high-profile one published by Indiana University researchers this week.

This study in many ways is a descendant of one conducted by the University of Chicago’s National Opinion Research Center in 1992. Such studies are so infrequent because sexual research that looks beyond fertility and sexual health issues can be hard to fund, researchers say.

The latest study emerges in a different research climate than the one that produced the 1992 study. The Indiana study makes use of online polling, which wasn’t an option last time around. Knowledge Networks, which conducted the survey, uses mailing addresses to recruit its panel and provides Internet access to households that need it. This helps overcome some potential methodological drawbacks to online research, which also has an important potential advantage for research asking about sensitive topics: It allows researchers to ask about sex without ever personally asking anyone about sex. On the phone or in person, there is “more risk that respondents and research subjects are going to be less honest in their responses,” because of their desire to please interviewers, or not be embarrassed with them, said J. Michael Dennis, executive vice president at Knowledge Networks.

Another important advantage for online polling is cost. “For face-to-face interviews, to get really large numbers, it is considerably expensive,” said Debby Herbenick, research scientist at Indiana University and co-author of the recent study.

Then there are all the inherent challenges with getting people to talk about sex, for which pollsters have devised a wide range of strategies. For the 1992 survey, interviewers avoided asking directly about sensitive topics, either by giving respondents some questions in writing or by using a clever way to cloak their responses. When asking about which kinds of sexual behavior was involved in respondents’ most recent sexual encounter, interviewers would hand over a card where letters corresponded to various options, such as oral sex, and respondents would tick off letters. Interviewers didn’t have the key to the card.

In addition to the interview mode, other characteristics differ between the two surveys — notably questions asked. That lessens the ability to track trends between the surveys.

Funding also differed. The older study was planned with the promise of Congressional money, but when North Carolina Republican Jesse Helms led an effort to block in the Senate, researchers had to delay and turn to other sources, including foundations. The newer study bypassed governmental sources, receiving funding for data collection from Church & Dwight Co., the makers of Trojan condoms. Condoms are a major topic in the research. Herbenick said the company didn’t dictate study design, though it did offer advice.

These surveys nonetheless are broader and in some ways more reliable than the federal government’s efforts to track sexual behavior, which generally are part of broader studies. The Centers for Disease Control and Prevention’s Youth Risk Behavior Surveillance System, administered in schools, asks just seven questions about sex. It also doesn’t define sex, out of fear that more schools or parents would opt their students out of responding. “We want to ask appropriate questions, but at the same time, we don’t want to create any unnecessary red flags,” said Laura Kann, who helps direct the survey.

Meanwhile, the National Survey of Family Growth, run by the CDC’s National Center for Health Statistics, focuses on reproduction, so it covers only ages 18 to 44, and only in 2002 did it begin to survey men in addition to women.

One finding many of these surveys share is that the stories men and women tell about their sex lives don’t always match up as we’d expect. Typically, men report more sex partners than women do. Also, in the latest survey and in recent studies of college students, more men report their partners achieve orgasm in their latest sexual encounters than the proportion of women who report achieving orgasm, a finding which has received extensive press coverage.

Researchers who have found these results attribute them generally to poor communication between men and women in bed. “We all need to do a better job of communicating,” Herbenick said.

But such findings may be problematic when certain groups are excluded. Robert T. Michael, an economist at the University of Chicago and a principal investigator on the study, pointed out that prostitutes are unlikely to tell strangers about their sex habits, and those with many clients may have a major effect on estimates of average number of sex partners.

His collaborator on the 1992 study, Edward Laumann, said that because of such discrepancies, “people could be describing the same thing,” yet their answers still could add up to different numbers for each gender.

William Mosher, a statistician at the National Center for Health Statistics, added that these surveys generally are fixed on a certain age range, and don’t ask people outside the country about their sex habits — both factors that could lead to mismatches between male and female responses.

This doesn’t necessarily mean that the orgasm reporting gap isn’t real, or even bigger than found — just that there are potential confounding factors. Stanford University sociologist Paula England, who has collaborated on the college studies, said these findings hold up among various subcategories of respondents — which suggests that at least in this population, if not necessarily in the overall population, sex isn’t always perceived the same way by both partners. And Pepper Schwartz, a sociologist at the University of Washington, said a study she worked on in the 1980s found a similar effect, though not as large, among couples — who presumably were reporting on the same sexual encounter. “The direction or percentage might have some wiggle room, but the message is pretty clear,” Schwartz said.

Copyright 2008 Dow Jones & Company, Inc. All Rights Reserved

ST: New system tracks where vehicles go

Singapore

Home > Singapore > Story

Oct 7, 2010

New system tracks where vehicles go

Satellite-based technology will also give drivers traffic updates

By Goh Chin Lian

The new system being studied here will replace the current 12-year-old one that uses physical gantries to detect vehicles entering the Central Business District and using expressways and major arterial roads. -- ST FILE PHOTO

 

THE satellite-based Electronic Road Pricing (ERP) system being studied here will not only charge motorists for using the roads, but also have new capabilities such as the ability to track the movements of vehicles.

It will be able to capture the location from which vehicles leave, the route they take, and their final destination, supplying valuable traffic data to transport planners.

In turn, drivers will receive more accurate real-time traffic information, such as which congested roads to avoid, through a device installed on board their vehicles.

Their current in-vehicle units for the gantry-based ERP cannot receive such alerts. They now learn about traffic conditions from electronic displays along the roads or radio broadcasts.

The Land Transport Authority (LTA) revealed these details when it sourced for contractors earlier this month for an evaluation test of a next-generation ERP system, which it described as a 'multimillion-dollar project'.

Only contractors with a net worth of no less than $30 million will be considered. They must have a track record of applying state-of-the-art technologies to transport-related solutions. The LTA said it may engage up to four contractors to provide solutions, paying each $1 million.

Contractors have until January next year to submit their bids.

The new system will be tried out at various sites and routes. The entire test project is scheduled to take 19 months from the award of the contract.

But Transport Minister Raymond Lim had indicated in June that the new system will take some years to develop and test before it is ready to be launched.

The next-generation system will replace the current 12-year-old one that uses physical gantries to detect vehicles entering the Central Business District and using expressways and major arterial roads.

The LTA does not think putting up more gantries - there are 68 islandwide now - is the answer in the long run.

The Straits Times understands it is looking at a new system that can handle at least one million vehicles on not only expressways and major arterial roads, but also all other types of roads including smaller access links to buildings.

Three forms of road pricing are under study.

Motorists could be levied a fee for crossing a particular point on the road or when they enter a zone, as is the practice now.

Or they could be, for the first time, charged for the distance travelled on a congested road.

Motorists may use a prepaid system like a stored-value card, or receive a monthly bill, or there could be a combination of both.

One technical challenge is to differentiate between vehicles on priced and non-priced roads, which may run parallel and close to one another.

The system also must not lose track of the vehicle when it zips into a tunnel, is under dense foliage or amid tall buildings in the Central Business District.

Transceivers installed along the road using short-range wireless communication serve as a backup if the satellite-positioning system goes awry or is inaccurate.

Deterring and spotting cheats is a key consideration. The system will detect any jamming of signals and tampering of the device in the vehicle.

Safeguarding the privacy of motorists is another concern. In tracking vehicles for traffic data, for instance, the LTA indicated that provision is to be made to preserve the anonymity of the vehicles.

Associate Professor Lee Der Horng from the National University of Singapore's civil engineering department said a system that can provide real-time traffic information by tracking all vehicles on the road will be 'very powerful'. It can gather more accurate data compared to the current reliance on a few thousand taxis installed with tracking devices, induction loops buried in roads and roadside cameras to gauge traffic flow.

Mr Ong Kian Min, an MP for Tampines GRC and a member of the Government Parliamentary Committee for Transport, said the Government should allay any public concern over privacy, by monitoring only heavily used roads and not small roads.

He said the system could also be helpful for security agencies, like the use of closed-circuit TV cameras in public areas. 'The merits outweigh the concerns about privacy,' he noted.

chinlian@sph.com.sg

---

One technical challenge is to differentiate between vehicles on priced and non-priced roads, which may run parallel and close to one another.

It also must not lose track of the vehicle when it zips into a tunnel, is under dense foliage or amid tall buildings in the Central Business District.

FT: When to turn a blind eye to the facts

When to turn a blind eye to the facts

By Philip Delves Broughton

Published: September 20 2010 23:10 | Last updated: September 20 2010 23:10

The good news from the Gulf of Mexico is that the worst predictions of environmental disaster after the Deepwater Horizon oil rig exploded are unlikely to be fulfilled. It’s still awful, but not as awful as first feared. But when you think back to the worst days, as BP struggled to cap the well against a cacophony of abuse, there were two kinds of decision making on display.

For the men and women fighting to stop the leak, there were the messy, practical challenges to be met, and decisions to be made based on the evidence. And then there were those who made the decision early to pound away at this latest example of “Big Oil” gone wild.

EDITOR’S CHOICE

Philip D Broughton: Beware the superstar chief executive - Sep-13

Richard Waters: Angels can beat deals with the devil - Aug-31

Philip D. Broughton: An old-fashioned view of hoarding - Aug-30

Tom Peters: Kindness can be the hardest word - Aug-23

Robert Sutton: Separating the best CEOs from the dolts - Aug-16

Octavius Black: ‘I don’t have time to think’ - Aug-09

President Barack Obama was caught between the two, trying to make decisions based on the complex set of facts, while others claimed to know exactly what decisions should be made, and compiled the evidence accordingly.

Businesses are often caught in the same trap. Ideally, you want to base your decisions on sound evidence. But often, managers make a decision then rustle up the evidence to support it. As Peter Tingling and Michael Brydon of Simon Fraser University wrote recently in the MIT Sloan Management Review, it is the difference between evidence-based decision making and its ugly sibling, decision-based evidence making.

British schoolchildren are taught the story of Lord Nelson at the Battle of Copenhagen. When told that his commander was signalling for him to retreat, Nelson raised a telescope to his blind eye and said: “I really do not see the signal.”

Within a few hours, the Danish fleet was defeated. He had made the decision to fight and moulded the evidence to fit. “If you get the outcome you want, then everything is fine,” says Prof Tingling.

But in a world that venerates data-driven decision making, whether in financial services or sports management, managers are often loath to admit that they still take the Nelson approach to make the decision first and find the evidence later.

Profs Tingling and Brydon found that evidence is used by managers in three different ways: to make; inform; or support a decision. If it is used to make a decision, it means the decision arises directly from the evidence. If it is used to inform a decision, evidence is mixed in with intuition or bargaining to lead to a decision. If it is used to support a decision, it means the evidence is simply a means to justify a decision already made. They also found that evidence is often shaped by subordinates to meet what they perceive to be the expectations of their bosses.

There are two dangers to letting decisions trump evidence. The first is when decision making is simply ill-informed. Ideally, a decision that contradicts the evidence is an inspired hunch, formed by experience, like Nelson’s. In the worst case, it is the product of ignorant bias.

The second danger is that once your employees know that you, as a manager, are more interested in finding evidence to fit your conclusions rather than seeking out truth, it infects a company with demoralising and destructive cynicism.

When Herman Miller designed the Aeron chair, consumer focus groups were hostile. But the company ignored them and went ahead with production anyway. The chair was a huge hit. The company was capable of taking evidence and successfully ignoring it.

In other companies, however, the cult of data-driven decision making leaves so little room for personal beliefs that people just tailor evidence to fit pre-made decisions.

At Ford in the 1950s, Robert McNamara, then chief executive, demanded data on everything. Interns would cut up newspapers and paste them into binders so executives could point to the voluminous research that went into each decision.

Vince Kaminski, who led Enron’s research division, has spoken of the frustrations of having 50 highly skilled mathematicians picking apart Enron’s risky deals only to be ignored by executives who prized volume above all else.

So what is a manager to do? How do you encourage the use of data, while leaving room for the occasional inspired decision? One solution is to be more flexible in how you categorise decisions. Not all will require the same degree of evidence.

Another is to weigh the costs of gathering evidence. Is it always worth it? If not, don’t fudge it for appearance’s sake. Admit that you are trusting your well-honed instincts. This is especially true for those within your company. Sometimes you will have to come up with tendentious evidence for an external audience, which demands at least a charade of evidence. But don’t ever pretend for those inside. They will know better and punish the slightest deceit either now or well into the future.

Copyright The Financial Times Limited 2010.

NY Times: Web Start-Ups Offer Bargains for Users' Data

May 30, 2010

Web Start-Ups Offer Bargains for Users’ Data

By STEPHANIE CLIFFORD

As concern increases in Washington about the amount of private data online, and as big sites like Facebook draw criticism that they collect consumers’ information in a stealthy manner, many Web start-ups are pursuing a more reciprocal approach — saying, in essence: give us your data and get something in return.

The budgeting Web site Mint.com, for example, displays discount offers from cable companies or banks to users who reveal their personal financial data, including bank and credit card information. The clothing retailer Bluefly could send offers for sunglasses to consumers who disclose that they just bought a swimsuit. And location-based services like Foursquare and Gowalla ask users to volunteer their location in return for rewards like discounts on Pepsi drinks or Starbucks coffee.

These early efforts are predicated on a shift in the relationship between consumer and company. Influenced by consumers’ willingness to trade data online, the sites are pushing to see how much information people will turn over.

“People are a lot more willing to give away a lot of stuff as long as it results in some benefits that they value,” said Stephen J. Hoch, a marketing professor and director of the Jay H. Baker Retailing Initiative at the University of Pennsylvania’s Wharton School.

WeShop.com, still in development, has built a system that allows people to spread information about their shopping habits.

New companies including WeShop, Aprizi, Blippy and Dopplr are trying to exploit the data that people seem so willing to give up. Some are even allowing shoppers to set what terms they want — free shipping, half-price discounts, only fair-trade products. They can also list what they are shopping for, like a gray cashmere sweater under $100, for instance, and let the retailers fight it out for the right to make a sale.

“The whole privacy debate has grown up around people using your data without your permission,” said Antony Lee, chief executive of WeShop. “If you want to use your data to your benefit, that’s for you to do,” Mr. Lee said.

Daniel Sjoberg, a 26-year-old in Manhattan, signed up for Mint as he was completing graduate school in biostatistics last summer. He allowed the site to pull information from his checking account, his credit card and his student loan account.

He meant to use Mint as a budgeting tool, but soon began browsing through offers Mint listed under “ways to save.” He signed up for a Charles Schwab checking account when he learned it would refund his A.T.M. fees, and an Ally savings account based on the interest rate.

Sponsors are allowed to include their logo and a link to their site, and they pay a referral fee if a consumer signs up, but offers from nonsponsors are listed, too. Mr. Sjoberg said he particularly liked how transparent Mint seemed to be.

“They put that very clearly for you to see — ‘We think Ally is good for you, and by the way, they’re endorsing us,’ ” Mr. Sjoberg said. “It’s refreshing to have it out in the open who’s giving them money and who’s not giving them money.”

Aaron Patzer, who founded Mint in 2007 (the company was acquired by Intuit in 2009 for $170 million), believed that users would give the site private information in return for allowing Mint to analyze their finances to alert them when they had exceeded their budget, or to send them offers from cable companies or banks.

“Most venture capitalists, when I was starting this company, said that no one would trust a start-up with their financial information,” said Mr. Patzer, now vice president and general manager of Intuit’s personal finance group. “In essence, we would data-mine your own data in order to help you.”

While data on Mint is kept private — there is no way to share financial details with other users — WeShop has built a system that allows people to spread information about their shopping habits. After a consumer gives WeShop access to an e-mail account, the system scans e-mail headers to find electronic receipts, then extracts what someone bought and what price they paid.

All that information is posted to the WeShop site as a kind of in-depth shopping history. A consumer can keep it private, or share some or all purchase data with other people in WeShop networks (using a nickname). That lets users compare prices and post messages about what Lego sets or prom dresses they are considering.

WeShop will soon give retailers access to the data and allow them to send specific offers. A retailer like Bluefly, which says it plans to test WeShop, could search for people who have bought something prom-related, and send those users e-mail messages with special offers on formal dresses. Retailers pay WeShop a percentage of the sale price when a consumer buys a product.

“Everyone’s kind of working on different angles of how we give the consumer more ways to target what they want, to ask for what they want, and get it,” said Bradford Matson, chief marketing officer of Bluefly.com. “This notion of how people shop is changing very quickly.”

Bun Lai, a chef at Miya’s Sushi in New Haven, said he liked the way WeShop put him in charge. “I wasn’t completely comfortable right from the start — it’s after I understood that you actually have control of the information that you’re going to allow the network to have, then I realized it was actually something more empowering,” he said.

But the upfront dealmaking does not quiet all the privacy concerns.

“The big problem is that these business models are not very stable. Companies set out privacy policies, consumers disclose data, and then the action begins,” Marc Rotenberg, executive director of the advocacy group Electronic Privacy Information Center, said in an e-mail message. “The business model changes. The companies simply want the data, and the consumer benefit disappears.”

But Giff Constable, co-founder of Aprizi, a shopping start-up that asks users to enter information about preferences, then selects products for them from throughout the Web, said a changing business model was unlikely, and even if it happened, the consumer could easily disengage.

“It’s opt-in, and it’s understood from the beginning what’s going on here,” Mr. Constable said. “That level of control is explicit: you can plug in and you can remove the information that’s there.

“It’s a very simple bargain — it’s, ‘You share with me, and I am going to make your life better.’ It’s not about privacy versus lack of privacy — it really does come down to control and giving people choice.”

That, and a perceived benefit. “I wasn’t really concerned about giving them information,” said Mr. Sjoberg, the Mint user. “I was only considering the things I was getting out of it.”

NY Times: Cellphones Now Used More for Data Than for Calls

May 13, 2010

Cellphones Now Used More for Data Than for Calls

By JENNA WORTHAM

Liza Colburn uses her cellphone constantly.

She taps out her grocery lists, records voice memos, listens to music at the gym, tracks her caloric intake and posts frequent updates to her Twitter and Facebook accounts.

The one thing she doesn’t use her cellphone for? Making calls.

“I probably only talk to someone verbally on it once a week,” said Mrs. Colburn, a 40-year-old marketing consultant in Canton, Mass., who has an iPhone.

For many Americans, cellphones have become irreplaceable tools to manage their lives and stay connected to the outside world, their families and networks of friends online. But increasingly, by several measures, that does not mean talking on them very much.

For example, although almost 90 percent of households in the United States now have a cellphone, the growth in voice minutes used by consumers has stagnated, according to government and industry data.

This is true even though more households each year are disconnecting their landlines in favor of cellphones.

Instead of talking on their cellphones, people are making use of all the extras that iPhones, BlackBerrys and other smartphones were also designed to do — browse the Web, listen to music, watch television, play games and send e-mail and text messages.

The number of text messages sent per user increased by nearly 50 percent nationwide last year, according to the CTIA, the wireless industry association. And for the first time in the United States, the amount of data in text, e-mail messages, streaming video, music and other services on mobile devices in 2009 surpassed the amount of voice data in cellphone calls, industry executives and analysts say.

“Originally, talking was the only cellphone application,” said Dan Hesse, chief executive of Sprint Nextel. “But now it’s less than half of the traffic on mobile networks.”

Of course, talking on the cellphone isn’t disappearing entirely. “Anytime something is sensitive or is something I don’t want to be forwarded, I pick up the phone rather than put it into a tweet or a text,” said Kristen Kulinowski, a 41-year-old chemistry teacher in Houston. And calling is cheaper than ever because of fierce competition among rival wireless networks.

But figures from the CTIA show that over the last two years, the average number of voice minutes per user in the United States has fallen.

Still, even the telephone design industry has taken note. Ross Rubin, a telecommunications analyst with the NPD Group, said cellphones outfitted with numerical keyboards — easiest for quickly dialing a phone number — were no longer in vogue. Touch screens, or quick messaging devices with full “qwerty” keyboards, on the other hand, are. On the newest phones, users must press several buttons or swipe through several screens to get to the application that allows them to make calls.

“Handset design has become far less cheek-friendly,” Mr. Rubin said. Mr. Hesse of Sprint said he expected that within the next couple of years, cellphone users would be charged by the data they used, not by their voice minutes, a prediction echoed by other industry executives.

When people do talk on their phones, their conversations are shorter; the average length of a local call was 1.81 minutes in 2009, compared with 2.27 minutes in 2008, according to CTIA. For some, the unused voice minutes mount up.

“I have thousands of rollover minutes,” said Zach Frechette, 28, editor of Good magazine in Los Angeles, who explained that he dialed only when he needed to get in touch with someone instantly, and limited those calls to 30 seconds. “I downgraded to the lowest available minute plan, which I’m not even getting close to using.”

Mr. Frechette said part of the reason he rarely talked on his phone was that he had an iPhone, with its notoriously spotty phone reception in certain locales. But also, he said, most of his day was spent swapping short messages through services like Gmail, Facebook and Twitter. That way, he said, “you can respond when it’s convenient, rather than impose your schedule on someone else.”

Others say talking on the phone is intrusive and time-consuming, while others seem to have no patience for talking to just one person at a time. They prefer to spend their phone time moving seamlessly between several conversations, catching up on the latest news and updates by text and on Facebook with multiple friends, instead of just one or two.

“Even though in theory, it might take longer to send a text than pick up the phone, it seems less disruptive than a call,” said Jefferson Adams, a 44-year-old freelance writer living in San Francisco. By texting, he said, “you can multitask between two or three conversations at once.”

Nicole Wahl, a 35-year-old communications manager at the University of Toronto, estimates she talks on her phone only about 10 minutes a month.

“The only reason I ever call someone anymore is if I don’t have their Twitter handle or e-mail address,” Ms. Wahl said. “Like my hairdresser to see if she has a last-minute appointment or my parents to say I’m dropping by.”

American teenagers have been ahead of the curve for a while, turning their cellphones into texting machines; more than half of them send about 1,500 text messages each month, according to a recent study by the Pew Research Center’s Internet and American Life Project.

Mrs. Colburn, from Massachusetts, said she caved to the pleading of her 12-year-old daughter Abigail for a cellphone to send text messages with her friends after she and her husband discovered it was hindering her from developing bonds with her classmates.

“We realized she was being excluded from party invitations and being in the know with her peers,” she said.

Mrs. Colburn said texting had also become a much easier way to stay in touch with her daughter and receive quick updates about after-school plans.

“The other night she texted me from upstairs to ask a vocabulary question,” she said with a laugh. “But I drew the line there. I went upstairs to answer it.”

NY Times: The Data-Driven Life

April 26, 2010

The Data-Driven Life

By GARY WOLF

Humans make errors. We make errors of fact and errors of judgment. We have blind spots in our field of vision and gaps in our stream of attention. Sometimes we can’t even answer the simplest questions. Where was I last week at this time? How long have I had this pain in my knee? How much money do I typically spend in a day? These weaknesses put us at a disadvantage. We make decisions with partial information. We are forced to steer by guesswork. We go with our gut.

That is, some of us do. Others use data. A timer running on Robin Barooah’s computer tells him that he has been living in the United States for 8 years, 2 months and 10 days. At various times in his life, Barooah — a 38-year-old self-employed software designer from England who now lives in Oakland, Calif. — has also made careful records of his work, his sleep and his diet.

A few months ago, Barooah began to wean himself from coffee. His method was precise. He made a large cup of coffee and removed 20 milliliters weekly. This went on for more than four months, until barely a sip remained in the cup. He drank it and called himself cured. Unlike his previous attempts to quit, this time there were no headaches, no extreme cravings. Still, he was tempted, and on Oct. 12 last year, while distracted at his desk, he told himself that he could probably concentrate better if he had a cup. Coffee may have been bad for his health, he thought, but perhaps it was good for his concentration.

Barooah wasn’t about to try to answer a question like this with guesswork. He had a good data set that showed how many minutes he spent each day in focused work. With this, he could do an objective analysis. Barooah made a chart with dates on the bottom and his work time along the side. Running down the middle was a big black line labeled “Stopped drinking coffee.” On the left side of the line, low spikes and narrow columns. On the right side, high spikes and thick columns. The data had delivered their verdict, and coffee lost.

He was sad but also thrilled. Instead of a stimulating cup of coffee, he got a bracing dose of truth. “People have such very poor sense of time,” Barooah says, and without good time calibration, it is much harder to see the consequences of your actions. If you want to replace the vagaries of intuition with something more reliable, you first need to gather data. Once you know the facts, you can live by them.

Five years ago, Ben Lipkowitz, who is now 28, was living with some friends in Bloomington, Ind., and he found himself wondering how much time he spent doing one of his roommates’ dishes. Lipkowitz had a handheld electronic datebook that he purchased on a trip to Tokyo, and on May 11, 2005, at 2:20 p.m., he started using it to keep a record of his actions. Instead of entering his future appointments, he entered his past activities, creating a remarkably complete account of his life. In one sense this was just a normal personal journal, albeit in a digital format and unusually detailed. But the format and detail made all the difference. Lipkowitz eventually transferred the data to his computer, and now, using a few keyboard commands, he can call up his history. He knows how much he has eaten and how much he has spent. He knows what books he has read and what objects he has purchased. And of course, he knows the answer to his original question. “I was thinking I was spending an hour a day cleaning up after this person,” Lipkowitz says. He shrugs. “It turned out it was more like 20 minutes.”

Another person I’m friendly with, Mark Carranza — he also makes his living with computers — has been keeping a detailed, searchable archive of all the ideas he has had since he was 21. That was in 1984. I realize that this seems impossible. But I have seen his archive, with its million plus entries, and observed him using it. He navigates smoothly between an interaction with somebody in the present moment and his digital record, bringing in associations to conversations that took place years earlier. Most thoughts are tagged with date, time and location. What for other people is an inchoate flow of mental life is broken up into elements and cross-referenced.

These men all know that their behavior is abnormal. They are outliers. Geeks. But why does what they are doing seem so strange? In other contexts, it is normal to seek data. A fetish for numbers is the defining trait of the modern manager. Corporate executives facing down hostile shareholders load their pockets full of numbers. So do politicians on the hustings, doctors counseling patients and fans abusing their local sports franchise on talk radio. Charles Dickens was already making fun of this obsession in 1854, with his sketch of the fact-mad schoolmaster Gradgrind, who blasted his students with memorized trivia. But Dickens’s great caricature only proved the durability of the type. For another century and a half, it got worse.

Or, by another standard, you could say it got better. We tolerate the pathologies of quantification — a dry, abstract, mechanical type of knowledge — because the results are so powerful. Numbering things allows tests, comparisons, experiments. Numbers make problems less resonant emotionally but more tractable intellectually. In science, in business and in the more reasonable sectors of government, numbers have won fair and square.

For a long time, only one area of human activity appeared to be immune. In the cozy confines of personal life, we rarely used the power of numbers. The techniques of analysis that had proved so effective were left behind at the office at the end of the day and picked up again the next morning. The imposition, on oneself or one’s family, of a regime of objective record keeping seemed ridiculous. A journal was respectable. A spreadsheet was creepy.

And yet, almost imperceptibly, numbers are infiltrating the last redoubts of the personal. Sleep, exercise, sex, food, mood, location, alertness, productivity, even spiritual well-being are being tracked and measured, shared and displayed. On MedHelp, one of the largest Internet forums for health information, more than 30,000 new personal tracking projects are started by users every month. Foursquare, a geo-tracking application with about one million users, keeps a running tally of how many times players “check in” at every locale, automatically building a detailed diary of movements and habits; many users publish these data widely. Nintendo’s Wii Fit, a device that allows players to stand on a platform, play physical games, measure their body weight and compare their stats, has sold more than 28 million units.

Two years ago, as I noticed that the daily habits of millions of people were starting to edge uncannily close to the experiments of the most extreme experimenters, I started a Web site called the Quantified Self with my colleague Kevin Kelly. We began holding regular meetings for people running interesting personal data projects. I had recently written a long article about a trend among Silicon Valley types who time their days in increments as small as two minutes, and I suspected that the self-tracking explosion was simply the logical outcome of this obsession with efficiency. We use numbers when we want to tune up a car, analyze a chemical reaction, predict the outcome of an election. We use numbers to optimize an assembly line. Why not use numbers on ourselves?

But I soon realized that an emphasis on efficiency missed something important. Efficiency implies rapid progress toward a known goal. For many self-trackers, the goal is unknown. Although they may take up tracking with a specific question in mind, they continue because they believe their numbers hold secrets that they can’t afford to ignore, including answers to questions they have not yet thought to ask.

Ubiquitous self-tracking is a dream of engineers. For all their expertise at figuring out how things work, technical people are often painfully aware how much of human behavior is a mystery. People do things for unfathomable reasons. They are opaque even to themselves. A hundred years ago, a bold researcher fascinated by the riddle of human personality might have grabbed onto new psychoanalytic concepts like repression and the unconscious. These ideas were invented by people who loved language. Even as therapeutic concepts of the self spread widely in simplified, easily accessible form, they retained something of the prolix, literary humanism of their inventors. From the languor of the analyst’s couch to the chatty inquisitiveness of a self-help questionnaire, the dominant forms of self-exploration assume that the road to knowledge lies through words. Trackers are exploring an alternate route. Instead of interrogating their inner worlds through talking and writing, they are using numbers. They are constructing a quantified self.

UNTIL A FEW YEARS ago it would have been pointless to seek self-knowledge through numbers. Although sociologists could survey us in aggregate, and laboratory psychologists could do clever experiments with volunteer subjects, the real way we ate, played, talked and loved left only the faintest measurable trace. Our only method of tracking ourselves was to notice what we were doing and write it down. But even this written record couldn’t be analyzed objectively without laborious processing and analysis.

Then four things changed. First, electronic sensors got smaller and better. Second, people started carrying powerful computing devices, typically disguised as mobile phones. Third, social media made it seem normal to share everything. And fourth, we began to get an inkling of the rise of a global superintelligence known as the cloud.

Millions of us track ourselves all the time. We step on a scale and record our weight. We balance a checkbook. We count calories. But when the familiar pen-and-paper methods of self-analysis are enhanced by sensors that monitor our behavior automatically, the process of self-tracking becomes both more alluring and more meaningful. Automated sensors do more than give us facts; they also remind us that our ordinary behavior contains obscure quantitative signals that can be used to inform our behavior, once we learn to read them.

“When you have small, distributed battery-powered sensors, you want to collect all biometric data,” says Ken Fyfe, one of the pioneers of wearable tracking devices. In the mid-’90s, Fyfe was teaching engineering at the University of Alberta in Edmonton, where his specialty was acoustics and vibration. He was also a runner, in a family of runners. His sons were national competitors at 400 and 800 meters. At the time, runners who wanted to know more about the mechanics of their performance — their stride, their cadence, the way their motion changed as they grew tired — had to go into a lab and be filmed. “You would run in a room on a treadmill with reflective stickers on your hips, knees, ankles and feet,” Fyfe recalls.

Taking video of people in motion, and then analyzing the video, seemed like a roundabout way to get data. Why not use an accelerometer, which can directly measure changes in speed and direction? Accelerometers had long been used in industry and cost several hundred dollars each. Then accelerometers were developed to trigger the air bags in cars. Massive purchases in the automotive industry drove the cost down. The size and power demands shrank, too. Suddenly, it seemed less crazy to put an accelerometer on your body.

Fyfe guessed that there would be plenty of interest in something like a personal speedometer, a wearable instrument that displayed how far you’d gone and your average speed. So he tried to invent one. “I worked on it every weekend for three years,” Fyfe says. He put accelerometers into a molded plastic insert. The insert fit into a shoe, and data were transmitted wirelessly to a sports watch. But there was a problem. The numbers produced by a motion sensor don’t necessarily say anything about a runner’s pace and distance. They give you the acceleration of a runner’s foot — that’s all. Some method — a formula or algorithm — is needed to translate the data into the information you want, and the method must work for almost everybody under a wide range of conditions: stopping and starting, jumping over a curb, limping because of an injury. Developing these algorithms took up most of Fyfe’s time during the years he perfected his system.

Thanks to faster computers and clever mathematical techniques, Fyfe and other inventors are turning messy data from cheap sensors into meaningful information. “The real expertise you need is signal processing and statistical analysis,” says James Park, the chief executive and co-founder of Fitbit, a company that makes a tracker released late last year. The Fitbit tracker is two inches long, half an inch wide and shaped like a thick paperclip. It tracks movement, and if you wear it in a little elastic wristband at night, it can also track your hours of sleep. (You are not completely still when sleeping. Your pattern of movement, however, can be correlated with sleeping and waking, just as the acceleration of a runner’s foot reveals speed.) Park and his partner, Eric Friedman, first showed their prototype at a San Francisco business conference in the summer of 2008. Five weeks later, Park and Friedman, who are both 33, had $2 million in venture capital, and they were flying back and forth to Singapore to arrange production. Last winter they shipped their first devices.

At nearly the same time, Philips, the consumer electronics company, began selling its own tiny accelerometer-based self-tracker, called DirectLife, which, like the Fitbit, is meant to be carried on the body at all times. Zeo, a company based in Newton, Mass., released a tracker contained in a small headband, which picks up electrical signals from the brain, and uses them to compile the kind of detailed record of light sleep, deep sleep and REM sleep that, until now, was available only if you spent the night in a sleep-research clinic. Lately I’ve been running into people who say they wear it every night. And Nike recently announced that its Nike+ system, one of the first personal speedometers, has been used by more than 2.5 million runners since its release in 2006.

Ken Fyfe’s accelerometer-based tracking system is used with sports watches by Adidas and Polar. In 2006 he sold his company, Dynastream, for $36 million to Garmin, which makes navigation equipment commonly used in cars and airplanes and which is now branching out into personal tracking. Fyfe’s former company stayed in Alberta, where it continues to sell tracking components. A low-power data-transmission protocol they invented is in new blood-pressure cuffs, glucose monitors, blood-oxygenation sensors, weight scales and sleep monitors, all of which are aimed at the consumer market.

Web entrepreneurs like to talk about democratizing communication. Fyfe’s dream is to democratize objective research on human subjects. “Until we came up with this technology, you couldn’t do this kind of analysis unless you could get into a lab,” he says. “Now you can.”

At the center of this personal laboratory is the mobile phone. During the years that personal-data systems were making their rapid technical progress, many people started entering small reports about their lives into a phone. Sharing became the term for the quick post to a social network: a status update to Facebook, a reading list on Goodreads, a location on Dopplr, Web tags to Delicious, songs to Last.fm, your breakfast menu on Twitter. “People got used to sharing,” says David Lammers-Meis, who leads the design work on the fitness-tracking products at Garmin. “The more they want to share, the more they want to have something to share.” Personal data are ideally suited to a social life of sharing. You might not always have something to say, but you always have a number to report.

This is how the odd habits of the ultrageek who tracks everything have come to seem almost normal. An elaborate setup is no longer necessary, because the phone already envelops us in a cloud of computing. This term, “the cloud,” has some specialized meanings among software architects, but fundamentally the cloud is just a poetic label for the global agglomeration of computer resources — the processors, hard drives, fiber-optic cables and so on — that allow us to access our private data from any Internet connection. We entrust all kinds of things to the cloud: our mail and our family photographs; the places we go and the list of people we call on the phone. When Jeff Clavier, the founder of SoftTech VC, a Silicon Valley venture capital firm, invested in a small financial company called Mint (now part of Intuit), he was warned that ordinary people were unlikely to trust their bank passwords and credit-card details to the cloud. “About 1.5 million people did it,” Clavier says.

One of the reasons that self-tracking is spreading widely beyond the technical culture that gave birth to it is that we all have at least an inkling of what’s going on out there in the cloud. Our search history, friend networks and status updates allow us to be analyzed by machines in ways we can’t always anticipate or control. It’s natural that we would want to reclaim some of this power: to look outward to the cloud, as well as inward toward the psyche, in our quest to figure ourselves out.

SOPHIE BARBIER, a 47-year-old teacher in Palo Alto, is a cyclist who regularly logs her time, distance and heart rate during a ride. “Training logs have been around forever,” she told me. “But the more variables I added, the more curious I got.” Along with her cycling stats, Barbier began scoring her mood, sleep and ability to focus, as well as her caffeine consumption, and noting the days her menstrual cycle began and ended.

After surgery for a back problem, Barbier had trouble sleeping. On CureTogether, a self-tracking health site, she learned about tryptophan, a common amino acid available as a dietary supplement. She took the tryptophan, and her insomnia went away. Her concentration scores also improved. She stopped taking tryptophan and continued to sleep well, but her ability to concentrate deteriorated. Barbier ran the test again, and again the graph was clear: tryptophan significantly increased her focus. She had started by looking for a cure for insomnia and discovered a way to fine-tune her brain.

It is tempting to dismiss reports of such experiments as trivial anecdotes, or the placebo effect. I took Barbier’s results to a friend of mine, Seth Roberts, an emeritus professor of psychology at the University of California, Berkeley, and an expert on self-experimentation. “There is a large difference between what Barbier did and the minimal story of somebody who takes a pill looking for a certain effect and then finds it,” he pointed out. “First, she wrote the numbers down, so the results are not subject to memory distortion. Second, she changed the conditions several times. Every switch is a test of her original theory.”

Roberts told me about his own method of measuring mental changes, a quick test he programmed on his computer that involves 32 easy arithmetic problems. The test takes about three minutes, and he has found that it can detect small changes in cognitive performance. He has used his self-tracking system to adjust his diet, learning that three tablespoons daily of flaxseed oil reliably decreases the amount of time it takes him to do math. Consuming a lot of butter also seems to have a good effect.

Self-experiments like Barbier’s and Roberts’s are not clinical trials. The goal isn’t to figure out something about human beings generally but to discover something about yourself. Their validity may be narrow, but it is beautifully relevant. Generally, when we try to change, we simply thrash about: we improvise, guess, forget our results or change the conditions without even noticing the results. Errors are possible in self-tracking and self-experiment, of course. It is easy to mistake a transient effect for a permanent one, or miss some hidden factor that is influencing your data and confounding your conclusions. But once you start gathering data, recording the dates, toggling the conditions back and forth while keeping careful records of the outcome, you gain a tremendous advantage over the normal human practice of making no valid effort whatsoever.

I recently received an e-mail message from a 26-year-old filmmaker named Toli Galanis, who keeps track of about 50 different streams of personal data, including activities, health, films watched and books read, the friends he talks with and the topics they discuss. While Galanis acknowledged that he gets pleasure from gathering data and organizing it intelligently, it was a different aspect of his report that caught my attention. “I know that immediately after watching a bad movie I am more apt to be negative about my career prospects as a filmmaker,” he wrote, explaining that tracking has made him better able to detect the influence of seemingly trivial circumstances on his mood and decisions.

The idea that our mental life is affected by hidden causes is a mainstay of psychology. Facility in managing the flow of thought and emotion is a sign of happiness and good adjustment. But how is it done? Nearly every therapeutic prescription involves an invitation to notice, to pay attention. Once we have a notion in our sights, we can attack it with an arsenal of tools: cognitive, psychoanalytic, even spiritual. But none of these will tell us if we’ve missed something. You may simply have failed to notice a debilitating habit, a negative correlation, a bad influence.

Galanis’s realization that bad movies subject him to professional discouragement is the type of insight that will seem accessible to anybody blessed with a modest amount of self-awareness; finding it is no more difficult than catching sight of a dollar in the street and picking it up. But for every one you grab, how many do you overlook?

It’s not only the context of our thoughts that escapes us. Our actions do, too. Since 2004, Terry Paul, an educational entrepreneur and philanthropist, has been working on a digital device that tucks into specially designed toddlers’ clothing and can be used to predict language development through tracking the number of conversational exchanges a child has with adults. It cost Paul $32 million to perfect the system that takes the noisy sounds of a baby’s environment and translates it into reliable data. As a commercial enterprise, it was unsuccessful. His device, called the LENA monitor, is used for academic research but never took off as a consumer product. When I tell parents about it, most of them are horrified. They imagine a nightmare of surveillance and an inducement to neurotic competition: who wants a digital recorder that grades you on how you talk to your kid?

Were we to submit to such a test, however, many of us would fare poorly. Parents, in fact, overestimate how much they talk to their preverbal children. Users of the LENA monitor can be awkwardly surprised. A mother I spoke with recently began monitoring after her daughter was prescribed a seizure medication that was associated with language delays. “It became very clear to us that my husband’s words were less than mine,” she said. He needed to try to talk to his daughter more. Until he saw the data, he had no idea that his attention was wandering.

Of course, sometimes we fail to notice what we do because we are motivated not to notice it. We are ashamed of ourselves, so we lie to ourselves. Shaun Rance started tracking his drinking two years ago, after his father was given a diagnosis of end-stage liver disease. He didn’t pledge to stop drinking; he didn’t do a searching moral inventory; he just started counting, using the anonymous Web site drinkingdiary.com. He found that his externalized memory was very powerful. Having a record of every drink he took sharpened his awareness and increased his feeling of self-mastery — and reduced his drinking. Because his tally is held by a machine, he doesn’t feel any of the social shame that might make him, consciously or not, underestimate his drinking. “I don’t lie to the diary,” he says. After all, it is silly to posture in front of a machine. The tracking system is an extension of a basic faculty of Rance’s consciousness, there to remind him where he stands, and it does its work without emotion. As far as he’s concerned, that’s a virtue.

There may be new domains of our biology that we can incorporate into our sense of self. “We know about asleep, awake, hungry, depressed, cold, drowsy, nauseous,” says Dave Marvit, a vice-president at Fujitsu Laboratories of America, where he is leading a research project on self-tracking. “But what about hypoxic, anemic, hyperglycemic?” If we had a gentle signal about how much sugar was in our blood, would we change how we ate? Would it change how we feel?

Drinking. Talking. Being discouraged by a movie. Giving a moment’s attention to a feeling of anger or elation, a small surge of energy or a metabolic dip. These are the materials of daily life. They barely stand out against the background of what we take for granted, and yet picking up these weak signals gives us leverage. Margaret Morris, a clinical psychologist and a researcher at Intel, recently ran a series of field trials using a mobile phone for tracking emotion. At random times, the phone rang and quizzed its owner about his or her mood. A man in one of Morris’s studies reviewed the trends in his data and noticed that his foul mood began at the same time every day. He had a rushed transition from work to home. While unfinished tasks were still on his mind, new demands crowded in. The stress followed him for the rest of the evening. The data showed him where the problem was. With help, he learned to take a short mental break right there. He was much relieved.

The contrast to the traditional therapeutic notion of personal development is striking. When we quantify ourselves, there isn’t the imperative to see through our daily existence into a truth buried at a deeper level. Instead, the self of our most trivial thoughts and actions, the self that, without technical help, we might barely notice or recall, is understood as the self we ought to get to know. Behind the allure of the quantified self is a guess that many of our problems come from simply lacking the instruments to understand who we are. Our memories are poor; we are subject to a range of biases; we can focus our attention on only one or two things at a time. We don’t have a pedometer in our feet, or a breathalyzer in our lungs, or a glucose monitor installed into our veins. We lack both the physical and the mental apparatus to take stock of ourselves. We need help from machines.

Watch out for those machines, though. Humans know a special trick of self-observation: when to avert our gaze. Machines don’t understand the value of forgiving a lapse, or of treating an unpleasant detail with tactful silence. A graph or a spreadsheet talks only in numbers, but there is a policeman inside all of our heads who is well equipped with punishing words. “Each day my self-worth was tied to the data,” Alexandra Carmichael, one of the founders of the self-tracking site CureTogether, wrote in a heartfelt blog post about why she recently stopped tracking. “One pound heavier this morning? You’re fat. Skipped a day of running? You’re lazy. It felt like being back in school. Less than 100 percent on an exam? You’re dumb.” Carmichael had been tracking 40 different things about herself. The data she was seeing every day didn’t respect her wishes or her self-esteem. It was awful, and she had to stop.

Electronic trackers have no feelings. They are emotionally neutral, but this very fact makes them powerful mirrors of our own values and judgments. The objectivity of a machine can seem generous or merciless, tolerant or cruel. Designers of tracking systems are trying to finesse this ambivalence. A smoking-cessation program invented by Pal Kraft, a Norwegian researcher at the University of Oslo, automatically calls people who are trying to quit, asking them every day whether they’ve smoked in the last 24 hours. When the answer is yes, a recorded voice delivers an encouraging message: All is well, take it easy, try again. This mechanical empathy, barely more human than a recorded voice on the customer-service line, can hardly be expected to fool anybody. But a long line of research in human-computer interaction demonstrates that when machines are given humanlike characteristics and offer emotional reassurance, we actually do feel reassured. This is humbling. Do we really feel better when a computer pats us on the back? Yes, we do.

Jon Cousins is a 54-year-old software entrepreneur and former advertising executive who was given a diagnosis in 2007 of bipolar affective disorder. Cousins built a self-tracking system to help manage his feelings, which he called Moodscope; now used by about 1,000 others, Moodscope automatically sends e-mail with mood-tracking scores to a few select friends. “My life was changed radically,” Cousins told me recently in an e-mail message. “If I got the odd dip, my friends wanted to know why.” Sometimes, after he records a low score, a friend might simply e-mail: “?” Cousins replies, and that act alone makes him feel better. Moodscope is a blended system in which measurement is supplemented by human sympathy. Self-tracking can sometimes appear narcissistic, but it also allows people to connect with one another in new ways. We leave traces of ourselves with our numbers, like insects putting down a trail of pheromones, and in times of crisis, these signals can lead us to others who share our concerns and care enough to help.

Often, pioneering trackers struggle with feelings of being both aided and tormented by the very systems they have built. I know what this is like. I used to track my work hours, and it was a miserable process. With my spreadsheet, I inadvertently transformed myself into the mean-spirited, small-minded boss I imagined I was escaping through self-employment. Taking advantage of the explosion of self-tracking services available on the Web, I started analyzing my workday at a finer level. Every time I moved to a new activity — picked up the phone, opened a Web browser, answered e-mail — I made a couple of clicks with my mouse, which recorded the change. After a few weeks I looked at the data and marveled. My day was a patchwork of distraction, interspersed with valuable, but too rare, periods of focus. In total, the amount of uninterrupted close attention I was able to muster in a given workday was less than three hours. After I got over the humiliation, I came to see how valuable this knowledge was. The efficiency lesson was that I could gain significant benefit by extending my day at my desk by only a few minutes, as long as these minutes were well spent. But a greater lesson was that by tracking hours at my desk I was making an unnecessary concession to a worthless stereotype. Does anybody really believe that long hours at a desk are a vocational ideal? I got nothing from my tracking system until I used it as a source of critical perspective, not on my performance but on my assumptions about what was important to track.

People are not assembly lines. We cannot be tuned to a known standard, because a universal standard for human experience does not exist. Bo Adler, a young computer scientist at Fujitsu Laboratories of America, is one of the most committed self-trackers I’ve ever met: during his most active phase he wore a blood-pressure cuff, pulse oximeter and accelerometer all day long, along with a computer on a harness to collect the data. Adler has sleep apnea, and he is trying to figure it out. When he became too self-conscious going to the gym in his gear, he wore a Google T-shirt to throw people off. Maybe he was a freak, but at least people could mistake him for a millionaire freak.

“Here’s what they told me was the normal surgical course of treatment,” Adler explained. “First they were going to cut out my tonsils, and if that didn’t work, they would break my jaw and reset it to reposition my tongue, and finally they would cut out the roof of my mouth. I had one question: What if my case is different? They said, ‘Let’s try the standard course of treatment first, and if that doesn’t work, then we’ll know your case is different.’ ” Adler recognized what this proposal meant: it meant that his doctors had no cure for different. They wanted to see him as a standard case, because they have treatments for the standard cases. Before Adler underwent surgery, he wanted some evidence that he was a standard case. Some of us aren’t standard, after all; perhaps many of us aren’t.

Adler’s idea that we can — and should — defend ourselves against the imposed generalities of official knowledge is typical of pioneering self-trackers, and it shows how closely the dream of a quantified self resembles therapeutic ideas of self-actualization, even as its methods are startlingly different. Trackers focused on their health want to ensure that their medical practitioners don’t miss the particulars of their condition; trackers who record their mental states are often trying to find their own way to personal fulfillment amid the seductions of marketing and the errors of common opinion; fitness trackers are trying to tune their training regimes to their own body types and competitive goals, but they are also looking to understand their strengths and weaknesses, to uncover potential they didn’t know they had. Self-tracking, in this way, is not really a tool of optimization but of discovery, and if tracking regimes that we would once have thought bizarre are becoming normal, one of the most interesting effects may be to make us re-evaluate what “normal” means.

“My girlfriend thinks I’m the weird person when I wear all these devices,” Bo Adler says. “She sees me as an oddity, but I say no, soon everybody is going to be doing this, and you won’t even notice.”

Gary Wolf writes about science and social issues for Wired, where he is a contributing editor. This is his first article for the magazine.