Does Chinese innovation have a dark side?

New York Times columnist Tom Friedman has emerged from a recent foray to China newly astonished by the breakneck pace of innovation here. In his latest column Friedman marvels that the Middle Kingdom has become a cashless society in which “everyone pays for everything with a mobile phone” and even beggars use QR codes.

After hobnobbing with the president of China’s main search engine and the founder of its largest mobile food delivery company, Friedman rhapsodizes that, “in an age when raw data from the Internet of people and the Internet of things is the new oil, the fact that China has 700 million people doing so many transactions daily on the mobile Internet means it’s piling up massive amounts of information that can be harvested to identify trends and spur new artificial intelligence operations.” His conclusion: Americans “underestimate China -- and attribute all of its surge in growth to unfair trade practices -- at our peril.”

Just so. And yet the arc of China’s rise -- particularly in areas like e-commerce, mobile payment, Big Data and the Internet of Things -- is troubling in two ways. The first Friedman acknowledges: as China’s economy has grown, its leaders have continued to protect local companies from global competition, restricting foreign firms’ access to Chinese customers, and forcing them to surrender intellectual property as the price of wider market access. The second reason for concern Friedman leaves unsaid: China’s rulers hope to use the trove of data its tech firms are collecting to tighten the grip of an already authoritarian state.

On Thursday, Alibaba offered further evidence of China’s growing tech prowess by forecasting its annual sales will surge to more than $34 billion in the 2018 fiscal year, a 45% gain over last year and far higher than analysts’ estimates. The announcement inspired rapture on Wall Street. In the first minutes of trading, investors bid BABA’s stock price up 12% to $140.84, an all-time high. The stock is up nearly 50% so far this year, confounding naysayers. Alibaba’s annual sales still trail Amazon’s $136 billion, but they are growing twice as fast.

The pace of innovation in China’s tech sector is so frantic because its largest players -- the big three Internet companies, Alibaba, Tencent and Baidu, and device makers like Huawei, Xiaomi and ZTE -- are locked in battle, and pumping billions into new growth opportunities at home and abroad. Alibaba last week reportedly paid $81 million for an 18% stake in Lianhua, one of China’s largest grocery chains. This week, Alibaba Pictures bought a majority stake in Indian ticketing startup TicketNew. The assumption is that bigger is better because Chinese consumers will gravitate to a one-stop-shopping model where a single platform connects them to all the goods and services they seek.

The race to build such platforms has spawned innovations that give Western competitors pause. The Financial Times reports this week that Chinese tech giants are barreling forward in the use of face recognition technologies. Alibaba’s mobile payments affiliate, Ant Financial, allows its 450 million users to log into their online wallets by taking a selfie. Baidu, China Construction Bank, and ride-hailing service Didi Chuxing use the technologies in identify employees as well as customers. A Beijing-based company called Face++ has raised $100 million in its third round of financing, according to the FT, and has licensed its software to Ant and Didi.

U.S. companies like , Nest and have been squeamish about pushing face recognition technologies, fearing backlash from customers concerned about privacy. But China’s citizens, who are obliged to slide ID cards into chip readers to set up mobile phone accounts, make travel reservations and book hotels, seem less fussed about surrendering personal data.

Meanwhile, Caixin, a Chinese business magazine, reports Ant is steadily expanding use of its “Sesame Credit” system, which assigns customers a “financial reliability” score according to criteria such as their online spending records, how regularly they pay their utility bills or credit cards, and other factors such as what city they live in, whether they own a house or car. Caixin says the ranking even factors in the scores of acquaintances.

This essay was originally published in our CEO Daily Newsletter. Subscribe.

Ant recently introduced a new service to expedite visa applications to Japan and Luxembourg for customers with high Sesame Credit scores. Caixin notes that Ant has offered “no clear indication of what exactly constitutes Sesame’s definition of an ‘extremely creditworthy person'” or whether the algorithms underlying the rankings are themselves reliable.

Should Chinese customers worry about those algorithms? After all, they make life more convenient, and Western financial institutions use credit rating systems too. Critics say gains in Big Data and artificial intelligence are more insidious in China than the West because China’s political system offers far fewer protections for individual rights. Some fear that in China, “Fourth Industrial Revolution” technologies are setting the stage for an Orwellian distopia.

On June 1, China implemented a sweeping new cybersecurity law the government says was designed to protect the personal information of private users. The first business to run afoul of those new rules? , the American company that defined its brand with iconic ads rebelling against “Big Brother.”