AI token usage is not productivity. What to measure instead

• Amazon and Meta both retired internal AI token usage leaderboards this year after employees gamed them
• Southeast Asian firms are setting their AI metrics now, with six in ten reporting under 5% EBIT impact

China’s daily AI token usage reached 140 trillion in March, up from roughly 100 billion at the start of 2024, National Data Administration director Liu Liehong told the China Development Forum in Beijing. He gave tokens an official Chinese name, ciyuanand described them as the settlement unit connecting technological supply with commercial demand. The administration now reports the number at policy forums, as other governments report industrial output.

That is a defensible measure at the national level. Token throughput tells you something real about how much inference an economy is running. The trouble starts when the same number travels down the org chart and lands on an individual employee, which is already happening.

The South China Morning Post reported last week that Beijing-based Kunlun Tech announced in February it would give engineers access to US coding agents while evaluating them on how many tokens they consumed. The same report describes Tencent moving the other way since June, halving internal quotas and requiring fresh approval every two weeks, according to an employee working on AI agents.

Both moves are worth watching, because the US ran this experiment first and finished it in about eight weeks.

What happened when AI token usage became the scoreboard

In April, The Information reported on an internal Meta dashboard called Claudeonomics, named after Anthropic’s Claude. It was not a management tool. An employee built it on the company intranet using company data, and it ranked the top 250 token consumers across more than 85,000 staff, handing out titles including Token Legend and Cache Wizard.

The leaderboard came down two days after the story ran. Meta told Fortune the employee took it down at their own discretion and that the company did not request it. Amazon’s version lasted slightly longer. KiroRank, a beta dashboard tied to the company’s Kiro developer platform, ranked employees by AI activity against a target of more than 80% of developers using AI each week.

The Financial Times reported that staff had begun assigning agents to unnecessary tasks purely to climb the table, a practice engineers named tokenmaxxing, and that the resulting compute bills were the reason it was switched off. Business Insider confirmed the 29 May shutdown with an Amazon spokesperson, who said the tracker was informal, created by a group of employees, and never meant to promote usage for its own sake.

Amazon senior vice-president Dave Treadwell told staff not to use AI just for the sake of using AI. It is important to note that neither company abandoned measurement; they merely changed the unit. Amazon replaced KiroRank with normalised deployments, which count AI-assisted code that actually reaches production. Meta moved in June to cap internal spending and route usage through a centralised platform with alerts for unusual spikes, according to an internal memo reported by The Information.

Staff shorthand shifted from tokenmaxxing to tokenminimizing. The failure was not that these companies measured AI. It was that they measured the input and called it the output. Token consumption records what an employee spent, not what the organisation got, and the gap between those two things is precisely where an ambitious engineer with an idle agent goes to work.

Why AI token usage is the wrong number for Southeast Asia

The region is about to make this decision, and it will make it during an agent rollout rather than after one. McKinsey, working with Singapore’s Economic Development Board and Tech in Asia, surveyed 330 companies across Indonesia, Malaysia, the Philippines, Singapore, Thailand and Vietnam for a report published in February. Roughly 46% have moved beyond piloting, marginally ahead of the global average.

More than six in ten are putting between 11% and 40% of their technology budgets into AI. Nearly nine in ten expect to experiment with AI agents this year. The returns have not arrived at the same speed. Around six in ten respondents reported less than 5% EBIT impact from their AI use, and close to one in five reported no discernible financial effect at all. Unclear ROI ranked among the top barriers to value capture, alongside talent shortages and integration complexity.

That combination is what makes the metric choice urgent rather than academic. A company with rising AI spend, imminent agent deployment and no bottom-line signal has a strong incentive to reach for a number that moves. Token consumption moves beautifully. It goes up every quarter, it is already in the billing data, and it costs nothing to instrument.

Grab’s group head of data and analytics, Nikhil Dwarakanath, told the researchers that “the Cambrian explosion of AI token usage is growing very quickly,” while cautioning that small independent entrepreneurs in the region should not be left behind by it. Growth in usage is the easy part.

The metrics that survived contact with reality

The same survey identifies what separates the small group of high performers from everyone else, and none of it is consumption. They are roughly twice as likely to fundamentally redesign workflows rather than layer AI onto existing processes. They invest at a different magnitudewith more than a third putting over 20% of their digital budgets into AI.

They formalise governance and define when human validation is required. Nearly half report senior leaders taking genuine ownership of AI initiatives. The report’s own guidance on measurement is pointed. It advises tracking usage and business outcomes together, not technical metrics alone.

Petronas offers the closest thing the region has to a worked example. Chief data scientist Rajamani Sambasivam told the researchers the company deliberately never wrote a separate AI strategy, treating the business strategy as the AI strategy, and warned that a technology-first approach produces misplaced focus and inflated expectations.

The company reports that more than 85% of the value delivered through its digital solutions comes from AI and data science work. Its citizen analytics programme has upskilled over 26,000 employees, with more than 5,000 trained to build machine learning models. That last figure is the one worth sitting with. It is a headcount numberand it is doing the job that a token count cannot.

At DBS Bank, group chief operating officer Derrick Goh framed the objective as serving customers better and operating more effectively rather than cutting headcount, with the largest use cases concentrated in call centres and code writing and the rest going toward augmenting what people already do.

None of this is an argument against instrumenting AI use. Finance directors need the consumption data, and the cost line is real enough that Meta built a monitoring platform around it. The argument is narrower: the number that belongs in a cloud invoice does not belong in a performance review, and the two companies that tried hardest to put it there have both quietly taken it back out.

Southeast Asian enterprises get to skip that entire process. The question their boards should be asking is not how many tokens the organisation burned last quarter. It is how much of that burn reached a customer, shipped to production, or closed a case, and whether anyone can currently tell.

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.