How to see the number of tokens (prompt_tokens, completion_tokens) consumed by the llm node?

[gukoff]

I have a flow with 2 nodes: llm node and python node that parses LLM response.

I can batch-run the flow, and it produces correct results:

pf run create -f flows/my_flow/batch_run.yaml --name kgukov_run_20250211134714 --data /path/to/data/dataset.csv

The problem is, promptflow reports 0 token usage by the LLM.

Same behavior is seen when:

• using local or AzureML connection provider to connect to the LLM
• running with PF_DISABLE_TRACING=false
• running with promptflow packages 1.16.0, 1.17.1 and 1.17.2

How do I get proper token usage metrics?

---

Flow:

Metrics after batch inference run:

Metrics from pf run show:

{
   ...
   "system_metrics": {
            "total_tokens": 0,
            "prompt_tokens": 0,
            "completion_tokens": 0,
            "duration": 52.001273
        }
}

Metrics from the trace UI

Package versions

promptflow==1.17.1
promptflow-azure==1.17.1
promptflow-core==1.17.1
promptflow-devkit==1.17.1
promptflow-tools==1.4.0
promptflow-tracing==1.17.1

[LauraDamianTNA]

I was able to reproduce it with 1.17.2 as well.