quandora backtest said +42% then the PE column was empty
Quandora
Open-source workflows for AI-assisted financial research
Ran the guided research workflow on Quandora last night against their "10 years of real market data" pitch. Agent wrote a mean-reversion note on three midcaps, backtest green, report looked polished.
Then I opened the evidence table. PE ratios blank on 2 of 3 names. One ticker was a delisted shell that still had price history. Kill switch never tripped because drawdown calc used the wrong NAV series.
Not throwing the project out — open source finance agents are rare — but if your harness can pass with empty fundamentals, you're grading the prose not the trade.
5 comments
Join the discussion
Log in to comment.
yeah the invented-citation problem is worse when the "citation" is a PE ratio. I tried their research report export into Notion and it confidently cited a 2021 10-K footnote that doesn't exist. Draft looked longer and worse, classic.
same class of bug on my side. paper run looked fine until I priced the fills myself — agent assumed mid, live would have slipped ~18 bps on the thin names. i'm keeping Quandora for factor mining but pinning every data field in the prompt now. empty PE = fail the run, no exceptions.
this is exactly the silent-nulls-as-product-bug thing. if the workflow's success metric is "report generated" instead of "every required column populated", it will keep shipping empty PE forever.
i'd fail the run in the harness before the LLM even drafts the narrative. polish is not a signal.
yeah. empty PE should be a hard assert in the agent CI, not a "nice to have" check after the PDF looks pretty. if you can pass with PE=null your eval suite is theater.
sorry for english — i checked one of their sample notebooks last week. when PE missing it forward-fills from last known value. that quietly turns blank fundamentals into stale ones.
for midcaps that is worse than empty cell. you think you have signal. i want hard fail on null PE before any backtest chart.