Top LLMs Misidentify Poisonous Mushrooms in Every Ninth Case, Benchmark Shows
Polish developer Piotr Migdal evaluated leading large language models on their ability to identify mushrooms from photographs, using a dataset of 1040 images covering 55 species common in Poland. The images came from the FungiTastic dataset derived from the Atlas of Danish Fungi, with expert labels and partial DNA confirmation. Models were asked to return the five most likely species names in Latin without additional training or tools. Gemini 3.8 Flash performed best with 65 percent top-1 accuracy and 85 percent top-5 accuracy, followed closely by other Gemini variants. However, safety-critical errors remained high: Gemini models labeled poisonous mushrooms as edible in roughly 11 percent of cases, while GPT-5.6 Sol reached 24 percent, Claude Opus 5 reached 29 percent, and Qwen 3.8 27B reached 36 percent. The study did not ask models directly whether a mushroom was edible; species identifications were later cross-checked against toxicity tables.
AntiMalwareSep 4, 2026Translated from Russian