Sam Lewis-Lim

I am a PhD student in the Natural Language Processing group at the University of Sheffield, supervised by Nikos Aletras and Cass Zhao. Before starting my PhD, I completed an MSc in Computer Science with Speech and Language Processing at Sheffield and previously worked as a software engineer at Dojo, a financial technology company in London.

My research focuses on trustworthy reasoning in LLMs, including monitoring, faithfulness and understanding how LLMs use Chain-of-Thought reasoning. I am also currently interested in LLM introspection and self-knowledge.

Feel free to contact me at: slewis-lim1[at]sheffield.ac.uk. (Note: The email listed in some of my papers is currently incorrect)

news

Oct 05, 2026	📄 New preprint looking the effect of efficient reasoning training on CoT faithfulness and monitorability: Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability
Sep 07, 2026	Think When Unsure: Leveraging Model Confidence to Decide When to Use Chain-of-Thought has been accepted to AACL Findings 2026!!
Aug 13, 2026	🌉🌉🌉 Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models has been accepted to COLM 2026!!! I will be presenting this in San Francisco in October
Aug 20, 2025	Our paper on Chain-of-Thought reasoning influence and faithfulness was accepted to EMNLP 2025 main conference! 🎉🎉🎉

selected publications

COLM

Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models

Danae Sanchez Villegas, Samuel Lewis-Lim, Nikolaos Aletras, and 1 more author

In Third Conference on Language Modeling, 2026

(Accepted to COLM 2026)

Abs arXiv

Recent advances in vision language models (VLMs) offer reasoning capabilities, yet how these unfold and integrate visual and textual information remains unclear. We analyze reasoning dynamics in 18 VLMs covering instruction-tuned and reasoning-trained models from two different model families. We track confidence over Chain-of-Thought (CoT), measure the corrective effect of reasoning, and evaluate the contribution of intermediate reasoning steps. We find that models are prone to answer inertia, in which early commitments to a prediction are reinforced, rather than revised during reasoning steps. While reasoning-trained models show stronger corrective behavior, their gains depend on modality conditions, from text-dominant to vision-only settings. Using controlled interventions with misleading textual cues, we show that models are consistently influenced by these cues even when visual evidence is sufficient, and assess whether this influence is recoverable from CoT. Although this influence can appear in the CoT, its detectability varies across models and depends on what is being monitored. Reasoning-trained models are more likely to explicitly refer to the cues, but their longer and fluent CoTs can still appear visually grounded while actually following textual cues, obscuring modality reliance. In contrast, instruction-tuned models refer to the cues less explicitly, but their shorter traces reveal inconsistencies with the visual input. Taken together, these findings indicate that CoT provides only a partial view of how different modalities drive VLM decisions, with important implications for the transparency and safety of multimodal systems.
EMNLP

Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?

Samuel Lewis-Lim, Xingwei Tan, Zhixue Zhao, and 1 more author

In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Aug 2025

(Accepted to EMNLP 2025 Main Conference)

Abs arXiv

Recent work has demonstrated that Chain-of-Thought (CoT) often yields limited gains for soft-reasoning problems such as analytical and commonsense reasoning. CoT can also be unfaithful to a model’s actual reasoning. We investigate the dynamics and faithfulness of CoT in soft-reasoning tasks across instruction-tuned, reasoning and reasoning-distilled models. Our findings reveal differences in how these models rely on CoT, and show that CoT influence and faithfulness are not always aligned.