An Educational Dialogue System for Visually Impaired People

Balestrucci, Pier Felice

;Di Nuovo, Elisa;Sanguinetti, Manuela;Anselma, Luca;Bernareggi, Cristian;Mazzei, Alessandro

2024-01-01

---

Abstract

Finite State Automata (FSA) serve as a fundamental mathematical model of computation, commonly explored in computer science degree programs. Traditionally, FSA are represented through state diagrams or tables, which, in educational settings necessitating assistive technologies, pose accessibility challenges, particularly for Visually Impaired People (VIP). This work is part of a broader initiative focused on creating tools for inclusive access to scientific information. Its main goal is to explore the effectiveness of a dialogue system (DS) as an alternative method for conveying FSA information to VIP, aiming to enhance their comprehension and user experience. To achieve this, a rule-based DS tailored for facilitating FSA access, with a primary focus on VIP as end users, was developed. The research involved an A/B test comparing participants’ comprehension of FSA using the rule-based DS versus standard tabular representations. Statistical analysis was also carried out to evaluate the performance differences between the two methods. The findings indicate that communication through the DS significantly outperforms the tabular representation, establishing it as a viable and effective alternative for both VIP and non-VIP. Although VIP participants displayed slightly varying performance depending on the questions, their feedback favoured the DS for its ease of use and overall user satisfaction. Additionally, the study analysed the capabilities and limitations of popular Large Language Models (LLMs) in describing graphical structures like FSA. Despite their general effectiveness in language tasks, LLMs proved inadequate for accurately and consistently describing FSA, highlighting the need for more controlled and explainable DS approaches in educational contexts.