Evaluation Framework
A structured approach to assessing the performance, usability, and impact of walking systems.
A structured approach to assessing the performance, usability, and impact of walking systems.
Evaluating walking systems requires more than testing technical accuracy; it involves assessing how systems reflect lived experiences, support diverse users, and contribute to broader well-being. An evaluation framework provides structure for this process, ensuring consistency and comparability across studies while remaining sensitive to contextual variation.
By combining quantitative and qualitative methods, evaluation frameworks can address three key dimensions: system performance, user experience, and societal impact. This balance prevents overemphasis on efficiency at the expense of inclusivity or meaning, creating a more holistic understanding of how walking systems function in practice.
Performance evaluation focuses on how effectively a system processes data and generates outputs. Metrics include routing accuracy, computational efficiency, data integration, and reliability under different conditions. For walking systems, performance must also account for fine-grained spatial requirements, such as correctly modelling crossings, gradients, and surfaces that significantly affect pedestrian experience.
Evaluation at this level often involves benchmarking algorithms, stress-testing databases, and validating outputs against known ground-truth datasets. While essential, performance evaluation alone is insufficient without user-centred perspectives.
User experience evaluation assesses how walkers perceive and engage with the system. This includes usability testing, accessibility reviews, and methods such as think-aloud studies or post-walk interviews. Metrics may cover clarity of interface, interpretability of recommendations, trust in outputs, and alignment with user goals.
For leisure walking in particular, evaluation must capture subjective dimensions such as pleasure, safety, or cultural resonance. Qualitative data can reveal mismatches between system design and lived experience, highlighting opportunities for redesign.
Evaluation also extends beyond individuals to consider collective and societal effects. Walking systems can influence public health, environmental sustainability, and mobility justice. They may encourage inclusive access to urban spaces or inadvertently reinforce inequalities if they fail to represent marginalised users. Evaluation frameworks should therefore incorporate measures of equity, participation, and contribution to wider policy goals such as the Sustainable Development Goals (SDGs).
By situating walking systems in their broader social context, evaluation frameworks can demonstrate real-world value and identify unintended consequences, supporting more ethical and sustainable development.