TY - GEN
T1 - Do Models Improve the Understanding of Safety Compliance Needs?
T2 - 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2016
AU - De La Vara, Jose Luis
AU - Marín, Beatriz
AU - Giachetti, Giovanni
AU - Ayora, Clara
PY - 2016/9/8
Y1 - 2016/9/8
N2 - Context. Many critical systems must meet safety compliance needs from safety standards. These standards are usually large textual documents whose compliance needs can be hard to understand. As a solution, the use of models has been proposed. Goal. We aim to provide evidence of the extent to which models improve the understanding of safety compliance needs. Method. We designed an experiment and ran a pilot to study the effectiveness, efficiency, and perceived benefits of understanding these needs, with the text of standards and with models in the form of UML object diagrams. Results. The overall results from 15 Bachelor students show that the effectiveness of understanding safety compliance needs increases very little with models (2%), and the efficiency even decreases (24%). Nonetheless, the results improve when the potential complexity in navigating the models is taken into account (15% effectiveness increase). The students find benefits in using the models but most consider that the models are hard to understand. Conclusions. The extent to which models improve the understanding of safety compliance needs seems to be lower than what the research community expects. New studies are necessary to confirm our initial insights.
AB - Context. Many critical systems must meet safety compliance needs from safety standards. These standards are usually large textual documents whose compliance needs can be hard to understand. As a solution, the use of models has been proposed. Goal. We aim to provide evidence of the extent to which models improve the understanding of safety compliance needs. Method. We designed an experiment and ran a pilot to study the effectiveness, efficiency, and perceived benefits of understanding these needs, with the text of standards and with models in the form of UML object diagrams. Results. The overall results from 15 Bachelor students show that the effectiveness of understanding safety compliance needs increases very little with models (2%), and the efficiency even decreases (24%). Nonetheless, the results improve when the potential complexity in navigating the models is taken into account (15% effectiveness increase). The students find benefits in using the models but most consider that the models are hard to understand. Conclusions. The extent to which models improve the understanding of safety compliance needs seems to be lower than what the research community expects. New studies are necessary to confirm our initial insights.
KW - Safety-critical system
KW - model
KW - pilot experiment
KW - safety compliance needs
KW - safety standard
KW - understanding
UR - http://www.scopus.com/inward/record.url?scp=84991669605&partnerID=8YFLogxK
U2 - 10.1145/2961111.2962621
DO - 10.1145/2961111.2962621
M3 - Conference contribution
AN - SCOPUS:84991669605
T3 - International Symposium on Empirical Software Engineering and Measurement
BT - 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2016
PB - IEEE Computer Society
Y2 - 8 September 2016 through 9 September 2016
ER -