SPLASH 2020
Sun 15 - Sat 21 November 2020 Online Conference
Tue 17 Nov 2020 23:20 - 23:40 at SPLASH - T-3-Onward
Tue 17 Nov 2020 11:20 - 11:40 at SPLASH - T-3-Onward

Lines-of-code metrics are commonly reported in PL, SE, and Systems papers. This convention has several different, often contradictory, goals, including demonstrating the “hardness” of a problem, and demonstrating the “easiness” of a problem. In many cases, the reporting of loc metrics is done not with a clearly communicated intention, but instead in an automatic, checkbox-ticking, manner.

In this paper we investigate the uses of code metrics in PL, SE, and System papers. We consider the different goals that reporting metrics aims to achieve, several various domains wherein metrics are relevant, and various alternative metrics and their pros and cons for the different goals and domains. We argue that communicating claims about research software is usually best achieved not by reporting quantitative metrics, but by reporting the qualitative experience of researchers, and propose guidelines for the cases when quantitative metrics are appropriate.

We end with a case study of the one area in which lines of code are not the default measurement—code produced by papers’ solutions—and identify how measurements offered are used to support an explicit claim about the algorithm. Inspired by this positive example, we call for other cogent measures to be developed to support other claims authors wish to make.

Tue 17 Nov
Times are displayed in time zone: Central Time (US & Canada) change

11:00 - 12:20: T-3-OnwardOnward! Papers / Onward! Essays at SPLASH +12h
11:00 - 11:20
Talk
Onward! Essays
Manas ThakurIIT Mandi
11:20 - 11:40
Talk
Onward! Essays
Kalev AlpernasTel Aviv University, Yotam FeldmanTel Aviv University, Hila PelegUniversity of California, San Diego
11:40 - 12:00
Talk
Onward! Essays
Jens DietrichVictoria University of Wellington, David PearceVictoria University of Wellington, New Zealand, Patrick LamUniversity of Waterloo
Pre-print
12:00 - 12:20
Talk
Onward! Papers
23:00 - 00:20: T-3-OnwardOnward! Papers / Onward! Essays at SPLASH
23:00 - 23:20
Talk
Onward! Essays
Manas ThakurIIT Mandi
23:20 - 23:40
Talk
Onward! Essays
Kalev AlpernasTel Aviv University, Yotam FeldmanTel Aviv University, Hila PelegUniversity of California, San Diego
23:40 - 00:00
Talk
Onward! Essays
Jens DietrichVictoria University of Wellington, David PearceVictoria University of Wellington, New Zealand, Patrick LamUniversity of Waterloo
Pre-print
00:00 - 00:20
Talk
Onward! Papers