The Wonderful Wizard of LoC: Paying Attention to the Man behind the Curtain of Lines-of-Code Metrics
Tue 17 Nov 2020 11:20 - 11:40 at SPLASH-III - T-3-Onward Chair(s): Stephen Kell, William Cook
Lines-of-code metrics (loc) are commonly reported in Programming Languages (PL), Software Engineering (SE), and Systems papers. This convention has several different,
often contradictory, goals, including demonstrating the hardness' of a problem, and demonstrating the
easiness' of a
problem. In many cases, the reporting of loc metrics is done not with a clearly communicated intention, but instead in an
automatic, checkbox-ticking, manner.
In this paper we investigate the uses of code metrics in PL, SE, and System papers. We consider the different goals that
reporting metrics aims to achieve, several various domains wherein metrics are relevant, and various alternative metrics
and their pros and cons for the different goals and domains. We argue that communicating claims about research software
is usually best achieved not by reporting quantitative metrics, but by reporting the qualitative experience of
researchers, and propose guidelines for the cases when quantitative metrics are appropriate.
We end with a case study of the one area in which lines of code are not the default measurement—code produced by
papers' solutions—and identify how measurements offered are used to support an explicit claim about the algorithm.
Inspired by this positive example, we call for other cogent measures to be developed to support other claims authors wish
to make.
Tue 17 Nov Times are displayed in time zone: Central Time (US & Canada) change
11:00 - 12:20: T-3-OnwardOnward! Papers / Onward! Essays at SPLASH-III +12h Chair(s): Stephen KellUniversity of Kent, William CookUniversity of Texas at Austin | |||
11:00 - 11:20 Talk | How (Not) to Write Java Pointer Analyses after 2020 Onward! Essays Manas ThakurIIT Mandi Link to publication DOI Pre-print | ||
11:20 - 11:40 Talk | The Wonderful Wizard of LoC: Paying Attention to the Man behind the Curtain of Lines-of-Code Metrics Onward! Essays Kalev AlpernasTel Aviv University, Yotam M. Y. FeldmanTel Aviv University, Hila PelegUniversity of California at San Diego Link to publication DOI | ||
11:40 - 12:00 Talk | Putting the Semantics into Semantic Versioning Onward! Essays Patrick LamUniversity of Waterloo, Jens DietrichVictoria University of Wellington, David J. PearceVictoria University of Wellington, New Zealand Link to publication DOI Pre-print | ||
12:00 - 12:20 Talk | Demystifying Dependence Onward! Papers Link to publication |
23:00 - 00:20: T-3-OnwardOnward! Papers / Onward! Essays at SPLASH-III Chair(s): Crista LopesUniversity of California, Irvine | |||
23:00 - 23:20 Talk | How (Not) to Write Java Pointer Analyses after 2020 Onward! Essays Manas ThakurIIT Mandi Link to publication DOI Pre-print | ||
23:20 - 23:40 Talk | The Wonderful Wizard of LoC: Paying Attention to the Man behind the Curtain of Lines-of-Code Metrics Onward! Essays Kalev AlpernasTel Aviv University, Yotam M. Y. FeldmanTel Aviv University, Hila PelegUniversity of California at San Diego Link to publication DOI | ||
23:40 - 00:00 Talk | Putting the Semantics into Semantic Versioning Onward! Essays Patrick LamUniversity of Waterloo, Jens DietrichVictoria University of Wellington, David J. PearceVictoria University of Wellington, New Zealand Link to publication DOI Pre-print | ||
00:00 - 00:20 Talk | Demystifying Dependence Onward! Papers Link to publication |