What makes good characteristic study paper? Interesting findings with practical implications for users/developers/policy makers.

1. What problem does the paper address?

Real world statistics to show current study on concurrent concurrency bugs is not enough, or can not cover very important perspectives.

13 findings in Table 1, only select interesting ones here.

• bug patterns: order violations bugs are important but not well studied. Atomicity violations and race conditions are also important. (By showing the numbers or portion.)
• manifestation: 1. certain partial order between 2 threads make most concurrent bugs appear. 2. concurrent access to multiple variables are important and not studied, probably because it’s hard.
• bug fix strategies: 1. fix deadlock bugs may introduce new non-deadlock but concurrency bugs.
• bug avoidance: for language designers, basically Transactional memory designers. order should also be considered to enforce checking.

2. How is it different from previous work, if any?

Even though many works on bug study has been done, but this one stress on concurrency bugs. To see why cc bugs are different, this one is a must read.

3. What is the approach used to solve the problem?

Section2 Methodology part specifies (1) bug sources and why these many bugs covers many software systems (2) categories and dimensions of study, specified in Table 2. (3) Threats to validity, frankly tell readers why it is limited. It makes reviewers stupid to point it out in reviews.

4. How does the paper support or otherwise justify its arguments and conclusions?

…

5. What do you like / dislike this paper?

• Show findings and implications in Table 1., really impressive.
• Very clear perspectives, insightful for users

6. Was the paper, in your judgement, successful in addressing the problem?

• yes, that’s why it gets so many citations.