What is empirical study? or characteristic study? It is a way of gaining knowledge by observations (empirical evidence) qualitatively or quantitatively. With collected evidence (data), empirical questions should be answered.

I read four papers of characteristic study papers in a few days, which covers from concurrency bugs[1], incorrect bug fixes[2], configuration errors[3, 4, 6], and log practices[5]. These excellent study papers cover different research themes and share something for common. Here are some notes.

1. All these studies are done with a specific perspective.

   It is not just the interesting/important topic, but also the findings with a special angle to look at. [1] for concurrency bugs with bug patterns, fix strategies, and manifestation method. [2] for quantitive analysis on significance, types, patterns, reasons of incorrect fixes. [3, 4, 6] are from different perspect of configurations. [3] looks at importance, types, reactions and cause of misconfig. [4] is more about number of configuration settings, and the percentage of parameters being userd well. Based on the findings to propose that we should simplify config and improve config navigation method. [6] focus on latent config errors. [5] is all about log practices, and one interesting finding that log is modified more than other code, which give insights on improving logging practices.

2. Structure

   Intro - Methodology - Perspect 1-N - Related work - Conclution

   First, be clear what is the problem/overall topic. It should have a strong focus.

   Then think about (0) significance or prevalence of this problem (1) causes/reasons for this problem, (2) types/categories of this problem, (3) characteristic for different types, (4) possible solutions discussion, can be simple to demostrate why the findings beneficial.

3. Angles for permissions
   Goal: provide insights for developers, users, policy makers! not a single app company.

   What is the problem?

   • Incorrect permission usage by developers, how many developers require too many permissions.
     • How to define permissions developers should not ask for?
       • Permissions that is not relevant with user required functionality.
     • What if it is an app with various functions, how to derermine? If it is done by human determinination, how to do it?

   • Incorrect permission understanding android users, they do not understand current permission design. fine-grained vs coarse grained

     Small perspects

   1. game apps tend to require more permissions, and users less care about it.
      • specify the number of dangerous permissions required by each app
   2. the more famous the app is, the less careful people are about the permissions
      • relationship between number of downloads to permission granting portion.

What data do we have?
- user - app - permission - grant/not 
- app - category - permission 

[1] Learning from Mistakes — A Comprehensive Study on Real World Concurrency Bug Characteristics, ASPLOS’08
[2] How Do Fixes Become Bugs? – A Comprehensive Characteristic Study on Incorrect Fixes in Commercial and Open Source Operating Systems, FSE’11, best paper
[3] An Empirical Study on Configuration Errors in Commercial and Open Source Systems, SOSP’11
[4] Hey, You Have Given Me Too Many Knobs! Understanding and Dealing with Over-Designed Configuration in System Software, FSE’15
[5] Characterizing Logging Practices in Open-Source Software, ICSE’12
[6] Early Detection of Configuration Errors to Reduce Failure Damage, OSDI’16 morning paper