西窗月

How to write a paper?

  • First sentence of each paragraph is the most important. If they feel convinced, just skip the paragraph.
  • No one read the paper from beginning to end. Usually people read introduction and related work
    first. If they are interested, they will read the evaluation to see if the evaluation is solid or not. The method to solve the problem is not as important.

Components of introduction.

  • Why your paper is important, short background in summary.
  • continued…

不插电学习法

可能是由于自己本科时经常去图书馆养成的学习习惯,发现自己在安静的、大家都在学习的环境下效率会更高。如果处于一个封闭的独处空间或者开放的办公室空间,在无deadline的情况下,很难进行长时间的专注。

不插电学习法,表示一个0)不携带手机,1)不联接网,2)不插电的工作学习方法,对于写作,写代码这种不太需要信息获取的工作效率提升明显(实验样本目前只有自己)。

不联网即完全隔绝网络上的信息,保持一个无人打扰的状态。没有email,没有sns,没有娱乐网站。如果进行编程的话,可以使用离线文档, 如dash。因为编程大多数时候bug是不需要从网上寻找解决方法的,同时对于自己的写bug能力也有提升。

不插电是为了保持合理的工作休息时间。对于一个老mac用户,不插电电脑可使用4.5小时左右(屏幕亮度在90%以上)。电量每下降30%左右可以休息一首歌的时间(保护视力)。这个方法和番茄工作法的区别之处是保持了更久的工作时间。在电量15%左右时返回一个有手机有网的环境(保护电脑的电池)。这个时候已经应该高效地工作了3个多小时,完成了一个battery time。如果一天能完成两个battery time,那就已经非常高效了。

在办公室的时候可以去楼下的study room或者library,需要和colleague讨论的时候返回实验室去讨论一番。在家也可以去小区的公开study room,效果比在家/实验室效率会高很多。

Disjoint-set (union find)

Union

Assume we initialize each node’s parent is itself. root[i] = i

Union operation is just to set one root to be a child of another root.

1
2
3
4
5
6
7
void union_root(int x, int y) {
int r1 = find_root(x);
int r2 = find_root(y);
if (r1 != r2) {
root[r1] = r2;
}
}

Find

The plain version is to find root of its parent until its parent is itself.

1
2
3
4
5
6
int find_root(int x) {
if (x!=root[x]) {
return find_root(root[x]);
}
return root[x];
}

Path compression (No reason not to use.)

1
2
3
4
5
6
int find_root(int x) {
if (x!=root[x]) {
root[x] = find_root(root[x]);
}
return root[x];
}

or

1
2
3
4
5
6
7
int find_root(int x) {
while (x!=root[x]) {
root[x] = root[root[x]];
x = root[x];
}
return root[x];
}

Reference

  1. princeton slide

Characteristic study

What is empirical study? or characteristic study? It is a way of gaining knowledge by observations (empirical evidence) qualitatively or quantitatively. With collected evidence (data), empirical questions should be answered.

I read four papers of characteristic study papers in a few days, which covers from concurrency bugs[1], incorrect bug fixes[2], configuration errors[3, 4, 6], and log practices[5]. These excellent study papers cover different research themes and share something for common. Here are some notes.

  1. All these studies are done with a specific perspective.

    It is not just the interesting/important topic, but also the findings with a special angle to look at. [1] for concurrency bugs with bug patterns, fix strategies, and manifestation method. [2] for quantitive analysis on significance, types, patterns, reasons of incorrect fixes. [3, 4, 6] are from different perspect of configurations. [3] looks at importance, types, reactions and cause of misconfig. [4] is more about number of configuration settings, and the percentage of parameters being userd well. Based on the findings to propose that we should simplify config and improve config navigation method. [6] focus on latent config errors. [5] is all about log practices, and one interesting finding that log is modified more than other code, which give insights on improving logging practices.

  2. Structure

    Intro - Methodology - Perspect 1-N - Related work - Conclution

    First, be clear what is the problem/overall topic. It should have a strong focus.

    Then think about (0) significance or prevalence of this problem (1) causes/reasons for this problem, (2) types/categories of this problem, (3) characteristic for different types, (4) possible solutions discussion, can be simple to demostrate why the findings beneficial.

  1. Angles for permissions
    Goal: provide insights for developers, users, policy makers! not a single app company.

    What is the problem?

    • Incorrect permission usage by developers, how many developers require too many permissions.
      • How to define permissions developers should not ask for?
        • Permissions that is not relevant with user required functionality.
      • What if it is an app with various functions, how to derermine? If it is done by human determinination, how to do it?
    • Incorrect permission understanding android users, they do not understand current permission design. fine-grained vs coarse grained

      Small perspects

    1. game apps tend to require more permissions, and users less care about it.
      • specify the number of dangerous permissions required by each app
    2. the more famous the app is, the less careful people are about the permissions
      • relationship between number of downloads to permission granting portion.
What data do we have?
- user - app - permission - grant/not 
- app - category - permission 



[1] Learning from Mistakes — A Comprehensive Study on Real World Concurrency Bug Characteristics, ASPLOS’08
[2] How Do Fixes Become Bugs? – A Comprehensive Characteristic Study on Incorrect Fixes in Commercial and Open Source Operating Systems, FSE’11, best paper
[3] An Empirical Study on Configuration Errors in Commercial and Open Source Systems, SOSP’11
[4] Hey, You Have Given Me Too Many Knobs! Understanding and Dealing with Over-Designed Configuration in System Software, FSE’15
[5] Characterizing Logging Practices in Open-Source Software, ICSE’12
[6] Early Detection of Configuration Errors to Reduce Failure Damage, OSDI’16 morning paper

Notes on "learn from mistakes"

What makes good characteristic study paper? Interesting findings with practical implications for users/developers/policy makers.


1. What problem does the paper address?

Real world statistics to show current study on concurrent concurrency bugs is not enough, or can not cover very important perspectives.

13 findings in Table 1, only select interesting ones here.

  • bug patterns: order violations bugs are important but not well studied. Atomicity violations and race conditions are also important. (By showing the numbers or portion.)
  • manifestation: 1. certain partial order between 2 threads make most concurrent bugs appear. 2. concurrent access to multiple variables are important and not studied, probably because it’s hard.
  • bug fix strategies: 1. fix deadlock bugs may introduce new non-deadlock but concurrency bugs.
  • bug avoidance: for language designers, basically Transactional memory designers. order should also be considered to enforce checking.

2. How is it different from previous work, if any?

Even though many works on bug study has been done, but this one stress on concurrency bugs. To see why cc bugs are different, this one is a must read.

3. What is the approach used to solve the problem?

Section2 Methodology part specifies (1) bug sources and why these many bugs covers many software systems (2) categories and dimensions of study, specified in Table 2. (3) Threats to validity, frankly tell readers why it is limited. It makes reviewers stupid to point it out in reviews.

4. How does the paper support or otherwise justify its arguments and conclusions?

5. What do you like / dislike this paper?

  • Show findings and implications in Table 1., really impressive.
  • Very clear perspectives, insightful for users

6. Was the paper, in your judgement, successful in addressing the problem?

  • yes, that’s why it gets so many citations.

Understanding KMP

Useful links for understanding.

  • Understand how kmp works, Post by Ruanyi Feng

  • Understand how to calculate the next table, post from Zhihu

My implementation of calculate next table.

1
2
3
4
5
6
7
8
9
10
11
vector<int> cal_next(string s) {
vector<int> next(s.size(), 0);

int j = 0; // keep track of the pointer it should match
for (int i = 1; i < s.size(); ++i) {
while (j > 0 && s[i] != s[j]) j = next[j-1];
if (s[i] == s[j]) { j++; }
next[i] = j;
}
return next;
}

Wiki Links

Links

Academic blogs

Articles

nachos project notes

proj0

Just find and replace… It is better to read the code through though.

proj1

1.1

In alarm.java, make current thread wait for some interval. Just need an external structure to store the thread according to the priority. And in timerInterrupt() to wake them up if current time is larger than the designated time for each thread.

Note that the structure involves read and write at the same time, need to disable interrupt before handle it, or use some concurrent structure maybe.

1.2

Need to has an external variable to record in which thread the join has been called. And a counter to record how many times has been called. But this is too cumbersome, since it involves atomicity. Maybe just test whether the join thread has been set or not.

1.3

Condition.java implements conditional variable (CV) using semaphores. Just open Semaphore.java and copy paste all the code into Condition2.java… Just kidding. Basicly, you do the samething. Use a simpler interrupt on a wait queue for threads trying to get the conditional variables. Think about that, when you wait (sleep) on some variables in one thread, need to put this thread into the wait queue. When others finished (call wake), then make this thread ready. Done.

Wait… Is that simple? What is the relationship with lock? Each CV has a lock. When wait (sleep) for some CV, you need first release the lock, let next thread N on the wait queue for the lock be ready. Then let the current thread C to sleep, put C on the wait queue for the conditional variable. Then let N acquire the lock, and do things.

It is a little complicated… Let’s think it fundementally. CV is a high level of synchronize primitive, which combines Lock with signal process. When you release the lock, you also signal others you are free. Put others on the wait queue for the lock to be ready. So when you wait for the conditional variables, you do not need to constantly check if the lock is free. Instead, you use a signal procedure to avoid this spinning waiting. Here is a helpful stack overflow link. differences-between-conditional-variables-mutexes-and-locks

1.4

Geoff’s homework is GameMatch, which is really geoff style. It does basicly the samething for locks and conditions. First N-1 players are waiting, then the Nth player comes to wake them all, and increase the match counter. The description is not so clear, so I assume it is first come, first serve order. However it is hard for testing if there are multiple rounds of matches.

proj 2

Since it involves mips gcc which can not run on my stupid macintosh, I switch to the ubuntu on my pc.

2.1 file system calls

Implement calls creat, open, read, write, close, and unlink, as specified in test/syscall.h.

Let’s do it step by step. Startring from the simple ones.

Personally I feel open, close are the simplest ones, from the length of description lol. You need to create a file descriptor table, to hold the fd. size is specified as 16. To read the file name, your need to call APIs, like readVirtualMemoryString in UserProcess.

read(), write should consider many corner cases; like what if read failed, or read only 0 byte, which might be failure, or just no read. ?

unlink(): let fs handle the problem of multiple open files

support multiprogramming

questions:

  1. what should be done, if there is not enough page for the process? My current solution is to allocate pages one by one, if receives an error in the process, free all allocated pages for the process.

Reference

[1] Nachos related document. Especially nachos javadoc for walk through the APIs.

note on "PRES" paper

  • What problem does the paper address?

    • reproduce concurrency bug in production on multi-processsors (multicore), with low overhead in a few coordinated attempts.
  • How is it different from previous work, if any?

    1. Traditional : deterministic replay - the non-determinism (1) input, eg. keyboard (2) system calls, eg. gettime() can be recorded for re-execution. (3) Timing variation from hardware, can be overcomed by logic time, eg. number of instructions executed. But (3) causes problems in multicore.

The most problematic source of non-determinism comes from timing variations caused by hardware, such as cache state, memory refresh-scrubbing, and timing variations on buses and devices.

    1. Timing variations exert an important influence on how threads concurrently interleaves each other, besides thread scheduling. When threads are on multicore, they interact with each other with sync or shared memory. This is different from uni-core.
    2. This can be solved by record all inter-thread interactions. (1) hardware assist to reduce overhead, but these hw do not exist. (2) software approach. 10x - 100x production run overhead. Something like I can save you but you have to stay in hospoital all your life.
      This picture is really helpful for people to understand!
      Fig 2.very clear object, really helpful for people to understand difference
  • What is the approach used to solve the problem?

    Fig 3. PRES approach overview

    • Not reproduce bug in one replay, which cause high overhead in production.
    • Just reproduce bug, not execution paths - how to make sure the bug is caused by the same execution path.
    • Sketches, are just record pointers, different sketch mechanism are just adding different event pointers.
    • It is confusing how a production run failure is reproduced the same. How to make sure they crush for the very same reason. Also for incorrect result, how to make sure it is recorded in the production run, since it’s just sketches not all info?
  • How does the paper support or otherwise justify its arguments and conclusions?

    • 11 opensource applications, 3 categories, but only 13 bugs? Are these bugs representitive? See the characteristic study.
    • very thorough experiment
  • What do you like / dislike this paper?

    • likes: writing very clear, problem important, solution interesting
    • dislike: I still doubt whether it will miss some bugs. The experiment has only 13 bugs. I understand it is hard to collect complicate concurrent bugs and even reproduce them. but …
  • Was the paper, in your judgement, successful in addressing the problem?

    • yes