軟體學徒forever

Descriptors in Python is an interesting approach to simplifying many difficult design problems. I would like to share one of my application of this useful feature. Consider the following code snippet: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 class MailEvent ( object ): def fire ( self , instance , cls ): print "handling mail event from " + str ( cls . __name__ ) + " " + str ( instance . name ) class LogEvent ( object ): def fire ( self , instance , cls ): print "handling log event from " + str ( cls . __name__ ) + " " + str ( instance . name ) # using inheritance class BaseWorker ( object ): def __init__ ( self , name ): self . name = name self . event_factory = { "mail" : MailEvent (), "log" : LogEvent ()} def event ( self , evt ): if evt in self . event_factory : self ....

There are many different ways to break dependencies when doing unit testing in C++. We can use approaches described in the book , including constructor injection, setter injection, extract and override, and...yes, the factory design pattern. They work well in some cases so that we should know the pros and cons of every tricks above. However, they have some common shortcomings: unnecessary run-time indirection by using inheritance (it is getting worse when critical execution path is on the objects) unnecessary interfaces should be added and used. (those interfaces are merely for unit test convenience, not so meaningful in production code) Those traditional tricks used in unit test are based on common OO features. If we focus on C++, can we solve these two problems by different thinking?  I gave template a try by writing a simple word count program. Before we go on, you can clone it to play. The main class (mwc) is a class template with a template parameter contai...

There are several ways to terminate threads. You can use pthread_cancel() to raise a request to them and they would be canceled at some cancellation points, or use pthread_kill() to send a signal to them asynchronously. However, I think they are both not practical . By using pthread_cancel(), we can be sure that the thread would be canceled at some specific POSIX functions. But what if there are other third-party libraries used and there is no way to make sure what POSIX functions are used by them? By using pthread_kill(), it is even worse. We might affect the original read()/write() behaviors when receiving SIGINT. Even we construct safe read()/write() and other functions safe for handling signals, we still have no idea what to cleanup in signal handlers. Signal handling is complex in process, not to mention how to do it well in thread environment which is much younger than the signal mechanism. So how to terminate a thread with minimum effort? It seems only one way: ...

One of my colleague came to me today and ask: I found the free memory is amazingly few even there is no process using memory by observing the result of "top". This leads to poor system performance when kernel starts to use swap. Why? Mmm...How to analyze this problem? My steps are: Since we could not see the cause in user space, it might be that kernel consumed the memory. We tried to see the result of "free". If the "cached" is large, means we need to use fadvise to handle it.  It turned out...not the case!! Ok, I tried to observe the result of the cached status of kernel by "cat /proc/slabinfo". Ooops, it shows dentry_cache occupied 2GB!! My colleague's program tried to generate a lot of new files, this caused a lot of dentries generated by kernel. Linux is not capable of predicting when to release dentry_cache, so it keeps them for performance. Since we realize the root cause, the solution is easy to find. echo 2 > /proc/sys/vm/...

Software watchdog is essential to monitor system's status, especially for those run for very long period of time, such as server programs. The basic idea is simple: construct a trustworth process or thread to monitor the real working process or thread. However, after googling a while, I could not get any satisfactory solution in Python. I think it's time to write this by myself. In this article, I will share how I implement a software watchdog in Python. Before we go on, you may want to pull my code from github to play with. The first thing is to determine what kind of model we want watchdog to run? There are two choices: be a thread, or be a process. Being a thread, we can simplify communication between watchdog and working thread. The problem is that if there are some resources hold on with working thread, watchdog might deadlock. Moreover if working thread is trapped in some loop, we have no graceful way to kill it. My choice is obvious: let watchdog and working on...

在 LWN 看到 一篇好文章 ，討論Linux kernel 中的 ACCESS_ONCE() macro 的用法與意義。文章討論到 multi-thread 程式共用變數所需要注意的一個細節：要使用 volatile 關鍵字去修飾變數以避免編譯器的最佳化造成的問題。 文中以一個在 Linux kernel 的例子來說明，但考慮到這個議題在任何 multi-thread 程式（或任何有 context-switch 的程式中）都會遇到，所以我用簡單的情況來舉例。考慮某個 thread 有如下程式碼片段： for   ( ;; )   { ...     if   ( global_status  ==  ST_MOVE )   {        do_something ( ) ;     } ... 它會依據 global_status 的值來決定程式流程，而這個變數可能會在另一個 thread function 中被修改（也可能在 signal handler 中被修改），所以每次 loop 時，可能會執行 do_something() 也可能不會執行它。問題在於編譯器的最佳化往往都會作在語言沒有清楚定義的地方。在 C 語言中，由於沒有規定 multi-thread 的共享變數的行為（在 C11 前幾乎對此沒有著墨），所以若 for loop 中沒有對 global_status 任何修改，編譯器有可能將上述 loop 中的判斷整個拿掉： // if global_status is ST_SLEEP for   ( ;; )   { ...     /*    if (global_status == ST_MOVE) {        do_something();    }*/ ... 所以一般要避免這種情況發生，都會以 volatile 修飾 globe_status 變數。這時，我們再來看看 ...